System and method for semi-automatic image segmentation with user interaction loop learning and related training methods - Patents.com

Abstract

The invention relates to a semi-automatic image segmentation system comprising a neural network (12) configured to receive, for each iteration, input data comprising at least one image (14) to be segmented and to provide output data comprising at least one segmentation mask (16) for each image, The semi-automatic image segmentation system further comprises a user interaction loop (18) with a user interaction module (20) configured to provide at least one user mask representing a real and / or simulated user interaction with each segmentation mask, the user interaction loop (18) being configured to provide as input data for the neural network (12) each segmentation mask provided by the neural network resulting from a previous iteration of the neural network (12), called past segmentation masks, and a user mask associated with each past segmentation mask.

Description

[Technical field]

[0001] The invention relates to a system for 2D or 3D image segmentation with a neural network, to an image segmentation method using this system and to a method for training the segmentation system, in particular the neural network. The invention is particularly intended for the segmentation of medical images, for example by magnetic resonance imaging (MRI) or computed tomography (CT), for the identification of organs in each of the cut images obtained by MRI or CT, in order to reconstruct a 3D modeling of the organs from these segmentations. [Background technology]

[0002] Image segmentation is a technique that makes it possible to associate to each pixel or voxel of a 2D or 3D image a label that represents the object, texture, condition, etc. that corresponds to said pixel or voxel.

[0003] Image segmentation is used in particular in the medical field to determine organs, tissues, fluids, etc. visible in 2D or 3D images. Such images are obtained, for example, by MRI, and make it possible to form a 3D model of the target organ and, for example, its environment and the tumor to be operated on. The reconstruction of this 3D model requires the segmentation of the images obtained by MRI.

[0004] Current segmentation techniques fall into several categories: manual segmentation, semi-automatic segmentation (also called interactive segmentation), or fully automatic segmentation.

[0005] The manual approach is time consuming and requires the user to classify each pixel or voxel one by one, potentially taking hours for a single cut.

[0006] Although automatic approaches allow segmentation without user intervention, the lack of control can adversely affect the results and their practical application. Furthermore, the results are never perfect and for better results, a human operator must maintain some degree of control for validation and to correct errors.

[0007] Semi-automatic methods allow interactivity, where the segmentation is mostly automatic, but with a degree of control by the user, who can act especially upon initialization and modification during certain steps.

[0008] Different segmentation procedures can be implemented using deep learning, in particular methods using artificial neural networks. Deep learning is used especially in fully automated methods, but semi-automated segmentation solutions using deep learning have received limited attention compared to the number of automated solutions. Neural networks have made it possible to significantly improve the performance of automatic segmentation solutions, but semi-automated segmentation has not benefited from such improvements due to the difficulty of achieving the integration of neural networks.

[0009] In particular, the use of neural networks in semi-automatic segmentation is problematic in that it takes into account user interactions during the training of the neural network, which creates problems in training the system because the training data that takes into account the user interactions must be adapted to the output of the neural network with the interactions applied, and the neural network is not trained at the beginning of the process, which results in a problematic dependency, where the input data required for training itself depends on the output data with the user interactions applied.

[0010] Solutions proposed to address these issues do not provide the desired level of performance, especially due to the fact that user interactions are generally not considered in a dynamic way and have little impact on the quality of training due to the focus on the input data. Proposed solutions generally address training neural networks for semi-automatic segmentation tasks in the same way as automatic segmentation. They generate user interaction masks from annotated data in a static way before training or in a dynamic way during training. Static generation methods form the input structure to the neural network closest to automatic segmentation and do not consider the dynamic nature of user interactions. Existing dynamic generation methods, although capable of diversifying the training data and improving performance, do not directly address the problem of dependencies between the output and input of a system that considers interactions. In particular, the user interactions used for dynamic processing do not reproduce typical user interaction sequences, since they do not consider the order of these interactions.

[0011] Therefore, the inventors sought to improve the semi-automatic segmentation solution so that it could better take user interaction into account, and in particular so that the influence of user interaction could be taken into account during training when the solution is implemented. [Prior art documents] [Non-patent literature]

[0012] [Non-Patent Document 1] He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770-778

Non-Patent Document 2

Non-Patent Document 3

Non-Patent Document 4

Non-Patent Document 5

[0013] The present invention aims to provide a system and method for semi-automatic image segmentation, particularly for medical images, such as cuts obtained by MRI.

[0014] The present invention also aims to provide a method for training such a segmentation system.

[0015] The present invention, in at least one embodiment, aims to provide a semi-automatic segmentation system that can be easily trained by simulating user interactions.

[0016] The present invention, in at least one embodiment, aims to provide a segmentation system that allows operation and training to take into account previous segmentation results.

[0017] The present invention, in at least one embodiment, aims to provide a segmentation system that allows operation and training to take previous user interactions into account. [Means for solving the problem]

[0018] To this end, the invention provides a semi-automatic image segmentation system comprising a neural network configured to receive, for each iteration, input data comprising at least one image to be segmented and to provide output data comprising at least one segmentation mask for each image, said system comprising: a user interaction loop with a user interaction module configured to provide at least one user mask representing a real and / or simulated user interaction with each segmentation mask, the user interaction loop being configured to provide as input data for the neural network each segmentation mask provided by the neural network resulting from a previous iteration of the neural network, called past segmentation masks, and a user mask associated with each past segmentation mask.

[0019] The image segmentation system according to the invention thus makes it possible to take into account user interactions simultaneously during training and use for image segmentation, and to associate these user interactions as new inputs for the neural network with the segmentation mask obtained at the output of the neural network.

[0020] In particular, the training data is not only obtained by the recovery of a large dataset, but is generated by the segmentation system itself, by a loop of user interaction. The user mask cannot be prepared in advance, since it is adapted depending on the output data of the network. Without this loop, like an automatic segmentation system, the system is only trained from the images to be segmented, and therefore cannot take into account the interaction with the user when it is used. Using only one input dataset without a loop will not give satisfactory results, regardless of the capacity of the input dataset. In particular, using only one input dataset without a loop will result in a system that does not allow correction of the obtained results. Such corrections are needed in many fields, especially in the medical field, where the user must verify or edit the results before using them in the treatment action.

[0021] Furthermore, using the output segmentation mask as input allows to introduce continuity in the processing of the data by taking into account past segmentation masks when defining better future segmentations. Using the output segmentation mask and user interactions as input data allows to configure an internal sequential memory that keeps a trace of past segmentations and user interactions in order to maintain the continuity of the system. In particular, the internal sequential memory keeps the previous segmentation masks obtained with every user interaction and the output of the neural network in order to provide them as new inputs for the neural network. The segmentation is sped up because successive user interactions have a greater impact on the result.

[0022] User interaction loops are used during training of the neural network, during testing of the segmentation system from test data, and during use of the segmentation system on new data.

[0023] Real user interaction is understood to be the interaction of a human user interacting directly with the segmentation mask provided at the output. Simulated user interaction is understood to be the simulation of interactions that a human user could perform, defined by a user interaction simulation algorithm. User interaction could for example be the assignment of specific labels to exact points in the segmented image.

[0024] User interaction in particular allows corrections to be made to the segmentation mask obtained at the output of the neural network, for example indicating incorrectly labeled zones of the 2D or 3D image. Taking the user into account from the time of training the system makes it possible to improve the impact of user corrections when using the system. Advantageously, and by means of its training method, the image segmentation system can itself influence the initial segmentation when it is trained and takes into account user corrections for improving the segmentation. The initial segmentation may be trivial, in particular one with little or no informative value, or one calculated by another segmentation method (typically a fully automatic method).

[0025] The system can be advantageously used for segmenting MRI images, but can also be applied to other types of 2D or 3D images, medical or not. The system can be integrated into a larger system using the results of the segmentation for specific applications, such as creating a 3D model from the segmented data and displaying the 3D model in augmented reality (AR) on a video.

[0026] The segmentation system does not depend on a specific neural network to be used. In fact, image segmentation systems are particularly characterized by a user interaction loop that is external to the neural network. The neural network can integrate modules of the Recurrent Neural Network (or RNN) type.

[0027] A module may consist of, for example, a computing device such as a computer, a group of computing devices, an electronic component or a group of electronic components, or, for example, a computer program, a group of computer programs, a library of computer programs, or functionality of a computer program executed by a computing device such as a computer, a group of computing devices, an electronic component or a group of electronic components.

[0028] Advantageously, and according to the invention, each user mask representing a simulated user comprises a simulation of at least one interaction brought about by the simulated operator on a segmentation mask associated with the user mask, the position of said interaction on the image being determined according to a difference between said output segmentation mask and a ground truth.

[0029] According to this aspect of the invention, the location of the simulated interactions is determined to approximate what a real user would do, i.e., to position his / her interactions in zones that are poorly segmented or labeled, or that are unlabeled.

[0030] Advantageously, and according to the invention, the position of the interaction is determined according to the centre of the area selected as needing to be rectified by user interaction.

[0031] Advantageously and according to the invention, the system comprises a graph cut module adapted to apply a graph cut to the output data of the neural network in order to obtain each segmentation mask. According to this aspect of the invention, this step makes it possible to smooth the output data using a graph cut algorithm in order to improve the relevance of the labels integrated in the segmentation masks. The graph cut algorithm is applicable for binary segmentation, while a derived algorithm is used for multi-class segmentation.

[0032] Advantageously, and according to the invention, the user interaction loop comprises an internal sequential memory in which each previous segmentation mask and each user mask obtained at each iteration is stored in turn.

[0033] Advantageously, and in accordance with this latter aspect of the invention, the internal sequential memory comprises a first-in-first-out, or FIFO, type file which stores, at each iteration, the segmentation mask and the user mask corresponding to that iteration.

[0034] The invention also relates to a method for training an image segmentation system according to the invention using at least one dataset comprising images to be segmented, comprising the steps of: - at least two iterations during which the neural network processes the input data to provide output data, namely a first iteration during which the input data comprises at least one image to be segmented and a predefined initialization mask, and subsequent iterations during which the input data comprises each image to be segmented, each previous segmentation mask obtained in a previous iteration, and a user mask representing the interactions of a real user and / or a simulated user; - updating the weights of the neural network from the output data after each iteration, or when a predetermined number of iterations have been performed by the neural network; The present invention relates to a method comprising the steps of:

[0035] The training method according to the invention makes it possible to simulate user interactions in order to maximize the effectiveness of the training and the relevance of the neural network resulting from the training. In contrast to the prior art models in which weights are updated after each iteration, the training method according to the invention advantageously uses the user interaction loop of the system in order to make it possible to use the user interaction simulation and the output data, i.e. the segmentation mask of the previous iteration, as input data for the system. In this way, the image segmentation system makes it possible to improve the user interaction simulation, taking into account past segmentation masks and user interactions, based on the quality control of the segmentation obtained. This training makes it possible to reduce the number of false positives and false negatives in the simulation that a user would make when checking the quality of the segmentation.

[0036] By repeating such training, the accuracy of the segmentation can be improved.

[0037] The weight update is for example implemented by a loss function minimization algorithm, the gradient of which is calculated for example by backpropagation. The weight update can be performed at each iteration, in particular each time output data is generated by the neural network, or it can be performed only at the end of every iteration, so that the weight update is only done from the best segmentation obtained.

[0038] The initialization mask may be, for example, a trivial mask, an empty mask, a mask obtained by automatic pre-segmentation, or a mask derived from the ground truth.

[0039] During the next iteration, the initialization mask is added to a user mask that indicates the error between the segmentation mask obtained at the output of the neural network and the result desired by the user.

[0040] Each dataset can comprise one or more images, which may or may not be different. Even if two datasets comprise the same images, the training method processes two identical images differently, with simulated user interactions allowing for modification of the training.

[0041] The simulated user interactions are obtained from a ground truth, which is formed by predefined labeled data, from which the error between the segmentation mask obtained at the output of the neural network and the desired result is determined. Thus, training is supervised by previously segmented data with labels on image regions.

[0042] In prior art segmentation methods, training is generally done only from these labeled data, which is not sufficient because the user and the segmentation masks of previous iterations are not taken into account in the training. Furthermore, our training method allows providing different simulated user interactions on the same image, as is the case for real users who never provide the same interaction on the same image.

[0043] Advantageously, according to the invention, the pre-defined number of iterations is determined for each image and each data set to be segmented independently of pre-defined numbers of iterations determined for other images and other data sets to be segmented.

[0044] According to this aspect of the invention, the number of iterations is predetermined during processing of the dataset or image to be segmented, but can be varied over time to replicate the behavior of real users who do not systematically perform the same number of corrections. The maximum number of iterations can be fixed so that training does not take too long. Fixing the maximum number of iterations also makes the network less dependent on interactions during the testing phase. Since the interactions are based on the ground truth, it is desirable to aim for minimal interactions during the training phase, followed by maximizing performance with minimal interactions during the testing phase.

[0045] The invention also relates to a semi-automatic image segmentation method, comprising the steps of processing input data comprising each image to be segmented by a neural network, in order to provide, at each iteration, output data comprising at least one segmentation mask for each image, the steps comprising: Each iteration is - receiving a user mask representing a real user interaction with each segmentation mask; - sending each segmentation mask and the user mask provided by said neural network as input data for a next iteration of the neural network; The present invention relates to a semi-automatic image segmentation method, comprising:

[0046] According to this aspect of the invention, the image segmentation method allows taking into account the user's interactions as many times as necessary until the user considers that the quality of the image segmentation meets the user's expectations, and if the quality is not sufficient, the user's interactions and the segmentation mask are sent as input data to provide a new segmentation mask.

[0047] Advantageously, the image segmentation system according to the invention is adapted to implement the image segmentation method according to the invention.

[0048] Advantageously, the image segmentation method according to the invention is implemented by an image segmentation system according to the invention.

[0049] The invention also relates to a computer program product for training an image segmentation system according to the invention, said computer program product comprising program code instructions for carrying out the steps of the training method according to the invention, when said computer program product is run on a computer.

[0050] The invention also relates to a computer program product for semi-automatic image segmentation, said computer program product comprising program code instructions for executing the segments of the image segmentation method according to the invention, when said computer program product is executed on a computer.

[0051] The invention also relates to an automatic segmentation system, an automatic segmentation method, an automatic segmentation computer program product, a training method and a training computer program product characterized by all or some combination of the features mentioned above or below.

[0052] Other objects, features and advantages of the present invention will become apparent upon reading the following description, given in a non-limiting manner only, and made with reference to the accompanying drawings, in which: [Brief description of the drawings]

[0053] [Figure 1] FIG. 1 is a schematic diagram of a semi-automatic image segmentation system according to an embodiment of the present invention. [Diagram 2] FIG. 1 is a schematic diagram of a semi-automatic image segmentation method according to an embodiment of the present invention. [Diagram 3] FIG. 2 illustrates diagrammatically the steps of a method for training a semi-automatic image segmentation system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0054] In the figures, for purposes of illustration and clarity, scale and proportions have not been strictly respected.

[0055] Furthermore, identical, similar or analogous elements are designated with the same reference numerals in all figures.

[0056] FIG. 1 illustrates a schematic of a semi-automatic image segmentation system 10 according to one embodiment of the present invention.

[0057] The segmentation system 10 comprises a neural network 12 configured to receive, at each iteration, input data comprising at least one image 14 to be segmented and to provide output data comprising at least one segmentation mask 16 for each image. The segmentation mask defines, for each part of the image, a label that allows to identify the region of the image that corresponds to a particular class. The segmentation can be a binary segmentation, but also a multi-class segmentation, making it possible to identify regions of more than two classes. At the output of the neural network, the output data can be smoothed by graph cuts in order to obtain the segmentation masks.

[0058] The segmentation system 10 is integrated around an existing automated learning model, for example using an existing encoder-decoder neural network architecture with a recurrent neural network module. In one embodiment, for example, a ResNet34 encoder, as described in the publication "He, K., Zhang, X., Ren, S., Sun, J., 2016, Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770-778.," is used, and a decoder with a pair of standard convolutional layers and recurrent long short-term memory (LSTM) layers corresponding to each step of the oversampling path.

[0059] To counteract the imbalance in the dataset, the system uses focal loss as described in Lin, TY, Goyal, P., Girshick, RB, He, K., Dollar, P., 2017.Focal loss for dense object detection. 2017 IEEE International Conference on Computer Vision (ICCV), 2999-3007. and pre-computed per-class weights to scale the dataset.

[0060] The neural network is pre-trained, for example, on the ImageNet dataset and then refined on a dataset dedicated to the desired task, which may be, for example, a dataset comprising MRI-type medical images of the female pelvis that have been manually segmented by manual segmentation software such as 3DSlicer and MITK.

[0061] The female pelvis MRI dataset created by the applicant contains 97 MRI series, which represents a total of 3,066 cuts. This female pelvis MRI dataset is used to train a neural network in the following manner. - 77 MRI series corresponding to 2,449 cuts for the training set, - 10 MRI series corresponding to 308 cuts for the validation set, and - 10 MRI series equivalent to 309 cuts for the test set.

[0062] A cut of the female pelvis MRI data set can be pre-processed via normalization, standardization and bias correction, e.g. vertical and horizontal turnover, intensity offset of brightness, gamma correction of contrast, as well as blurring and soft filters to adjust sharpness, etc. Random augmentation of the data can be predicted.

[0063] In the scope of MRI type medical images for gynecological applications, examples of labels could be uterus, bladder, possible tumor, cavity, background, etc. forming segmentation classes.

[0064] The image segmentation system 10 differs from the prior art in that it further comprises a user interaction loop 18 with a user interaction module 20 configured to provide at least one user mask representing real and / or simulated user interaction with each segmentation mask.

[0065] The user interaction loop 18 provides input data for the neural network: each segmentation mask 16 provided by said neural network from a previous iteration of said neural network 12, called past segmentation mask, as indicated by the arrow 22; a user mask associated with each previous segmentation mask 16, obtained by a user interaction module 20, as indicated by arrow 24; is configured to provide.

[0066] The segmentation mask 16 and the user mask are related via a state module 26 to form a state 28 formed from the combination of the segmentation mask and the user mask to form new input data. During the first iteration, the state 28 is fed by an equivalent empty structure 30, a mask obtained by an automatically obtained pre-segmentation, or a mask obtained from the ground truth when the system is being trained. In this way, the state 28 integrates all past segmentation masks and user masks and forms the internal sequential memory of the segmentation system.

[0067] The user interaction loop 18 is external to the neural network and does not depend on a particular neural network architecture, although examples of neural networks that may be used are given above.

[0068] The memory-free prior art interaction loop takes user interaction into account by providing the image to be segmented and the user interaction as input to the neural network. The user interaction is represented by N binary masks, where N is the number of segmentation classes. The input of the neural network is therefore concatenated into a tensor of size HxLxC, where H is the height of the image, L is the width of the image, and Cbase is the number of channels, which is equal to 1+N in the memory-free interaction loop, corresponding to the channels of the image and the N channels of the N segmentation classes.

[0069] Prior art solutions propose the addition of an internal memory characterized by a group of N binary masks for each segmentation class where user interactions are accumulated. Thus, the input tensor is reduced to C, since groups of user interactions are merged. CIM = 1 + N channels. This type of method can be called "accumulation interaction memory" in light of its principle of operation and has been implemented according to different variations, notably in the following publications: - Amrehn, M., Gaube, S., Unberath, M., Schebesch, F., Horz, T., Strumia, M., Steidl, S., Kowarschik, M., Maier, A., 2017. UI-Net: Interactive Artificial Neural Networks for Iterative Image Segmentation Based on a User Model, in: Eurographics Workshop on Visual Computing for Biology and Medicine, The Eurographics Association; - Zhou, B., Chen, L., Wang, Z., 2019. Interactive deep editing framework for medical image segmentation, in: Shen, D., Liu, T., Peters, TM, Staib, LH, Essert, C., Zhou, S., Yap, PT, Khan, A. (Eds.), Medical Image Computing and Computer Assisted Intervention - MICCAI2019, Springer International Publishing, Cham. pp. 329-337; - Liao, X., Li, W., Xu, Q., Wang, X., Jin, B., Zhang, X., Zhang, Y., Wang, Y., 2020. Iteratively-refined interactive 3d medical image segmentation with multi-agent reinforcement learning. 2020 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 9391-9399.

[0070] Thus, the image segmentation system 10 according to the present invention, thanks to the preservation of the state of previous interactions, SIM = 1 + 2DN channels, with sequentiality being stored in the input tensor, D being the stored state number, i.e., the number of interactions performed. The factor 2 corresponds to the presence of N interaction masks and N probability maps corresponding to the intermediate segmentation results.

[0071] The internal sequential memory is thus formed by a FIFO type stack in which successive states corresponding to interactions are recorded, this memory being external to the neural network and having no interaction with the internal memory of the neural network, in particular with the weights assigned to each node of the neural network specific to this neural network and to the data type.

[0072] FIG. 2 illustrates generally a semi-automatic image segmentation method 40 according to one embodiment of the present invention.

[0073] The segmentation method includes a step 42 of receiving an image to be segmented for presentation to a neural network, as described in the segmentation system.

[0074] The method includes a step 44 of processing input data comprising each image to be segmented by the neural network to provide, at each iteration, output data comprising at least one segmentation mask for each image.

[0075] If the quality of the segmentation is deemed adequate in the quality control step 46, the segmentation method ends and in a final step 52 a segmentation mask associated with the final segmentation is provided.

[0076] If the quality of the segmentation is not adequate, each next iteration further comprises a step 48 of receiving a user mask representing a real user's interaction with each segmentation mask, and a step 50 of transmitting each segmentation mask and the user mask provided by the neural network as input data for the neural network in the next iteration.

[0077] FIG. 3 illustrates diagrammatically the steps of a method for training a semi-automatic image segmentation system according to an embodiment of the present invention.

[0078] The training method 100 includes a predetermined number of iterations for each image in the dataset, each iteration being marked from I1 to In.

[0079] Each iteration comprises a step 112 of processing input data 114 by the neural network to provide output data 116. In a first iteration I1, the input data comprises at least the image to be segmented and a predefined initialization mask 110. The predefined initialization mask 110 is in this case for example an empty mask, a trivial mask, a mask obtained by automatic pre-segmentation or a mask derived from a ground truth.

[0080] In subsequent iterations, I2 to In, the input data comprises each image to be segmented, each previous segmentation mask obtained in the previous iteration, and a user mask representing real and / or simulated user interactions, as represented by the dotted arrows linking the output data 116 of an iteration to the input data 114 of the next iteration.

[0081] The user mask can be obtained, for example, by comparison of the segmentation mask with a ground truth, i.e., previously labeled images, allowing supervised learning. In this way, the user mask simulates the interaction of a user who knows the desired outcome of the segmentation.

[0082] The user mask is, for example, a binary mask with segmentation classes, representing the user's clicks in the foreground. The location of the click is chosen randomly in the maximal region of the class according to a probability map that is maximal in the center of the region, decreases away from the center, and decreases significantly outside the region.

[0083] Training with an internal sequential memory as described above allows the D state memory to be filled with realistic values ​​generated by virtual users. The D states are formed by D iterations to fill the internal sequential memory before the weights are updated. Preferably, the states are formed by inference through the neural network, i.e., without updating the neural network weights.

[0084] The method also includes a step 120 of updating the neural network weights from the output data after each iteration or when a predetermined number of iterations have been performed by the neural network, where in the embodiment shown in Figure 3, the step 120 of updating the neural network weights is performed in the last iteration based on the last output data 116.

[0085] The predefined number of interactions is determined independently for each class and each image set. The maximum number of interactions is typically 3 interactions to simulate a traditional user, and the minimum is 0.

[0086] The probability of adding a supplemental interaction decreases linearly after each interaction, reaching zero when the maximum number of interactions is reached.

[0087] In one embodiment of the present invention, classes are randomly selected in each epoch of the training method using a dataset where no user interaction is generated.

[0088] In one embodiment of the present invention, a percentage of the interactions generated is retained, typically 20%.

Claims

1. 1. A semi-automatic image segmentation system comprising a neural network (12) configured to receive, for each iteration, input data comprising at least one image (14) to be segmented and to provide output data comprising at least one segmentation mask (16) for each image, a user interaction loop (18) comprising a user interaction module (20) configured to provide at least one user mask representing an interaction of a real user and / or a simulated user with each segmentation mask; The user interaction loop (18) receives as input data for the neural network: each segmentation mask provided by said neural network (12) resulting from a previous iteration of said neural network (12), called a past segmentation mask; said user mask associated with each previous segmentation mask; configured to provide A semi-automatic image segmentation system comprising:

2. each said user mask representing a simulated user comprises a simulation of at least one interaction performed by a simulated operator on the segmentation mask (16) associated with said user mask, the location of said interaction in said image being determined according to a difference between said output segmentation mask (16) and a ground truth; 2. The image segmentation system of claim 1.

3. a graph cut module configured to apply graph cuts to output data of the neural network (12) to obtain each of the segmentation masks (16).

3. An image segmentation system according to claim 1 or 2, characterized in that

4. said user interaction loop (18) comprising an internal sequential memory in which each previous segmentation mask and each user mask obtained at each iteration are stored in turn; An image segmentation system according to any one of claims 1 to 3.

5. A method for training an image segmentation system according to any one of claims 1 to 4 using at least one dataset comprising an image to be segmented, comprising: For each image in the dataset, at least two iterations (I1, I2, In) during which said neural network (12) processes input data to provide output data, namely a first iteration (I1) during which said input data comprises at least one image to be segmented and a predefined initialization mask (110), and a subsequent iteration (I2) during which said input data comprises each image to be segmented, each previous segmentation mask obtained in a previous iteration, and a user mask representative of the interactions of a real and / or simulated user; - updating (120) the weights of the neural network from the output data after each iteration, or when a predetermined number of iterations have been performed by the neural network; The method of claim 1, further comprising:

6. 6. The method of claim 5, wherein the predetermined number of iterations is determined for each image and each data set to be segmented independently of predetermined numbers of iterations determined for other images and other data sets to be segmented.

7. 1. A method for semi-automatic image segmentation comprising the steps of processing input data comprising each image to be segmented by a neural network to provide, in each iteration, output data comprising at least one segmentation mask for each image, Each iteration is receiving a user mask representative of a real user's interaction with each segmentation mask; sending each segmentation mask provided by said neural network and said user mask as input data of said neural network in a next iteration; 1. A semi-automatic image segmentation method comprising:

8. A computer program product for training an image segmentation system according to any one of claims 1 to 4, comprising program code instructions for carrying out the steps of the training method according to any one of claims 5 or 6, when said computer program product is run on a computer.

9. A computer program product for semi-automatic image segmentation, comprising program code instructions for carrying out the steps of the image segmentation method according to claim 7, when said computer program product is executed on a computer.

Images