Medical image processing

Abstract

A method of processing detected images comprises obtaining measurement data comprising at least one image generated based on a signal captured by a detector, theat least one image comprising a plurality of data values, each data value being associated with a spatial location having a distance to the detector. A first segmentation comprises first segmentation values, each first segmentation value being associated with the spatial location having the distance. A second segmentation comprises second segmentation values, each second segmentation value being associated with the spatial location having the distance. Weights are based on the distances of the spatial locations to the detector. The first segmentation values compared to the second segmentation values obtains comparison results associated with the spatial locations. A quantification of a difference between the first segmentation and the second segmentation based on the weights and the comparison results is calculated.

Description

[0001] Medical image processing

[0002] FIELD OF THE INVENTION

[0003] The invention relates to medical image processing. The invention further relates to a method of processing detected images. The invention further relates to a system for processing detected images.

[0004] BACKGROUND OF THE INVENTION

[0005] Cancer is an often diagnosed disease worldwide, and incidence rates are rising every year. Treatment of cancer often involves surgery. Cancer surgery is characterized by a delicate balance between radical tumor resection and sparing healthy tissue and critical anatomical structures. Adequate surgical tumor resection, with a sufficient margin of healthy tissue both around and distally to the tumor, is crucial for patient prognosis. However, tumor boundary identification during surgery can be challenging without any intra-operative imaging guidance. The current golden standard for resection margin evaluation based on pathological examination takes place days after surgery. Unfortunately, a positive resection margin is still regularly found in e.g. uterine, prostate, breast and colorectal cancers. For example, a positive resection margin is still found in Dutch hospitals in 4.7% and 12.7% of T1-3 and T4 rectal tumors respectively, worsening patient prognosis and warranting an improved method for intra-operative margin assessment.

[0006] Several methods have been developed for intra-operative tumor localization, such as frozen sections, fluorescence guidance, marker placement, and surgical navigation technology. However, these techniques all have certain drawbacks, including radiation exposure, long duration, non-optimal accuracy, complexity and the need for special instruments or technical support. On the other hand, many of the aforementioned methods are mainly focused on general tumor localization and are less capable of providing any information about the tumor boundaries. There is a lack of a technique that is precise, non-invasive, simple and provides real-time automatic analyses for intraoperative margin assessment.

[0007] Ultrasound (US) is a non-invasive and real-time imaging technique, that is widely used in health care for various purposes. Many studies have shown that ultrasonic wave propagation in tissues is strongly dependent on histological features including tissue microstructure, and tissue heterogeneity. Unfortunately, to the best of our knowledge, no studies have been performed on the use of intra-operative ultrasound (IOUS) for realtime resection margin assessment during cancer surgery. For example, in breast cancer ultrasound is used to localize breast tumors, but margin assessment by ultrasound is less well investigated. A similar application for ultrasound is used in colorectal cancer. A couple of case studies and feasibility studies have used ultrasound imaging during colorectal cancer surgery to localize the tumor region, with or without pre-operative endoscopic clip placement. However, this concerned rough localizations of colorectal tumors and was not aimed at assessment of the tumor resection margin. Moreover, US reading based on human interpretation of the US images can be challenging and relies on training and experience.

[0008] When developing deep learning models for ultrasound interpretation, loss functions may be an important aspect of developing such models. During the model training, the loss function estimates the error between the predictions of the model and the ground truth in order to adjust all the weights and biases inside the hidden layers of the model. During the model validation, loss values are calculated on the validation set to estimate the performance on data the model has not seen during training, in order to assess the generalization abilities of a model. The loss function can be tailored to the task of the model, such as classification or segmentation. The loss function provides an estimation of the error in the predictions of the model, to guide the learning process.

[0009] Overlapping losses, such as the Dice loss, are known to be used to assess the context and shape of segmentation results.

[0010] SUMMARY OF THE INVENTION

[0011] It would be advantageous to provide an improved tool for developing learning models to segment image data.

[0012] To better address this concern, a method of processing detected images, is provided the method comprising obtaining measurement data comprising at least one image generated based on a signal captured by a detector, the at least one image comprising a plurality of data values, each data value being associated with a spatial location having a distance to the detector; obtaining a first segmentation of the at least one image, the first segmentation comprising first segmentation values, each first segmentation value being associated with the spatial location having the distance; obtaining a second segmentation of the at least one image, the second segmentation comprising second segmentation values, each second segmentation value being associated with the spatial location having the distance; assigning weights to the spatial locations based on the distances of the spatial locations to the detector; comparing the first segmentation values to the second segmentation values associated with the same spatial locations, to obtain comparison results associated with the spatial locations; calculating a quantification of a difference between the first segmentation and the second segmentation based on the weights and the comparison results.

[0013] By virtue of the weights, which depend on the distance to the detector, the quantification of the difference becomes more focused. This helps to concentrate on the most important features, and / or features for which the measurement data is most accurate: for example the tumor boundary closest to the detector. In doing so, the method helps to identify the quality of the segmentation of features close to the detector or at a given focus distance from the detector. In view of this, the method helps the detection of tumor margin.

[0014] The data values associated with a smaller distance may be weighted more heavily in the quantification of the difference. For example, the weights assigned to spatial locations closer to the detector may be larger than the weights assigned to spatial locations further away from the detector. This may help to improve the detection of important features close to the detector.

[0015] The obtaining the first segmentation may comprise generating the first segmentation by applying a learned model to the at least one image. This helps to assess the quality of the learned model by calculating the quantification for one or more images.

[0016] The method may comprise updating the learned model based on the calculated quantification of the difference. Experiments have shown that the learning of a model based on the quantification set forth provides improved learned models.

[0017] The detector may comprise an ultrasound detector or a reflectance-based detector. The technique was found to be particularly helpful to improve segmentation of ultrasound images. Other reflectance-based detectors, for example optical detectors, such as optical coherence tomography, may benefit from the technique as well.

[0018] The first segmentation or the second segmentation may represent a tumor tissue, in particular a boundary of the tumor. The technique was found to strongly improve the results of cancer segmentation, of e.g. ultrasound images. For example, a learned model may be trained by providing ground truth segmentations (e.g. manually created segmentations) as second segmentations, and generating the first segmentations by the learned model, and computing the quantification of the difference, and updating the learned model based on the quantification of the difference.

[0019] The first segmentation or the second segmentation of the at least one image may represent a classification of the data values into at least a first category and a second category, and the method may further comprise determining a minimal distance of the spatial location of the data values of the first category or second category; and determining a maximal distance of the spatial location of the data values of the first category or second category, wherein the weight of data values decreases as the associated distance increases in between the minimal distance and the maximal distance. This helps the quantification of the difference to enhance the differences regarding the margin of the first category or second category. For example, the segmentation representing a ground truth (e.g. the second segmentation) may be used to set the weights. For example, the first category represents a segmented region. The weights of data values associated with distances smaller than the minimal distance or larger than the maximum distance may be set to a fixed value, which may be zero. Moreover, any data value not part of the first category may be assigned a weight with a value of zero, whereas the weights of the data values of the first category may be increasing with the distance to the detector.

[0020] The method may further comprise providing a predicted probabilistic map P for predicting a label k over N image elements; providing a ground truth map G over the N image elements extracted from the image IGk, and / or providing a gradient-weighted map T over N image elements extracted from the image ITk.

[0021] The step of calculating the quantification may be performed based on a quotient wherein: k denotes class labels; N denotes a number of elements along a first two dimensions of the image; pkndenotes a value of each element n for label k gkndenotes a value of each element n for label k tkndenotes a gradient-transformed pixel value of each element n for label k and wkis an optional class-specific weighting factor. This may provide particularly efficient training results when using the quantification as the loss function of a training algorithm.

[0022] According to another aspect, a system for processing detected images is provided, the system comprising a memory configured to store measurement data comprising at least one image generated based on a signal captured by a detector, the at least one image comprising a plurality of data values, each data value being associated with a spatial location having a distance to the detector, wherein the memory is further configured to store a first segmentation of the at least one image, the first segmentation comprising first segmentation values, each first segmentation value being associated with the spatial location having the distance, wherein the memory is further configured to store a second segmentation of the at least one image, the second segmentation comprising second segmentation values, each second segmentation value being associated with the spatial location having the distance; and a processor configured to: assign weights to the spatial locations based on the distances of the spatial locations to the detector; compare the first segmentation values to the second segmentation values associated with the same spatial locations, to obtain comparison results associated with the spatial locations; and calculate a quantification of a difference between the first segmentation and the second segmentation based on the weights and the comparison results.

[0023] The known Dice loss function is derived from the Dice similarity coefficient and divides the overlapping area of the ground truth mask and the predicted mask by the total area of both masks. In this loss function, all segmented pixels have the same importance impact on the loss calculation. However, the claimed invention provides improved weight by emphasizing the most important parts.

[0024] The method set forth may be implemented in form of a computer program stored on a non-tangible computer readable media.

[0025] The person skilled in the art will understand that the features described above may be combined in any way deemed useful. Moreover, modifications and variations described in respect of the method may likewise be applied to the system and to the computer program product, and modifications and variations described in respect of the method may likewise be applied to the system and to the computer program product.

[0026] BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In the following, aspects of the invention will be elucidated by means of examples, with reference to the drawings. The drawings are diagrammatic and may not be drawn to scale. Throughout the drawings, similar items may be marked with the same reference numerals.

[0028] Fig. 1 shows a data acquisition workflow.

[0029] Fig. 2 illustrates gradient weighting of ground truth mask in a custom loss function.

[0030] Fig. 3 shows a visualization of a tumor margin error metric.

[0031] Fig. 4 shows a visualization of tumor segmentations when using different loss functions.

[0032] Fig. 5 illustrates a dice and tumor margin error.

[0033] Fig. 6 shows Table 1 : Exemplary tumor segmentation performance on a test set, for several example loss functions.

[0034] Fig. 7 shows a flowchart illustrating aspects of a method of processing detected images.

[0035] Fig. 8 shows a block diagram illustrating aspects of a system for processing detected images.

[0036] DETAILED DESCRIPTION OF EMBODIMENTS

[0037] Certain exemplary embodiments will be described in greater detail, with reference to the accompanying drawings.

[0038] The matters disclosed in the description, such as detailed construction and elements, are provided to assist in a comprehensive understanding of the exemplary embodiments. Accordingly, it is apparent that the exemplary embodiments can be carried out without those specifically defined matters. Also, well-known operations or structures are not described in detail, since they would obscure the description with unnecessary detail.

[0039] Fig. 7 shows a flowchart of a method of processing detected images. The method may be performed by a computer, for example. Alternatively, the method may be performed by an imaging device, such as an ultrasound imaging device.

[0040] The method may comprise step 10 of obtaining measurement data comprising at least one image generated based on a signal captured by a detector 150. This detector may be an ultrasound detector, for example. Alternatively, the detector may be an optical detector. The at least one image may comprise a plurality of data values, each data value being associated with a spatial location having a distance to the detector 150. This distance may be measured, for example, from reference point or reference surface of the detector 150. For example, the distance may be measured from a surface 151 of the detector 150, the surface 151 facing a field of view of the detector 150. Alternatively the distance may be measured from a focal spot of the detector 150 or be measured from an internal component (i.e. a sensor) inside the detector 150.

[0041] The method may comprise step 20 of obtaining a first segmentation of the at least one image. The first segmentation may comprise first segmentation values, each first segmentation value being associated with a respective spatial location, and each respective spatial location may have a corresponding distance to the detector 150.

[0042] The method may further comprise step 30 of obtaining a second segmentation of the at least one image. The second segmentation may comprise second segmentation values, each second segmentation value being associated with a respective spatial location, and each respective spatial location may have a corresponding distance to the detector 150. The locations and distances of the first segmentation values may match the locations and distances of the second segmentation values. This makes it possible to compare the first and second segmentation values of the same spatial location.

[0043] The method may further comprise step 40 of assigning weights to the spatial locations based on the distances of the spatial locations to the detector.

[0044] It is observed that, in certain embodiments, the at least one image may be onedimensional, two-dimensional, or three-dimensional, in each case with data values associated with three-dimensional real world locations having different distances to the detector 150. It is observed that the spatial locations and distances may not have to be known in the correct scale. In certain cases, only the relative distance of the data values is taken into account.

[0045] The method may further comprise step 50 of comparing the first segmentation values to the second segmentation values associated with the same spatial locations, to obtain comparison results associated with the spatial locations. For example, a difference may be observed between the first segmentation and the second segmentation at certain locations, whereas the segmentations are identical or nearidentical at certain other locations. These comparison results may be stored in a memory 102. For example, a 1 may be stored for locations where the segmentation values differ, whereas a 0 may be stored for locations where the segmentation values are identical.

[0046] Next, in step 50 the method may calculate a quantification of a difference between the first segmentation and the second segmentation based on the weights and the comparison results. For example, a weighted sum of the comparison results may be calculated using the weights.

[0047] The quantification calculated in step 50 may be highly valuable to assess the quality of one segmentation against the other. This may be used to compare segmentation methods, for example. In certain embodiments, the second segmentation is regarded a ground truth (e.g. verified by a human expert) and the first segmentation is made by a segmentation algorithm. The quantification calculated in step 50 may thus be a quantification of an error or difference between these segmentations, thus giving information about the quality of the segmentation algorithm used to make the first segmentation. Since the method weighs the values depending on the distance to the detector, the method allows the quantification to ‘focus’ on certain elements at a distance that is relevant to a certain diagnosis, for example.

[0048] In certain embodiments, the data values associated with a smaller distance to the detector are weighted more heavily in the quantification of the difference. For example, in step 40 the weights assigned to locations with a smaller distance to the detector may be larger than the weights assigned to locations with a greater distance to the detector.

[0049] In certain embodiments, step 50 of calculating the quantification comprises step 51 of identifying a first segmented region of the first segmentation and identifying a second segmented region of the second segmentation. That is, the first segmented region is a region that contains e.g. a particular material, tissue type or object according to the fist segmentation. For example, the first segmented region may correspond to locations where the first segmentation indicates a second segmentation class different from a first segmentation class.

[0050] The second segmented region is a region that contains that particular material, tissue type or object according to the second segmentation. For example, the second segmented region may correspond to locations where the second segmentation indicates the second segmentation class.

[0051] In step 52, an overlapping region may be detected, that is a region where the first segmented region overlaps the second segmented region.

[0052] In step 53, a total region (or union region) may be identified. That is, the total region is the region that results from joining the first segmented region and the second segmented region.

[0053] In step 54, a quantification may be calculated. This quantification is a value representing a difference (or error) between the first segmented region and the second segmented region. This quantification may be calculated based on the overlapping region, the total region, and the weights. The quantification may alternatively represent a similarity (i.e., a value that is greater if the error is less). The quantification may be referred to as a loss function throughout the present disclosure. In certain embodiments, step 40 of assigning the weights may comprise step 41 of assigning a weight of zero to spatial locations where the second segmentation indicates a first segmentation class. Further, step 41 may comprise assigning a nonzero weight to spatial locations where the second segmentation indicates a second segmentation class different from the first segmentation class. These non-zero weights assigned to the spatial locations are dependent on the distance of those spatial locations to the detector.

[0054] In certain embodiments, step 20 of obtaining the first segmentation comprises step 21 of generating the first segmentation by applying a learned model to the at least one image. This learned model may be in the process of being trained, for example. The quantification that is calculated by the method may be used to refine the learned model by adjusting the model parameters of the learned model in step 60. In this regard, for example, the second segmentation represent may a ground truth segmentation. Ground truth segmentations may be generated in several different ways, such as manual creation and / or manual verification, manual correction, or a particular method that may have a high quality but that may be more expensive or more computer intensive than the learned model, for example.

[0055] The training of the learned model may proceed in a way that is known to the person skilled in the art per se, wherein the loss function may be used as the error function. For example, a number of training data may be provided, comprising example images (e.g. ultrasound images of an object such as a tumor) together with ground truth segmentations of the object. The ground truth segmentation may be referred to as the second segmentation. The segmentation generated by the learned model may be referred to as the first segmentation. The quantification indicates the difference between the first segmentation and the second segmentation. Based on the quantification in respect of one image or based on aggregate value of the quantifications in respect of a plurality of images, the parameters of the learned model may be updated. For example, this updating may be performed based on the updating step of any known training method, such as gradient descent. For example, the learned model is a neural network and the model parameters include the coefficients of the connections between the units of the neural network. The gradient descent method may update a model parameter based on a partial derivative of the quantification or aggregate value with respect to the model parameter.

[0056] The detector 150 may be any kind of detector. In particular an ultrasound detector or another type of reflectance-based detector may be particularly suited to be used in conjunction with the method and system as described herein. Such reflectance- based detector may be an optical detector having a light emitter, for example. In particular, in cases where quality of detection of the tissues closer to the detector 150 are of more interest than the quality of detection of tissues further away from the detector 150, the present techniques may improve the quality of learned models trained in step 60 using the quantification generated in step 50. A typical application domain of segmentation models trained in the way set forth, may be segmentation of tumor tissue detected by the detector 150 (e.g. ultrasound detector or optical detector).

[0057] As mentioned above, the first segmentation of the at least one image may represent a classification of the data values into at least a first category and a second category. For example, the first category may be a background, and the second category may be a material, object, or tissue of interest, such as a tumor tissue. Similarly, the second segmentation may represent a similar classification of the same data values into the same categories.

[0058] For example, step 40 of assigning the weights may comprise step 42 of identifying a minimal distance of the spatial locations of the data values of the second category. This minimal distance may be the distance from the detector 150 to the spatial location of the second category closest to the detector 150. Step 40 may further comprise step 43 of identifying a maximal distance of the spatial location of the data values of the second category. This maximal distance may be the distance from the detector 150 to the spatial location of the second category furthest away from the detector 150. The weights assigned to locations associated with distances smaller than the minimal distance may be smaller than the weights assigned to locations associated with distances above the maximal distance, and the weight assigned to the locations may decrease as the associated distance increases from the minimal distance to the maximal distance.

[0059] In certain embodiments, these weights are determined for each segmentation separately; that is, different weights may be assigned to the first segmentation and to the second segmentation. In other embodiments, the same weights may be assigned to both the first segmentation and the second segmentation. In the latter case, the minimal / maximal distance may be found based on either the first segmentation or the second segmentation. Alternatively, the minimal / maximal distance may be found based on the total region or the overlapping region.

[0060] The method described above may be implemented as a computer program product containing software, which may take the form of computer executable instructions stored on a tangible or non-tangible computer readable media. Fig. 8 shows a block diagram of a system 101 for processing detected images. The figure also shows a detector 150 placed on an excised specimen 190 with a tumor 191. The minimal distance 155 from the surface 151 of the detector to the tumor 191 is shown in the drawing, as well as the maximal distance 156 from the surface 151 of the detector to the tumor 191. Since the first and second segmentations try to identify the tumor 191 , these maximal and minimal distances may correspond roughly to the minimal and maximal distances of the segmented regions to the detector, which were mentioned above.

[0061] Fig. 8 illustrates the detector 150 placed on the surface of an excised specimen 190 to perform the measurement. However, this is not a limitation. Alternatively, the detector 150 may be placed on the surface of a patient, for example. Yet alternatively, the detector 150 may be placed on a resection surface inside the patient, during surgery. In all cases it is possible that there is a wall in between the detector 150 and the tissue; for example, a protective wall of a specimen container.

[0062] The system 101 may be configured to perform the method of Fig. 7 or another method set forth herein.

[0063] The system 101 may comprise a memory 102, which may be random access memory or read-only memory or any computer memory. The memory 102 may be configured to store measurement data. The measurement data may be received from an external source using a communication module 104. Communication module 104 may comprise a network interface for wired or wireless communication. The detector 150 may be directly connected to the communication module 104. Alternatively, the detector 150 may be connected to a server, which may forward the measurement data to the communication module 104. The measurement data may comprise at least one image generated based on a signal captured by the detector 150. This at least one image may comprise a plurality of data values, each data value being associated with a spatial location having a distance to the detector 150. Each data value may be representative of a certain physical property that was sensed by the detector 150. The data values relating to specific spatial locations may be computed (for example by the detector 150, or alternatively by the system 101) from a measurement signal generated by a sensor of the detector 150, for example using tomography or known ultrasound calculations.

[0064] The memory 102 may be further configured to store a first segmentation of the at least one image, the first segmentation comprising first segmentation values, each first segmentation value being associated with a particular spatial location having a particular distance to the detector 150. The memory 102 may be further configured to store a second segmentation of the at least one image, the second segmentation comprising second segmentation values, each second segmentation value being associated with a particular spatial location having a particular distance to the detector.

[0065] The first segmentation and the second segmentation may be segmentations that localize the same or similar features in the measurement data (the image). For example, both segmentations indicate a particular region that would represent the tumor 191 , but there may be a difference between the two segmentations due to different algorithms or different methods used to identify the tumor (or other structure of interest).

[0066] The system 101 may further comprise a processor 103. This processor 103 may be a computer processor or control unit, or a dedicated electronic circuit. The processor 103 may be configured to assign weights to the spatial locations based on the distances of those spatial locations to the detector 150.

[0067] The processor may be further configured to compare the first segmentation value associated with each spatial location to be evaluated to the corresponding second segmentation value associated with each spatial location to be evaluated. This way, comparison results are obtained that are associated with the spatial locations. The processor may be further configured to calculate a quantification of a difference between the first segmentation and the second segmentation based on the weights and the comparison results. The processor may be further configured to implement aspects of the method described in the present disclosure. For example, the processor or control unit may be configured to perform each of the steps of the method described with respect to Fig. 7. For example, step 10 of obtaining measurement data may comprise receiving, by the processor or control unit, the measurement data.

[0068] Deep learning-based approaches could facilitate the automatic analysis of tumor images, such as e.g. ultrasound images of tumor tissue, or colorectal ultrasound images during surgery. Convolutional neural networks (CNNs) are widely used in medical image analysis for image interpretation through automatic classification and segmentation. However, CNNs need a plethora of labeled data samples for adequate training which is often scarce in the medical field, especially in the case of intra-operative imaging of cancer or ex vivo imaging of surgical specimens of cancer tissue where imaging is not routinely used in standard clinical care. Consequently, no public (labeled) datasets are available either.

[0069] This study will present a deep learning model for tumor segmentation in colorectal US images, using pre-trained CNNs. Some of the contributions of the present disclosure can be summarized as follows: • We acquired the first annotated ultrasound dataset for colorectal cancer, with ground truth tumor annotations based on a correlation with histopathology results.

[0070] • We developed a new custom loss function (Gradient Weighted Dice), which puts more emphasis on the clinically relevant top margin of the tumor during the training of the segmentation networks.

[0071] • This is the first study on automatic colorectal ultrasound segmentation, which can be used for real-time intra-operative margin assessment.

[0072] Loss functions are important in developing learned models, such as statistical models, neural networks, and in particular deep learning models. During the model training, the loss function estimates the error between the predictions of the model and the ground truth in order to adjust all the weights and biases inside the hidden layers of the model. Loss values are calculated on the validation set to estimate the performance on data the model has not seen during training also called the generalization abilities of a model. The loss function is dependent on the task of the model, such as classification or segmentation, where it has a goal of estimating the error in the predictions of the model as accurately as possible to guide the learning process.

[0073] Overlapping losses such as the Dice loss are commonly used to assess the context and shape of segmentation results. This loss is derived from the Dice similarity coefficient and divides the overlapping area of the ground truth mask and the predicted mask by the total area of both masks. In this loss function, all segmented pixels have the same importance impact on the loss calculation.

[0074] Although resection margin assessment benefits from accurate tumor segmentation, the main clinical focus may be, in certain embodiments, the correct identification of the top margin of the tumor. Therefore, we proposed a new custom loss function called the Gradient Weighted Dice loss function (GWDL) based on an expansion of the generalized Dice loss. In order to emphasize the importance of the top border, weights were introduced that applied a gradient, for example an exponential gradient, in the vertical direction to the ground truth mask, see FIG. 2. Weights were applied in a way that tumor weights at the bottom of the tumor stayed at the original weight assigned by the Generalized Dice loss. Weights for the pixels at the top of the image were increased by a factor of two which exponentially decreased to one over the height of the tumor, as shown in algorithm 1 . Instead of a factor of two, any other factor may be used. Instead of exponential decreasing weights, other decreasing weights may be used, such as linearly decreasing weights or polynomially decreasing weights.

[0075] IGkrepresents the ground truth mask with two channels, channel 1 ( / G1) represents background ground truth while channel 2 ( / G2) represents tumor ground truth. Each channel is a binary mask with 0 or 1 values showing the correct assigned labels. Herein, a value of 1 means that the pixel belongs to that class, and a value of 0 means that the pixel does not belong to that class. ITksimilar to IGk, denotes a double channel weight map for both labels (background and foreground).

[0076] For example, the new loss function can be obtained based on a calculated weighted gradient mask ( / Tk): and can be rewritten as:

[0077] Herein: k is the class labels;

[0078] N is the number of elements along the first two dimensions of the image (i.e., total number of pixels in the image);

[0079] P is the predicted probabilistic map for predicting label k over N image elements, the value of each element n for label k is denoted by pkn,

[0080] G is the ground truth map over N image elements extracted from the image IGk, the value of each element n for label k is denoted by gkn, T is the gradient-weighted map over N image elements extracted from the image / Tk;the gradient-transformed pixel value of each element n for label k is denoted by tkn, wkis an optional class-specific weighting factor that controls the contribution each class makes to the score to compensate for unbalanced segmentations. This weighting helps counter the influence of larger regions on the generalized Dice score. wkis typically the inverse area of the expected region:

[0081] FIG. 2 shows gradient weighting of ground truth mask in the custom loss function. Fig. 2a shows an exemplary original US image. Fig ,2b shows a ground truth mask as used in the known dice loss function. Fig. 2c shows a gradient-weighted ground truth mask as used in an exemplary embodiment of the proposed Gradient weighted Dice loss function.

[0082] Using the new Gradient loss function resulted in a reduction of the error in margin assessment by 40% compared to the original Dice loss function.

[0083] The gradient-weighted loss function may be generalized to other loss functions. The gradient, for example the gradient shown in Algorithm 1 , may be employed to similarly augment other types of loss function. For example, gradient weighted binary cross entropy (GWBCE) may be defined as follows:

[0084] Herein, the notations are the same as in the previous equations.

[0085] In the next paragraphs, we will first describe the workflow of colorectal ultrasound image collection and corresponding tumor annotations based on histopathology results. Then, the necessary image pre-processing steps will be performed, after which we will use transfer learning to optimize models pre-trained on breast ultrasound data for our colorectal ultrasound dataset. The added value of using a custom gradient-based loss function will be examined. Finally, the tumor margin prediction performance will be assessed.

[0086] Freshly excised colorectal cancer specimens from 78 patients undergoing surgery at Antoni van Leeuwenhoek hospital-Netherlands Cancer Institute (AvL-NKI) for colorectal cancer between April 2019 and April 2022 were included in this study, under the approval of the Hospital Ethics Review Board. Patients were included when diagnosed with a tumor of at least stage T2 in the colon, sigmoid or rectum, based on preoperative examinations. Both patients with and without neoadjuvant therapy were included in this study. All patients have given permission for the further use of their data and biological materials for scientific research.

[0087] Ultrasound images were acquired using an ultrasound high-frequency transducer (7-15 MHz), which was placed directly on the specimen surface. This transducer is well suited for high-resolution superficial measurements, with an imaging depth of only 3 cm. Per patient, one to three cross-sectional US images were acquired depending on the tumor size, which resulted in 179 US images in total (Figure 1a). The data set was divided patient-wise into a training set of 121 images, a validation set of 28 images and a test set of 30 images. All images have a size of 430 x 344 pixels.

[0088] Fig. 1 shows an example data acquisition workflow. This data acquisition workflow may be used to create training data and test data to be used in creating a learned model. Fig. 1a illustrates performing an ultrasound acquisition on a colorectal specimen. Fig. 1b illustrates to mark the image location with ink marks. Fig. 1c illustrates slicing the specimen at the locations of the ink marks. Fig. 1d illustrates microscopic analysis of the sliced specimen. Fig. 1e illustrates ground truth labelling of the ultrasound image, using the result of the microscopic analysis of the specimen. In a study performed by the inventors, one to three US images are acquired on the freshly excised colorectal specimens, depending on the tumor size (a). Immediately after each acquisition, the measured imaging plane was marked with ink to allow correlation with histopathological results (b). During the histopathology process, the specimen is sliced at the locations of the ink marks (c), after which these slices are microscopically analyzed and the tumor is delineated by a pathologist (d). Based on these histopathological results and the location of the ink marks, the tumor was manually delineated in the acquired US image (e).

[0089] Interpreting ultrasound images and annotating the tumor area can be challenging, even for an experienced radiologist. In order to obtain accurate labels, the measurement plane was marked with ink after the acquisition of each US image to correlate the data with histopathology results (Fig. 1 b). To be able to retrieve the orientation during further processing, two black ink marks and one purple ink mark were placed.

[0090] After data acquisition, the specimens were brought to the pathology department for further processing according to standard protocols. Here, they were fixated in formalin for 48 hours, after which the specimens were dissected in slices in such a way that each row of ink marks ended up in a separate slice. From these slices, the areas with ink marks on the surface were sampled in cassettes (Fig. 1c). The final H&E stained sections were digitally scanned for microscopic analysis (Fig. 1d). The tumor area was delineated by a pathologist in all digital slices. The black and purple ink marks, representing the exact US measurement locations, can be found back in the digital tissue slices. Similarly, the position of these locations in the US image is known, since the US probe was fixated using a mold during data acquisition and the same mold was used as a reference for marking the locations. Based on this correlation, the tumor was manually delineated in the acquired US image as ground truth (Fig. 1e).

[0091] Since this study is focused on resection margin assessment close to the specimen surface, the US images were cropped to the top half of the images and subsequently resized to 128 x 128 pixels. All images were normalized to a pixel intensity range of 0 to 1 before training the models.

[0092] During training, data augmentation was applied to generalize the models and reduce the risk of overfitting. Augmentation methods included vertical flipping, rotation, and gamma correction. Image rotation angles between -5 and 5 degrees were used. Gamma correction was applied according to the formula Pout = (Pin) , in which y ranged from 0.8 to 1.2. These gamma values were based on the distribution of pixel intensities in the images.

[0093] Although deep learning models have proven very useful in medical image segmentation, training them involves a plethora of labeled data samples for adequate training, which is often scarce in the medical field. In order to combat data scarcity, a framework leveraging transfer learning was used, using pre-trained CNNs for tumor segmentation in breast US images.

[0094] Wilfrido Gomez-Flores and Wagner Coelho de Albuquerque Pereira (“A comparative study of pre-trained convolutional neural networks for semantic segmentation of breast tumors in ultrasound”, Computers in Biology and Medicine, 126:104036, 11 2020) trained different convolutional neural network architectures for semantic segmentation of breast tumors in ultrasound images. These publicly available models were trained on 3061 ultrasound images of different public datasets. We used five pre-trained models on breast US images for transfer learning with the colorectal ultrasound data set acquired in this study: MobilenetV2 (Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang Chieh Chen. MobileNetV2: Inverted Residuals and Linear Bottlenecks. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 4510-4520, 2018), Resnet18, Resnet50 (Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 770-778. IEEE Computer Society, 12 2016), U-net (Olaf Ronneberger, Philipp Fischer, and Thomas Brox. Il-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI 2015), pages 234-241 , 2015), and Xception (Frangois Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, pages 1800-1807, 2017).

[0095] In the transfer learning process, all model architectures were prototyped with hyperparameter tuning through an exhaustive grid search. The best-performing model architecture was chosen based on the highest Dice similarity coefficient. Learning rates for transfer learning were a factor of 10 smaller than the learning rates of the original networks to retain previously learned features. All training was performed in MATLAB 2022a (MathWorks, Natick, Massachusetts).

[0096] Overlapping losses such as the Dice loss are commonly used to assess the context and shape of segmentation results. This loss is derived from the Dice similarity coefficient and divides the overlapping area of the ground truth mask and the predicted mask by the total area of both masks. To combat class imbalance between the foreground and background, a class-weighted variant of the Dice loss called the Generalized Dice Loss (GenDice) was used. See (William R. Crum, Oscar Camara, and Derek L.G. Hill. Generalized overlap measures for evaluation and validation in medical image analysis. IEEE Transactions on Medical Imaging, 25(11): 1451— 1461 , 2006) and (Carole H. Sudre, Wenqi Li, Tom Vercauteren, Sebastien Ourselin, and M. Jorge Cardoso. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), volume 10553 LNCS, pages 240-248. Europe PMC Funders, 2017). This adaptation of the Dice loss weighs the contribution of each class by the inverse of the area of the class in the ground truth mask, to counter the influence of larger regions on the Dice score, see Equation 1 and 2: with tn the pixel values of the true tumor mask, pn the pixel values of the predicted tumor mask, and wk the class weights for each class / c: It may be observed that in alternative implementations, wk may have a different value or may be omitted, by substituting it for a value of 1.

[0097] Although resection margin assessment benefits from accurate tumor segmentation, the main clinical focus is the correct identification of the top margin of the tumor. To obtain a better identification of the top margin of the tumor, a custom loss function called Gradient Weighted Dice Loss (GWDL) loss function was developed in this study based on an expansion of the Generalized Dice loss (GenDice). In order to emphasize the importance of the top border, weights were introduced which applied an exponential gradient in the vertical direction to the ground truth mask, see Fig. 2. Weights were applied in a way that tumor weights at the bottom of the tumor stayed at the original weight assigned by the Generalized Dice loss. Weights for the pixels at the top of the image were increased by a factor of two which exponentially decreased to one over the height of the tumor, as shown in algorithm 1 .

[0098] An example of the new loss function is obtained based on a calculated weighted gradient mask ( / T):

[0099] With tnthe gradient-transformed pixel values of the true tumor mask ( / T(x,y)), gnthe value of ground truth (0 or 1), pnthe pixel values of the predicted tumor mask, and wkthe class weights for each class k, and tg the gradient-transformed pixel values of the true tumor mask.

[0100] Fig. 2 shows an example of Gradient weighting of ground truth mask in the custom loss function: (a) Original US image, (b) Ground truth mask as used in GenDice loss function, (c) Gradient-weighted ground truth mask as used in GWDL loss function.

[0101] The performance of all network architectures was evaluated using the Dice similarity coefficient, the tumor margin error, and the area under the curve (AUG). The Dice similarity coefficient is a commonly used performance metric in medical image segmentation, measuring the amount of overlap between two segmentation masks (ranging from 0 to 1). A tumor margin error metric was devised to assess the accuracy of the top tumor margin prediction, which is calculated as the vertical distance between the top tumor pixel in the ground truth mask and the top tumor pixel in the predicted mask (in millimeters), see Fig. 3. For comparing the performance, the average tumor margin error was calculated for all test images. The AUG represents the area under the receiver operating characteristic (ROC) curve, measuring the performance of the model irrespective of what classification threshold is chosen (ranging from 0 to 1). For the Dice and AUG, a larger value is indicating a better performance, while for the tumor margin error a smaller value is indicating better performance.

[0102] Fig. 3 shows a visualization of the tumor margin error metric. On the left the ground truth is shown, and on the right the prediction. TMgt shows the ground truth tumor margin, TMp shows the predicted tumor margin. The tumor margin error, E, is the absolute difference between the ground truth and predicted tumor margin (in millimeters).

[0103] Comparison between GenDice and GWDL loss functions: The performance of the custom Gradient weighted Dice loss function was compared to the commonly used GenDice loss function for all five individual models, see Table 1 in Fig. 6. The GWDL loss function slightly increased the mean Dice score from 0.78 to 0.80, while the mean tumor margin prediction error clearly decreased from 1.08 mm to 0.92 mm. This shows that the custom loss function is able to improve the tumor margin prediction by emphasizing the top margin of the tumor, without compromising the segmentation of the overall tumor contour. Examples of segmentation differences between both loss functions are visualized in Fig. 4. The biggest differences between the two loss functions can be seen at the top border of the tumor mask.

[0104] Table 1 (Fig. 6) shows tumor segmentation performance on the test set of the standard GenDice loss function compared to the custom GWDL loss function.

[0105] Fig. 4 shows a visualization of tumor segmentations using the standard GenDice loss function and the custom GWDL loss function, for the individual Resnet50 network. The leftmost column of Fig. 4 shows the ultrasound image. The middle column shows the result using GenDice. The rightmost column shows the result using GWDice. Fig. 4 shows different gray tones for the ground truth segmentation 501 , the prediction 502, and the overlap 502.

[0106] Optimization: Choosing the optimal output probability threshold for the networks involves a trade-off between the best Dice coefficient and tumor margin error. The results presented in the previous paragraphs were based on a threshold for an optimal balance between the Dice and tumor margin error. However, when for the final application one of them is most important, the thresholds can be further optimized for the Dice or tumor margin specifically. Fig. 5 shows the resulting Dice and tumor margin error for the classification method, using every possible output probability threshold between 0 and 1. When optimizing purely the Dice coefficient, a maximum Dice of 0.84 can be achieved compared to 0.83 previously reported using ensemble method. When optimizing purely the tumor margin prediction error, a minimum tumor margin error of 0.64 mm can be achieved compared to 0.67 mm previously reported. Fig. 5 illustrates dice and tumor margin error for every possible output probability threshold between 0 and 1. The solid line and shaded area represent the average and standard deviation of all images in the test set, respectively. The upper curve and lefthand vertical axis shows Dice. The lower curve with the righthand vertical axis shows tumor margin error in mm. The horizontal axis shows the output probability threshold.

[0107] Discussion: In the present disclosure, a model for automatic tumor segmentation in colorectal US images was developed, to provide real-time guidance on resection margins using intra-operative US. A new custom gradient-based loss function was assessed and the resection margin prediction accuracy based on segmentation results was evaluated.

[0108] Due to data scarcity in the field of intra-abdominal colorectal ultrasound images, convolutional neural network (CNN) models pre-trained for tumor segmentation in breast US were used as a starting point. After re-training these models with our colorectal US dataset based on the loss function disclosed herein, the segmentation performance (mean Dice of 0.78) was achieved. The available pre-trained models for tumor segmentation in breast US concern long-established neural network architectures.

[0109] A new custom GWDL loss function was introduced to put more emphasis on the top tumor border, which is clinically most important for intra-operative resection margin assessment. The new loss function provided exactly the desired effect; improving the top tumor margin prediction from 1.08 mm to 0.92 mm, while preserving a good general overlapping score (mean Dice score increased from 0.78 to 0.80), see Table 1 (see Fig. 6) and Fig. 4. Currently, a vertical exponential gradient was used to adjust the weights in the ground truth mask. In future research, it might be interesting to explore other intensity profiles as well, in which the weights of the ground truth mask decrease less or more rigorous in the vertical direction.

[0110] Using an ensemble learning technique by combining predictions from the five individual models and the use of the loss function disclosed herein further increased the Dice score from 0.80 to 0.84 and decreased the tumor margin error from 0.92 to 0.67 mm, compared to using one separate model.

[0111] To provide intra-operative guidance on resection margins, accurate tumor margin prediction is important. In this study, an average tumor margin prediction accuracy of 0.67 mm was achieved. This value is in the same order of magnitude as the US resolution (» 0.5 mm). The use of high-frequency ultrasound, with a higher spatial resolution and lower penetration depth, may achieve a slightly higher tumor margin prediction accuracy. In addition, the accuracy of 0.67 mm falls within the resection margin of 1 mm which is generally used in colorectal cancer surgery. Larger errors for deeper tumor margins might be caused by a decrease in US signal deeper in the tissue. However, given the fact that tumor detection close to the resection surface (up to 5 mm in depth) is clinically most relevant, this is not a major concern.

[0112] Although a correlation with histopathology was performed to obtain accurate tumor annotations, small errors might have been made during manual annotations. The Dice coefficient of 0.84 achieved with our automatic segmentation model was equal to the human observers agreement of 0.84. In case of the resection margin prediction, an accuracy of 0.67 mm was achieved with our automatic segmentation model, compared to an inter-observer variability of 0.59 mm. This shows that the automatic segmentation model achieves comparable results to human observers, indicating that intra-operative US in combination with an automatic tumor segmentation model seems promising and may contribute to more accurate colorectal tumor resections.

[0113] Fig. 5 showed that the optimal output probability threshold depends on the metric used to evaluate the performance of the segmentation model. The thresholds can be optimized specifically for the final application and user wishes; e.g. fully focused on achieving the best tumor margin, the entire tumor contour, or a combination. A similar trade-off can be made between sensitivity and specificity.

[0114] In the above example, the US images were acquired on specimens from patients undergoing surgery for colorectal cancer, directly after surgical resection. However, the invention is not limited to colorectal cancer. The techniques disclosed herein, including the loss function, can be used for a wide variety of tumors, including breast tumors and soft tissue tumors. In addition, the method and system disclosed herein may be applied, for example, during surgery on intra-operatively acquired US images. In addition to tumor segmentation, the automatic US analysis can be extended towards tumor detection for a wide variety of tumors or different tissue types. By retraining the networks on a dataset that also includes healthy colorectal US images, the techniques introduced in this paper may be used for distal margin assessment as well.

[0115] Some or all aspects of the invention may be suitable for being implemented in form of software, in particular a computer program product. The computer program product may comprise a computer program stored on a non-transitory computer- readable media. Also, the computer program may be represented by a signal, such as an optic signal or an electro-magnetic signal, carried by a transmission medium such as an optic fiber cable or the air. The computer program may partly or entirely have the form of source code, object code, or pseudo code, suitable for being executed by a computer system. For example, the code may be executable by one or more processors. The examples and embodiments described herein serve to illustrate rather than limit the invention. The person skilled in the art will be able to design alternative embodiments without departing from the spirit and scope of the present disclosure, as defined by the appended claims and their equivalents. Reference signs placed in parentheses in the claims shall not be interpreted to limit the scope of the claims. Items described as separate entities in the claims or the description may be implemented as a single hardware or software item combining the features of the items described.

[0116] Certain aspects are disclosed in the following clauses.

[0117] 1 . A method of processing detected images to obtain a quantification of a difference between segmentations for training a learned model to do the segmentation, the method comprising obtaining (10) measurement data comprising at least one image generated based on a signal captured by a detector, the at least one image comprising a plurality of data values, each data value being associated with a spatial location having a distance to the detector; obtaining (20) a first segmentation of the at least one image, the first segmentation comprising first segmentation values, each first segmentation value being associated with the spatial location having the distance; obtaining (30) a second segmentation of the at least one image, the second segmentation comprising second segmentation values, each second segmentation value being associated with the spatial location having the distance; assigning (40) weights to the spatial locations based on the distances of the spatial locations to the detector; comparing (50) the first segmentation values to the second segmentation values associated with the same spatial locations, to obtain comparison results associated with the spatial locations; and calculating (50) a quantification of a difference between the first segmentation and the second segmentation based on the weights and the comparison results.

[0118] 2. The method of clause 1 , wherein the data values associated with a smaller distance to the detector are weighted more heavily in the quantification of the difference.

[0119] 3. The method of any preceding clause, wherein calculating (50) the quantification comprises: determining (51) a first segmented region of the first segmentation and a second segmented region of the second segmentation; determining (52) an overlapping region of the first segmented region and the second segmented region; determining (53) a total region of the first segmented region and the second segmented region; and calculating (54) the quantification of the difference based on the overlapping region, the total region, and the weights.

[0120] 4. The method of any preceding clause, wherein assigning (40) the weights comprises: assigning (41) a weight of zero to spatial locations where the second segmentation indicates a first segmentation class, and assigning a nonzero weight dependent on the distance to spatial locations where the second segmentation indicates a second segmentation class different from the first segmentation class.

[0121] 5. The method of any preceding clause, wherein both the first segmentation and the second segmentation are indictive of both a tumor and a boundary of the tumor.

[0122] 6. The method of any preceding clause, wherein the obtaining (20) the first segmentation comprises generating (21) the first segmentation by applying a learned model to the at least one image.

[0123] 7. The method of clause 6, further comprising training (60) the learned model based on the calculated quantification of the difference.

[0124] 8. The method of any preceding clause, wherein the second segmentation represents a ground truth.

[0125] 9. The method of any preceding clause, wherein the detector comprises an ultrasound detector or a reflectance-based detector.

[0126] 10. The method of any preceding clause, wherein the first segmentation or the second segmentation represents a tumor tissue.

[0127] 11. The method of any preceding clause, wherein the first segmentation or the second segmentation of the at least one image represents a classification of the data values into at least a first category and a second category, the first category representing a segmented region, and the step of assigning (40) the weights further comprises determining (42) a minimal distance of the spatial location of the data values of the first category to the detector; and determining (43) a maximal distance of the spatial location of the data values of the first category to the detector, wherein the weight of data values decreases as the associated distance increases in between the minimal distance and the maximal distance.

[0128] 12. The method of any preceding clause, further comprising: providing a predicted probabilistic map P for predicting a label k over N image elements; providing a ground truth map G over the N image elements extracted from the image IGk, and providing a gradient-weighted map T over N image elements extracted from the image ITk, wherein the calculating (50) of the quantification is performed based on a quotient wherein: k denotes class labels;

[0129] N denotes a number of elements along a first two dimensions of the image; pkndenotes a value of each element n for label k gkndenotes a value of each element n for label k tkndenotes a gradient-transformed pixel value of each element n for label k and wkis an optional class-specific weighting factor.

[0130] 13. A system (101) for processing detected images to obtain a quantification of a difference between segmentations for training a learned model to do the segmentation, the system comprising a memory (102) configured to store measurement data comprising at least one image generated based on a signal captured by a detector (150), the at least one image comprising a plurality of data values, each data value being associated with a spatial location having a distance to the detector, wherein the memory (102) is further configured to store a first segmentation of the at least one image, the first segmentation comprising first segmentation values, each first segmentation value being associated with the spatial location having the distance, wherein the memory (102) is further configured to store a second segmentation of the at least one image, the second segmentation comprising second segmentation values, each second segmentation value being associated with the spatial location having the distance; and a processor (103) configured to: assign weights to the spatial locations based on the distances (155, 156) of the spatial locations to the detector (150); compare the first segmentation values to the second segmentation values associated with the same spatial locations, to obtain comparison results associated with the spatial locations; and calculate a quantification of a difference between the first segmentation and the second segmentation based on the weights and the comparison results.

Claims

CLAIMS:1 . A method of processing detected images to obtain a quantification of a difference between segmentations for training a learned model to do the segmentation, the method comprising obtaining (10) measurement data comprising at least one image generated based on a signal captured by a detector, the at least one image comprising a plurality of data values, each data value being associated with a spatial location having a distance to the detector; obtaining (20) a first segmentation of the at least one image, the first segmentation comprising first segmentation values, each first segmentation value being associated with the spatial location having the distance; obtaining (30) a second segmentation of the at least one image, the second segmentation comprising second segmentation values, each second segmentation value being associated with the spatial location having the distance; assigning (40) weights to the spatial locations based on the distances of the spatial locations to the detector; comparing (50) the first segmentation values to the second segmentation values associated with the same spatial locations, to obtain comparison results associated with the spatial locations; and calculating (50) a quantification of a difference between the first segmentation and the second segmentation based on the weights and the comparison results, wherein the first segmentation or the second segmentation of the at least one image represents a classification of the data values into at least a first category and a second category, the first category representing a segmented region, and the step of assigning (40) the weights further comprises determining (42) a minimal distance of the spatial location of the data values of the first category to the detector; and determining (43) a maximal distance of the spatial location of the data values of the first category to the detector, wherein the weight of data values decreases as the associated distance increases in between the minimal distance and the maximal distance.

2. The method of claim 1 , wherein the data values associated with a smaller distance to the detector are weighted more heavily in the quantification of the difference.

3. The method of any preceding claim, wherein calculating (50) the quantification comprises: determining (51) a first segmented region of the first segmentation and a second segmented region of the second segmentation; determining (52) an overlapping region of the first segmented region and the second segmented region; determining (53) a total region of the first segmented region and the second segmented region; and calculating (54) the quantification of the difference based on the overlapping region, the total region, and the weights.

4. The method of any preceding claim, wherein assigning (40) the weights comprises: assigning (41) a weight of zero to spatial locations where the second segmentation indicates a first segmentation class, and assigning a nonzero weight dependent on the distance to spatial locations where the second segmentation indicates a second segmentation class different from the first segmentation class.

5. The method of any preceding claim, wherein both the first segmentation and the second segmentation are indictive of both a tumor and a boundary of the tumor.

6. The method of any preceding claim, wherein the obtaining (20) the first segmentation comprises generating (21) the first segmentation by applying a learned model to the at least one image.

7. The method of claim 6, further comprising training (60) the learned model based on the calculated quantification of the difference.

8. The method of any preceding claim, wherein the second segmentation represents a ground truth.

9. The method of any preceding claim, wherein the detector comprises an ultrasound detector or a reflectance-based detector.

10. The method of any preceding claim, wherein the first segmentation or the second segmentation represents a tumor tissue.11 . The method of any preceding claim, wherein the obtaining (20) the first segmentation comprises providing a predicted probabilistic map P for predicting a label k over N image elements; wherein the obtaining (30) the second segmentation comprises providing a ground truth map G over the N image elements extracted from the image IGk, and wherein the assigning (40) the weights comprises providing a gradient-weighted map T over N image elements extracted from the image ITk, wherein the calculating (50) of the quantification is performed based on a quotientwherein: k denotes class labels;N denotes a number of elements along a first two dimensions of the image; pkndenotes a value of each element n for label k gkndenotes a value of each element n for label k tkndenotes a gradient-transformed pixel value of each element n for label k and wkis an optional class-specific weighting factor.

12. A system (101) for processing detected images to obtain a quantification of a difference between segmentations for training a learned model to do the segmentation, the system comprising a memory (102) configured to store measurement data comprising at least one image generated based on a signal captured by a detector (150), the at least one image comprising a plurality of data values, each data value being associated with a spatial location having a distance to the detector, wherein the memory (102) is further configured to store a first segmentation of the at least one image, the first segmentation comprising first segmentation values, each first segmentation value being associated with the spatial location having the distance, wherein the memory (102) is further configured to store a second segmentation of the at least one image, the second segmentation comprising second segmentation values, each second segmentation value being associated with the spatial location having the distance; anda processor (103) configured to: assign weights to the spatial locations based on the distances (155, 156) of the spatial locations to the detector (150); compare the first segmentation values to the second segmentation values associated with the same spatial locations, to obtain comparison results associated with the spatial locations; and calculate a quantification of a difference between the first segmentation and the second segmentation based on the weights and the comparison results, wherein the first segmentation or the second segmentation of the at least one image represents a classification of the data values into at least a first category and a second category, the first category representing a segmented region, and the processor is configured to, when assigning the weights: determine a minimal distance of the spatial location of the data values of the first category to the detector; and determine a maximal distance of the spatial location of the data values of the first category to the detector, wherein the weight of data values decreases as the associated distance increases in between the minimal distance and the maximal distance.

Images