Generating and using synthetic training data

By generating synthetic training data and using benchmark ground truth segmentation masks and natural image data, the image degradation process of medical imaging modalities is simulated, which solves the problem of training data shortage, improves the robustness and segmentation performance of machine learning algorithms, and reduces the dependence on real medical images.

CN123125618APending Publication Date: 2026-11-17KONINKLIJKE PHILIPS NV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202580027238.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-04-10
Filing Date
2025-04-03
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In existing technologies, the shortage of training data for medical imaging modalities leads to poor generalization and low segmentation performance of AI models, especially due to privacy regulations and X-ray radiation restrictions, making it difficult to collect sufficient quantity and quality of training data.

Method used

By generating synthetic training data, using benchmark ground truth segmentation masks and natural image data, the image degradation process of medical imaging modalities is simulated, providing realistic training data pairs, including noise, artifacts, and degradation processing, applicable to different medical imaging modalities.

Benefits of technology

It provides more diverse training data, improves the robustness and segmentation performance of machine learning algorithms, reduces reliance on real medical images, protects privacy, and lowers data acquisition costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123125618A_ABST
    Figure CN123125618A_ABST
Patent Text Reader

Abstract

A computer-implemented method for generating synthetic training data for training a machine learning segmentation algorithm (1) of a software module, wherein the software module is configured to generate segmentation masks for images acquired by a medical imaging modality, the method comprising the steps of: receiving a ground truth segmentation mask (4) that identifies at least one major structure by means of labels; generating synthetic image data (5) by using image pattern filling of synthetic image data according to the labels of the ground truth segmentation mask (4); degrading the image quality of the synthetic image data (5) to obtain an input training image (6); and providing the input training image (6) together with the ground truth segmentation mask (4) as a training pair for the training data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a computer-implemented method for generating synthetic training data, a computer-implemented method for generating training weights, and a computer program product for implementing any of these methods. Background Technology

[0002] Computed tomography (CT) is a widely used medical imaging modality that typically images across a cross-section of a patient's body. During image acquisition, the patient is irradiated with X-rays, which are then measured by a detector unit after passing through the patient. Recovering information about the scanned volume, as well as image reconstruction and semantic segmentation, are important factors, for example, when aiming to reduce the X-ray dose level applied to the patient and improve the final image quality. Semantic segmentation of images is the cornerstone of automated medical classification. Due to recent advances in artificial intelligence (AI), segmentation problems are often solved using deep learning models. However, machine learning algorithms must be trained using training data during the training phase. This requires labeled data, i.e., pairs of images and ground truth (GT) segmentation masks, to derive the optimal segmentation model through an optimization (learning) process. In particular, large amounts of training data are often required to achieve high-quality AI-based segmentation tools. Data shortages lead to poor generalization (overfitting) of AI models and low segmentation performance.

[0003] The requirement for training data often hinders development because the amount of available CT image data is limited, for example, due to privacy regulations regarding access to medical data, such as the EU's General Data Protection Regulation (GDPR). Furthermore, the generation of CT image data is further restricted due to the negative effects of X-ray radiation on the human body. Labeling the data can be very time-consuming and therefore costly.

[0004] One way to address data scarcity is to artificially increase data variability by augmenting the training data. Augmentations can represent, for example, rigid or deformable deformations, noise or blurring damage, or variations in contrast or intensity. More advanced methods can rely on conditional adversarial networks (GANs) trained to learn the distribution of images conditioned on semantic labels. However, the application of GANs is often limited by data requirements and training instability.

[0005] Therefore, collecting diverse training datasets is extremely laborious, and for some problems, obtaining sufficient quantity and quality of training data using existing methods may simply be impossible. This problem applies not only to CT training data but also specifically to other imaging modalities that apply potentially harmful radiation, such as X-ray radiation. Furthermore, the difficulty in acquiring sufficient training data (e.g., limitations due to privacy concerns) can generally be applied to image processing related to any imaging modality.

[0006] The article "Less Is More: Unsupervised Mask-Guided Annotated CT Image Synthesis With Minimum Manual Segmentations" by Xiaodan Xing et al. (IEEE Transactions on Medical Imaging, Vol. 42, No. 9, September 2023) discloses a medical image synthesis strategy (i.e., unsupervised mask (UM)-guided synthesis) to obtain synthesized images and segments using limited manual segmentation labels. It uses a superpixel-based algorithm to generate unsupervised structure guidance and a conditional generative model to simultaneously synthesize images and annotations from unsupervised masks in a semi-supervised multi-task setting. Purpose of the invention

[0007] Therefore, the object of this invention is to provide a better method for solving the above-mentioned problems. Furthermore, it is desirable to find a solution that allows for sufficient training of machine learning algorithms and provides such well-trained machine learning algorithms. In particular, it is desirable to find a method that can provide more training data. Summary of the Invention

[0008] To better address this objective, methods according to claim 1, computer programs according to claim 11, methods according to claim 12, and data processing systems according to claim 14 are provided. Advantageous embodiments are set forth in the dependent claims. Any features, advantages, or alternative embodiments described herein with respect to the claimed methods also apply to other categories of claims, and vice versa.

[0009] According to a first aspect, a computer-implemented method is provided for generating synthetic training data for training a machine learning algorithm of a software module. The software module is configured to perform image processing on images acquired by a medical imaging modality to generate a segmentation mask. The medical imaging modality may in particular be an X-ray-based medical imaging modality, such as computed tomography (CT) imaging. The method includes the steps of: receiving a reference ground truth segmentation mask that identifies the main structures inside the image by at least one label; generating synthetic image data by filling synthetic image data (5) with an image pattern according to at least one label of the reference ground truth segmentation mask (4); degrading the image quality of the synthetic image data (5) to obtain an input training image (6); and providing the input training image together with the reference ground truth segmentation mask as a training pair for the training data, wherein the image pattern used to fill the synthetic image data may be selected from various options: (1) a natural image; (2) a natural video, wherein the temporal dimension of the natural video is converted into a third spatial dimension to obtain three-dimensional image data; and / or (3) image data selected from medical images.

[0010] According to an embodiment of the present invention, the dynamic range of the image pattern is adjusted based on the dynamic range of typical medical image data values, based on the corresponding label in the segmentation mask for at least one image pattern in the image pattern.

[0011] Optionally, only a subset of the segmentation labels in the segmentation mask may be used to mimic missing organs in the image training data.

[0012] Advantageously, the method of the present invention allows for the provision of training data pairs that provide a realistic downgraded version of a synthetic image (input training image) with a segmentation mask (i.e., a baseline ground truth image). The method of the present invention is applicable to various medical imaging modalities; that is, the training data can be used to train a machine learning algorithm to generate a segmentation mask obtained from any imaging modality. The method of the present invention may be particularly useful for imaging modalities involving the application of X-ray radiation to an object or patient (such as X-ray imaging or computed tomography (CT)), because the amount of available training data for these imaging modalities is often particularly limited due to the adverse effects of X-ray radiation on the human body (which ethically limits the possibility of acquiring image data). However, the method can also be applied to other medical imaging modalities, such as magnetic resonance imaging (MRI), ultrasound imaging, or positron emission tomography (PET). The method can also have training data for obtaining training data for training image-guided therapy (IGT) methods.

[0013] The term "synthetic image" will be interpreted broadly in the context of this application. It generally refers to an image generated at least partially by means other than medical imaging scans utilizing an imaging modality, particularly an image generated by a software module. For example, a synthetic image may also be partially synthesized and partially obtained through an imaging modality. Furthermore, the synthetic image may have been derived from images obtained through medical imaging scans, but these images have been processed, for example, to degrade image quality or to generally alter the appearance of the image. Training pairs can be provided so that they can be used for training, particularly supervised training. Alternatively, it can be specified that the training pairs are to be used to validate a trained machine learning algorithm. In the context of this invention, this alternative may also be included by the term "training data." Therefore, the term "training data" may also include the use provided for validation purposes.

[0014] Training data can be used to train machine learning algorithms for software modules. Machine learning algorithms can be, for example, neural networks (NNs), particularly convolutional neural networks (CNNs). In this context, the term "software module" must be understood broadly, typically describing a component of a software architecture that includes one or more software modules. Optionally, a software module can also be the only software module in the software architecture. The software module is configured to segment images acquired by a medical imaging modality. Therefore, the software module can include program code configured for segmentation. In particular, the software module includes machine learning algorithms.

[0015] Synthetic images can be obtained entirely or partially from natural image data. The term "natural image data" will be understood broadly in the context of this invention and generally describes images that humans might observe in the real world. For example, a natural image may depict a landscape, object, or animal, or parts thereof. For example, a natural image may be a depiction of sand or a depiction of a part of a tiger. Specifically, natural image data can be or includes optical images acquired with an optical camera (including infrared cameras and video streams). Thus, natural images can also include frames acquired from optical / natural video. Therefore, the term "natural image data" can describe image data not acquired by a medical imaging modality, particularly not by an imaging modality of the same kind as the medical imaging modality from which the training data is to be used. Surprisingly, it has been found that training data generated from natural images can be used to achieve feasible training results, thus making the provision of actual medical images unnecessary. Therefore, when using natural image data, access to high-resolution imaging data, such as CT projection data acquired at high X-ray doses, is not necessarily required. Advantageously, natural image data can often be provided in large quantities and at high resolution. Prior to their application, the synthesis can be converted into a specific color code corresponding to the typical color code (e.g., grayscale) of the medical imaging modality. Additionally or alternatively, training data based on natural image data can also be used to supplement training data based on images obtained from the medical imaging modality.

[0016] To obtain the input training images, the image quality of the synthetic images is downgraded. Therefore, the input training images are downgraded or more realistic versions of the synthetic images. Degradation can generally be understood as any kind of reduction in image quality and / or perceptible image information. Degradation can be selected to correspond to one or more image degradations typically associated with a medical imaging modality. Therefore, the degradation can be selected based on the type and / or quality level of the medical imaging modality to be used. Thus, the image quality of the downgraded image can correspond to the typical image quality available in the medical imaging modality. Therefore, the training pairs can be adapted to train machine learning algorithms to segment images from the medical imaging modality.

[0017] According to embodiments, degrading image quality includes: adding noise to image data, adding image artifacts to image data, and / or reducing the resolution of image data. When adding image artifacts, training pairs can be used to train a machine learning algorithm to provide a segmentation mask even in the presence of image artifacts. In CT-related cases, image artifacts can be, for example, cone-beam artifacts. Adding noise can be used to generate training pairs that are used to train a machine learning algorithm to provide a segmentation mask in the presence of noise. For example, noise can be added by adding noise with statistical characteristics typical of the imaging modality (e.g., statistical characteristics typical of CT images). Additionally or alternatively, additional degradation can be applied. For example, motion can be added to the image data as a supplement and alternative to the above degradation. For example, motion can be generated by using dynamic content from natural video. Degradation can include applying only one or a combination of degradations.

[0018] According to an embodiment, degrading the image quality includes: forward projecting image data into a projection space of an artificially designed imaging modality, degrading the image quality of the projected image data in the projection space, and reconstructing the projected and degraded image data back into the image space.

[0019] In the context of this invention, the projection space will be understood broadly. The projection space can describe any kind of image space corresponding to raw image data (particularly image data prior to image reconstruction) obtained by an artificially designed imaging modality. The projection space can also be referred to as the projection domain. For example, in the case of CT imaging, the projection space can be the imaging space including, for example, the X-ray projection detected by the X-ray detector of a CT system. It has been found that degrading image quality in the projection space can achieve more realistic image degradation compared to degrading directly in the image space. This is due to the fact that the degradation is applied to a more “raw” state corresponding to the actual reconstructed data in a real system, and therefore corresponds better to realistic image degradation. By forward-projecting the image back into the image space, the artificial quality loss is less “obvious” for the image improvement algorithm. Therefore, the algorithm must learn to process and segment realistic images. For example, more realistic resolution loss or more realistic noise can be simulated. Therefore, the training data obtained through this embodiment can thus produce better training results, such as more robust trained segmentation algorithms.

[0020] According to an embodiment, in the projection space, image quality is degraded by performing a low-dose simulation, which introduces noise corresponding to the medical imaging modality. The medical imaging modality may have its own type of noise with characteristics distinguishable from other types of noise. For example, the noise may have a characteristic distribution in the frequency space. The power of noise at different frequencies may differ for different modalities. Therefore, to create realistic data, it may be advantageous to apply noise that is typically corresponding to the imaging modality. The noise may be even more realistic when applied in the projection space (i.e., noise corresponding to the original data) than attempting to apply noise in the reconstructed image.

[0021] According to an embodiment, the synthesized images are obtained at least in part from image data taken from natural videos, wherein the temporal dimension of the natural videos is transformed into a third spatial dimension to obtain three-dimensional image data. Advantageously, natural videos can provide high image resolution and low noise levels, and are therefore well-suited for this purpose. The temporal dimension of the video (i.e., the third dimension) can be used to generate third-dimensional data of the three-dimensional image data (i.e., the three-dimensional image volume). Therefore, using natural videos in this way advantageously allows for the generation of synthetic training data with relatively low effort, even for three-dimensional volumes. Prior to their application, the natural videos can be converted into a specific color code corresponding to the typical color code (e.g., grayscale) of the medical imaging modality.

[0022] According to an embodiment, a synthetic input image is generated by receiving a medical image obtained from a medical imaging modality or another medical imaging modality and replacing at least a portion of the medical image with at least one image portion obtained from another image source (particularly from at least one natural image). Preferably, several or all portions may be replaced. Thus, the ground truth image is partially synthetic and partially based on real medical images. In particular, the ground truth image may preserve the basic structure of the medical image, but replace the content of the structure with texture obtained from another image source (particularly from one or more natural images). Replacing portions of the medical image allows for a larger amount of training data while still maintaining realistic anatomical structures in the image. Thus, a larger training dataset can be generated based on a small amount of available training data. It has been found that using natural images can be particularly advantageous due to the generally more diverse fine structures and textures on natural images. Therefore, a rich variety of data can be generated. Training using these partially replaced images can yield even more robust trained algorithms by benefiting from the varying structures of natural images.

[0023] According to an embodiment, at least one region outside the segmentation mask is clustered together into at least one clustering region, and wherein pixels or voxels within the same clustering region are assigned a constant pixel or voxel value, optionally the average value of the original pixels or voxels within the clustering region. In many cases, segmenting each organ on a medical image may be impractical. Therefore, for the remainder of the medical image, clustering according to this embodiment can be applied. This can allow, or even fill, unsegmented regions. For example, pixels or voxels in a cluster can be assigned the average value within the clustering region. This may result in a segmented, constant approximation of unsegmented regions. Advantageously, the assignment of pixel or voxel values ​​can further allow for the labeling of corresponding regions. Clustering and applying new pixel / voxel values ​​have the advantages of reducing connections to the original patient from whom the image was acquired. On the one hand, this can therefore generate a more diverse training dataset. On the other hand, this can therefore better protect the privacy rights of the original patient.

[0024] According to an embodiment, the dynamic range of at least one image portion of at least one natural image is adjusted based on the dynamic range of medical image data within a segmentation mask. This embodiment can also be applied when the image portion is taken from any other source (i.e., not from a natural image). Depending on the corresponding medical image and depending on the corresponding segment being segmented (e.g., an organ), the dynamic range of the original portion of the medical image can be different. For example, some body parts may appear bright, while others may appear quite dark. Furthermore, typical variations in brightness and / or contrast within different segments can be different. On the other hand, random natural images can have different dynamic ranges. Therefore, by adjusting the dynamic range of the image portion obtained from the natural image, the result can be more realistic, such that the synthesized image will have a dynamic range comparable to that of a real medical image. This may lead to better training results when training a machine learning algorithm using the corresponding training data. For example, in a first sub-step, the dynamic range of the medical image data within the segmentation mask can be calculated based on the original imaging data values. In subsequent sub-steps, the dynamic range of the image portion can be adjusted. For example, adjustment can be provided by matching the histogram of the dynamic range and / or by rescaling the dynamic range according to the quantiles of the dynamic range distribution. Additionally or alternatively, image portions can be further adjusted. For example, image portions can be flipped, rotated, and / or deformed. This can provide a wider variety of training data.

[0025] According to an embodiment, at least one property of the segmentation mask is modified within a predefined boundary before filling image data with an image pattern. Modifying the property of the segmentation mask can advantageously allow for a more diverse dataset. For example, in this way, a limited imaging dataset can be expanded to a much larger and richer imaging dataset. Predefined boundaries can be defined such that the changes remain within realistic properties. In particular, the shape, position, orientation, and / or size of at least one segmentation mask can be modified. The position can be changed by shifting the segmentation mask within the medical image. The maximum permissible shift can be contracted, for example, relative to the size of the segmentation or by absolute value. The orientation of the segmentation mask can be changed by rotating the segmentation mask. The rotation may be limited by a maximum permissible rotation, for example, a specific value given in degrees. The size of the segmentation mask can be changed by adjusting the size of the mask. The size adjustment can be limited by, for example, a maximum permissible increase or decrease in size relative to the original size of the segmentation mask. For example, the shape of the segmentation mask can be changed by distorting it. One, several, or all of the changes in shape, position, orientation, and size can be applied. Other properties can also be altered; for example, the dynamic range within a segmentation mask can be adapted, for instance, by the percentile of the original dynamic range within the segmentation mask. This change can be performed randomly within predefined boundaries. Which changes can be applied can be selected randomly, including changes to shape, position, orientation, and / or size. The number of changes applied can be selected randomly.

[0026] According to an embodiment, before generating synthetic image data, at least one property of the segmentation mask is changed within a predefined boundary, wherein in particular, the shape, position, orientation and / or size of at least one segmentation mask is changed.

[0027] According to another aspect of the invention, a method is provided for generating training weights to be used in a software module including a machine learning algorithm. The training weights are generated by training the machine learning algorithm using synthetic training data obtained as described herein by the method for generating synthetic training data. The machine learning algorithm may optionally be trained using additional training data, such as based on real medical image data and segmentation masks. The machine learning algorithm can be trained to correct cone-beam artifacts in CT image data. The machine learning algorithm can be trained to perform deblurring of image data. The machine learning algorithm can be trained to perform deblurring in an image space with reconstructed image data or in a projection space (particularly with raw image data obtained by an imaging device). Training data in the projection space can be generated by forward projecting image data from the image space into the projection space. The machine learning algorithm can be trained to upsample image data to a higher image resolution. The machine learning algorithm can be trained to perform magnification in an image space with reconstructed image data or in a projection space (particularly with raw image data obtained by an imaging device). The machine learning algorithm can be trained to perform motion correction of image data. Motion in the training data can be generated by manipulating segmentation masks (e.g., shifting, rotating, and / or distorting masks) or by using dynamic content from natural videos.

[0028] According to another aspect of the present invention, a computer program product is provided comprising a computer program code module, which, when executed on a computing device having a processing system, causes the processing system to perform all steps of the computer-implemented method according to any one of claims 1 to 8.

[0029] Advantageously, the quality of the trained machine learning algorithm in the software module can be improved because a larger amount of training data can be provided by using synthetic training data. Furthermore, it has been found that using natural images as the basis for training data, as described in this paper, can enhance the robustness of machine learning segmentation algorithms even when applying a considerable amount of training data; this may be attributed to the greater structural detail of natural images. Therefore, the provision of synthetic training data can still have an effect on the trained algorithm even during its application.

[0030] Features and advantages of different embodiments can be combined. Features and advantages of one aspect of the invention (e.g., a method for generating synthetic training data) can be applied to other aspects, such as methods for generating training weights, training weights, software modules, image processing apparatus, uses, and / or non-transient storage media, and vice versa. Attached Figure Description

[0031] The invention will now be described with reference to embodiments in the accompanying drawings, in which:

[0032] Figure 1 A flowchart is shown of a computer-implemented method for generating synthetic training data for training machine learning segmentation algorithms, according to an embodiment of the present invention.

[0033] Figure 2 A diagram illustrates a computer-implemented method for generating synthetic training image data for training machine learning algorithms in software modules;

[0034] Figure 3 A flowchart is shown of a computer-implemented method for generating synthetic training data for training machine learning segmentation algorithms, and a method for using the training data to train weights;

[0035] Figure 4 An example of training image data generated by a method according to an embodiment of the present invention is shown;

[0036] Figure 5 The segmentation result and the baseline ground truth segmentation mask are shown using the segmentation algorithm according to the present invention. Detailed Implementation

[0037] Throughout the accompanying drawings, the same or corresponding features / elements in various embodiments are designated by the same reference numerals.

[0038] Figure 1 A flowchart 100 illustrates a computer-implemented method for generating synthetic training data according to an embodiment of the present invention. This synthetic training data is used to train a machine learning segmentation algorithm 1 of a software module for image processing of images acquired from a medical imaging modality. The method relies on the simulation of actual acquisition and a known ground truth segmentation mask. In a first step 110, a ground truth segmentation mask 4 is received. In a second step 120, a synthetic image is generated based on the ground truth segmentation mask. In a further step 130, the image quality of the synthetic image 5 is downgraded to obtain an input training image 6. For example, downgrading may include one or more of adding noise to the image data, adding image artifacts to the image data, and reducing the resolution of the image data. Thus, the downgrading step simulates the physical limitations of actual image modality acquisition and therefore forces prior knowledge about the actual acquisition, thereby achieving a perceived image quality similar to that of real images. In a further step 140, the input training image 6 is provided together with the ground truth segmentation mask 4 as a training pair.

[0039] Figure 2A diagram illustrates a computer-implemented method for degrading the image quality of a synthesized image. In a first step 210, synthesized image data 5 is received. In this example, synthesized image 5 is based on a natural image 3, which is based on a label applied to a segmentation mask for identifying a liver on a cylinder, wherein the liver is filled with constant values, and the remainder of the cylinder is filled with values ​​from artificial data, random or non-random portions from medical images, and / or from the natural image. Optionally, the image can be obtained at least partially from image data taken from a natural video, such that the temporal dimension of the natural video is transformed into a third spatial dimension to obtain three-dimensional image data, i.e., a three-dimensional image volume. In a further step 221, the image data is forward-projected into the projection space of an artificially designed imaging modality, thereby generating projected image data 51. The artificially designed imaging modality can be, for example, a computed tomography detector with any chosen resolution. In a further step 220, the image quality of the projected image data 51 is degraded in the projection space. This is done by downsampling the projected image data 51 to an imaging modality with a lower resolution. Furthermore, low-dose simulation can be applied to introduce noise into the image data to obtain noisy, low-resolution projected image data 61. Therefore, this embodiment can be applied to reduce resolution only, add noise only, or both. In a further step 222, the projected and downgraded image data 61 is reconstructed back into the image space to obtain the input training image 6. Compared to the synthetic image data 5, the input training image 6 is more blurred and / or noisier. In a further step 230, the input training image 6 is provided together with the baseline ground truth segmentation mask 4 as a training pair against the training data.

[0040] Figure 3 A flowchart of a computer-implemented method for generating synthetic training data according to another embodiment of the present invention is shown. This synthetic training data is used to train a machine learning algorithm 1 of a software module for image processing of images acquired by a medical imaging modality. In a first step 311, a reference ground truth segmentation mask 4 obtained from the medical imaging modality or another medical imaging modality is received.

[0041] Alternatively, segmentation methods using convolutional neural networks (CNNs), as known in the art, can be applied to medical images to generate a baseline ground truth segmentation mask. The segmentation steps can be performed according to methods known in the prior art. In particular, the method can be based on the application of a trained convolutional network. An example of a segmentation method that can be applied similarly or in combination with variations is the TotalSegmentator, as published by Jakob Wasserthal et al. in “TotalSegmentator: robust segmentation of 104 anatomical structures in CT images” (arXiv:2208.05868[eess.IV], 2022.). Thus, almost all organs in the human body can be segmented. This approach is likely most advantageous when multiple segments (preferably as many as possible) are applied. Applying more segments and thus replacing more parts of the image can lead to more varied training data and therefore better training results. However, this approach may already be advantageous when only a few or even just one segment is applied.

[0042] Furthermore, in another step 312, an image pattern, such as artificial or natural image data, or a (random or non-random) portion of a medical image, is obtained. The natural image 3 can be enhanced and / or rescaled. Enhancement may include, for example, flipping, rotating, and / or deforming. In another optional step 313, labels for the segmentation mask are selected. For example, some labels may be randomly omitted to broaden the diversity in the training data to simulate missing organs. In another example, the baseline ground truth segmentation mask 4 may be provided from a database or generated by manually labeling medical images from the same or different image modalities. Optionally, in an optional step 314, at least one property of the corresponding segmentation mask 4 may be changed within predefined boundaries. For example, the shape, position, orientation, and / or size of the segmentation mask 4 may be changed. In another step 315, the dynamic range of one or more natural images 3 or portions of natural images 3 may be adjusted based on the dynamic range of the medical image data within the segmentation mask 4. For example, the dynamic range within the segmentation mask 4 can be calculated based on the original imaging values, and the dynamic range of the natural image 3 or a portion thereof can then be adjusted based on this calculation. This adjustment can be achieved, for example, by matching histograms or by rescaling according to quantiles. In a further step 316, image data corresponding to at least some of the labels in the segmentation mask 4 of the medical image are replaced with image patterns obtained from the artificial, medical, or natural image 3 or portions thereof. Preferably, the selection of the natural image 3, adjustment of the dynamic range, and replacement of the segmentation mask 4 are applied to each label of the segmentation mask 4, respectively. If regions exist outside the segmentation mask(s) 4, for example, if the segmentation mask only covers a portion of the image, these unsegmented regions can be clustered together into at least one cluster region and pixels or voxels, similarly filled with the artificial, medical, or natural image 3. This can be an option, for example, if segmenting each organ is not feasible in some cases. For example, a constant pixel or voxel value can be the average of the original pixels or voxels within the cluster region. In particular, if several cluster regions are applied, this may result in a segmented constant approximation of the unsegmented regions. Therefore, the image obtained by replacing parts of a medical image with an image pattern can then be used as synthetic image data 5. In a further step 320, the image quality of the synthetic image data 5 is downgraded to obtain the input training image 6. It has been found that the resulting image is high-quality image training data 6 consistent with the real image already obtained from the corresponding imaging modality. Due to the natural noise texture and the global structure forced by the segmentation mask but filled with arbitrary local structures, the result is similar to a real CT image. The entire process can be repeated multiple times for the same set of selected labels. Finally, in a further step 330, the input training image 6 is provided together with the baseline ground truth image 5 as a training pair for the training data.

[0043] Figure 3A method for generating training weights to be used in a software module including machine learning segmentation algorithm 1 is also shown. The training weights are generated by training machine learning segmentation algorithm 1 using synthetic training data comprising a training pair including input training image 6 and a baseline ground truth segmentation mask image 4. Therefore, machine learning algorithm 1 can be trained using this training pair and more equivalent training pairs.

[0044] Figure 4 An example of an input training image 6 generated by a method according to an embodiment of the invention is shown. The first image (a) shows a computed tomography input training image 6, generated by a CT simulation of a synthetic image 5 of a liver segmentation with constant values ​​pasted into a cylinder filled with values ​​from a natural image (using forward projection of a selected geometry (e.g., slice-by-slice axial or spiral), simulation of CT noise assuming a specific tube current distribution, and subsequent image reconstruction). In the second image (b), the input training image is based on a synthetic image 5 made using constant values ​​of various segmentation labels in a segmentation mask 4. In the third image (c), the input training image is a downgraded version of the synthetic image 5 made by filling various segmentation labels into a natural image. The dynamic range may have been adjusted based on the actual CT values ​​within the corresponding segmentation mask before pasting the natural image data into the CT scan. Thus, an input training image with a realistic overall structure and dynamic range, different from the internal structure of the organ, has been provided. Therefore, the new image is suitable to be used as part of the training dataset for training the machine learning segmentation algorithm 1. The resulting input training image is a high-quality CT image consistent with the real image. By simulating different CT contribution degradations, a very suitable pair of images can be obtained for further learning-based methods (such as CNNs).

[0045] Synthetic training data generated using the method according to embodiments of the present invention was tested to verify the usefulness of the method. Using data corresponding to… Figure 4 Several strategies are used to generate the data from the three images shown to train the segmentation model. Figure 5 The results of segmentation algorithm 1 segmenting the liver on a CT scan are shown when tested on a real CT scan. Figure 5 (d) shows the baseline truth segmentation mask, and Figure 5 (a) shows liver segmentation using a segmentation algorithm trained with an input training image based on a synthetic image generated by filling the liver region with a constant value and filling the non-liver region with a natural image. Figure 5 (b) shows liver segmentation using segmentation algorithm 1 trained with input training images based on synthetic images generated from segmentation masks with multiple labels, the regions of which have been filled with random constant values ​​(one value per region). Figure 5(c) shows liver segmentation using segmentation algorithm 1 trained with input training images based on synthetic images generated from segmentation masks with multiple labels, the regions of which have been filled with corresponding rescaled natural video.

[0046] The above discussion is intended to illustrate the method and system only and should not be construed as limiting the appended claims to any particular embodiment or group of embodiments. Therefore, the specification and drawings should be considered illustrative and not intended to limit the scope of the appended claims. By studying the drawings, disclosure, and appended claims, those skilled in the art can understand and implement other variations of the disclosed embodiments in practicing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the words "a" or "an" do not exclude multiple. A single processor or other unit can perform the functions of several items recited in the claims. Although specific measures are recited in mutually different dependent claims, this does not indicate that combinations of these measures cannot be advantageously used. Computer programs can be stored / distributed on suitable media, such as optical storage media or solid-state media provided with or as part of other hardware, but computer programs can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems. Any reference numerals in the claims should not be construed as limiting the scope.

Claims

1. A computer-implemented method for generating synthetic training data for training a machine learning algorithm (1) of a software module, in, The software module is configured to generate segmentation masks for images acquired by medical imaging modalities. The method includes the following steps: Receive (110) a reference truth segmentation mask (4), the reference truth segmentation mask including at least one label that identifies the main structure; The synthetic image data (120) is generated by using image pattern filling to synthesize the synthetic image data (5) based on at least one label of the segmentation mask (4) according to the reference truth value; The image quality of the synthesized image data (5) is downgraded (130) to obtain the input training image (6); and The input training image (6) and the baseline ground truth segmentation mask image (4) are provided together (140) as a training pair for the training data, characterized in that the image pattern used to fill the synthetic image data is selected from: Natural Images (3) Natural video, wherein the temporal dimension of the natural video is converted into a third spatial dimension to obtain three-dimensional image data, and Image data from medical images.

2. The method according to claim 1, in, The dynamic range of the image pattern is adjusted based on the dynamic range of typical medical image data values, using the corresponding label in the segmentation mask for at least one of the image patterns.

3. The method according to any one of claims 1 to 2, wherein, Only a subset of the segmentation labels in the segmentation mask are used to mimic missing organs in the image training data.

4. The method according to any one of claims 1 to 2, in, Degrading image quality includes at least one of the following operations: Noise is added to the synthesized image data (5). Add image artifacts to the synthesized image data. Reduce the resolution of the synthesized image data. The synthesized image data is forward-projected into the projection space of an artificially designed imaging modality; The image quality of the projected image data (51) in the projection space is downgraded, preferably downgraded to an image resolution that can be obtained by the medical imaging modality; and the projected and downgraded image data (61) is reconstructed back into the image space.

5. The method according to claim 4, in, In the projection space, the image quality is degraded by performing a low-dose simulation, which introduces noise corresponding to the medical imaging modality.

6. The method according to any one of the preceding claims, in, The baseline true value segmentation mask (4) is generated by receiving a medical image obtained from the medical imaging modality or another medical imaging modality and segmenting the medical image.

7. The method according to any one of claims 1 to 6, in, Before generating the synthetic data image, at least one property of the segmentation mask (10) is changed within a predefined boundary, wherein in particular, the shape, position, orientation and / or size of the at least one segmentation mask (10) is changed.

8. A computer-implemented method for generating training weights to be used by a machine learning algorithm (1), in, The training weights are generated by training the machine learning algorithm (1) using synthetic training data, wherein the synthetic training data is obtained by any of the preceding claims.

9. A computer program product comprising a computer program code module, which, when executed on a computing device having a processing system, causes the processing system to perform all the steps of the computer-implemented method according to any one of claims 1 to 8.