Devices for generation of synthetic 3D representations of myelin content

Abstract

The present invention relates to a device for training a generative model configured to generate at least one synthetic 3D representation of myelin content and to a device and a computer-implemented method for obtaining at least one synthetic 3D image representative of myelin content for a patient using a trained generative model having training parameters obtained using the device for training.

Description

DEVICES FOR GENERATION OF SYNTHETIC 3D REPRESENTATIONS OFMYELIN CONTENTFIELD OF INVENTION

[0001] The present invention relates to the technical field of image processing and more particularly to a device for training a generative model configured to generate at least one synthetic 3D representation of myelin content. The invention also relates to a device and computer-implemented method for obtaining at least one synthetic 3D image representative of myelin content for a patient using a trained generative model having training parameters obtained from said device for training.BACKGROUND OF INVENTION

[0002] Demyelination is a process in which the protective myelin sheath surrounding nerve fibers in the central nervous system (CNS) or peripheral nervous system (PNS) becomes damaged, leading to disruptions in the transmission of nerve impulses and a range of neurological symptoms.

[0003] Demyelination is observed in several demyelinating diseases such as antibody- mediated inflammatory demyelinating diseases (e.g. Myelin oligodendrocyte glycoprotein antibody disease (MOGAD) or Neuromyelitis optica), Multiple Sclerosis (MS), Acute Disseminated Encephalomyelitis (ADEM), Progressive Multifocal Leukoencephalopathy (PML), genetic and acquired leukodystrophies such as Metachromatic Leukodystrophy (MLD) or Krabbe Disease.

[0004] Magnetic Resonance Imaging (MRI) is a key tool for visualizing demyelinating lesions in the CNS. While MRI in a clinical setting provides an accessible noninvasive tool for patient’s routine follow-up, it doesn’t give access to specific information about the biological mechanisms underlying the disease such as the myelin content.

[0005] Myelin content refers to the amount and distribution of myelin in the CNS. It may be used by physicians to diagnose a disease and assess its progression and to make informed decisions about treatment.

[0006] Positron Emission Tomography (PET) with Pittsburgh Compound B (PiB) is particularly adapted to obtain myelin content. However, this technique is not available in clinical routine because of the costs of the procedure and the risks associated with the exposure to ionizing radiations for patients.

[0007] Therefore, there is a need for a non-invasive clinical routine approach capable of quantifying myelin content in patients presenting demyelination.SUMMARY

[0008] This invention thus relates to a device for training a generative model configured to generate at least one synthetic 3D representation of myelin content, using 3D images of at least one portion of a brain of a patient having white matter demyelinating lesions, said device comprising: at least one input configured to receive, for each subject of a plurality of subjects presenting white matter demyelinating lesions, a first ensemble comprising at least three 3D images including an acquired first 3D image, an acquired second 3D image and an acquired third 3D image, wherein said first 3D image and said second 3D image are 3D magnetic resonance routine images representative of brain anatomical structures, and said third 3D image is representative of myelin content; at least one processor configured to: o generate a training dataset by: for each subject of said plurality of subjects:■ co-registering the at least three 3D images of each first ensemble in a reference space;■ extracting, in each co-registered 3D image of each first ensemble, at least one same volume comprising at least one portion of at least one white matter demyelinating lesion, so as to obtain corresponding patches fromeach co-registered 3D image of each first ensemble, so as to define at least one set of lesions patches;■ extracting, in each co-registered 3D image of each first ensemble, at least one same volume comprising at least one portion of white matter so as to obtain corresponding patches from each co-registered 3D image of each first ensemble, so as to define at least one set of white matter patches; and■ generating a set of training image pairs including the co-registered first 3D image paired with the co-registered second 3D image for each first ensemble; o train a Conditional Generative Adversarial Network to obtain training parameters for said generative model, wherein during the training:■ said generative model is fed (optionally fed forward) with said set of training image pairs from the training dataset so as to provide as an output at least one synthetic 3D image representative of myelin content, and■ at least one discriminator receives as input:• said at least one synthetic 3D image representative of myelin content provided by the generative model,• said sets of training image pairs, and• conditional information including said at least one set of lesion patches and / or said at least one set of white matter patches according to a curricular training strategy; at least one output configured to provide said obtained training parameters of said generative model for generation of at least one synthetic 3D representation of myelin content.

[0009] Advantageously the device of the present invention provides training parameters for a generative model (i.e., Conditional Generative Adversarial Network) configured to generate synthetic 3D representations of myelin content using as input only routine images representative of brain anatomical structures easily available for all patients. It has to be noted that these routine images representative of brain anatomical structures and allowing to locate MS lesions (for FLAIR) do not allow trained physicians to evaluate the myelin content or the demyelination level into white matter lesions, and indeed are notcomprised into the medical guidelines as viable images to analyze in case of white matter demyelinating lesions diseases. The specific construction of the training dataset and training process herein proposed advantageously allows to obtain a robust generative model, capable of effectively representing the amount and distribution of myelin with a high specificity, similar to the specificity that can be obtained using imaging techniques designed to capture myelin content such as PET images acquired using a myelin specific radiotracer, Magnetization Transfer Ratio images or T1 relaxometry image. Furthermore, the curricular training strategy provides enhanced performance compared to using a standard 3D patches U-Net. Indeed, training the generative model using lesion and white matter patches following a curricular training strategy provides an adaptable training allowing to focus the training on the discriminator and to choose a simpler machine learning structure for said generative model reducing the risks of overtraining the generative model.

[0010] According to other advantageous aspects of the invention, the device comprises one or more of the features described in the following embodiments, taken alone or in any possible combination.

[0011] According to one embodiment, said first 3D image is a T1 -weighted image or a MP2RAGE sequence image, said second 3D image is a FLAIR image or a T2-weighted image, and said third 3D image is one of the following: a PET image acquired using a myelin specific radiotracer or a Magnetization Transfer Ratio image or a T1 relaxometry image or a Myelin Water Fraction image or inhomogeneous Magnetization Transfer Ratio (ihMTR) image.

[0012] According to one embodiment, said at least one first ensemble further comprises a magnetic resonance T2 sequence 3D image, the at least one processor being further configured to: segment white matter demyelinating lesions on each T2 sequence 3D image; obtain a segmentation mask of the white matter demyelinating lesions and a segmentation mask of the white matter; wherein the position of said at least one same volume in the reference space, so that it comprises at least one portion of white matter demyelinating lesion or it comprises at leastone portion of white matter, is chosen using the segmentation mask for the white matter demyelinating lesions and the white matter mask respectively. According to one alternative embodiment the segmentation is performed on a FLAIR image instead of a T2 sequence.

[0013] According to one embodiment, said curricular training strategy comprises: during a first epoch of the training, feeding to the discriminator conditional information comprising exclusively patches selected from the at least one set of lesion patches, and increasing throughout epochs the proportion of patches selected from the at least one set of white matter patches comprised in the conditional information.

[0014] Advantageously, the present invention proposes this “adapted” curricular training strategy wherein the discriminator receives, from the beginning of the training, mostly the patches comprising the lesions (i.e., more difficult examples to learn for the discrimination) and then progressively receives more and more of the white matter patches, which might or might not comprise the lesions (i.e., easier examples to learn).

[0015] According to one embodiment, the proportion of sets of white matter patches comprised in the conditional information linearly increases throughout epochs until it reaches 100% for the final training epoch.

[0016] According to one embodiment, the at least one discriminator is a patch-based convolutional neural network.

[0017] According to one embodiment, the first ensemble comprises at least one fourth 3D image representative of myelin content, wherein the acquisition parameters and / or the imaging technique used to acquire said at least one fourth 3D image is(are) different from the imaging technique used to acquire said third 3D image. In this embodiment, the generative model has a machine learning architecture comprising one encoder and at least two decoders including a first decoder and a second decoder, wherein the first decoder is associated to the at least one third 3D image and the second decoder is associated to the at least one fourth 3D image.

[0018] In that case, the Conditional Generative Adversarial Network may comprise at least two discriminators, including a first discriminator and a second discriminator wherein the first discriminator is associated to the first decoder and the second discriminator is associated to the second decoder, wherein during the training the first discriminator uses as conditional information the patches of the sets obtained using the first 3D images, the second 3D images and the third 3D images, and the second discriminator uses as conditional information the patches of the set obtained from the first 3D images, the second 3D images and the fourth 3D images.

[0019] According to one example of this embodiment, during training, said encoder and said at least two decoders are jointly trained, so that the training parameters are iteratively updated at the end of a training epoch only when all outputs of said at least two decoders have been obtained.

[0020] According to one embodiment, the generative model is a 3D U-Net. The generative model may be notably derived from a U-Net- Variational Auto-Encoder (VAE), with one encoder and at least two decoders. Advantageously, having only one encoder allows to extract common latent characteristics of the image at each stage based on the clinical available inputs (e.g., T1 and FLAIR). Using a VAE architecture in the bottleneck helps to structure the latent space with reduced parameters, preventing a sequence- specific sub-mapping of the latent space. Moreover, the latent space vector may be calculated an analyzed as a global latent myelinic biomarker for a patient.

[0021] The present invention further relates to a device for obtaining at least one synthetic 3D image representative of myelin content for a patient, using a trained generative model having training parameters obtained from a device for training such as described above, wherein the device comprises: at least one input configured to receive, for said patient, a second ensemble comprising an acquired first 3D image and an acquired second 3D image, wherein said first 3D image and said second 3D image are 3D magnetic resonance routine images representative of brain anatomical structures; at least one processor configured to:o co-register the first 3D image and second 3D image into a same reference space; o generate at least one synthetic 3D image representative of myelin content by feeding forward said co-registered first and second 3D images to said trained generative model; at least one output configured to provide said at least one synthetic 3D image representative of myelin content.

[0022] According to one embodiment, the at least one processor is further configured to calculate a position in a representation space of white matter demyelinating patients, said position being obtained from a latent space of the generative model feed forward with said co-registered first and second 3D images. Advantageously, this representation of patients in the multidimensional latent space allows to isolate one group of patients presenting white matter demyelinating lesions as it seems to occupy a specific region of the latent space. This specific localization of patients with white matter demyelinating lesions provides meaningful information for physicians in the diagnosis of white matter demyelinating lesions disease.

[0023] The present invention further relates to a computer-implemented method for obtaining at least one synthetic 3D image representative of myelin content for a patient using a trained generative model having training parameters previously obtained, wherein the method comprises: receiving, for said patient, a second ensemble comprising an acquired first 3D image and an acquired second 3D image, wherein said first 3D image and said second 3D image are 3D magnetic resonance routine images representative of brain anatomical structures; co-registering the first 3D image and second 3D image into a same reference space; generating at least one synthetic 3D image representative of myelin content by feeding forward said co-registered first and second 3D images to said trained generative model; outputting said at least one synthetic 3D image representative of myelin content.

[0024] In addition, the disclosure relates to a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method for obtaining at least one synthetic 3D image representative of myelin content described above.

[0025] The present disclosure further pertains to a computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method for obtaining at least one synthetic 3D image representative of myelin content described above.

[0026] The present disclosure further pertains to a non-transitory program storage device, readable by a computer, tangibly embodying a program of instructions executable by the computer to perform a method for obtaining at least one synthetic 3D image representative of myelin content, compliant with the present disclosure.

[0027] Such a non-transitory program storage device can be, without limitation, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device, or any suitable combination of the foregoing. It is to be appreciated that the following, while providing more specific examples, is merely an illustrative and not exhaustive listing as readily appreciated by one of ordinary skill in the art: a portable computer diskette, a hard disk, a ROM, an EPROM (Erasable Programmable ROM) or a Flash memory, a portable CD-ROM (Compact-Disc ROM).DEFINITIONS

[0028] In the present invention, the following terms have the following meanings:

[0029] The terms “adapted” and “configured” are used in the present disclosure as broadly encompassing initial configuration, later adaptation or complementation of the present device, or any combination thereof alike, whether effected through material or software means (including firmware).

[0030] The term “processor” should not be construed to be restricted to hardware capable of executing software, and refers in a general way to a processing device, which can for example include a computer, a microprocessor, an integrated circuit, or aprogrammable logic device (PLD). The processor may also encompass one or more Graphics Processing Units (GPU), whether exploited for computer graphics and image processing or other functions. Additionally, the instructions and / or data enabling to perform associated and / or resulting functionalities may be stored on any processor- readable medium such as, e.g., an integrated circuit, a hard disk, a CD (Compact Disc), an optical disc such as a DVD (Digital Versatile Disc), a RAM (Random- Access Memory) or a ROM (Read-Only Memory). Instructions may be notably stored in hardware, software, firmware or in any combination thereof.

[0031] “Machine learning (ML)” designates in a traditional way computer algorithms improving automatically through experience, on the ground of training data enabling to adjust parameters of computer models through gap reductions between expected outputs extracted from the training data and evaluated outputs computed by the computer models.

[0032] A “hyper-parameter” presently means a parameter used to carry out an upstream control of a model construction, such as a remembering-forgetting balance in sample selection or a width of a time window, by contrast with a parameter of a model itself, which depends on specific situations. In ML applications, hyper-parameters are used to control the learning process.

[0033] “Datasets” are collections of data used to build an ML mathematical model, so as to make data-driven predictions or decisions. In “supervised learning” (i.e. inferring functions from known input-output examples in the form of labelled training data), three types of ML datasets (also designated as ML sets) are typically dedicated to three respective kinds of tasks: “training”, i.e. fitting the parameters, “validation”, i.e. tuning ML hyperparameters (which are parameters used to control the learning process), and “testing”, i.e. checking independently of a training dataset exploited for building a mathematical model that the latter model provides satisfying results.

[0034] A “neural network (NN)” designates a category of ML comprising nodes (called “neurons”), and connections between neurons modeled by “weights”. For each neuron, an output is given in function of an input or a set of inputs by an “activation function”.Neurons are generally organized into multiple “layers”, so that neurons of one layer connect only to neurons of the immediately preceding and immediately following layers.

[0035] The above ML definitions are compliant with their usual meaning, and can be completed with numerous associated features and properties, and definitions of related numerical objects, well known to a person skilled in the ML field. Additional terms will be defined, specified or commented wherever useful throughout the following description.BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The present disclosure will be better understood, and other specific features and advantages will emerge upon reading the following description of particular and non-restrictive illustrative embodiments, the description making reference to the annexed drawings wherein:

[0037] Figure 1 is a block diagram representing schematically a particular mode of a device for obtaining a trained generative model for generation of at least one synthetic 3D representation of myelin content compliant with the present disclosure;

[0038] Figure 2 is a is a flow chart showing successive steps executed with the device for obtaining a trained generative model for generation of at least one synthetic 3D representation of myelin content of Figure 1;

[0039] Figure 3 is a block diagram representing schematically a particular mode of a device for obtaining at least one synthetic 3D image representative of myelin content using the trained generative model from the device represented in Figure 1, compliant with the present disclosure;

[0040] Figure 4 is a flow chart showing successive steps executed with the device for obtaining at least one synthetic 3D image representative of myelin content of Figure 3;

[0041] Figure 5 shows an image A and an image B corresponding to a first 3D image and a second 3D image captured on a same patient, an image D corresponding to an overlapping brain mask, white matter mask and lesions mask of the patient, an image Ecorresponding to a synthetic 3D image representative of myelin content obtained using a curricular training, and an image F corresponding to a to synthetic 3D image representative of myelin content obtained using an anti-curricular training.

[0042] Figure 6 shows a representation of the distribution of patients with white matter demyelinating lesions (cross symbols) and healthy subjects (dots symbols) from a testing cohort in the latent space after Principal Component Analysis.

[0043] Figure 7 shows the correlation at lesion level between synthetic and actual PiB DVR in the testing cohort for the curricular training approach and the opposite approach (anti curricular training).

[0044] Figure 8: Architecture of the MS-SQUID-Net with the algorithm used for training.

[0045] Figure 9: Synthetic (top) and actual (bottom) images for a pwMS and HC atlases from MIMS Database.

[0046] Figure 10: A: Multimodal synthetic myelinic atlas from Genova cohort HCs. B: Lesional atlas from the three testing cohorts in MNI152 (number of lesions detected per voxel in hot colors), with 6HDs sample position (squares) with MOG expression intensity (diverging colors) from the Brain Allen Atlas. C: Spatial correlations for all genes of interest expression and all atlases intensities (synthetic atlases from Genova cohort and actual atlases from MIMS) among samples with a non-null MS-lesion probability, intensities represent beta coefficient, and values the p-values of the model, significant correlations after multiple comparison correction are marked with *.

[0047] Figure 11: A: Lesion-level correlation between synthetic and actual MTR signal in MS lesions for the Inflasep testing database. B: Patient-level correlation between synthetic MTR LMI and actual MTR LMI in the Inflasep testing database. C: Patientlevel correlation between synthetic MTR LMI and actual PiB SRTM2 LMI in the MIMS testing database. D: Patient-level correlation between synthetic multimodal LMI and actual multimodal LMI in the MIMS testing database. E: Patient-level correlation between longitudinal evolution of multimodal LMI and stepwise EDSS change (y-axis)for the Inflasep testing research cohort. F : Patient- level correlation between longitudinal evolution of multimodal LMI and stepwise EDSS change (y-axis) for the Genova clinical cohort.

[0048] Figure 12: A: atlas of demyelination (pooled zscores among patients) for first visit of Inflasep cohort, based on actual MTR. B: atlas of demyelination (pooled zscores among patients) for first visit of Inflasep cohort, based on multimodal synthetic myelinic maps. C: Maps of synthetic dynamic remyelination and remyelination among patients on Inflasep cohort. D: gradient of remyelination from lateral ventricles to deep white matter based on synthetic myelinic maps.

[0049] Figure 13: A: Actual image of Florbetaben SUVr of a pwMS. B: Synthetic image of Florbetaben SUVr of the same pwMS given by the MS-SQUID-Net newly trained Florbetaben decoder. C: Examples of two other pwMS correlation between after synthetic and actual normalized Florbetaben SUVr mean value in MS-lesions, showing strong and significant correlation for each patient.

[0050] Figure 14 is a table comprising information concerning dataset with training (first four lines) and testing (last three lines) cohorts.

[0051] Figure 15 is a table comprising: A: Comparison of EMI correlation between MS- SQUID-Net and other architectures for both monomodal and multimodal EMI in the MIMS cohort. B and C: Fidelity metrics (MAE, MSE, PSNR, SSIM) for different ROIs in the INFLASEP (Figure 15B) and MIMS (Figure 15C) testing cohorts, comparing MS- SQUID-Net with other architectures based on absolute synthetic and actual signal values for each myelinic modality. Results are displayed, among patients, as : mean (standard deviation).ILLUSTRATIVE EMBODIMENTS

[0052] The present description illustrates the principles of the present disclosure. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the disclosure and are included within its scope.

[0053] All examples and conditional language recited herein are intended for educational purposes to aid the reader in understanding the principles of the disclosure and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions.

[0054] Moreover, all statements herein reciting principles, aspects, and embodiments of the disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.

[0055] Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein may represent conceptual views of illustrative circuitry embodying the principles of the disclosure. Similarly, it will be appreciated that any flow charts, flow diagrams, and the like represent various processes which may be substantially represented in computer readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0056] The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which may be shared.

[0057] It should be understood that the elements shown in the figures may be implemented in various forms of hardware, software or combinations thereof. Preferably, these elements are implemented in a combination of hardware and software on one or more appropriately programmed general-purpose devices, which may include a processor, memory and input / output interfaces.

[0058] The present disclosure will be described in reference to a particular functional embodiment of a device 1 for obtaining a trained generative model for generation of at least one synthetic 3D representation of myelin content, as illustrated on Figure 1.

[0059] The device 1 is adapted to output training parameters 30 for a generative model 20 for generation of at least one synthetic 3D representation of myelin content. This generative model 31 is configured to receive as input only a first 3D image 32 and a second 3D image 33 obtained from one patient, wherein the first 3D image 32 and the second 3D image 33 are 3D magnetic resonance routine images representative of brain anatomical structures.

[0060] In the context of the present invention, “3D magnetic resonance routine” images refer to images acquired using common and basic MRI sequences, that are generally considered standard procedures in MRI examinations. They provide valuable structural information and are relatively straightforward to acquire. These 3D images may be acquired using magnetic resonance imaging (MRI) scan. These scans are conducted using standard imaging sequences and protocols that are commonly used in clinical practice, such as a T1 -weighted imaging, MP2RAGE sequence, T2- weighted imaging, or FLAIR. Conventional MRI primarily relies on the detection of differences in water content and the relaxation properties of tissues. It provides excellent contrast between tissues with varying water content, such as between gray matter and white matter in the brain. However, myelin is a fatty substance and does not contain significant water, making it impossible to quantify in standard MRI images.

[0061] Thanks to the training proposed in the present invention, using these routine 3D images 32, 33, non-specific to myelin content but highly available, the trained generative model 31 is capable of generating as an output at least one synthetic 3D image 61 representative of myelin content.

[0062] To that end, the device 1 is configured to generate a training dataset and train a Conditional Generative Adversarial Network 25 to obtain the training parameters 30 for said generative model 20.

[0063] The device 1 for training the untuned generative model 20 is associated with a device 2, represented on Figure 3, for obtaining at least one synthetic 3D image 61representative of myelin content for a patient from a first 3D image 32 and a second 3D image 33, using the trained generative model 31 obtained from the device 1, which will be subsequently described.

[0064] Though the presently described devices 1 and 2 are versatile and provided with several functions that can be carried out alternatively or in any cumulative way, other implementations within the scope of the present disclosure include devices having only parts of the present functionalities.

[0065] Each of the devices 1 and 2 is advantageously an apparatus, or a physical part of an apparatus, designed, configured and / or adapted for performing the mentioned functions and produce the mentioned effects or results. In alternative implementations, any of the device 1 and the device 2 is embodied as a set of apparatus or physical parts of apparatus, whether grouped in a same machine or in different, possibly remote, machines. The device 1 and / or the device 2 may have functions distributed over a cloud infrastructure and be available to users as a cloud-based service, or have remote functions accessible through an API.

[0066] The device 1 for training the generative model and the device 2 for obtaining at least one synthetic 3D image representative of myelin content may be integrated in a same apparatus or set of apparatus, and intended to same users. In other implementations, the structure of the device 2 may be completely independent of the structure of the device 1, and may be provided for other users. For example, the device 2 may have a trained generative model available to operators for Synthetic 3D images of myelin content generation, wholly set from previous training effected upstream by other players with the device 1.

[0067] In what follows, the modules are to be understood as functional entities rather than material, physically distinct, components. They can consequently be embodied either as grouped together in a same tangible and concrete component, or distributed into several such components. Also, each of those modules is possibly itself shared between at least two physical components. In addition, the modules are implemented in hardware, software, firmware, or any mixed form thereof as well. They are preferably embodied within at least one processor of the device 1 or of the device 2.

[0068] The device 1 comprises a module 11 for receiving, for each subject of a plurality of N subjects presenting white matter demyelinating lesions, at least one first ensemble of images 24 and the untuned generative model 20, stored in one or more local or remote database(s) 10. The latter can take the form of storage resources available from any kind of appropriate storage means, which can be notably a RAM or an EEPROM (Electrically- Erasable Programmable Read-Only Memory) such as a Flash memory, possibly within an SSD (Solid-State Disk). Alternatively, module 11 may be configured to receive as an input directly a training dataset (i.e.; from a database 10) instead of the information comprised the first ensemble of images 24.

[0069] More precisely, when the device 1 is configured to construct the training dataset, the module 11 may be configured to receive as an input at least one first ensemble 24 comprising at least three 3D images including a first 3D image 21, a second 3D image 22 and a third 3D image 23 acquired on a same subject. The module 11 may be configured to receive only one first ensemble 24 acquired on a subject, but more preferably, the device 1 is configured to receive multiple first ensembles 24 acquired on multiple subjects.

[0070] In the present invention, the first 3D image 21 and the second 3D image 22 are 3D magnetic resonance routine images wherein brain anatomical structures are observable. These 3D images may be acquired using magnetic resonance imaging (MRI) scan conducted using standard imaging sequences and protocols. Preferably, the first 3D image may be a T1 -weighted image or a MP2RAGE sequence image and the second 3D image may be a FLAIR image or a T2-weighted image.

[0071] The training dataset comprises multiple training samples, and each training sample may be associated to each subject of the plurality of subjects, notably of a cohort. Each training sample may comprise at least one set of lesions patches, at least one set of white matter patches and a set of training image pairs.

[0072] The third 3D image 23 may be a PET image previously acquired using a myelin specific radiotracer, a Magnetization Transfer Ratio image or a T1 relaxometry image or a Myelin Water Fraction image.

[0073] In the present disclosure a PET image refers to an image of a subject acquired using a PET scan. During such PET scan, a subject is injected with a small amount of a radioactive substance called a radiotracer. The radiotracer is a molecule that contains a positron-emitting isotope. In the case of demyelinating diseases, the radiotracer is chosen to specifically be taken up by the myelin. Multiple myelin specific radiotracers are currently available such as [HC]Pittsburgh Compound B ([HC]-PiB), [HC]Benzenesulfonamide ([11C]-BMB), [18F]Florbetaben, [18F]-Florbetapir, [18F]Flutemetamol, or [HC]Methyl-6-(2-(4-(2-(2-(2-18F-Fluoroethoxy)ethoxy)ethoxy)phenyl)-lH-imidazol-4-yl)pyridin-3-amine ([11C]- MeDAS). It as to be noticed that any new radio tracer developed for target myelin content in brain and known by the person skilled in the art could be used in the context of the present invention. The PET scan detects the radiations emitted by the radiotracers uptaken by white matter demyelinating lesions. In the present disclosure the PET images are acquired so to comprise the brain region.

[0074] Alternatively, the third 3D image 23 may be a Magnetization Transfer Ratio image, which is a specialized type of magnetic resonance imaging (MRI) that provides information about the properties of tissues in the human body, particularly in the brain. MTR imaging is used to highlight and study the presence of macromolecules, such as myelin. The third 3D image 23 may also be a T1 relaxometry image, which is a magnetic resonance imaging (MRI) technique that focuses on measuring the T1 relaxation time of tissues within the body. T1 relaxometry imaging is used to create T1 maps, which provide quantitative information about tissue characteristics, such as myelin content.

[0075] Advantageously, each first ensemble 24 may also comprise a magnetic resonance T2 sequence 3D image 25.

[0076] Additionally, the first ensemble 24 may also comprise at least one fourth 3D image 26. Notably, the first ensemble 24 may comprise also a fifth, sixth, ... nth3D images representative of myelin content, like the third 3D image. The at least one fourth 3D image, and ithimages 26 are preferably a 3D images representative of myelin content acquired with different acquisition parameters and / or a different imaging technique than the one used to acquire the third 3D image 23. In the same way, the ith3D images are acquired with different acquisition parameters and / or a different imaging technique thanthe others. For example, the third 3D image may be a PET image, the fourth image an MTR image and the fifth image is a T1 relaxometry image.

[0077] The device 1 further comprises optionally a module 12 for preprocessing the 3D images. The module 12 may be adapted to preprocess the at least three 3D images 21-23, 25, 26 of the first ensemble 24. For instance, the at least three 3D images 21-23, 25, 26 may be cropped to focus on the signal within the brain and re-sized, for instance to 128x128x128 pixels voxels. Advantageously, the first 3D image 21 and the second 3D image 22 may be rescaled within a predefined range, and the extreme predefined percentage of values may be removed. For example, the extreme 5% may be removed.

[0078] According to one example, the module 12 is configured to correct for the N4 bias field the first 3D image 21 (e.g., Tl) and the second 3D image 22 (e.g., FEAIR). Then to robustly scale them on the brain mask, meaning that their intensity histograms are linearly redistributed to [-1, 1], excluding the 5% most extreme voxel intensities. The module 12 may be further configured to apply a common preprocessing for myelinic modalities (23, 25, 26). First, voxel-level quantification of third 3D image 23 (e.g., [nC]-PiB PET image) may be performed to extract DVRs using the Eogan graphical method, and with the Simplified Reference Tissue Model 2 (SRTM2). These quantification methods employ a supervised clustering algorithm at the voxel level to extract a reference region based on the Time Activity Curve of each voxel. Those quantification procedures provided the PET Eogan and PET SRTM2 ground truth (GTH), respectively. MP2RAGE images were processed allowing us to extract a Tl relaxometry, corresponding to our Tlmap ground truth.

[0079] As ground truth images (e.g., PET Logan, PET SRTM2, MTR, and Tlmap) are quantitative, they were treated as absolute values. Due to differences in modality scales, fixed coefficients we used based on the median signal value in the white matter of healthy controls. Advantageously this approach allows to prevent bias toward any specific myelinic sequence during training. After inference, these fixed coefficients may be applied to rescale the network's output to directly usable values.

[0080] The device may comprise a module 13 for the construction of the training dataset based on the received at least three 3D images 21-23, 25, 26 of the first ensemble 24.Preferably, multiple first ensembles 24 are acquired from multiple subjects are used in this module 13. The module 13 is configured to construct a training dataset comprising at least one set of lesions patches, at least one set of white matter patches and a set of training image pairs.

[0081] A first step for the generation of the training sample of the training dataset comprises co-registering the at least three 3D images in a same reference space. The reference space may be the space of one of the at least three 3D images, such as the space of the third 3D image 23, or an entirely new space.

[0082] Module 13 is then configured to generate a set of training image pairs including the co-registered first 3D image paired with the co-registered second 3D image for each first ensemble 24.

[0083] Module 13 is further configured to generate, for each first ensemble 24 associated to one patient, one or more sets of lesions patches. This step is performed by extracting, in the first, second and third co-registered 3D images of each ensemble 24, a same volume comprising at least one portion of at least one white matter demyelinating lesion. The extracted volumes should have the same coordinates in the reference space for each coregistered 3D image. The extracted volumes are also called patches. Preferably, for the lesion patches, the volume is selected so to be centered on the center of gravity of a randomly chosen lesion. Advantageously, several patches may be extracted in the different co-registered 3D images of the first ensemble 24. In order to guide the choice of the special position of these extracted volumes, a segmentation mask for the white matter demyelinating lesions of the associated patient may be used.

[0084] The at least one set of white matter patches may be obtained by extracting, in each co-registered 3D image of the first ensemble 24, a same volume comprising at least one portion of white matter. The extracted volumes should have the same coordinates in the reference space for each co-registered 3D image of the first ensemble 24. The extracted volumes are also called patches. Advantageously, several patches may be extracted in the different co-registered 3D images of the first ensemble 24. The patches included in the at least one set of white matter patches may overlap with the patches in the at least one set of lesions patches. In order to guide the choice of the special positionof these extracted volumes, a segmentation mask for the white matter of the associated patient may be used. Preferably, for the white matter patches, the volume is selected so to be centered on random points within the white matter mask.

[0085] The position of the same volume may advantageously be determined using the magnetic resonance T2 sequence 3D image 25 (or a FLAIR image) received as an input by the device 1. Indeed, the module 13 may be configured to segment the white matter demyelinating lesions on each co-registered magnetic resonance T2 sequence 3D image of each first ensemble 24 in order to obtain two segmentation masks: a segmentation mask of the white matter demyelinating lesions and a segmentation mask of the white matter. Both the segmentation mask of the white matter demyelinating lesions and the segmentation mask of the white matter may be used to determine the position of the same volume comprising at least one portion of white matter and the same volume comprising at least one portion of at least one white matter demyelinating lesion. For instance, the segmentation of the white matter demyelinating lesions may be performed using a 3D Linet. A mask of the brain may be extracted using affine registration to an atlas using Ants. The white matter segmentation may for instance be performed using the Fastsurfer CNN, a deep-learning pipeline for the fully automated processing of structural human brain MRIs.

[0086] Module 13 is further configured to aggregate, for each patient, the obtained sets of training image pairs (i.e., co-registered first and second image), the obtained at least one set of white matter patches and at least one set of lesions patches into one training sample.

[0087] The device 1 further comprises a module 14 configured to train a Conditional Generative Adversarial Network 27 using the training dataset constructed by the module 13(or received by module 11) to obtain training parameters for the untuned generative model 20.

[0088] By definition, a Conditional Generative Adversarial Network (cGAN) 27 is a generative model that comprises at least one generator and at least one discriminator, that works in a competitive manner during the training. The generator creates data samples, while the discriminator tries to distinguish between real data and generated data. A cGANis configured to incorporate additional conditional information (i.e., sets of white matter and lesions patches) during both the training and generation processes. This conditional information advantageously help controlling and guiding the generation of specific content or characteristics in the generated data (i.e., synthetic 3D representation of myelin content).

[0089] During the training, the generative model 20 is fed (notably fed forward) with the sets of training image pairs from the training dataset so as to provide as an output at least one synthetic 3D images representative of myelin content. The training may be performed till convergence using an Adam optimizer.

[0090] The cGAN 27 comprises at least one discriminator configured to receive the at least one synthetic 3D images representative of myelin content provided by the generative model 20. The cGAN 27 is also configured to receive the sets of training images pairs and conditional information including said at least one set of lesion patches and / or said at least one set of white matter patches.

[0091] The conditional information may be received according to a curricular training strategy. In other words, the proportion of patches coming from sets of lesion patches and the proportion of patches coming from sets of white matter patches evolves throughout the training process. This advantageously allows to focus the training on the discriminator, which is different from the usual training strategies for GAN structures, that focus on generators (i.e., generative model). This allows to choose a simpler machine learning structure for the generative model 20 and to reduce the risks of overtraining (also known as overfitting) the generative model.

[0092] In one advantageous example, during the first epoch (e.g. during the first pass through the entire training dataset of the training), the conditional information provided to the discriminator comprises exclusively patches from the set of lesion patches. Throughout the epochs, the proportion of patches selected from the at least one set of white matter patches increases until it reaches 100% in the final training epoch. Indeed, the inventors have surprisingly found that this strategy, which starts by focusing the training on difficult examples and then slowly introduces easier example, provides better performances that the opposite approach (starting by learning easier examples andprogressing afterwards to more complex examples). By guiding the learning process with a curriculum, the cGANs can learn to capture a wide range of lesions variations and generate more realistic and diverse synthetic 3D representation of myelin content. The increase in the proportion of patches selected from the at least one set of white matter patches may be linear.

[0093] The architecture of at least one discriminator may be a patch-based convolutional neural networks.

[0094] The architecture of the generative model 20, trained in module 14, may be a neural network and may be a convolutional neural network (CNN) or an attention based neural network, such as a transformer, or any neural network for which the last layer outputs an activate map that has the size of a 3D image.

[0095] In one example, the generative model 20 has an architecture comprising two main parts: at least one encoder (contracting path) and at least one decoder (expansive path). The at least one encoder captures the contextual information of the input image through a series of convolutional and pooling layers. This part of the network reduces the spatial dimensions of the input while extracting high-level features. The at least one decoder aims to recover the spatial information and to produce a segmentation map. It uses a series of upsampling and convolutional layers to gradually increase the spatial resolution.

[0096] In a particular embodiment of the invention wherein at least one fourth 3D image is received, the generative model 20 has a machine learning architecture comprising one encoder and at least two decoders, including a first decoder and a second decoder. The first decoder is associated to the third 3D images 23 from the at least one first ensembles 24 in order to generate a first synthetic 3D image representative of myelin content and the second decoder is associated to the fourth 3D images 25 from the at least one first ensembles 24 in order to generate a second synthetic 3D image representative of myelin content.

[0097] As a direct consequence of this particular embodiment of the invention, the Conditional Generative Adversarial Network 27 may comprise at least two discriminators, including a first discriminator and a second discriminator, wherein the first discriminator is associated to the first decoder and the second discriminator isassociated to the second decoder. During the training, the first discriminator uses as conditional information the patches from the at least one set of white matter patches and / or the patches from the at least one set of lesions patches obtained using the first 3D images 21, second 3D images 22 and third 3D images 23 of the first ensemble 24. The second discriminator uses conditional information the patches from the at least one set of white matter patches and / or the patches from the at least one set of lesions patches obtained using the first 3D images 21, second 3D images 22 and fourth 3D images 25 of the first ensemble 24. Therefore, the type of myelin content represented in the first and second synthetic 3D image is different because these synthetic images are generated so as to mimic the myelin content respectively of the third 3D images 23 and fourth 3D images 25. 25 (which have been acquired with different acquisition parameters and / or different imaging technique).

[0098] If the first ensemble 24 comprises a fourth and a fifth 3D images, the generative model has a machine learning architecture comprising three decoders wherein the second decoder is associated to the fourth 3D images from the at least one first ensembles 24 in order to generate a second synthetic 3D image representative of myelin content and the third decoder is associated to the fifth 3D images in order to generate a third synthetic 3D image representative of myelin content. Similarly to case of two decoders, the type of myelin content represented in the first, second and third synthetic 3D image is different as these synthetic images are generated so as to mimic the myelin content respectively of the third 23, fourth 25 and fifth 3D images.

[0099] In that case, the Conditional Generative Adversarial Network 27 comprises at least three discriminators, including a first discriminator, a second discriminator and a third discriminator, wherein the first discriminator is associated to the first decoder, and the second discriminator is associated to the second decoder and the third discriminator is associated to the third decoder. During the training, the first discriminator uses as conditional information the patches from the at least one set of white matter patches and / or the patches from the at least one set of lesions patches obtained using the first 3D images 21, second 3D images 22 and third 3D images 23 of the first ensemble 24. The second discriminator uses as conditional information the patches from the at least one set of white matter patches and / or the patches from the at least one set of lesions patchesobtained using the first 3D images 21, second 3D images 22 and fourth 3D images 25 of the first ensemble 24. The third discriminator uses conditional information the patches from the at least one set of white matter patches and / or the patches from the at least one set of lesions patches obtained using the first 3D images 21, second 3D images 22 and fifth 3D images 25 of the first ensemble 24.

[0100] The number of decoders in the machine learning architecture of the generative model may increase proportionately with the number of additional ith3D images. In the same manner, the number of discriminators may increase proportionately with the number of decoders.

[0101] Advantageously, the encoder and the at least two decoders of the generative model 20 are jointly trained, so that the training parameters are iteratively updated at the end of a training epoch only when all outputs of said at least two decoders have been obtained.

[0102] In a particular embodiment, the generative model architecture may be a 3D U- Net model. The 3D U-Net architecture, which is a CNN, is advantageously well-suited for tasks where the input image needs to be segmented into distinct regions or objects. The U-Net architecture includes skip connections between the encoders and decoders to enable the network to retain and merge high-resolution features with the upsampled information. The skip connections are a key feature of the U-Net architecture, allowing the network to capture both local and global context. They enable the precise localization of objects and help address the problem of information loss during the downsampling process. In one example, the generative model architecture may be derived from a U-Net- Variational Auto-Encoder (VAE) with one encoder and at least two decoders which combines the U-Net architecture with a VAE framework.

[0103] According to one embodiment, particularly adapted to the generative model architecture being derived from a U-Net-VAE, the training is performed in two parts: the first part involving simultaneous multimodal training focused on training the encoder, while the second part involving refining a given modality decoder branch. In both parts of the training, a curricular approach is employed, applied at the patient level for the first part of the training and at the Region Of Interest (ROI) level for the second part.

[0104] The first part of the training aims to develop the best encoder to extract the global myelinic latent space. For this purpose, only databases that could provide at least two myelinic sequences are used, ensuring that at each iteration, the U-Net-VAE-based architecture was trained to extract characteristics capable of simultaneously recreating multiple myelinic maps. The U-Net-VAE-based architecture was trained to minimize the following loss at each iteration, focusing only on the brain parenchyma by applying the loss on the brain mask only: Losspartl= KLdatentiter, N(ji100, <J100) + Ssegi2;osspredseq,gthseq)' Kullback-Leibler divergence, V(100, a100)the gaussian distribution based on vectors p and c of size 100, latentiteris the latent tensor at a given iteration, and the sum term corresponds to the average of L2 loss between prediction and ground truth for each myelinic sequence. The training routing is configured to ensures that the U-Net-VAE-based architecture has seen all four myelinic sequences to compute its loss before backpropagating. A dropout regularization may be employed.

[0105] The second part of the training aims to enhance the decoders by providing them with additional examples. To achieve this, two extra cohorts, comprising more MTR and T1 maps, are utilized for training. To leverage these new examples for improving the MTR and Tlmap decoders, while retaining the benefits of the first training phase that required multimodal myelin maps, all weights and biases in the U-Net-VAE-based architecture’s encoder, bottleneck, and other decoders were frozen. This meant no gradients or optimizations were applied to these components during this phase. For the decoded sequences (MTR and then Tlmap), the U-Net-VAE-based architecture - with all parameters frozen except for the sequence decoder - functions as a generator. This generator is incorporated into the training loop of the conditional Generative Adversarial Network (cGAN). In this embodiment, the generator produced a synthetic MTR map (MTRsynth), while a patch-based discriminator attempted to classify the map as synthetic or real, using the (Tl, FLAIR) data as conditioning inputs. Batches of 48 patches of size 16x16x16 may be used for the discriminator and a batch size of 1 may be used for the generator (representing the entire image). The loss functions for the generator and discriminator may be as follows:

[0106] where D(MTR) corresponding to the label attributed by the discriminator to a given MTR image based on its (Tl, FLAIR) condition, labelstrue and labelsfake respectively describes the labels attributed to true and fake MTR. To facilitate training, the labels were smoothed with a Gaussian filter with a mean of 0.8 for labelstrue and 0.2 for labelsfake, and standard deviation of 0.3 for both labels.

[0107] In this second part of the training, for MTR as for Tlmap, the conditional Generative Adversarial Network (cGAN) was trained using a curriculum approach at the region-of-interest (ROI) level. A discriminator's patch sampling rule was used to guide the training process, emphasizing specific ROIs. Initially, the discriminator was trained with batches containing patches sampled predominantly from multiple sclerosis (MS) lesions. Over time, the proportion of patches randomly sampled from the white matter was gradually increased, reaching 100% by the 300th epoch.

[0108] Once the training is completed, the module 14 is configured to output the obtained training parameters 30 of said generative model for generation of at least one synthetic 3D representation representative of myelin content. The training parameters 30 may then by stored in one or more local or remote database(s) 10. The latter can take the form of storage resources available from any kind of appropriate storage means, which can be notably a RAM or an EEPROM (Electrically-Erasable Programmable Read-Only Memory) such as a Flash memory, possibly within an SSD (Solid-State Disk).

[0109] In its automatic actions, the device 1 may for example execute the following process (Figure 2):- receiving the at least one first ensemble 24 comprising the at least three 3D images including the first 3D images 21, the second 3D images 22, third 3D images 23 and the untuned generative model 20 (step 41),- preprocessing the at least three 3D images of the at least one first ensemble 24, including the first 3D images 21, second 3D images 22, third 3D images 23 (step 42),- constructing the training dataset using the preprocessed at least three 3D images of the at least one first ensemble 24, including the preprocessed first 3D images 21, second 3D images 22, third 3D images 23 (step 43),- training the Conditional Generative Adversarial Network 27 using the untuned generative model 20 and the constructed training dataset so as to obtain training parameters for the generative model 20 (step 44).

[0110] The present invention also relates to a device 2 for generating Synthetic 3D images of myelin content 61 from a first 3D image 32 and a second 3D image 33, acquired on one subject, using the trained generative model 30 associated to the training parameters30 obtained from the device 1, as described above. The device 2 will be described in reference to a particular function embodiment as illustrated in Figure 3.

[0111] The device 2 is adapted to provide as an output at least one synthetic 3D images of myelin content 61 for the subject using at least the first 3D image 32, the second 3D image 33 and the trained generation model 31.

[0112] The device 2 comprises a module 15 for receiving the trained generation model31 and the first 3D image 32 and a second 3D image 33, stored in one or more local or remote database(s) 10. The latter can take the form of storage resources available from any kind of appropriate storage means, which can be notably a RAM or an EEPROM (Electrically-Erasable Programmable Read-Only Memory) such as a Flash memory, possibly within an SSD (Solid-State Disk). In advantageous embodiments, the trained generation model 31 and all its parameters may have been previously generated by a system including the device 2 for training. Alternatively, the untuned generation model and its training parameters 30 may be received by the device 2 from a communication network.

[0113] The device 2 further comprises optionally a module 16 for preprocessing the 3D images comprised in the second ensemble 34, including the first 3D image 32 and the second 3D image 33. For instance, the 3D images 32, 33 may be cropped to focus on the signal within the brain and re-sized, for instance to 128x128x128 pixels voxels. Advantageously, the first 3D image 32 and the second 3D image 33 may be rescaledwithin a predefined range, and the extreme predefined percentage of values may be removed. For example, the extreme 5% may be removed.

[0114] The device 2 further comprises a module 17 configured to provide the preprocessed 3D images from the second ensemble 34, including the preprocessed first 3D image 32 and the preprocessed second 3D image 33 as input to said trained generative model 31 so as to generate a corresponding synthetic 3D image of myelin content 61.

[0115] Further to the generation of the synthetic 3D image 61, the device 2 may be configured to calculate a position in a representation space of white matter demyelinating patients, said position being obtained from a latent space of the generative model feed forward with said co-registered first and second 3D images. For instance, the position in the latent space may be put in correlation with information about the patients, such as the age, level of disability, lesion load or myelin content. These data may be compared to healthy patients. Predefined groups may be identified, such as the group including healthy patients or the group including patients with a low level of disability. Afterward, each new tested patient may have its position in the latent space compared to said predefined groups in order to identify the group from which he is the closest. For that, a classical machine learning algorithm may be used to define the groups. Moreover, a K nearest neighbor approach may be used to classify the new patient in a group given his position in the latent space. Figure 6 shows an example of a distribution of patients with white matter demyelinating lesions (cross symbols) and healthy subjects (dots symbols) from a testing cohort in the latent space after Principal Component Analysis.

[0116] The device 2 may be configured to calculate a Patient Lesional Myelinic Index (LMI) according to the following

[0117]

[0118] where LMI represents the Lesional Myelinic Index for a given myelinic modality, numiesions patient is the total number of lesions above 50 mm3for the patient, mean(signal)iesion and volumeiesion are respectively the mean z-score and the volume of the lesion, and lesion total volumepatient corresponds to the total lesion volume for the patient.

[0119] This LMI reflects the overall level of demyelination in lesions for a given myelinic modality. The LMIs may be extracted using both actual myelinic sequences (actual LMIs) and synthetic myelinic sequences (synthetic LMIs). For both synthetic and actual data, LMIs may be averaged across the available myelinic modalities to create multimodal LMIs. The LMI may be calculated after removing from the images lesions smaller than 50 mm3.The LMI may be considered as an index reflecting the intensity and the volume of the lesions.

[0120] The device 2 may interact with a user interface 18, via which information can be entered and retrieved by a user. The user interface 18 includes any means appropriate for entering or retrieving data, information or instructions, notably visual, tactile and / or audio capacities that can encompass any or several of the following means as well known by a person skilled in the art: a screen, a keyboard, a trackball, a touchpad, a touchscreen, a loudspeaker, a voice recognition system.

[0121] In its automatic actions, the device 2 may for example execute the following process (Figure 4):- receiving the second ensemble 34 comprising at least the first 3D image 32 and the second 3D image 33 (step 51),- preprocessing the 3D images of the second ensemble 34, including the first 3D image 32 and the second 3D image 33 (step 52),- providing the preprocessed 3D images of the second ensemble 34, including the preprocessed first 3D image 32 and second 3D image 33 to said trained generation model 31 so to generate at least one synthetic 3D image 61 of myelin content (step 53).

[0122] The present invention also relates to a method for predicting disability progression in a patient having white matter demyelinating lesions, said method comprising: determining a first patient lesional myelinic index for a second ensemble 34 obtained for a first moment and determining a second patient lesional myelinic index for a second ensemble 34 obtained for a second moment in time using the device 2 as described herein; wherein the first moment is anterior to the second moment in time; comparing the first and second patient lesional myelinic indexes;predicting that the patient disability worsens if the second patient lesional myelinic index increases with respect to the first patient lesional myelinic index.

[0123] It will be understood to the skilled artisan in the art that the method for predicting disability progression in a patient as described herein may present the advantage of being independent of volumetric predictors generally used to predict disability worsening. Examples of volumetric predictors include, without limitation, thalamic volume change or cortical volume change.

[0124] The patient as described herein have white matter demyelinating lesions. White matter demyelinating lesions may be detected by imaging, and in particular, with MRI scans.

[0125] In one embodiment, the patient having white matter demyelinating lesions is a patient suffering from a myelin pathology.

[0126] As used herein, a myelin pathology refers to any disease or pathology of the nervous system in which the myelin sheath of neurons is damaged. Examples of myelin pathologies include, without limitation, multiple sclerosis, optic neuritis, transverse myelitis, neuromyelitis optica or adrenoleukodystrophy.

[0127] In one embodiment, the patient as described herein suffers from multiple sclerosis.

[0128] It may be predicted that the patient disability worsens when the second patient lesional myelinic index is about 10% or more higher than the first patient lesional myelinic index, such as about 20, 30, 40, 50, 60, 70, 80, 90% or more higher than the first patient lesional myelinic index.

[0129] Disability progression may be measured by changes in Expanded Disability Status Scale (EDSS). As used herein, the Expanded Disability Status Scale (EDSS) is a method of quantifying disability in multiple sclerosis (MS).

[0130] The EDSS provides a total score on a scale that ranges from 0 to 10. The first levels 1.0 to 4.5 refer to patients with a high degree of ambulatory ability and the subsequent levels 5.0 to 9.5 refer to the loss of ambulatory ability.

[0131] The criteria used in EDSS are presented in Table 1 below.

[0132] Table 1* Excludes cerebral function grade 1.

[0133] The present invention also relates to a method for identifying if a test compound is a pro-remyelinating compound, wherein the method comprises: receiving, for at least one patient, a first (e.g., prior) second ensemble 34 and a second (e.g., subsequent) second ensemble 34, wherein each of the first 34 and second second ensemble 34 comprises an acquired first 3D image 32 and an acquired second 3D image 33, wherein said first 3D image 32 and said second 3D image 33 are 3D magnetic resonance routine images representative of brain anatomical structures; wherein the first second ensemble 34 has been acquired before administration of a test compound to the at least one patient and the second second ensemble 34 has been acquired after administration of the test compound to the at least one patient; determining a first patient lesional myelinic index for said first second ensemble 34 using the device 2 as described herein; determining a second patient lesional myelinic index for said second second ensemble 34 using the device 2 as described herein; concluding that the test compound is a pro-remyelinating compound if the second patient lesional myelinic index is inferior to the first patient lesional myelinic index.

[0134] As used herein, a pro-remyelinating compound is a compound capable of promoting remyelination.

[0135] It may be concluded that the test compound is a pro-remyelinating compound when the second patient lesional myelinic index is about 10% or more lower than the first patient lesional myelinic index, such as about 20, 30, 40, 50, 60, 70, 80, 90% or more lower than the first patient lesional myelinic index.

[0136] In one embodiment, the at least one patient has white matter demyelinating lesions. In one embodiment, the at least one patient suffers from a myelin pathology as described herein.

[0137] The present invention also relates to the use of a patient lesional myelinic index as described herein as a predictor of disability in patients having white matter demyelinating lesions.

[0138] The present invention also relates to a method for assessing whether a patient having white matter demyelinating lesions is achieving a response with a pro- remyelinating compound, wherein the method comprises: receiving, for said patient, a first second ensemble (34) and a second second ensemble (34), wherein each of the first (34) and second second ensemble (34) comprises an acquired first 3D image (32) and an acquired second 3D image (33), wherein said first 3D image (32) and said second 3D image (33) are 3D magnetic resonance routine images representative of brain anatomical structures; wherein the first second ensemble (34) has been acquired before administration of the pro-remyelinating compound to the patient and the second second ensemble (34) has been acquired after administration of the pro-remyelinating compound to the patient; determining a first patient lesional myelinic index for said first second ensemble (34) using the device (2) as described herein; determining a second patient lesional myelinic index for said second second ensemble (34) using the device (2) as described herein; concluding that the patient is achieving a response with the pro-remyelinating compound if the second patient lesional myelinic index is inferior to the first patient lesional myelinic index.

[0147] It may be concluded that the patient is achieving a response with the pro- remyelinating compound when the second patient lesional myelinic index is about 10% or more lower than the first patient lesional myelinic index, such as about 20, 30, 40, 50, 60, 70, 80, 90% or more lower than the first patient lesional myelinic index.

[0148] It will be understood to the skilled artisan in the art that the method as described herein may enable to monitor whether a patient achieves a response when treated with a pro-remyelinating compound.

[0139] A particular apparatus may embody the device 1 as well as the device 2 described above. It corresponds for example to a workstation, a laptop, a tablet, a smartphone, or a head- mounted display (HMD).

[0140] That apparatus is suited to generation of segmentation mask and to related Machine Learning training. It comprises the following elements, connected to each other by a bus of addresses and data that also transports a clock signal:- a microprocessor (or CPU);- a graphics card comprising several Graphical Processing Units (or GPUs) and a Graphical Random Access Memory (GRAM); the GPUs are quite suited to image processing, due to their highly parallel structure;- a non-volatile memory of ROM type;- a RAM;- one or several I / O (Input / Output) devices such as for example a keyboard, a mouse, a trackball, a webcam; other modes for introduction of commands such as for example vocal recognition are also possible;- a power source; and- a radiofrequency unit.

[0141] According to a variant, the power supply is external to the apparatus.

[0142] The apparatus also comprises a display device of display screen type directly connected to the graphics card to display synthesized images calculated and composed in the graphics card. According to a variant, a display device is external to the apparatus and is connected thereto by a cable or wirelessly for transmitting the display signals. The apparatus, for example through the graphics card, comprises an interface for transmission or connection adapted to transmit a display signal to an external display means such as for example an LCD or plasma screen or a video -projector. In this respect, the RF unit can be used for wireless transmissions.

[0143] It is noted that the word "register" used hereinafter in the description of memories can designate in each of the memories mentioned, a memory zone of low capacity (some binary data) as well as a memory zone of large capacity (enabling a whole program to be stored or all or part of the data representative of data calculated or to be displayed). Also, the registers represented for the RAM and the GRAM can be arranged and constituted in any manner, and each of them does not necessarily correspond to adjacent memory locations and can be distributed otherwise (which covers notably the situation in which one register includes several smaller registers).

[0144] When switched-on, the microprocessor loads and executes the instructions of the program contained in the RAM.

[0145] As will be understood by a skilled person, the presence of the graphics card is not mandatory, and can be replaced with entire CPU processing and / or simpler visualization implementations.

[0146] In variant modes, the apparatus may include only the functionalities of the device 1, and not the learning capacities of the device 2. In addition, the device 1 and / or the device 2 may be implemented differently than a standalone software, and an apparatus or set of apparatus comprising only parts of the apparatus may be exploited through an API call or via a cloud interface.EXAMPLES

[0147] The present invention is further illustrated by the following examples.Example 1Materials and Methods

[0148] For this example, two independent cohorts were used: two independent cohorts. The two datasets were acquired on different scanners and in different countries (France and Brazil). In both cohorts, the patients underwent Tlw and FLAIR MRI and [1 lC]-PiB PET quantified by Distribution Volume Ratios (DVR). The study was approved by the respective Institutional Ethical Committees and patients gave written informed consent.

[0149] The training / validation cohort comprises a total of 17 patients with an active relapsing remitting form of MS among which 15 patients that underwent a total of 30 dynamic [11C]- PiB PET scans performed with a HHRT camera and 3D T1 MRI scans and FLAIR scans performed with a 3 Tesla Siemens MRI scan with MS. MRI images were acquired on a Siemens 3T machine and PET images were acquired on a Siemens High Resolution Research Tomograph (HRRT). The remaining 2 patients underwent only one dynamic [11C]- PiB PET scan, one 3D T1 MRI scan and one FLAIR scan. These 2 patients constitute the validation set.

[0150] The testing cohort comprises 44 patients with MS who underwent [1 lC]-PiB PET scans and 3D T1 MRI scans and FLAIR scans performed on a General Electric PET-MR system.

[0151] MS lesions were manually segmented on MRI by trained raters on T2 sequences. White matter (WM) segmentation and brain mask extraction was done using Freesurfer after lesion filling. Normal appearing white matter (NAWM) was defined as the white matter mask after removing MS lesions. Tlw and FLAIR images were linearly coregistered onto the PET space for each patient using Advanced Normalization Tools (ANTS). All images were cropped to focus on the signal within the brain, and re-sized to a final dimension of 128x128x128 voxels. Each subject was processed in its own PET space. Tlw and FLAIR images were rescaled between [-1, 1] excluding the 5% of extreme values in their native intensity histograms. PiB-DVR values were not altered nor adjusted in any way.

[0152] A Conditional Generative Adversarial Network was trained to generate, as Synthetic 3D images of myelin content, [HC]-PiB DVR maps. A classical 3D U-net generative model was used and a patch-based discriminator. The discriminator uses patches of size 16x16x16. A patch can be either centered on the center of gravity of a randomly chosen lesion (lesion patch) or centered on a random point within the white matter mask that could be lesional or not (white matter patch). To encourage the network to focus on MS lesions, the proportion of patches centered on lesions that the discriminator used during training was progressively increased, using a curricular learning approach. At the first epoch only lesion patches are considered for training, and the proportion of white matter patches linearly increases throughout epochs until itreaches 100% at epoch 150. This approach was compared to an anticurricular training strategy, which is the exact opposite: starting with white matter patches and progressively giving only lesion patches. The generator loss is the sum of the pixel loss (LI) and the adversarial loss (Mean Squared Error (MSE) between the prediction of the discriminator for generated patches and the true label).Results

[0153] A Pearson correlation was computed between synthetic and true PiB images at two levels: lesion level and patient level. At the lesion level, all lesions of all subjects of the testing cohort were pooled (suppressing those with size below 50mm3 which is the resolution of the PET image, resulting in 528 lesions above 50mm3). At the patient level, the mean difference between actual and synthetic PiB DVR signal of all lesions for a given patient, weighted by lesion’s volume was computed.

[0154] At the lesion level, results of the curricular training showed a higher correlation (r = 0.70, p < 10-5) between the mean true PiB DVR and the mean synthetic PiB DVR compared to the anti-curricular training (r = 0.62, p < 10-5). At the patient level, a stronger correlation was also found for curricular training (r = 0.74, p < 10-5) than for anticurricular training (r = 0.61, p < 10-5). An example of result in an individual patient from the testing set is shown on Figure 5, wherein image A corresponds to the Tlw image, image B corresponds to the FLAIR image, image C corresponds to a real PiB DVR image, image D includes the brain mask, white matter mask and lesions mask, image E corresponds to the synthetic PiB DVR obtained using a curricular training, and image F corresponds to the synthetic PiB DVR obtained using an anti-curricular training.

[0155] Therefore, the trained generative model is capable of generating myelin PET maps from clinical T1 and FLAIR MRI images of patients with MS. Moreover, the trained generative model is capable of generating satisfactory results when testing images are captured using a different MRI device than the one used for training.

[0156] Additionally, curricular training improves myelin content synthesis, with a strong correlation between synthetic and true lesional PiB DVR, both at lesional level and at patient level. A way to understand the superiority of curricular training is that lesions display more variability in their signal, so that to fool the discriminator, the generator hasto start by providing credible outputs to the discriminator in the most variable regions. On the contrary, starting from randomly sampled patches in the WM (anti-curricular training) pushes the generator towards learning mainly the general anatomical relationship between Tlw and FLAIR images and PiB DVR at the expense of pathological regions of interest like the demyelinated lesions.

[0157] Figure 7 shows the correlation at lesion level between synthetic and actual PiB DVR in the testing cohort for the curricular training approach and the opposite approach (anti curricular training). For the 528 lesions above 50mm3of the 44 patients with Multiple Sclerosis, the x-axis corresponds to the mean actual PiB DVR of the lesion and in the y-axis corresponds to the mean synthetic PiB DVR obtained with the GAN.

[0158] The curricular approach gave better results than the anti-curricular approach with a better correlation: at the lesion level, results of the curricular training showed a higher correlation (r = 0.70, p < 10-5) between the mean true PiB DVR and the mean synthetic PiB DVR compared to the anti-curricular training (r = 0.62, p < 10-5). At the patient level, we also found a stronger correlation (r = 0.74, p < 10-5) for curricular training than for anti-curricular training (r = 0.61, p < 10-5).

[0159] Example 2Material and methods1. Dataset

[0160] Datasets include patients with multiple sclerosis (pwMS) and healthy controls (HCs), all of whom have at least one clinical MRI acquisition with T1 and FLAIR sequences, as these are the only inputs required by the network. Detailed acquisition parameters are provided in the supplementary materials. Cohorts’ characteristics are provided in Figure 14.1 -a) Training cohorts:

[0161] In total, the network was trained on data from 178 scans, which included {Tl, FLAIR] and at least one myelin- specific modality, from 137 subjects (121 pwMS and 16 HCs).

[0162] For the first phase of training, cohorts with multiple myelin- specific modalities were required to extract common information between those modalities from the input data. In contrast, the second phase of training only needed cohorts with at least one myelin- specific modality to enhance its synthesis. From the available training cohorts containing the MTR sequence, those with similar MTR acquisition parameters were selected to ensure homogeneity in the MTR intensity histograms, facilitating more effective training. For the first phase (see Methods 4-b), the "Shadowtep" and "Onstim" cohorts were used, which include (MTR, Piblogan, PibSRTM2) and (MTR, Tlmap), respectively, as they each provide at least two myelin- specific modalities. For the second phase (see Methods 4-c), the "Flumatep" and "SEPbioprogress" cohorts were used, which contain MTR and Tlmap sequences, respectively, in addition to T1 and FLAIR.1-b) Testing cohorts:

[0163] In total, the network was tested on 422 acquisitions using only {Tl, FLAIR] as input, from 265 subjects (193 pwMS and 72 HCs). Testing the algorithm was done two main objectives:

[0164] 1) Testing myelinic modality reconstruction fidelity in cohorts acquired on different machines than those used for training, to assess the generalizability of this procedure. Lor this, the Brazilian research cohort “MIMS” was used (described below), which includes MTR, PiB Logan, and PiB SRTM2 sequences on a General Electrics PET MRI system, whereas training was performed on Siemens 3T MRI and High Resolution Research Tomograph (HRRT) PET.

[0165] 2) Assessing the clinical relevance of the synthetic cartographies. A Parisian research cohort was used (“Inflasep” cohort, described below), which includes MTR, a two-year follow-up with clinical scores, and also a real-life hospital cohort (“Genova” cohort, described below), which does not include myelinic modality data but does include a two-year follow-up with clinical scores. This second cohort enabled us to evaluate the clinical utility of the algorithm in everyday practice.2) Preprocessing2-a) Common preprocessing characteristics

[0166] All pre-processing procedures are performed in the subject's native T1 space, with rigid registration of all images from the same patient to their T1 using ANTs. For training and inference, images were cropped to a box centered on the brain and then linearly resampled to a size 128x128x128, allowing for size compatibility across different brains.

[0167] Common preprocessing for T1 and. FLAIR:

[0168] T1 and FLAIR images are corrected for the N4 bias field using SimplelTK. They are then robustly scaled on the brain mask using TorchlO, meaning that their intensity histograms are linearly redistributed to [-1, 1], excluding the 5% most extreme voxel intensities.

[0169] Common preprocessing for myelinic modalities:

[0170] Voxel-level quantification of [HC]-PiB PET was performed to extract DVRs using the Logan graphical method, and with the Simplified Reference Tissue Model 2 (SRTM2). These quantification methods employ a supervised clustering algorithm at the voxel level to extract a reference region based on the Time Activity Curve of each voxel. Those quantification procedures provided the PiB Logan and PiB SRTM2 ground truth (GTH), respectively. Magnetization Transfer images were processed resulting in the MTR ground truth. MP2RAGE images were processed allowing to extract a T1 relaxometry, corresponding to Tlmap ground truth.

[0171] As PiB Logan, PiB SRTM2, MTR, and Tlmap ground truth images are quantitative, they were treated as absolute values. Due to differences in modality scales, fixed coefficients were used based on the median signal value in the white matter of healthy controls. This approach was designed to prevent bias toward any specific myelinic sequence during training. After inference, these fixed coefficients were applied to rescale the network's output to directly usable values.2-b) Preprocessing particularities for research databases

[0172] Brain masking was obtained using the “Bet” tool from LSL. MS lesions were manually contoured by trained neurologists using Jim ®. Pollowing a lesion filling procedure, an automatic segmentation of all brain anatomical structures is performed with Preesurfer.2-c) Preprocessing particularities for clinical databases.

[0173] The idea behind this processing pipeline is to provide a fully automatic and quick method to evaluate the real-life applicability of the algorithm, aiming to extract a Lesional Myelinic Index as described in section METHODS-5d for a given patient in less than 10 minutes. Brain masking is performed using an affine registration with ANTs to the MNI152 atlas by using the atlas brain mask. MS lesions are segmented using an in-house 3D UNet, based on FLAIR. Anatomical segmentations are done using FastSurfer. New lesions are segmented based on baseline and follow-up FLAIR, using a deep learningbased method.3) Network

[0174] The architecture of the network, called the MS-SQUID-Net for Synthetic Quantitative mUltimodal Index of Demyelination, is described in Figure 8. It is derived from UNet- Variational Auto-Encoder (VAE), with one encoder and four decoders, resulting in 42,274,104 trainable parameters.

[0175] The rationale behind this architecture is that all decoded sequences — PiB Logan, PiB SRTM2, MTR, and Tlmap — are supposed to be specific to myelin and correspond to different representations of the same myelin content. Having only one encoder allows us to extract common latent characteristics of the image at each stage based on the clinically available inputs: T1 and FLAIR.

[0176] Using a VAE architecture in the bottleneck helps to structure the latent space with reduced parameters, preventing a sequence-specific sub-mapping of the latent space. Moreover, the latent space vector p can be further studied as a global latent myelinic biomarker for a subject. The encoder and decoder blocks, as well as skip connections, and are implemented on Py torch.4) Training procedure4-a) Common characteristics of the training

[0177] The MS-SQUID-Net was trained on an Nvidia T4 GPU with 15.0 GB RAM using Google Colab® Pro+. Training continued until convergence using the Adam optimizer, with a learning rate starting at 10-4 and decreasing at each plateau of validation lossduring training. Data augmentation was provided by the TorchlO library, as described in the supplementary materials.

[0178] The training was split into two parts: the first part involved simultaneous multimodal training focused on training the encoder, while the second part involved refining a given modality decoder branch. This approach allowed us to utilize more subjects for training by leveraging the Flumatep database. In both parts of the training, a curricular approach was employed, applied at the patient level for the first part of the training and at the Region Of Interest (ROI) level for the second part.4-b) First part of the training - multimodal encoder

[0179] In the first part of the training, the aim was to develop the best encoder to extract the global myelinic latent space. For this purpose, only databases that could provide at least two myelinic sequences were used, ensuring that at each iteration, the MS-SQUID- Net was trained to extract characteristics capable of simultaneously recreating multiple myelinic maps. The network was trained based on 71 acquisitions from both HCs and pwMS in the Shadowtep and Onstim databases. Specifically, there were 6 HCs from Shadowtep and 10 from Onstim, with 14 used for training and 2 for validation. For pwMS, there were 32 from Shadowtep and 23 from Onstim, with 50 used for training and 5 for validation. The acquisitions from Shadowtep consistently included {MTR, PiB Logan, PiB SRTM2] and from Onstim included {MTR, Tlmap}. The network was trained to minimize the following loss at each iteration, based on VAE, focusing only on the brain parenchyma by applying the loss on the brain mask only:

[0180] Where KLd is the Kullback-Leibler divergence, !V(pl00,<jl00) the gaussian distribution based on vectors p and c of size 100, latentiter is the latent tensor at a given iteration, and the sum term corresponds to the average of L2 loss between prediction and ground truth for each myelinic sequence.

[0181] To ensure simultaneous learning that benefits from all myelinic sequences at each iteration, the algorithm described in Figure 8 was used with a batch size of 1. This training algorithm ensures that the network has seen all four myelinic sequences tocompute its loss before backpropagating. Dropout regularization was employed, using the TorchlO implementation, with a dropout probability of 0.2 between each fully connected layer.4-c) Second part of the training - sequence specific decoders refinement

[0182] In the second part of the training, it is aimed to enhance the decoders by providing them with additional examples. To achieve this, two extra cohorts for training are utlized: Flumatep for MTR and SEPbioprogress for Tlmap. To leverage these new examples for improving the MTR and Tlmap decoders, while retaining the benefits of the first training phase that required multimodal myelin maps, all weights and biases in the SQUID-Nset’s encoder were frozen, bottleneck, and other decoders. This meant no gradients or optimizations were applied to these components during this phase. For the decoded sequences (MTR and then Tlmap), the network — with all parameters frozen except for the sequence decoder — functions as a generator. This generator was incorporated into the training loop of a conditional Generative Adversarial Network (cGAN).

[0183] To refine the MTR decoder, data from all pwMS in the Shadowtep and Onstim cohorts are used, supplemented with additional data from the Flumatep cohort. This amounted to 117 acquisitions, with 9 used for validation (including 2 HCs and 7 pwMS). For refining the Tlmap decoder, data from all pwMS in the Onstim cohort and added data from the SEPbioprogress cohort are utilized, totaling 94 acquisitions, with 3 used for validation (all pwMS).

[0184] The same process described for MTR decoder refinement was applied to the Tlmap decoder. In this approach, the generator produced a synthetic MTR map (MTRsynth), while a patch-based discriminator attempted to classify the map as synthetic or real, using the (Tl, FEAIR) data as conditioning inputs. Batches of 48 patches of size 16x16x16 for the discriminator and a batch size of 1 for the generator (representing the entire image) were utilized. The loss functions for the generator and discriminator are as follows:_ MSE(D(mtrsynth), labelsfake) + MSE(D(mtrgtl), labelstrue)LoSSpart2dlscriminator-7

[0185] D(MTR) corresponding to the label attributed by the discriminator to a given MTR image based on its (Tl, FLAIR) condition, labelstrue and labelsfake respectively describes the labels attributed to true and fake MTR. To facilitate training, labels were smoothed with a Gaussian filter with a mean of 0.8 for labelstrue and 0.2 for labelsfake, and standard deviation of 0.3 for both labels.

[0186] The discriminator architecture is the following (resulting in 4,653,441 trainable parameters): B(64) B(128) B(256) B(512) L(l)

[0187] with B(n) := {3D Convolution(n), striding = 3 — > Batch Normalization — LeakyReLU } and L(n) := {F latten, Linear(n), Sigmoid}. In the above, n represents the number of filters for convolutive blocks (B Block) or the number of output neurons for Linear blocks (L), J, indicates max pooling.

[0188] In the second part of the training, for MTR as for Tlmap, the conditional Generative Adversarial Network (cGAN) was trained using a curriculum approach at the region-of-interest (ROI) level. The discriminator's patch sampling rule is used to guide the training process, emphasizing specific ROIs. Initially, the discriminator was trained with batches containing patches sampled predominantly from multiple sclerosis (MS) lesions. Over time, the proportion of patches randomly sampled from the white matter is increased, reaching 100% by the 300th epoch.

[0189] Since the generator had already been trained in the first phase, to prevent imbalance during adversarial training — where a strong generator could overpower a naive discriminator — a learning rate of 10A-3 for the discriminator and 10A-4 for the generator were used for a start. The learning rates were reduced for both the discriminator and the generator whenever a plateau in validation loss was observed during training.4-d) Training for comparative architectures

[0190] To compare the performance of the MS-SQUID-Net with other networks for this task, three comparative architectures were selected: monomodal 3D-UNets, monomodal cGANs, and multimodal 3D-UNet. For each comparative architecture, networks were trained on the same number of patients as used for MS-SQUID-Net.

[0191] For monomodal 3D-UNets and monomodal cGANs, the same generator architecture for each myelinic modality was utilized: a 3D-UNet with 1,030,701 trainable parameters :

[0192] B(ri) := 2x {Group Normalization —> 3DConvolution(n) —> ReLU]{B(128)— >{B(64)— >{B(32)TB(16) — >Conv(l)

[0194] where the numbers in the parentheses are the number of filters, { indicates max pooling and { indicates trilinear upsampling. For monomodal cGANs, the same discriminator architecture as described in the METHODS section 4-c) for MS-SQUID- Net is employed. Training was conducted on the same machine as MS-SQUID-Net, using the same optimizer and an adaptive learning rate, as detailed in METHODS section 4-a), starting from 10 -4 for all networks.

[0195] For monomodal 3D-UNets, each myelinic modality was trained during 300 epochs to minimize the LI loss between the rescaled ground truth (with the scaling factor described) and the predicted synthetic image within the brain mask. For monomodal cGANs, the loss functions used were identical to those described for MS-SQUID-Net in METHODS section 4-c), across all myelinic modalities. The same curricular approach for the discriminator is used as described in METHODS section 4-c).5) Post-processing in testing cohorts5-a) Inference

[0196] Each inference is conducted in the patient’s T1 space, to which the patient’s FLAIR and brain mask are rigidly co-registered using ANTs. The T1 and FLAIR images are then cropped on the brain mask with a 2- voxel security margin and linearly resampled to a resolution of 128x128x128. Inside the brain mask, voxel intensities of T1 and FLAIR are robustly scaled using TorchlO to rescale the intensities to the range [-1,1], excluding the 2.5% of minimal and maximal voxels.

[0197] The network's prediction based on this input is then split into four images: predicted MTR, predicted PiB Logan, predicted PiB SRTM2, and predicted Tlmap. Forcomparative monomodal networks (monomodal 3D-Unet and monomodal cGAN), the only difference with the MS-SQUID-Net is that each trained monomodal network, for each myelinic modality, runs its prediction based on T1 and FLAIR separately. These images are linearly resampled from the 128x128x128 brain mask-cropped box back to the native T1 space and saved as NIfTI images using Nibabel.5-b) Atlases constitution

[0198] All ground truth (GTH) images of myelinic modalities were non-linearly registered, as well as prediction images, MS lesions, and anatomical masks, to a common standard space, the MNI 152 using ANTs SynMetric registration with trilinear interpolation. For masks, Nearest Neighbor Interpolation was used. Following spatial normalization, for each voxel in the MNI 152 space, the inter- HC mean and standard deviation were computed for each myelinic modality at that brain location, resulting in 3D atlases in MNI 152 of mean (as described in Figure 9) and standard deviation for each modality.

[0199] This procedure was applied both to the actual myelinic modalities, resulting in actual atlases, and to the synthetic maps created, resulting in synthetic atlases. Aa pooled multimodal myelinic atlas was created by averaging the MTR, Tlmap, Piblogan, and PibSRTM2 atlases for each voxel in the MNI152 space. This averaging was performed after applying the scaling coefficients described in Methods subsection 2a, as shown in Figure 10A.5-c) Z- score computation

[0200] For each pwMS and for each myelinic modality in MNI 152 space, lesion-level z-scores were computed using the following equation:

[0201] For each modality, the signalmap corresponds to the myelin signal for this modality in pwMS, while atlasmean and atlas std represent the mean and standard deviation, respectively, of the myelin signal in HCs atlases computed as explained in section 5-b for this sequence. For actual maps of myelin content, actual atlases were usedto compute z-scores, and for synthetic maps, synthetic atlases were used to compute z- scores (cf. Figure 9).

[0202] Using synthetic atlases to compute z-scores for synthetic images served as a method to normalize the variability in the T1 and FLAIR input across datasets. This ensures that the comparison between pwMS myelin maps and HC atlases is based on the same synthesis from T1 and FLAIR.5-d) Patient Lesional Myelinic Index (LMI)

[0203] For each pwMS, the average lesion z-score at the patient level was calculated by excluding lesions smaller than 50 mm3, due to the resolution limitations of PET images. Then the lesions were weighted by their volume using the following equation:where LMI represents the Lesional Myelinic Index for a given myelinic modality, numiesionSpatientis the total number of lesions above 50 mm3for the patient, mean(signal) lesion and volumeiesion are respectively the mean z-score and the volume of the lesion, and lesion total volumepatient corresponds to the total lesion volume for the patient.

[0204] This LMI reflects the overall level of demyelination in lesions for a given myelinic modality. LMIs were extracted using both actual myelinic sequences (actual LMIs) and synthetic myelinic sequences (synthetic LMIs). For both synthetic and actual data, all LMIs were averaged across the available myelinic modalities to create multimodal LMIs.5-e) Other testing metrics

[0205] To quantify the fidelity of the synthetic maps, the following metrics were calculated for MS-SQUID-Net and comparative networks across both the MIMS and INFLASEP testing cohorts. These metrics assess the differences between synthetic myelinic modality III and the ground truth image K :

[0001] Mean Ab solute Error(MAE) =where Nx, Ny and Nz are the dimension of the 3D images, I(i,j,k) and K(i,j,k) are respectively the voxel intensity value at position i,j,k for synthetic image I and ground truth image KMean Squared Error (MSE) =where Nx, Ny and Nz are the dimension of the 3D images, I(i,j,k) and K(i,j,k) are respectively the voxel intensity value at position i,j,k for synthetic image I and ground truth image K

[0003] Structural Similarity Index(SSIM) = where pl and K are the mean voxel intensities of the patches in the original and distorted 3D images, ol2and oK2are the variances of voxel intensities in the original and distorted 3D images (representing contrast). oIK is the covariance between the original and distorted voxel patches (representing structural similarity). Cl and C2 are stabilization constants. Considering a data range of L = (MAX-MIN) with MAX and MIN respectively maximum and minimum voxel value in the image, here Cl = (0.01*L)2and C2 = (0.03*L)2.Peak Signal to Noise Ratio (PSNR) =where MAX is the maximum voxel value in the image and MSE corresponds to Mean Squared Error.

[0206] These metrics were computed for several ROIs: Brain Mask (BM), MS Lesions (Les), Normal Appearing White Matter (NAWM, i.e., White Matter without MS Lesions), and Cortex (Cx). For each pwMS, the appropriate binary mask was applied to the images and extracted the mean value within each ROI when relevant for the testing metric.6) Gene expression correlations

[0207] To assess whether the spatial distribution of myelin-related gene expression correlates with the synthetic myelinic signal, the HC synthetic myelinic atlas from the Genova database was used (the database with the highest number of healthy controls). This atlas was obtained as described in Methods subsection 5.

[0208] First the T1 MRI scans of 6 healthy donors from Allen brain Atlas Database (HDs) were non-linearly registered to the MNI152 space using ANTs, converting their stereotaxic coordinates from the HD native space to the MNI152 space, which is consistent with the actual and synthetic atlases constituted from the cohorts. For studying myelin-related gene expression, structural and functional myelinic proteins expressed by oligodendrocytes were focused on: Proteolipoprotein 1 (PLP1), Myelin Basic Protein (MBP), Myelin- Associated Glycoprotein (MAG), and Myelin Oligodendrocyte Glycoprotein (MOG), Cyclic nucleotide 3'-phosphodiesterase (CNP) and Myelin associated Oligodendrocyte Basic Protein (MOBP). As a negative control, the serotonin receptor HTR1A gene was included, not linked to myelin expression in human brain, and PDGF receptor alpha (PDGFRA), known to be an oligodendrocyte precursor protein expressed in white matter although not directly linked to myelin. The probes used to assess gene expression are detailed in Supplementary Table 3.

[0209] For each gene of interest, probes with expression levels not significantly different from background noise was removed then the mean expression among the remaining probes was computed.

[0210] Since most of the samples from Allen brain Atlas Database were collected from cortical, central nucleus, and brainstem regions, and not from ROIs typically associated with MS lesions, samples from locations with a non-zero lesion probability were focused on to assess gene expression correlations in MS-relevant areas.

[0211] In the MNI152 common space, a lesion probability map was created by aggregating all MS lesions from all the testing cohorts (see Figure 10B). Then samples from healthy donors (HDs) were excluded if no lesions were found in the corresponding location. In those position with a non-zero lesional probability, gene expression valueswere extracted as previously described and correlated them directly with the synthetic myelinic atlas signal. Both the stereotaxic sampling and the atlas have millimetric resolution in the MNI152 space, facilitating this direct comparison.7) Transversal and longitudinal statistics

[0212] Statistics were run in Scipy. stats library on Python 3. Type 1 error rate was set to 0.05, using Bonferroni correction when several statistical tests were done on the same cohort. Normality was assessed using Shapiro test. When data were normally distributed, univariate correlations were run using Pearson correlation test, whereas using Spearman correlation test when data were not normally distributed. Multivariate analyses were conducted using the Global Linear Model from SciPy's 'stats' module, adhering to the guideline of having no more than a 1:10 ratio of covariates to subjects. When the number of subjects permitted the inclusion of up to 10 covariates, adjustments were made for the following: age, number of lesions at baseline, volumetric lesion load at baseline, longitudinal change in lesion load, thalamic volume at baseline, cortical volume at baseline, longitudinal change in cortical volume, longitudinal change in thalamic volume, and new lesion volume at follow-up. When the number of subjects allowed for only two covariates, adjustment was made for age and lesion load to ensure inclusion of both a clinically relevant predictor and an imaging-derived predictor.

[0213] For transversal statistical studies, the correlation at lesion level was considered by pooling all lesions above 50 mm3in each testing cohort and looking at the correlation for each modality between the mean actual and the mean synthetic signal in MS lesions. At patient level, the multimodal LMI was extracted for each patient and computed the correlation between the multimodal LMI for actual images and for synthetic images.

[0214] For longitudinal results, the correlation between the difference in patient’s multimodal LMI at two years and at baseline, with the evolution of his EDSS were computed. EDSS evolution was computed as step wise EDSS change, a common manner to account for the non-linearity of the scale, in details all EDSS change was doubled when EDSS was superior to 5.5.

[0215] For analyses related to the Allen Brain Atlas, the mean gene expression values were extracted from the probes for the 6 HDs and the corresponding atlas signals at the same MNI152 locations. Then the correlation between each gene's expression and each synthetic atlas signal was examined using a mixed-effects model. This model included a random effect for individual patients and anatomical regions, determined by FastSurfer, as well as a fixed effect for the signal values. The mixed-effects analysis was conducted using StatsModels in Python 3. Prior to analysis, each gene expression value and atlas intensity were standardized by a z-normalization using Scikit- learn. The results, including beta coefficients and p- values, were visualized on a heatmap.Results1) Synthetic and actual myelinic signals are strongly correlated in MS lesions

[0216] Examples of mean atlases for MTR, PiB Logan, PiB SRTM2, and Tlmaps on HCs from the MIMS testing cohort, as well as individual predicted and actual MTR, PiB Logan, PiB SRTM2, and Tlmaps for pwMS, are depicted in Figure 9. By comparing the HC atlas of mean and standard deviation, lesion-wise maps of z-scores are computed for each patient, enabling the extraction of mean values in MS lesions and the computation of Lesional Myelinic Index (LMI) at the patient level.

[0217] In the french Inflasep testing cohort on a 3T Siemens Machine, consisting of 2950 lesions analyzed > 50 mmA3 across 32 patients at the first visit, a strong correlation at the lesion level was observed between synthetic and actual z-score for MTR (r = 0.77 p<10e- 10), and an even stronger correlation at lesion level between actual MTR z-score and multimodal synthetic z-score (r=0.80, p<10e-10). At patient level, after computing LMIs for the 32 patients, a strong and significant correlation was observed between actual LMI (based on MTR which is the only acquired myelinic sequence in this testing cohort), and synthetic MTR LMI (r = 0.85, p=6,2.10e-10), as well as with synthetic multimodal LMI (r = 0.85, p=6,8.10e-10, Figure 10A). Only one patient over 32 was under agreement limits on Bland Altman plot (Figure 10B).

[0218] In the Brazilian MIMS testing cohort, acquired with another machine for PET and MRI to assess the generalizability of the approach (General-Electrics PET MRIsystem), 45 patients were analyzed, with a total of 1553 lesions above 50mm3. For each modality, a strong correlation at patient level was observed between actual and synthetic LMI, and for two over three evaluable myelinic modalities (MTR and piblogan), this correlation was stronger using multimodal synthetic LMI rather than monomodal synthetic LMI. In details, for MTR, on 45 pwMS, a correlation of r = 0,866, p = 1,6 e- 14 between actual and synthetic MTR LMI was observed, and a correlation of r = 0,873, p = 5,76e-15 between actual MTR LMI and multimodal synthetic LMI was observed.

[0219] For Piblogan, correlation between actual and synthetic Piblogan LMI was r = 0.576, p = 2.24 e-5, and increased too while correlating actual Piblogan LMI with multimodal synthetic LMI (r=0,694, p=1.23 e-07). For PibSRTM2 however, correlation between actual and synthetic PibSRTM2 LMI was r= 0.808, p=6.56 e-12, whereas correlation between actual pibSRTM2 LMI and multimodal synthetic LMI was r=0.764, p =9.88e-10. For Tlmap, as this modality was not present in testing set, only synthetic Tlmap LMI with multimodal actual LMI were correlated, giving a correlation of r = - 0.772, p=5.23e-10 (the sign of the correlation is negative as Tlmap has an anti-anatomic signal, Tanner et al, 2012). The correlation between multimodal predicted LMI and multimodal actual LMI was r = 0.869, p= 1.05e-14 (Figure 10C). The Bland-Altlman plot between actual multimodal and synthetic multimodal LMI is shown on Figure 10D.2) In HCs, synthetic multimodal myelin atlas spatially correlates with structural myelinic genes expression

[0220] To assess the correlation between the synthetic myelinic signals and myelin- related gene expression, from the 6 healthy donors (HDs) in the Allen Human Brain Atlas, after selecting samples with non-zero lesion probability based on the lesion probability map from the testing cohorts, 556 MS-relevant samples were kept among the 3369 potential samples (minimum MS-relevant sample per HD = 48, maximum MS-relevant per HD = 170), cf. Figure 10.B.

[0221] For each gene of interest among PLP1, MBP, MAG, MOG, CNP, MOBP, HTR1A, A1BG, gene expression was assessed by averaging the value of gene related probes, when their expression was significantly different from background. Then thisgene expression was correlated with atlas signal value at the same MNI152 location. That correlation was run for each gene of interest with mixed-effects model with HD and anatomical region (as determined by FastSurfer) as random effects, and atlas signal value as fixed effect. The correlation was tested both on synthetic atlases from HCs from Genova database (synthetic MTR, Tlmap, piblogan, andpibSRTM2 atlases and synthetic multimodal atlas), as well as on actual atlases from MIMS database (actual MTR, piblogan, pibSRTM2 and actual multimodal atlas). To do the correlation, a Bonferroni correction was used for multiple comparisons lead to a significance threshold of ( / corrected = 6,94 .10-4.

[0222] Results are shown in heatmap Figure 10.C. A significant correlation was observed for all synthetic and actual atlases with myelinic genes, and no significant correlation with non-myelinic genes HTR1A and PDGFRA except for actual pibSRTM2 in MIMS database with PDGFRA, but with a low beta coefficient (0,156). As expected, all correlations with myelinic genes are positive except for synthetic Tlmap as it has an anti-anatomic signal. Interestingly, two of the monomodal synthetic atlases: MTR and pibSRTM2 has a stronger correlation with myelinic gene expression than actual MTR and actual pibSRTM2 atlas. The multimodal synthetic myelinic atlas as a stronger correlation than the actual multimodal synthetic atlas, and two of the monomodal synthetic atlases (MTR and Tlmap) have a stronger correlation with myelinic gene expression than the actual multimodal atlas.3) In pwMS, the longitudinal evolution of LMIs serves as an independent predictor of disability worsening in both research and clinical cohorts

[0223] First looking at the clinical correlates of the synthetic maps in the Inflasep research testing database, comprising 23 analyzable patients with a 2-year follow-up, a significant negative correlation was observed between disease worsening (indicated by an increase in stepwise Expanded Disability Status Scale (EDSS) change) and actual index remyelination (reflected by an increase in MTR LMI), with r = -0.63 and p = 0.001 in univariate analysis. A significant negative correlation was also observed between disease worsening and longitudinal increase in multimodal synthetic LMI (multimodal synthetic remyelination), r=-0.52, p= 0.008. This correlation between disease worseningand multimodal synthetic remyelination was still significant in multivariate analysis after adjusting on sex and lesion load: P = - 0, 58 p = 0.003, r2=0.386. For monomodal synthetic remyelination, univariate correlation was significant for between synthetic pibSRTM2 assessed remyelination and disease worsening (r=-0.61, p =0.002), didn’t survive multiple comparison correction for MTR (r=-0.51, p=0.013) and for Tlmap (r=0.43, p=0,038, correlation is positive as expected as Tlmap is inversely related to myelin content), and no significant correlation for synthetic piblogan assessed remyelination was observed.

[0224] Then verification that negative correlation between multimodal synthetic remyelination index and disease worsening in a real-life clinical cohort was done. Over 125 pwMS of the Genova database, 109 had at exploitable 3D gradient echo T1 and 3D Flair at both time points and at least one MS lesion above 50mm3at baseline. On those 109 pwMS, a significant negative correlation was observed between EDSS stepwise change and multimodal longitudinal LMI evolution over 2 years in univariate analysis, with a magnitude close to the Inflasep database: r = -0.48 and p = 1.56 e-07. Monomodal synthetic remyelination indices were also significantly correlated to disease worsening but with smaller magnitude (with MTR only: r=-0.45, p=l.le-06, with piblogan only: r=- 0.35, p=2.2e-04, with pibSRTM2 only: r=-0.43, p=3.0e-06, with Tlmap only: r=0.37, p=6.6e-05). (Figure 10F). Similar to the research cohort, an assessment in a multivariate model if that result remained significant after adjustment for clinical and volumetric covariates was done. The negative correlation between multimodal synthetic remyelination index and disease worsening was still significant (P = 0.43, p <10e-06, r2= 0,320), after adjustment on age, sex, number of lesions at baseline, volumetric lesion load at baseline, longitudinal evolution of lesion load, cortical volume at baseline, thalamic volume at baseline, longitudinal evolution of thalamic volume, longitudinal evolution of cortical volume, new lesions at follow-up volume. On multivariate model, thalamic volume at baseline, longitudinal thalamic volume change and longitudinal cortical volume change were also significantly associated with disease worsening, in agreement with MS literature.4) Comparison with other architectures (cf. Figure 15)

[0225] When analyzing the LMIs, which are the metric of interest, MS-SQUID-Net shows higher correlation values between synthetic and actual LM for each myelinic modality in the MIMS cohort compared to the Monomodal 3D U-Net. With the exception of pibSRTM2 with MS-SQUID-Net, all other myelinic modalities across all networks exhibit higher correlation in the MIMS cohort between multimodal synthetic LMI and monomodal actual LMI than between monomodal synthetic and actual LMI. Overall, the correlation between multimodal synthetic and actual LMIs is very close for MS-SQUID- Net and pooled Monomodal 3D-UNet, but slightly better for MS-SQUID-Net.

[0226] Regarding absolute signal values (as opposed to LMIs, which are computed relative to Healthy Controls), the highest fidelity for MS lesions is observed with MTR in the INFLASEP cohort. This is due to the use of a 3T Siemens machine with acquisition parameters closely matching or identical to those used in the training database (with patients scanned on PRISMA and TRIO machines), achieving the highest PSNR and SSIM values of 28 and 85.4%, respectively, for MS lesions with the monomodal 3D- UNet. Conversely, in the MIMS cohort, where MTR histograms differ significantly due to acquisition on a PET-MRI GE machine, PSNR and SSIM drop to a maximum of 10.8 and 71.1%, respectively, for MS lesions with the monomodal 3D-UNet. For myelinic PET, PSNR and SSIM values are 19.8 and 64% with pibSRTM2, and 23.4 and 75.6% with piblogan in MS lesions of the MIMS cohort (with monomodal 3D-UNet).Discussion

[0227] This is the first time that synthetic multimodal mappings have been established to extract demyelination maps and individual remyelination profiles from clinical MRI data using a fully practical and applicable real- world procedure. By testing the approach on diverse datasets, including both research and real-world cohorts with varied disability phenotypes and lesion loads — completely separate from the training data and originating from different machines and countries — the robustness and generalizability of the network was demonstrated. The method synthesizes four myelin-related modalities (MTR, Tlmap, PibLogan, and PibSRTM2) based solely on T1 and FLAIR images, showing strong correlations at patient level compared to ground truth. Synthetic modalities exhibit spatial correlation with myelin-related gene expression and serve asindependent predictors of disability progression. Pooling multimodal synthesis was superior to monomodal methods as no monomodal synthesis has outperformed the multimodal average myelin content map both in producing a more robust synthetic demyelination index in patients (as demonstrated in the MIMS and INFLASEP testing cohorts), achieving better correlations with gene expression in healthy subjects (based on the Allen Human Brain Atlas), and also enhancing disability prediction in real-world settings (as seen in the Genova cohort). Interestingly, compared to monomodal networks, the network demonstrated higher generalizability with respect to LMI in databases acquired from different machines for each myelinic modality. Given that the network was trained with the same number of examples for each myelinic modality as the monomodal networks, it can reasonably be hypothesized that the performance improvement is due to the integration of examples from other modalities. This suggests that the synergistic nature of the network’s synthesis capability is beneficial. In other words, learning to synthesize MTR probably simultaneously enhances the synthesis of myelinic PET, as both modalities share common myelinic information. Remarkably, although the monomodal networks showed lower correlations at the patient level for demyelination indices (LMI) compared to the MS-SQUID-Net, they performed better overall on fidelity metrics when comparing absolute signal values between synthetic and actual myelinic modalities. This suggests that simultaneous synthesis may lead the network to prioritize capturing the essential myelin-related information, rather than precisely reproducing the specific contrast of a given modality.

[0228] Indeed, compared to previous monomodal myelin synthesis algorithms, the architecture of the MS-SQUID-Net offers an important advantage in explainability by extracting and mapping a common latent information that is sufficient to decode various myelin-related modalities. Previous approaches, which focused solely on synthesizing [1 lC]-PiB PET images quantified with the Logan graphical method, raised concerns that the MRI-derived data used for translation might not specifically reflect the necessary myelin information. This concern is analogous to translating a sentence from one language to another (e.g., French to Latin), where a simple word-to-word translation might succeed without fully capturing the syntax and semantics. A true understanding is better demonstrated by the ability to translate the same sentence into multiple, diverselanguages (e.g., Norwegian, Japanese, and Latin), indicating a deeper extraction of syntactic and semantic content from the original. This concept was applied in a current work by designing MS-SQUID-Net to extract latent information from input MRI data and use it to simultaneously reconstruct various myelin maps from the same clinical MRI input. It is hypothesized that by requiring the network to perform these multiple reconstructions, it will be forced to extract genuine myelin-related information. Since the latent information is extracted once and must suffice to reconstruct all different myelin maps, the network focuses on the shared, fundamental characteristics of the input data. This adds a constraint that enhances the explainability of the information extracted by the network. The concept of a shared multimodal latent biomedical information space has of course already been theorized and applied in neuroscience as well as extra-neurological tasks, but has never been implemented in this architectural form or applied to this task. Moreover, in an additional validation approach (Figure 13), the same encoder was able to extract identical latent myelin characteristics from the input MRI, and then decode a new myelin modality (PET with [18F]-Florbetaben). This not only supports the myelinic character of the extracted characteristics but also demonstrates that the approach is not limited to the four myelin modalities used for training. It shows potential for generalization to emerging and more advanced myelin imaging techniques in the future.

[0229] One of the key results is the correlation between the synthetic multimodal remyelination index and disability progression in real-life cohorts, independent of volumetric predictors. To date, the primary radiological prognostic factors currently available on clinical MRI are volume-based, and here it is added for the first time a quantitative signal-based prognostic factor available in clinical routine. The findings demonstrate that the global automatic pipeline for clinical cohorts can easily extract these volumetric factors in a clinical setting but more importantly, it can also include myelin content, which serves as an independent predictor of disability progression. As applied in the Genova cohort, the pipeline is now fully equipped to capture both volumetric and myelinic information from clinical MRIs, making it ready for deployment in therapeutic trials and large-scale clinical databases.Conclusion

[0230] MS-SQUID-Net provides an explainable and robust architecture to create synthetic multimodal maps of myelin content from clinical MRI and identify remyelination profiles among pwMS, allowing tomorrow to identify pro-remyelinating drugs on large clinical databases. Supplementary materials1) Data augmentation implementation

[0231] During training procedure, the following data augmentation procedure was applied, meaning that at each iteration, one of the following transformations will be randomly chosen, and then applied with the associated probability:Supplementary Table 1: data augmentation procedure.2) Probes used for gene expression correlations

[0232] Here are the probes used to assess each gene expression in Results part 2.Supplementary Table 2: probes used for gene expression assessment.3) Gradient of remyelination at distance of ventricles:

[0233] In the Inflasep cohort, all actual voxel-level MTR z-scores were mapped (Figure12A) and multimodal synthetic z-scores (Figure 12B) in MNI 152 space. This allowed us to observe similar patterns of demyelination near the lateral ventricles. For each patient, the voxel-level difference between follow-up and baseline multimodal synthetic z-scores was calculated, resulting in a voxel-level remyelination index (with higher values indicating greater remyelination). These voxel-level indices were compiled in MNI 152 space, as shown in Figure 12C. Notably, based on the synthetic maps, most remyelinated voxels are located further from the lateral ventricles, in contrast to the demyelination map (Figure 12B). This qualitative observation is quantitatively confirmed in Figure 12D, which displays the distribution of the remyelination index across varying distances from the lateral ventricles, clearly indicating a gradient where remyelination increases with distance from the ventricles. These findings, based on synthetic demyelination and remyelination maps, align well with existing literature on the spatial distribution of remyelination.4) Decoding a new modality: [18F]-Florbetaben4a) Methods:

[0234] To verify the hypothesis of extracting a common myelinic latent space, a demonstration that the trained network could reconstruct a new myelinic modality it had never seen before was achieved, using the same latent space. For this purpose, an additional research dataset called “Smart in MS” was utilized, which features a recent myelin- specific modality, the [18F]-Florbetaben radiotracer. To test this hypothesis, a new decoder was trained solely based on the same latent space using data from the HCs in the “Smart in MS” cohort. Then it was tested on pwMS from “Smart in MS” to assess the generalizability of the procedure to a new myelinic modality. This approach ensures that the network had never encountered MS lesions with this [18F]-Florbetaben before testing.

[0235] In details: “Smart in MS”: (Tl, FLAIR) + [18F]-Florbetaben, 9 scans (3 HCs, 6 pwMS);

[0236] Tl and Flair processing is as explained in Methods. To treat [18F]-Florbetaben SUVr, as it doesn’t correspond to an absolute quantification, its intensities were robustly rescaled on brain mask, meaning that [18F]-Florbetaben SUVr intensity histogram within brain masks was linearly redistributed to [-1, 1], excluding the 5% most extreme voxel intensities.

[0237] To evaluate whether the network could decode a new myelinic modality without retraining the latent space, the weights and biases of the encoder and bottleneck were frozen and connected them to a new decoder. This new decoder, with the same architecture as the other decoders, was initialized with random weights and biases, that were learnable. This decoder was trained, connected to the trained and frozen encoder, using data from only the three healthy controls (HCs) in the Smart in MS database.

[0238] The Adam optimizer was used with a batch size of 1 and a high learning rate of 10-3 for 100 epochs, minimizing the LI loss on the brain mask between predicted and actual [18F]- Florbetaben late SUVrs. This training was conducted on a very smallnumber of HCs (n=3) and subsequently tested on pwMS from the Smart in MS database. This approach ensured that neither the decoder nor the encoder had encountered any multiple sclerosis lesions with this new myelinic modality during testing.4b) Results:

[0239] After training the new decoder solely on HCs with the modality [18FJ-Florbetaben, its predictions were tested on 6 pwMS from the Smart in MS database (Figure 13). Examining the correlation between predicted and actual mean [18F]- Florbetaben Standardized Uptake Value ratio (SUVr) in MS lesions, a significant correlation was found for each patient (all p < 0.05). Across pwMS, the mean correlation coefficient (r) was 0.78 with a standard deviation of 0.04.

[0240] Across pwMS, the mean correlation coefficient (r) was 0.78 with a standard deviation of 0.04.

[0241] The trained network can be extended by adding a new decoder for a new myelin modality, which could come from PET, as shown here in the proof of concept with Florbetaben, or from MRI in future work (such as Myelin Water Fraction), while keeping the pre-trained encoder and only training a new decoder corresponding to this new modality on a small number of images.

Claims

CLAIMS1. A device (1) for training a generative model (20) configured to generate at least one synthetic 3D representation of myelin content, using 3D images of at least one portion of a brain of a patient having white matter demyelinating lesions, said device comprising: at least one input configured to receive, for each subject of a plurality of subjects presenting white matter demyelinating lesions, a first ensemble (24) comprising at least three 3D images (21-23, 25, 26) including an acquired first 3D image (21), an acquired second 3D image (22) and an acquired third 3D image (23), wherein said first 3D image (21) and said second 3D image (22) are 3D magnetic resonance routine images representative of brain anatomical structures, and said third 3D image (23) is representative of myelin content; at least one processor configured to: o generate a training dataset by: for each subject of said plurality of subjects:■ co-registering the at least three 3D images (21-23, 25, 26) of each first ensemble (24) in a reference space;■ extracting, in each co-registered 3D image of each first ensemble (24), at least one same volume comprising at least one portion of at least one white matter demyelinating lesion, so as to obtain corresponding patches from each co-registered 3D image of each first ensemble (24), so as to define at least one set of lesions patches;■ extracting, in each co-registered 3D image of each first ensemble (24), at least one same volume comprising at least one portion of white matter so as to obtain corresponding patches from each co-registered 3D image of each first ensemble (24), so as to define at least one set of white matter patches; and■ generating a set of training image pairs including the co-registered first 3D image paired with the co-registered second 3D image for each first ensemble (24);o train a Conditional Generative Adversarial Network (27) to obtain training parameters (30) for said generative model (20), wherein during the training:■ said generative model (20) is fed with said sets of training image pairs from the training dataset so as to provide as an output at least one synthetic 3D image representative of myelin content, and■ at least one discriminator receives as input:• said at least one synthetic 3D image representative of myelin content provided by the generative model (20),• said sets of training image pairs, and• conditional information including said at least one set of lesion patches and / or said at least one set of white matter patches according to a curricular training strategy; at least one output configured to provide said obtained training parameters (30) of said generative model (20) for generation of at least one synthetic 3D representation of myelin content.

2. The device according to claim 1, wherein said first 3D image (21) is a T1 -weighted image or a MP2RAGE sequence image, said second 3D image (22) is a FLAIR image or a T2- weighted image, and said third 3D image (23) is one of the following: a PET image acquired using a myelin specific radiotracer or a Magnetization Transfer Ratio image or a T1 relaxometry image or a Myelin Water Fraction.

3. The device according to claims 1 or 2, wherein said at least one first ensemble (24) further comprises a magnetic resonance T2 sequence 3D image (26), the at least one processor being further configured to: segment white matter demyelinating lesions on each T2 sequence 3D image (26); obtain a segmentation mask of the white matter demyelinating lesions and a segmentation mask of the white matter; wherein the position of said at least one same volume in the reference space, so that it comprises at least one portion of white matter demyelinating lesion or it comprises at least one portion of white matter, is chosen using the segmentationmask for the white matter demyelinating lesions and the white matter mask respectively.

4. The device according to any one of claims 1 to 3, wherein said curricular training strategy comprises: during a first epoch of the training, feeding to the discriminator conditional information comprising exclusively patches selected from the at least one set of lesion patches, and increasing throughout epochs the proportion of patches selected from the at least one set of white matter patches comprised in the conditional information.

5. The device according to claim 4, wherein the proportion of sets of white matter patches comprised in the conditional information linearly increases throughout epochs until it reaches 100% in the final training epoch.

6. The device according to any one of claims 1 to 5, wherein the at least one discriminator is a patch-based convolutional neural network.

7. The device according to any one of claims 1 to 6, wherein: the first ensemble (24) comprises at least one fourth 3D image (25) representative of myelin content, wherein the acquisition parameters and / or the imaging technique used to acquire said at least one fourth 3D image (25) is different from the imaging technique used to acquire said third 3D image (23); the generative model (20) has a machine learning architecture comprising one encoder and at least two decoders including a first decoder and a second decoder, wherein the first decoder is associated to the at least one third 3D image and the second decoder is associated to the at least one fourth 3D image, and the Conditional Generative Adversarial Network (27) comprises at least two discriminators, including a first discriminator and a second discriminator wherein the first discriminator is associated to the first decoder and the second discriminator is associated to the second decoder, wherein during the training the first discriminator uses as conditional information the patches of the setsobtained using the first 3D images (21), the second 3D images (22) and the third 3D images (23), and the second discriminator uses as conditional information the patches of the set obtained from the first 3D images (21), the second 3D images (22) and the fourth 3D images (25).

8. The device according to claim 7, wherein, during training, said encoder and said at least two decoders are jointly trained, so that the training parameters are iteratively updated at the end of a training epoch only when all outputs of said at least two decoders have been obtained.

9. The device according to any one of claims 1 to 8, wherein the generative model (20) is a 3D U-Net.

10. A device (2) for obtaining at least one synthetic 3D image representative of myelin content (61) for a patient using a trained generative model (31) having training parameters (30) obtained from a device according to any of claims 1 to 9, wherein said device (2) comprises: at least one input configured to receive, for said patient, a second ensemble (34) comprising an acquired first 3D image (32) and an acquired second 3D image (33), wherein said first 3D image (32) and said second 3D image (33) are 3D magnetic resonance routine images representative of brain anatomical structures; at least one processor configured to:• co-register the first 3D image (32) and second 3D image (33) into a same reference space;• generate at least one synthetic 3D image (61) representative of myelin content by feeding forward said co-registered first and second 3D images to said trained generative model (31); at least one output configured to provide said at least one synthetic 3D image (61) representative of myelin content.

11. The device (2) according to claim 10, wherein the at least one processor is further configured to calculate a position in a representation space of white matterdemyelinating patients, said position being obtained from a latent space of the generative model feed forward with said co-registered first and second 3D images.

12. The device (2) according to claim 10, wherein the at least one processor is further configured to calculate a patient lesional myelinic index.

13. A computer-implemented method for obtaining at least one synthetic 3D image (61) representative of myelin content for a patient using a trained generative model (31) having training parameters (30) obtained according to any of claims 1 to 9, wherein the method comprises: receiving, for said patient, a second ensemble (34) comprising an acquired first 3D image (32) and an acquired second 3D image (33), wherein said first 3D image (32) and said second 3D image (33) are 3D magnetic resonance routine images representative of brain anatomical structures; co-registering the first 3D image (32) and second 3D image (33) into a same reference space; generating at least one synthetic 3D image (61) representative of myelin content by feeding forward said co-registered first and second 3D images to said trained generative model (31); outputting said at least one synthetic 3D image (61) representative of myelin content.

14. A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of claim 12.

15. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of claim 12.

16. A method for predicting disability progression in a patient having white matter demyelinating lesions, said method comprising: determining a first patient lesional myelinic index for a second ensemble 34 obtained for a first moment and determining a second patient lesional myelinic index for a second ensemble 34 obtained for a second moment in time usingthe device 2 as described herein; wherein the first moment is anterior to the second moment in time; comparing the first and second patient lesional myelinic indexes; predicting that the patient disability worsens if the second patient lesional myelinic index increases with respect to the first patient lesional myelinic index.

17. A method for identifying if a test compound is a pro-remyelinating compound, wherein the method comprises: receiving, for at least one patient, a first second ensemble (34) and a second second ensemble (34), wherein each of the first (34) and second second ensemble (34) comprises an acquired first 3D image (32) and an acquired second 3D image (33), wherein said first 3D image (32) and said second 3D image (33) are 3D magnetic resonance routine images representative of brain anatomical structures; wherein the first second ensemble (34) has been acquired before administration of a test compound to be tested to the at least one patient and the second second ensemble (34) has been acquired after administration of the test compound to be tested to the at least one patient; determining a first patient lesional myelinic index for said first second ensemble (34) using the device (2) according to claim 12; determining a second patient lesional myelinic index for said second second ensemble (34) using the device (2) according to claim 12; and concluding that the test compound is a pro-remyelinating compound if the second patient lesional myelinic index is inferior to the first patient lesional myelinic index.

Images