Methods for the refinement of functional magnetic resonance images

Abstract

A method (100) for processing functional magnetic resonance images of an organ or anatomical part of an individual using a machine learning algorithm via a neural network architecture (1), comprising the steps of acquiring (S101) three-dimensional functional magnetic resonance image scan data (2) providing information about the organ or anatomical part, the information being spatio-temporal data (S101), acquiring optical flow data (3) of the three-dimensional image (S102), simultaneously applying (S103) a machine learning algorithm to the scan data (2) and to the optical flow data (3), and acquiring output information (4) about the organ or anatomical part, the learning algorithm being trained using adversarial learning, in training the learning algorithm desired output variables (16) are set, and the three-dimensional magnetic resonance image scan data (2) is reprocessed based on the desired output variables (16).

Description

[Technical field]

[0001] The present invention refers to a method for processing functional magnetic resonance images of an individual's organs or anatomical parts using machine learning algorithms, in particular using neural network architectures. Furthermore, the present invention relates to an image processing system or data processing device comprising means for carrying out the steps of said method. Furthermore, the present invention relates to a computer program capable of carrying out the steps of said method, and to a computer readable medium comprising a program for implementing said method. [Background technology]

[0002] Magnetic resonance is a radiological technique based on the physical properties of magnetic fields, which allows visualization of the inside of an individual's body without performing surgery or administering ionizing radiation, for example in computed tomography using X-rays. During magnetic resonance, an individual is in a very strong magnetic field that activates the atoms that make up the body and orients them according to the direction of the magnetic field. When the magnetic field is deactivated, the atoms return to their natural orientation, releasing their stored energy and emitting a signal. Thanks to sophisticated systems, it is possible to intercept this signal and convert it into a magnetic resonance image (imaging).

[0003] Magnetic resonance is used in many fields of application, for example in the fields of neurology, neurosurgery, traumatology, oncology, orthopedics, cardiology, gastroenterology, etc. Indeed, in addition to being a relatively safe investigation technique for the patient compared to other imaging methods, the information obtained from the images is of a different nature, making it possible to distinguish tissues based on their biochemical composition. Furthermore, it is possible to obtain images of the torso in three different planes (axial, frontal, sagittal). There are several applications of magnetic resonance imaging, for example diffusion magnetic resonance imaging and functional magnetic resonance imaging (fMRI). The latter is a biomedical imaging technique that consists in the use of magnetic resonance imaging to evaluate the functionality of an organ or system, complementary to structural imaging.

[0004] Recently, functional magnetic resonance has been used to diagnose and classify neurodegenerative diseases such as multiple sclerosis or Alzheimer's disease. For such applications, machine learning algorithms based on the use of neural networks are often used.

[0005] These techniques are advantageous because they can autonomously learn complex operations that allow them to process images and extract the desired information from them, without the developer having to enumerate the details of the operations to be performed. The goal of "autonomously learning" is reached by means of a so-called "training phase". Prior to the training process, the neural network algorithm consists of a sequence of layers that perform different mathematical operations, such as convolution with random parameters, pooling, padding, etc., which means that the values ​​of these parameters (often called "weights") are not adjusted to correctly process the input data to extract the desired information. During the "training phase", the algorithm is given pairs of input data and the corresponding desired output. The desired output is an instance of a variable that the user believes can be extracted from the input data. For example, an algorithm can be trained to recognize the mood of a subject based on a picture containing the subject's facial expression. The parameters or weights are gradually adjusted during the training phase to minimize the error between the desired output and the one actually produced by the algorithm. Different metrics can be used to define the error, the most common being the "mean absolute error" and the "mean squared error". It should be noted that one skilled in the art can define different metrics for measuring error that are appropriate for particular data and applications.

[0006] The training process does not have a predefined end and the user can stop the training process when the error reached is satisfactorily low according to the user's requirements or based on available time and computational resources.

[0007] Once an algorithm has been trained, it can be applied to new input data where the desired output information is not available. The user can expect the errors in the obtained output to be similar to those measured during the training phase. This second type of use of the algorithm is called the "inference phase".

[0008] However, the analysis of magnetic resonance images using neural networks is often complicated by various technical aspects related to the way these images are generated. Indeed, magnetic resonance is currently a non-quantitative technique, i.e., it does not directly measure specific tissue properties, but rather metrics related to these properties. These relationships can be arbitrarily complex and are influenced by a large number of technical variables related to the equipment and acquisition modality. As neural networks autonomously adjust parameters to process fMRI images and extract information from them, several studies have shown that the output of the algorithm often incorrectly depends on these technical variables, even when these variables have no causal relationship with the information to be extracted, i.e., the desired output. This phenomenon of neural networks learning to extract incorrect information also occurs for biological variables, such as the sex and age of the subject, among other variables. In the context of neural networks, these variables, i.e., both technical and biological variables, are generally referred to as confounding factors or variables, and the phenomenon is called the confounding effect.

[0009] Confounding effects occur because the finite nature of the training set allows for unwanted biases that generate spurious associations between confounding variables and the desired output variables that would not be present in a larger, well-sampled data set. For example, if the goal of an algorithm is to distinguish fMRIs belonging to subjects with a particular disease and those without the disease, but in the training set 80% of the subjects with the disease were captured using the same scanner, the algorithm will learn a pattern according to which data from that scanner are predicted to belong to diseased subjects.

[0010] Confounding variables are often already known to those familiar with the type of data to be analyzed, but can also be identified using the so-called confounding index, described in the paper "Measuring the effects of confounders in medical supervised classification problems: the Confounding Index (CI)" by E. Ferrari, A. Retico, and D. Bacciu (Artificial intelligence in medicine 103(2020):101804).

[0011] To this problem there is an additional complication due to the fact that the result of a functional magnetic resonance scan is a 3D image of an individual organ, e.g. the brain. In other words, the scan combines both spatial and temporal information. Extracting information from data with both spatial and temporal dimensions is considerably complicated, since spatial and temporal patterns are usually treated differently. Indeed, it is desirable to find spatial patterns that are invariant under rigid rotations or translations, whereas time clearly has a specific direction. Moreover, the identification of a spatial pattern can also depend strongly on its temporal duration. For example, a pattern relating to a moving object (metaphorically speaking) has a spatial dimension larger than that of the object itself, since the distance traveled must also be considered. [Prior art documents] [Non-patent literature]

[0012] [Non-Patent Document 1] E. Ferrari, A. Retico, and D. Bacciu, “Measuring the effects of confounders in medical supervised classification problems: the Confounding Index (CI)” (Artificial intelligence in medicine 103(2020):101804) Summary of the Invention [Problem to be solved by the invention]

[0013] It is therefore an object of the present invention to provide a method for processing functional magnetic resonance images, which partially or completely solves the problems of the prior art. In particular, it is an object of the present invention to provide a strategy for processing functional magnetic resonance images, with the goal of defining a protocol that allows taking into account the differences in the type of information carried by the spatial and temporal dimensions of the data. It is also an object of the present invention to reduce (i.e. to minimize) the dependence of the learned patterns, and therefore of the actual final output, on a list of variables, for example set by a user. This list of variables set by the user can be defined as "user-defined confounding variables". [Means for solving the problem]

[0014] The above mentioned objects are achieved by a method, by a system, by a computer program and by a computer readable medium as claimed in the appended claims.

[0015] In one aspect of the invention, there is provided a method for processing functional magnetic resonance images of an organ or anatomical region of an individual using a machine learning algorithm via a neural network architecture, the method comprising the steps of acquiring 3D functional magnetic resonance image scan data providing information about the organ or anatomical region, the information being spatio-temporal data, acquiring optical flow data of the 3D image, applying a machine learning algorithm to the scan data and to the optical flow data simultaneously, and acquiring output information about the organ or anatomical region, the learning algorithm being trained using adversarial learning, in training the learning algorithm desired output variables are set, and the 3D magnetic resonance image scan data is reprocessed based on the desired output variables.

[0016] The neural network architecture is based on the simultaneous analysis of 3D images (4D fMRI) and their optical flow (OF), which can either be calculated internally by the algorithm or taken as input. The optical flow is a representation of the image that enhances moving parts. The exact computation of the optical flow of a 3D image is a statically indeterminate problem. Therefore, different mathematical approximations to it exist, such as the Lucas-Canade and Horn-Schunk methods.

[0017] Thanks to the possibility of processing both the video scan data and its optical flow, the learning of spatiotemporal aspects is improved.

[0018] According to one example, information about an organ or anatomical site depends at least in part on a set of confounding variables, and the dependence of the output information on said set of confounding variables is reduced, particularly minimized. In particular, the set of confounding variables can be identified by a user together with the desired output variables.

[0019] Specifically, adversarial learning allows for the reduction, i.e., minimization, of the degree to which learned patterns depend on confounding variables, i.e., user-defined confounding variables.

[0020] In general, adversarial learning can be defined as a method in which a set of two or more models learn together by pursuing competing goals, usually defined in a single data instance. For example, a method for detecting spam emails based on adversarial learning is assembled by two subnetworks: a "generator" that learns to generate spam emails, and a "detector" that learns to distinguish the spam emails output by the generator from real emails. Training the "generator" and "detector" with competing tasks allows to develop a "detector" that is much more robust against spam emails with respect to its counterpart developed without the adversarial learning approach.

[0021] It should be noted that the unique properties of functional magnetic resonance images make the separate use of the two techniques, one after the other, ineffective (i.e., adversarial learning and simultaneous processing of the video and its optical flow).

[0022] If we were to first proceed with the first technique, i.e., simultaneous processing of the fMRI video and its optical flow, and then proceed with the second technique, i.e., the adversarial model, the output data from the first technique would undergo a dimensionality reduction that is not driven by adversarial learning, and thus information that does not depend on confounding variables may have been completely or largely discarded. Thus, the subsequent application of the second technique would lead to a performance much lower than that obtained with the architecture described here. If instead we were to first apply the second technique, i.e., adversarial learning, to the 3D video and then simultaneously process the output data from the second technique and the fMRI optical flow, this approach would be ineffective for two reasons. The output data from the second technique has been processed and no longer contains the same information as the original video. This may make it difficult to identify spatiotemporal patterns, which was the purpose of the first technique, which instead simultaneously processed the video and its optical flow. Furthermore, information contained in the optical flow that has not been processed with adversarial training continues to depend on confounding variables, and therefore the information emerging from the system consisting of the second technique and a succession of the first technique has not been sufficiently processed to minimize dependence on confounding variables, i.e., user-defined confounding variables.

[0023] The advantage of using the two techniques together and discontinuously is that the use of optical flow gives the adversarial component a better chance of identifying and reducing (i.e., minimizing) the dependence of the output on confounding factors, providing an alternative representation of the data. In particular, this representation emphasizes (metaphorically) the motion of the object, making the network better able to learn temporal patterns. In the case of fMRI, for example, the flow of oxygenated and deoxygenated blood in the brain is emphasized. Feeding this new representation to the neural network, obtained through highly nonlinear and complex computations, significantly reduces the network's training time.

[0024] The use of four-dimensional (4D) data, i.e. spatiotemporal data, offers several advantages over alternatives commonly explored in research. Usually, in practice, there are two options: (1) A neural network processes a vector of image features extracted through an engineering algorithm. (2) The 4D image is segmented into multiple parts of dimension-reduced portions that are processed separately.

[0025] In the first case, a portion of information is arbitrarily selected that may be irreparably corrupted by dependence on confounding variables and / or from which it is not possible to further extract the information of interest, and in the second case, it is not possible to discern patterns across several sections.

[0026] In another aspect of the invention there is provided an image processing system or data processing apparatus including means, in particular a processor, for carrying out the steps of the processing methods described herein.

[0027] In a further aspect of the invention, there is provided a computer program product capable of carrying out the steps of the processing methods described herein.

[0028] In a further aspect of the invention there is provided a computer readable medium comprising a computer program for practicing the methods described herein.

[0029] These and other aspects of the present invention will become more apparent upon consideration of the following description of certain preferred embodiments of the invention, as set forth below. [Brief description of the drawings]

[0030] [Figure 1A] 1 is a flowchart of a method according to an example. [Figure 1B] 1 is a flowchart of a method according to an example. [Figure 2A]FIG. 1 illustrates a schematic representation of a network architecture according to an example. [Figure 2B] FIG. 1 illustrates a schematic representation of a network architecture according to an example. [Figure 3A] FIG. 1 illustrates a schematic representation of the phases of the error backpropagation process applied in a network architecture as well as correlation of predicted vectors, according to an example. [Figure 3B] FIG. 1 illustrates a schematic representation of the phases of the error backpropagation process applied in a network architecture as well as correlation of predicted vectors, according to an example. [Figure 4] FIG. 13 shows a schematic representation of an error back-propagation process applied to a network architecture according to another example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] FIG. 1 shows a flow chart describing a functional magnetic resonance imaging method 100. The resonance images preferably refer to organs or anatomical sites of an individual or patient. For example, images can be directed to the brain to study the flow of oxygenated and deoxygenated blood in the brain, which gives an indirect indication of brain activity. Images can be directed to the heart to study how venous and arterial blood flows between valves. Images can also be directed to many other organs, such as the lungs and kidneys, where it is interesting to study not only the shape but also the functionality.

[0032] These and other organs can be studied in humans or animals.

[0033] The method 100 utilizes a machine learning algorithm based on a neural network architecture 1.

[0034] During a first step (or phase) S101 of the method 100, scan data 2 of functional magnetic resonance 3D images are acquired, which provide information about the patient's organs, such as the brain. This information is structured by spatiotemporal data. In step S102, data 3 of the optical flow of the 3D images are acquired. The optical flow is determined from the magnetic resonance images and can be represented by a tensor, describing a dense motion field with a motion vector at each pixel in the sequence of image frames. Then, in step S103, the method 100 includes the simultaneous application of a machine learning algorithm to both the scan data 2 and the optical flow data 3. Advantageously, the method 100 is based on an adversarial learning framework that is used to obtain (in step S104) output information 4 about the individual's organs or anatomical parts, for example, so as to minimize the dependency of this information on user-defined confounding variables. The output information 4 is a variable or element that can be generated from the functional magnetic resonance images, for example, considering both the spatial and temporal dimensions of the data about the patient's organ. The output information 4 can also be a reprocessed version of the original functional magnetic resonance image, where the reprocessing takes into account the spatiotemporal characteristics of the original image. Along with the spatiotemporal characteristics of the original image, the reprocessing can also take into account user-defined confounding variables.

[0035] During the "training phase", the desired output variables 16 are set and pairs assembled by the input data (scan data 2 and optical flow data 3) and their corresponding desired output variables 16 are given to the algorithm, or triplets assembled by the input data (scan data 2 and optical flow data 3), their corresponding desired output variables 16 and their corresponding user-defined confounding variables 17 are given to the algorithm. The desired output variables 16 represent what the user believes can be extracted from the input data, i.e. the scan data 2. For example, the algorithm is trained to recognize a disease based on said scan data 2.

[0036] User-defined confounding variables 17 are variables that affect the scan data 2 and whose effect on the output information 4 is intended by the user to be minimized.

[0037] During the "training phase", the 3D magnetic resonance image scan data 2 is reprocessed based on the desired output variables 16 and on the user-defined confounding variables 17 to minimize the error (e.g., mean absolute error, mean squared error or other metric) between the desired output variables 16 and the valid output information 4 produced by the algorithm, and to minimize the dependence of the valid output information 4 on the user-defined confounding variables 17.

[0038] The reprocessing of the scan data 2 and the optical flow 3 is shown diagrammatically in Fig. 2A, both of which are inserted as inputs (INPUT) into a neural network architecture 1 for the application of machine learning algorithms. The input data is paired with desired output variables 16. The input data can also be paired with user-defined confounding variables 17. Output information 4 is generated at the OUTPUT (OUTPUT).

[0039] By simultaneously processing both the video scan data 2 and the optical flow data 3 of said scan data 2, it is possible to improve the learning of spatiotemporal aspects.

[0040] According to an example, the information about the patient's organs assembled by the spatio-temporal data depends at least in part on a set of user-defined confounding variables 17. In particular, the method is trained to be flexible with respect to these variables using an adversarial learning framework, so that the dependency of the acquired output information 4 on said confounding variables 17 is reduced, in particular minimized. As mentioned above, the confounding variables 17 can be identified (set) by the user together with the desired output variables 16 and are associated with the input data, i.e. the scan data 2. However, the confounding variables 17 can also be identified (set) in advance by the user. During the training phase, the scan data 2 and its optical flow data 3, which depend on the confounding variables 17, are reprocessed based on the desired output variables 16. The (useful) output information 4 is extracted so as to minimize the dependency on the confounding variables 17. In other words, the algorithm is trained to eliminate or strongly reduce the influence of the confounding variables 17 on the output information 4. It should be noted that while the input data depends on the confounding variables 17, the output data 4, thanks to the application of the algorithm, no longer depends (or depends to a lesser extent) on the set of confounding variables 17.

[0041] As described below, the learning algorithm is trained using adversarial learning and during training the desired output variables 16 are set. The method includes a step of reprocessing the magnetic resonance imaging scan data 2 based on the desired output variables 16 so as to make the method robust to confounding variables 17. In other words, the trained method is applied to determine the output information 4 so that the scan data 2 and thus the functional magnetic resonance images can be interpreted correctly, i.e. without being affected by the confounding variables 17.

[0042] If we want to extract functional properties of the brain from each fMRI, e.g., regarding the average, maximum and minimum velocity of blood, and if we assume that the available data were captured by two different machines, one of which presents artifacts in key parts of the brain, then there are literally two possible approaches to analyze the data.

[0043] A possible standard approach to the problem (1) is to use engineering algorithms to calculate retrievable image properties (e.g., average gray level per region, brain size, etc.), and then process these data with neural networks to extract more complex information, such as blood velocity. Another standard approach (2) is to use a network that receives 4D images and is trained to extract the desired velocity properties.

[0044] In both situations (1) and (2), the calculation of these properties is done on the whole brain, and therefore data captured with a machine that exhibits significant dilation artifacts will have very different values ​​that are not comparable to those captured with other machines, and therefore the final extracted properties will depend on the capture machine.

[0045] Instead, by using the method 100 described herein, only a specific set of information is selected from the initial image, thereby minimizing the ability to discern the data with respect to "acquisition machine" confounding variables. Blood velocity properties in the brain are then extracted based on artifact-free, comparable brain portions.

[0046] The output information 4 can be, for example, a categorical variable indicating whether an individual presents a particular pathology based on one or more functional magnetic resonance images. However, the output information 4 can be any element that can be learned from data, with any dimensionality. For example, this information can be a value, a vector, an image, a video, a tensor, or even a graph.

[0047] A confounding variable 17 is a well-known concept in statistics and is a variable that influences the functional magnetic resonance 3D imaging scan data 2. In particular, a confounding variable can be any variable that has an influence on the values ​​of the input data and that the user does not want to influence the output of the network.

[0048] In one example, the confounding variables 17 may include at least technical variables related to the equipment and techniques for capturing functional magnetic resonance images (such as the scanner, capture parameters, capture modality, such as whether the subject has their eyes open or closed during scanning) and biological variables related to the characteristics of the organ or anatomical site analyzed (such as the patient's age or sex).

[0049] According to one example, the acquired scan data 2 refers to unprocessed functional magnetic resonance images. In particular, the acquired scan data 2 refers to functional magnetic resonance images that maintain their original size without distortion.

[0050] In other words, the input functional magnetic resonance images can be raw images, i.e., "raw", i.e., without modification, simply as produced by the scanner, or minimally processed in the sense that minimal processing operations (noise reduction, skull removal, intensity modification, etc.) can be performed, provided that the data remains 4D, i.e., spatiotemporal in nature. It should be noted that in practice, to be able to function properly, the method 100 uses information that includes a time dimension in addition to a spatial dimension. Using raw images reduces the complexity of the method. Furthermore, each preprocessing step introduces new noise and removes information from the original data, thereby increasing the probability that the preprocessed data does not contain any more information to be extracted, or that the margin of error when information can be extracted is greater.

[0051] It should be noted that this method may also be advantageously used for the diagnosis of behavioral, neurodevelopmental or neurodegenerative disorders, such as Alzheimer's disease.

[0052] In fact, the diagnosis of various psychiatric disorders / illnesses is mainly based on the assessment of behavioral characteristics that are difficult to measure, since quantitative biomarkers that characterize them have not yet been found. Problems in diagnosis are accompanied by problems in treatment. In fact, due to the heterogeneity that characterizes these disorders, individuals with psychiatric disorders may require different care and treatment.

[0053] Because these disorders manifest at the behavioral level, complex biomarkers (i.e., those that can vary significantly between individuals) can be defined using artificial intelligence techniques on functional data that directly or indirectly measure brain activity at rest or during task performance. Such data should have both spatial and temporal components, such as functional magnetic resonance imaging (fMRI) and positron emission tomography (PET). The identification of complex biomarkers would also make it possible to stratify populations affected by a particular disease in order to identify groups of individuals that will likely respond similarly to a treatment or treatment.

[0054] Thanks to the advantages provided by the present fMRI image processing method 100, the dependency of the output information 4 on a set of confounding variables 17 (related to the acquisition device and / or to the sex and age of the subject) is reduced, i.e. minimized. Furthermore, it is possible to distinguish the types of information carried by the spatial and temporal dimensions of the data. Thus, in addition to processing fMRI images for more accurate interpretation, the present method 100 can also be applied to the identification of quantitative biomarkers useful for the diagnosis and stratification of psychiatric disorders. For example, thanks to the application of an interpretability algorithm (or explainability) to an algorithm trained according to the present method, it is possible to identify for each subject the brain regions most important for diagnosis.

[0055] 2A and 2B show, in a block system, the insertion of scan data 2 and of optical flow 3 inside a neural network architecture 1 and output information 4 that is independent (or almost independent) of confounding variables 17. In particular, FIG. 2B shows in detail the structure of the neural network architecture 1 shown in FIG. 2A.

[0056] According to one example, applying a machine learning algorithm to the scan data 2 and to the optical flow data 3 simultaneously includes feeding said data into a feature extractor 5 and extracting a reduced size vector 6 (step S105 in FIG. 1B), where the feature extractor 5 includes a multi-channel, in particular a two-channel structure.

[0057] The two-channel structure of the feature extractor 5 serves to input and simultaneously process the scan data 2 and the optical flow data 3. As shown in Fig. 2B, the scan data 2 and the optical flow data 3 are each input into one of the two channels of the feature extractor 5, respectively. The feature extractor 5 (FE) serves to reduce the input information (fMRI+OF) into a reduced size vector 6, which thus contains a compressed representation of the input.

[0058] 1B and 2B, the method 100 further includes applying a machine learning algorithm to the reduced size vector 6 and feeding the reduced size vector 6 into a first processing module 7 to obtain output information 4 about the organ or anatomical part (step S106). The first processing module 7 can be a prediction module or predictor (P) that uses the representation created by the feature extractor 5 to generate a desired output and processes the information appropriately.

[0059] Further, the method 100 includes applying a machine learning algorithm to the reduced size vector 6 and inserting the reduced size vector 6 into a second processing module 8 to obtain a prediction vector 9 for confounding variables 17 associated with organ information or anatomical sites (step S107). The second processing module 8 can be a confounding variable prediction module or confounder predictor (C) that uses the representation created by the feature extractor 5 to predict confounding factors associated with each subject / individual, such as age, image capture modality, etc. The confounding factors can be selected by a user. However, in an alternative form of the method 100, the confounding factors can be automatically selected by a processor or electronic device.

[0060] As shown in FIG. 2B, the scan data 2 may include a set of fMRI images of the individual's brain as a function of time t. These data are fed into the feature extractor 5 as they come out of the magnetic resonance scanning machine, i.e. without the need for any particular pre-processing operations. However, the images must all be of the same size, so spatio-temporal resampling may be necessary. The optical flow data 3 is the scan data 2 on which the optical flow has been calculated. There are several implementations of optical flow calculation, and the method 100 is compatible with all of the calculation methods. The optical flow data 3 has the same dimensions as the scan data 2.

[0061] The method 100 uses deep neural networks (also called deep learning) and is based on a sequence of various mathematical operations called layers that result in increasingly abstract levels of representation of the input. In particular, both of the two channels of the feature extractor 5 include a convolutional layer, a pooling layer and a dropout layer 12. Furthermore, the feature extractor 5 includes a shared module having a dense layer 13. Both the first processing module 7 and the second processing module 8 include a dense layer 13.

[0062] Overall, the neural network is trained in an adversarial manner. In particular, according to one example, the adversarial learning consists in favoring the learning of a first processing module 7 and opposing the learning of a second processing module 8. The weights of the network are updated during the training process through backpropagation of the error using three loss functions (I to III).

[0063] With reference to FIG. 3A, the parameters associated with the feature extractor 5 and the first processing module 7 are optimized to minimize the error of the first processing module 7, i.e. the error between the output information 4 and the desired output variables 16, with respect to the input information, specified by the user. Furthermore, the parameters associated with the second processing module 8 are optimized to minimize the error of the second processing module 8, i.e. the error between the prediction vector 9 and the vector of values ​​of the user-defined confounding variables 17 with respect to the input information. Finally, the parameters associated with the feature extractor 5 are optimized to maximize the error of the second processing module 8. The parameters of the network (number of layers, type of layers, number of neurons in each layer, activation functions, etc.) are determined using a preliminary search (also called grid search). In other words, different configurations are tried and the user usually chooses the one that allows to obtain the highest performance.

[0064] In an example training scheme, the weights are updated through a specific sequence (I through III), although different sequences are possible.

[0065] First (I), the weights of the feature extractor 5 and the first processing module 7 are updated based on a loss function to minimize the error in the output information 4. Then (II), the weights of the second processing module 8 are updated based on a loss function to minimize the error in the prediction vector 9. Finally (III), the weights of the feature extractor 5 are updated based on a loss function to maximize the error in the prediction vector 9. This function is weighted with a hyperparameter that indicates how much we want the network output to depend on the confounding variables 17. Usually, this is a compromise, i.e., the more we want to depend on the confounding variables 17, the larger the error in the final prediction. Note that whenever we refer to minimizing or maximizing the error of a module, we mean minimizing or maximizing the error made in estimating the vector or output information from that module.

[0066] The same error metric or different metrics can be used to update the various modules. For example, a different metric can be used to update the feature extractor 5 than for updating the second processing module 8. Some metrics that can be used are cross entropy for categorical variables, mean squared error or mean absolute error for continuous variables. If the information to be extracted is a vector containing both categorical and continuous variables, the correlation between the predicted vector and the vector containing the desired output can be used.

[0067] In one example, the confounding variables 17 are represented by a vector of confounding variables 10. Referring to Figure 3B, the method 100 further comprises a step of performing a correlation between the vector of confounding variables 10 and the predictor vector 9 (step S108) and measuring a correlation value 11 at the end of the training of the neural network, where the output information 4 about the organ or anatomical site depends on the vector of confounding variables 10 in proportion to said correlation value 11 (step S109).

[0068] The predictor vector 9 can be correlated with a vector 10 of confounding variables in a correlation module 14, and the measurement of the correlation value 11 can take place in a measurement module 15. It is noted that this correlation value 11 is an index of the goodness of applying the machine learning algorithm to the fMRI images.

[0069] If the measurement module 15 uses Pearson's correlation as a metric, the index assessing how dependent the output 4 is on the confounding variable 17 can be said to be between 0 and 1, where 0 indicates no dependence and 1 indicates perfect proportionality. Other indices with different ranges are possible.

[0070] In other words, this value indicates to what extent the output information 4 depends on confounding variables 17. For example, fMRI images of a patient's brain can be used to determine whether (and to what extent) the patient is affected by a particular disease, such as a neurodegenerative disease. If the fMRI images were analyzed without applying the method described here, the information would be affected by confounding variables 17, such as the type of machine used to capture the images. By applying the method 100 instead, more accurate information can be obtained, reducing errors due to the presence of confounding variables 17 (e.g., the type of machine used).

[0071] Figure 4 describes a representation of the adversarial learning process applied to a network architecture according to an alternative approach compared to that of Figure 3A. Different from the configuration of Figure 3A, the reduced size vector 6 can be inserted into multiple second processing modules 8 (three second processing modules 8 are shown in parallel in the figure), each dedicated to output a subset of user-defined confounding variables 17 associated with organ information or anatomical sites.

[0072] The parameters associated with the feature extractor 5 and the first processing module 7 are optimized to minimize the error of the first processing module 7, i.e., the error between the output information 4 and the desired output variables 16, with respect to the input information, specified by the user. Furthermore, the parameters associated with each of the second processing modules 8 are optimized to minimize the error of each second processing module 8, i.e., the error between each prediction vector 9 and the vector of values ​​of each subset of user-defined confounding variables 17 with respect to the input information. In other words, each second processing module 8 is configured to learn one confounding variable or a group of confounding variables, so that the total amount of confounding variables is distributed among multiple second processing modules 8, rather than being learned by a single second processing module 8, as in FIG. 3A. Finally, the parameters associated with the feature extractor 5 are optimized to maximize the error of each second processing module 8.

[0073] Regarding the optimization of the weights, first (I), the weights of the feature extractor 5 and the first processing module 7 are updated based on a loss function so as to minimize the error in the output information 4. Then (II), the weights of each second processing module 8 are updated based on one or more loss functions so as to minimize the error of each second processing module 8. Finally (III), the weights of the feature extractor 5 are updated based on a loss function so as to maximize the error in the different prediction vectors 9. In this alternative representation of the adversarial learning process, it is possible to define a loss for one or more of the second processing modules 8 that prioritizes the maximization of the error over the others. It is also possible to develop each second processing module 8 with a different number of internal parameters. The more internal parameters there are in a second processing module, the more complex the information contained in the prediction vector 9 of that module may be. The measurement of the correlation value as shown in FIG. 3B also applies to the representation of FIG. 4. However, the measurement should be intended considering the correlation step for each of the multiple second processing modules 8.

[0074] The method can be applied to any image with the same dimensions as the image used to train the algorithm, regardless of the organ or anatomical part shown in the image, but if the algorithm is trained on an image showing a different organ or anatomical part, the output information 4 will be less likely to be the desired output information 16.

[0075] Generally, the method (100) can only be successfully applied to images of an organ or anatomical region for which the algorithm was pre-trained, and the selection of the desired output variables 16 and of the confounding variables 17 should be made by the user according to the organ or anatomical region intended to be analyzed, although the method can always be trained on any organ or anatomical region.

[0076] In one example, if one of the user-defined confounding variables 17 is an organ / anatomical region variable, the method (100) may be successfully applied to any organ or anatomical region. For example, it may be possible to train the algorithm to make the extraction of this information independent of the organ shown in the image by identifying a property in the image, such as the average water density shown in the video, and setting the organ variable as one of the user-defined confounding variables 17. In this case, the algorithm may be successfully applied to images of the lungs and kidneys, even if it was trained on images of the brain and heart.

[0077] In another example, the method (100) may be successfully applied to any organ or anatomical part, where the output variables can take on undefined values. For example, it may be possible to train an algorithm to classify the organ or anatomical part shown in the images. The algorithm is trained to output "a" for images showing the heart, "b" for images showing the brain, and "c" (the undefined value) for images showing any other organ or anatomical part.

[0078] In this case, even if the training set contains data representative of only the heart, brain, lungs and kidneys, the method may successfully output a "c" when applied to any other organ.

[0079] Those skilled in the art may make numerous further modifications and variations to the above-mentioned method in order to meet further fortuitous needs, all of which fall within the protection scope of the present invention as defined by the appended claims. [Explanation of symbols]

[0080] 1. Neural Network Architecture 2. Scanning Data 3. Optical flow data 4 Output information 5. Feature Extractor 6 Reduced Size Vector 7 First Processing Module 8 Second Processing Module 9 Predictive Vector 10 Vector of confounding variables 11 Correlation value 12 Dropout Layer 13 Dense Layer 14 Correlation Module 15 Measurement Module 16 Output Variables 17 Confounding variables

Claims

1. A method (100) for processing functional magnetic resonance images of an organ or anatomical part of an individual using machine learning algorithms via a neural network architecture (1), comprising: acquiring (S101) 3D functional magnetic resonance imaging scan data (2) providing information about said organ or anatomical region, said information being spatiotemporal data; A step (S102) of acquiring optical flow data (3) of the 3D image; applying (S103) the machine learning algorithm simultaneously to the scan data (2) and to the optical flow data (3); and obtaining (S104) output information (4) about the organ or anatomical part, wherein the learning algorithm is trained using adversarial learning, and during the training of the learning algorithm desired output variables (16) are set, and the scan data (2) of the 3D magnetic resonance image is re-processed based on the desired output variables (16).

2. 2. The method (100) of claim 1, wherein the information about the organ or anatomical site depends at least in part on a set of confounding variables (17), and the dependence of the output information (4) on the set of confounding variables (17) is reduced, in particular minimized.

3. 3. The method (100) of claim 2, wherein the set of confounding variables (17) together with the desired output variables (16) are identified by a user.

4. The method (100) according to any one of claims 1 to 3, wherein the step of simultaneously applying (S103) the machine learning algorithm to the scan data (2) and the optical flow data (3) comprises the step of feeding the data into a feature extractor (5) and extracting (S105) reduced size vectors (6), the feature extractor (5) comprising a multi-channel, in particular a two-channel structure.

5. applying (S106) the machine learning algorithm to the reduced size vector (6) and feeding the reduced size vector (6) into a first processing module (7) to obtain output information (4) about the organ or anatomical part; The method (100) according to any one of claims 2 to 4, further comprising the step (S107) of applying the machine learning algorithm to the reduced size vector (6) and inserting the reduced size vector (6) into a second processing module (8) to obtain a prediction vector (9) for the confounding variable (17) associated with the information about the organ or anatomical site.

6. The confounding variable (17) is defined by a vector of confounding variables (10), and the method (100) comprises: performing a correlation (S108) between said vector of confounding variables (10) and said predictor vector (9); 6. The method (100) of claim 5, further comprising a step (S109) of measuring a correlation value (11) at the end of the training of the neural network, wherein the output information (4) for the organ or anatomical site depends on the vector (10) of confounding variables in proportion to the correlation value (11).

7. a. the confounding variable (17) is a variable that influences the scanning data (2) of the functional magnetic resonance three-dimensional image; and / or b. The method (100) of any one of claims 1 to 6, wherein the confounding variables (17) include at least technical variables related to the equipment and techniques for acquiring the functional magnetic resonance images and biological variables related to the characteristics of the organ or anatomical site analyzed.

8. a. the acquired scan data (2) refers to raw functional magnetic resonance images; and / or b. The method (100) according to any one of claims 1 to 7, wherein the acquired scan data (2) refers to a functional magnetic resonance image that maintains its original size without distortion.

9. 9. The method (100) according to any one of claims 5 to 8, wherein the adversarial learning consists in favouring the learning of the first processing module (7) and in opposing the learning of the second processing module (8).

10. a. parameters associated with the feature extractor (5) and the first processing module (7) are optimized to minimize the error of the first processing module (7); and / or b. parameters associated with the second processing module (8) are optimized to minimize errors of the second processing module (8); and / or c. The method (100) of any one of claims 1 to 9, wherein parameters associated with the feature extractor (5) are optimized to minimize an error of the first processing module (7) and maximize an error of the second processing module (8).

11. 11. An image processing system or data processing device comprising means, in particular a processor, for implementing the steps of the method (100) according to any one of the claims 1 to 10.

12. A computer program capable of implementing the steps of the method (100) according to any one of claims 1 to 10.

13. A computer readable medium comprising a computer program for implementing the method (100) of any one of claims 1 to 10.

14. 11. Use of the method (100) according to any one of claims 1 to 10 for the diagnosis of a behavioral, neurodevelopmental or neurodegenerative disorder, such as Alzheimer's disease.

Images