Fmri task set utilizing machine learning

Abstract

The invention relates to a medical imaging method comprising: receiving (201) a set of object parameters describing an object; receiving (205) a predicted task from a trained deep neural network, DNN, in response to inputting (203) the set of object parameters to the trained DNN; presenting the task to the object; controlling (207) an MRI system (700) to acquire fMRI data from the object in response to the predicted task being performed by the object during acquisition.

Description

TECHNICAL FIELD

[0001] The present invention relates to a scanning imaging system, in particular to a medical analysis system for obtaining fMRI data using a task setting obtained by machine learning. BACKGROUND

[0002] Magnetic resonance imaging (MRI) scanners rely on a large static magnetic field (B0) to align the nuclear spins of atoms, which is part of the process of producing images in a patient. These images can reflect various quantities or properties of the subject. For example, the hemodynamic response of brain activity causes magnetic and electrical changes in active brain regions. MRI allows visualizing the magnetic changes, for example based on the blood flow or blood oxygen level dependent (BOLD) effect. The latter is often referred to as functional MRI (fMRI). One major problem of fMRI is the selection of the optimal task or task parameters for an individual. SUMMARY

[0003] Various embodiments provide a medical analysis system, method and computer program product as described by the subject matter of the independent claims. Advantageous embodiments are described in the dependent claims.

[0004] Disease classification, treatment selection or disease progression prognosis can have a higher specificity if well-controlled tasks are used in task-based experiments. Furthermore, the signal-to-noise ratio (SNR) in task-based experiments is typically high when average over subjects can be applied. However, it is still difficult to select the correct task or task parameters for an individual. The present subject matter can select or fine-tune subject-specific fMRI tasks or task parameters based on meta-features of subject data to make more precise disease classification, treatment selection or disease progression prognosis feasible and enhanced. For example, a set of subject data describing a subject, such as age and body type, can be input to a trained neural network in order to predict a task to be performed by the subject by the trained neural network. The predicted task can be performed by the subject when fMRI data is acquired.

[0005] According to one aspect, the invention relates to a medical imaging system. The medical imaging system comprises a magnetic resonance imaging, MRI, system configured to acquire functional magnetic resonance imaging, fMRI, data from a subject within an imaging region; a memory storing machine executable instructions; a processor configured to control the medical imaging system, wherein execution of the machine executable instructions causes the processor to: receive a set of predefined subject data describing the subject; receive a predicted task from a trained deep neural network, DNN, in response to inputting the set of subject data to the trained DNN. The predicted task can be a task that triggers brain activity. The MRI system can be controlled to acquire fMRI data from the subject in response to the predicted task being performed by the subject during acquisition.

[0006] The subject data can be defined offline, for example, using a system other than the medical imaging system and / or in a time period prior to (a day or a month prior to) acquiring fMRI data from the subject in response to the subject performing a predicted task during acquisition. Defining the subject data offline can save processing time by expediting the acquisition of fMRI data. This can be particularly advantageous in the case of brain tissue damage during a stroke, where there is only a treatment window of a few hours. For example, a subject database can be provided. At least a portion of the contents of the subject database can be used to train a DNN. For the application phase of the trained DNN, only data for the subject of interest may be required and can be collected from the subject database. The subject database includes metadata such as age, disease, gender, handedness, and body type for different subjects, including the subject of the present embodiment. Receiving a set of subject data can include reading a set of subject data describing the subject from the subject database. The set of subject data can include values ​​for a set of predefined subject parameters. Using the object database to retrieve the subject data offline can be advantageous because it can save processing resources that would otherwise be required to determine the values ​​for the set of subject parameters using multiple data sources rather than a single database. For example, the time required to read data from the database (e.g., as a single entry) can be less than the time required to collect each value in the set of subject parameters from the corresponding one or more sources.

[0007] fMRI data can be collected from a region of interest (imaging region) of a subject. For example, the region of interest can be the subject's brain.

[0008] The present subject matter may be advantageous because it can enable fully automated fMRI data acquisition. The present subject matter can automate data acquisition, thereby reducing the need for operator intervention. This may be particularly advantageous as the amount and complexity of subject data increases and threatens the user's ability to interpret the subject data.

[0009] This topic can simplify and shorten fMRI data acquisition because the task is precisely defined and thus can prevent unnecessary repeated scans with multiple task attempts. This can be advantageous, for example, in cases where brain tissue is damaged by stroke, which only has a treatment window of a few hours.

[0010] Using a DNN can be advantageous because it can model linear as well as complex nonlinear relationships. This can be particularly advantageous when the amount of object data is large.

[0011] For example, the predicted task can be such that it enables identification of brain regions in the nervous system of the subject. Example tasks can include visual cues, tapping the left index finger, squeezing the left toe, moving the tongue, etc. In another example, the predicted task can be a cognitive task. For example, the predicted task can be an executive function. For example, the cognitive task can seek certain aspects of problem solving, planning, organizational skills, selective attention, inhibitory control, and short-term memory. For example n-back, paced auditory serial addition test (PASAT), Boston Naming Test, and Test of Memory Malingering (TOMM).

[0012] According to one embodiment, the trained DNN is a recurrent neural network RNN, the set of subject data comprises a set of fMRI images of the subject in a resting state, wherein the input of the set of subject data comprises inputting the set of fMRI images to the trained DNN. The set of fMRI images can be acquired by the MRI system or another MRI system in the resting state of the subject.

[0013] This can enable learning of individual task settings based on previous resting state settings. Each person can have a different fMRI resting state signature, i.e. fingerprint. This signature can be related to the best task or best task parameter settings for that particular subject. This relationship can be learned by a recurrent artificial neural network. For example, a recommendation system can be used to label subject data, e.g. “people with this resting state signature usually react well to these task parameters”. For example, if a subject has a very weak BOLD response and thus a poor SNR of the auditory regions, a visual task can be chosen if the overall study protocol allows it. This embodiment can be advantageous because resting state data can enable a reference point for the setting of task-based fMRI data.

[0014] Using a RNN can be advantageous because it can be trained to recognize patterns across time. In particular, patterns in fMRI time series are able to capture BOLD signal correlations across successive scans.

[0015] According to one embodiment, the set of subject data comprises values of a set of predefined subject parameters, wherein a parameter of the set of subject parameters is a subject’s age, disease, gender, handedness, or body type. In this case, for example, the DNN can advantageously be a fully connected neural network, wherein information flows along the forward direction, because the set of subject data can have no temporal dependency at least during acquisition of the fMRI data. According to one embodiment, the trained DNN is a convolutional neural network CNN.

[0016] According to one embodiment, the trained DNN is configured to output a predicted task associated with a set parameter of the task. For example, a given type of task can have different set parameters. The DNN can have an output layer that represents different task types and different set parameters for each task type. This embodiment can further improve the accuracy and efficiency of the predicted task, and thus the accuracy and efficiency of the acquired fMRI data. By using a well-defined task according to the present subject matter, efficiency can be improved as repetitive unnecessary scans (e.g., trying different tasks) can be avoided.

[0017] In another example, presenting the predicted task to the subject according to the set parameter, wherein execution of the machine executable instructions causes the processor to: control the MRI system to acquire training fMRI data in a resting state of the subject; predict a value of the set parameter by inputting the training fMRI data to another trained DNN and receiving, in response, a value of the set parameter from the other trained DNN.

[0018] In another example, the other trained DNN can be configured to receive the same input data (the set of subject data) as the trained DNN and provide a value of a set parameter of the predicted task. Thus, the predicted task of the trained DNN can be presented to the subject according to the set parameter predicted by the other trained DNN.

[0019] For example, a plurality of other trained DNNs can be provided, wherein each other trained DNN corresponds to a respective type of task, e.g., one trained DNN for visual tasks, another trained DNN for audio tasks, etc. That is, each other trained DNN is trained to predict a set parameter of a given type of task. To this end, each of the other trained DNNs can be trained using a training set comprising pairs of sets of subject data and associated values of task parameters for that type of task, e.g., an entry of the training set indicates which settings of a visual task are most suitable for a given subject having a set of subject data. The output layer of the other trained DNN can include a node for each set parameter of the task associated with the other trained DNN.

[0020] According to one embodiment, the predicted task has a set parameter, wherein execution of the machine executable instructions causes the processor to determine a value of the set parameter as a predefined value associated with a task type of the predicted task. For example, if the predicted task is a visual stimulus, a value of brightness can be associated with that type of task. According to one embodiment, the set parameter indicates at least one of a visual grating and a brightness and an auditory stimulus and a volume.

[0021] According to an embodiment, the fMRI images comprise 2D or 3D fMRI images. According to the present subject matter, 2D fMRI images can speed up the overall process of fMRI acquisition. 3D fMRI images can improve the accuracy of task definition and thus can improve the accuracy of acquired fMRI data according to the present subject matter.

[0022] According to an embodiment, the execution of the machine executable instructions causes the processor to: receive a training set indicative of a set of subject parameters associated with respective tasks; use the training set to train the DNN, thereby generating a trained DNN. In one example, the training set can be updated using the set of processed subject parameters and the associated processed outcome (i.e. predicted task); and using the updated training set to generate an updated trained DNN to process further subject data. This can implement a self-improving system and can further improve the accuracy of determining tasks according to the present method. The updated trained DNN can also be used for further received sets of subject parameters in order to predict tasks according to the present subject matter.

[0023] In another aspect, the present invention relates to a medical imaging method comprising: receiving a set of subject data descriptive of a subject; receiving a predicted task from a trained deep neural network, DNN, in response to inputting the set of subject data to the trained DNN;

[0024] presenting the task to the subject; controlling an MRI system to acquire fMRI data from the subject in response to the predicted task being performed by the subject during acquisition.

[0025] In another aspect, the present invention relates to a computer program product comprising machine executable instructions for execution by a processor, wherein execution of the machine executable instructions causes the processor to perform at least part of the method according to the preceding embodiments.

[0026] It should be understood that one or more of the preceding embodiments of the present invention can be combined, as long as the combined embodiments are not mutually exclusive. BRIEF DESCRIPTION OF DRAWINGS

[0027] In the following, preferred embodiments of the present invention will be described by way of example only and with reference to the drawings, in which:

[0028] Figure 1 is a schematic illustration of a control system according to the present subject matter.

[0029] Figure 2 is a flowchart of a medical imaging method according to an example of the present subject matter.

[0030] Figure 3 is a flowchart of a medical imaging method according to an example of the present subject matter.

[0031] Figure 4 is a flowchart of a method for training a DNN according to the present invention.

[0032] Figure 5 is a block diagram illustrating a DNN according to an example of the present subject matter.

[0033] Figure 6 A cross-section and functional diagram of an MRI system are shown.

[0034] Reference Signs List

[0035] 100 Medical System

[0036] 101 Scanning Imaging System

[0037] 103 processor

[0038] 107 Memory

[0039] 108 Power Supply

[0040] 109 bus

[0041] 111 Control System

[0042] 121 Software

[0043] 125 Display

[0044] 129 User Interface

[0045] 133 Database

[0046] 201-405 Methods and Steps

[0047] 500 Deep Neural Networks

[0048] 501 Input Layer

[0049] 502 hidden layers

[0050] 503 hidden layers

[0051] 505 Output Layer

[0052] 700 MRI system

[0053] 704 magnet

[0054] 706 Magnet Bore

[0055] 708 Imaging Area

[0056] 710 Magnetic Field Gradient Coil

[0057] 712 Magnetic Field Gradient Coil Power Supply

[0058] 714 radio frequency coil

[0059] 715 RF amplifier

[0060] 718 subject DETAILED DESCRIPTION

[0061] Hereinafter, elements that have the same function or are the same as or similar to the elements having the same number in the drawings are not repeatedly described. If the function is equivalent, elements already discussed will not necessarily be discussed later in the drawings.

[0062] Various structures, systems and devices are schematically depicted in the drawings for purposes of explanation only and so as not to obscure the present disclosure with details that are well known to those skilled in the art. Nevertheless, the attached drawings are included to describe and explain illustrative examples of the disclosed subject matter.

[0063] Figure 1 is a schematic diagram of a medical analysis system (or medical imaging system) 100. The medical analysis system 100 includes a control system 111 configured to connect to a scanning imaging system (or acquisition component) 101. The control system 111 includes a processor 103, a memory 107, each of which is capable of communicating with one or more components of the medical analysis system 100. For example, the components of the control system 111 are coupled to a bidirectional system bus 109.

[0064] It will be understood that the methods described herein are at least partially non-interactive and automated by a computer system. For example, the methods can be further implemented in software 121 (including firmware), hardware, or a combination thereof. In an exemplary embodiment, the methods described herein are implemented in software form as executable programs and run by a special or general digital computer (e.g., a personal computer, a workstation, a minicomputer, or a mainframe computer).

[0065] The processor 103 is a hardware device for executing software, particularly that stored in the memory 107. The processor 103 can be any custom made or commercially available processor, a central processing unit (CPU), an auxiliary processor associated with the control system 111, a semiconductor-based microprocessor (in the form of a microchip or chip set), a microprocessor, or generally a device for executing software instructions. The processor 103 can control the operation of the scanning imaging system 101.

[0066] The memory 107 can include any one or combination of a volatile storage element (e.g., random access memory (RAM, such as DRAM, SRAM, SDRAM, etc.)) and a non-volatile storage element (e.g., ROM, erasable programmable read only memory (EPROM), electronically erasable programmable read only memory (EEPROM), programmable read only memory (PROM)). Note that the memory 107 can have a distributed architecture, where different components are situated remote from one another, but can be accessed by the processor 103. The memory 107 can store instructions or data related to at least one other component of the medical analysis system 100.

[0067] The control system 111 can further include a display device 125 that displays characters and images, etc. on a user interface 129, for example. The display device 125 can be a touch screen display device.

[0068] The medical analysis system 100 can further include a power source 108 for powering the medical analysis system 100. The power source 108 can be a battery or an external power source, such as power supplied by a standard AC outlet, for example.

[0069] The scan imaging system 101 can include at least one of an MRI, CT, and PET-CT imager. The control system 111 and the scan imaging system 101 can or can not be integral parts. In other words, the control system 111 can or can not be external to the scan imaging system 101.

[0070] The scan imaging system 101 includes components that are controllable by the processor 103 to configure the scan imaging system 101 to provide image data to the control system 111. The configuration of the scan imaging system 101 can enable operation of the scan imaging system 101. The operation of the scan imaging system 101 can be automatic, for example. Figure 6 An example of components of the scan imaging system 101 as an MRI system is shown.

[0071] The connection between the control system 111 and the scan imaging system 101 can include a total Ethernet connection, a WAN connection, or an internet connection, for example.

[0072] In one example, the scanning imaging system 101 can be configured to provide output data such as images in response to a specified measurement. The control system 111 can be configured to receive data such as MR image data from the scanning imaging system 101. For example, the processor 103 can be adapted to receive information from the scanning imaging system 101 in a compatible digital form (automatically or upon request) so that the information can be displayed on the display device 125. Such information can include operational parameters, warning notices, and other information related to the use, operation, and function of the scanning imaging system 101.

[0073] The medical analysis system 100 can be configured to communicate with other scanning imaging systems 131 and / or databases 133 via a network 130. The network 130 includes, for example, a wireless local area network (WLAN) connection, a WAN (wide area network) connection, a LAN (local area network) connection, or a combination thereof. The databases 133 can include information related to patients, scanning imaging systems, anatomical structures, scanning geometry configurations, scanning parameters, scans, etc. The databases 133 can include, for example, an electronic medical record (EMR) database including EMRs of patients, a radiology information system database, a medical image database, a PACS, a hospital information system database, and / or other databases including data for planning scanning geometry. The databases 133 can include, for example, an object database with appropriate metadata, such as age, disease, gender, dominant hand, body size, etc., and related tasks (e.g., visual grating, auditory stimulus) and / or task parameters (e.g., brightness, volume).

[0074] The memory 107 can further include an artificial intelligence (AI) component 150. The AI component 150 can or can not be part of the software component 121. The AI component 150 can be configured for training a DNN according to the present subject matter and providing the trained DNN for further use. For example, if the control system 111 is not part of the scanning imaging system 101, the trained DNN can be provided to the scanning imaging system 101 so that it can be used at the scanning imaging system 101 to determine a task for fMRI data by the scanning imaging system 101.

[0075] Figure 2 is a flowchart of a medical imaging method according to an example of the present subject matter. The method can be a task-based functional magnetic resonance imaging (tfMRI) method that enables the study of functional brain networks under task performance of an object. For the sake of illustration, the DNN of the method can be a fully connected neural network.

[0076] In step 201, a set of predefined object data describing an object may be received, for example, by the control system 111. The set of object data may, for example, have values ​​of object parameters indicating at least one of the following: age, disease, gender, handedness, body shape of the object. For example, the object data may be defined offline. For example, an object database may be provided, such as 133. The object database includes metadata such as age, disease, gender, handedness, body shape of the object. The step of receiving the object data set may include reading the object database to access the set of predefined object data describing the object. This may be advantageous because it may save processing resources that would otherwise be needed to determine the values ​​of the collective set of parameters online (on the fly), for example, the time required to read data from the database may be less than, for example, collecting each of the values ​​in the set of parameters from one or more sources.

[0077] The set of object data may be input into a trained DNN in step 203. The DNN may be trained as described in reference Figure 4 The trained DNN can be trained based on a description of the object. For example, the input layer of the trained DNN may include a node for each object parameter, such as a node for an age parameter and a node for a gender parameter. The values ​​of the object parameters can be input to each node of the input layer. Inference on the trained DNN using the input values ​​may result in a prediction task.

[0078] To this end, the trained DNN may be downloaded, for example, from a remote computer system such as 111 where it is stored. The download may be performed Figure 2 The method is performed in a system such as 101. This can enable the method to be run locally, for example, without sending data over a network to use the trained DNN. In another example, a set of object data can be sent over a communication network (such as the Internet) between, for example, the system executing the method to a remote computer system storing the trained DNN, and in response to sending the set of object parameters, the set of object parameters can be input into the trained DNN at the remote computer system. This can enable remote and centralized use of the trained DNN, for example, multiple users can use the trained DNN. This can enable consistent task definition and, therefore, consistent fMRI data acquisition between different users.

[0079] In response to inputting the set of object data into the trained DNN, a predicted task may be received from the trained DNN in step 205. The trained DNN may, for example, have an output layer having nodes associated with each type of task. Each node in the output layer may be associated with a value indicating the probability that the task associated with the node is appropriate for the current topic. For example, the task associated with the maximum value may be the predicted task received in step 205.

[0080] In one example, the predicted task can be associated with a setting parameter (which is also predicted as part of the predicted task), such as volume of audio stimulus or brightness of visual stimulus, etc. In another example, another trained DNN can be used to predict the setting parameter. The other trained DNN can be configured to receive the same input data as the trained DNN (step 203) and predict the setting parameter of the predicted task. This can allow the use of multiple subsequent networks, one network outputting the task itself and a subsequent network specific to the task and outputting a specific task parameter. The input data of the two networks can be the same. This can be particularly advantageous as different tasks can have different parameters or different number of parameters, and therefore it can be beneficial to use a subsequent network.

[0081] The MRI system can be controlled to acquire fMRI data from the subject in step 207 in response to the predicted task being performed by the subject during acquisition. For example, the predicted task for enabling fMRI data acquisition can be presented to the subject. The predicted task can be presented to the subject in accordance with the setting parameter of the predicted task. In one example, the subject can be informed of the predicted task he or she will perform prior to data acquisition for fMRI data acquisition.

[0082] Figure 3 is a flowchart of a medical imaging method according to an example of the present subject matter. The method can enable performing task-based functional magnetic resonance imaging (tfMRI), thereby enabling studying functional brain networks under task performance by a subject. In Figure 3 In this example of the present subject matter, the DNN can be a recurrent neural network. Using an RNN can be advantageous as it can capture BOLD signal correlations in successive scans.

[0083] In step 301, a set of pre-defined subject data describing a subject can be received, e.g., by the control system 111. The set of subject data can comprise, for example, fMRI time series data acquired in a resting state of the subject. For example, the set of subject data can be a set of 2D fMRI images (e.g., each time point in the time series represents one scan and one image).

[0084] The subject data can be defined, for example, offline. For example, the other scanning imaging system 131 can be used to acquire the fMRI time series while the subject is in a resting state and can be configured to transmit the acquired fMRI time series to the control system 111. In another example, the subject data can be determined or generated at a previous point in time prior to performing the present method. This can be advantageous as it can save processing resources that would otherwise be needed by determining the values of the set of parameters online (on the fly).

[0085] The set of fMRI images received in step 301 can be input to a trained RNN. Each image in the set of images can be an image at one time point in the fMRI time series. The RNN is trained to receive as input an fMRI time series and provide or predict a corresponding task. The RNN can be trained, for example, using a training set comprising multiple sets of fMRI images, where each set of images is labelled with a label indicating a task (e.g. and parameters of the set task) suitable for a subject, the set of fMRI images being acquired from the subject. In one example, the RNN can be a long short-term memory (LSTM) network.

[0086] In another example, the set of input images can be 3D images. This can provide 4D data, i.e. 3D images and a time dimension. In this case, a combination of CNN and RNN can be used, for example a temporal convolutional network (TCN), and the set of fMRI images received in step 301 can be input to a trained TCN. The TCN is trained to receive as input an fMRI time series and provide or predict a corresponding task. The particular architecture of the network (TCN or RNN), e.g. the number and type of layers, can depend on the input data to the network.

[0087] The trained RNN (or trained TCN) can be downloaded, for example, from a remote computer system (e.g. 111) in which it is stored. The download can be performed in a system performing the method, e.g. 101. This can enable the method to be run locally, e.g. without the need to send data over a network to use the trained RNN. In another example, the set of subject data can be sent over a communication network (e.g. the internet) between a system performing the method to a remote computer system storing the trained RNN, and in response to sending the set of subject data, the set of subject data can be input to the trained RNN at the remote computer system. This can enable remote and centralised use of the trained RNN, e.g. multiple users can use the trained RNN. This can enable consistent task definition, and therefore consistent fMRI data acquisition between different users. Figure 3

[0088] In response to inputting the set of fMRI images into the trained RNN, the trained RNN can output a predicted task in step 305. For example, the set task parameter values of the predicted task can also be provided by the trained RNN.

[0089] The MRI system, e.g. 101, can be controlled to acquire fMRI data from the subject in step 307 in response to the predicted task being performed by the subject during acquisition.

[0090] ​Figure 4 is a flowchart of a method for training a DNN according to the present application. The DNN can be a fully connected neural network.

[0091] In step 401, a training set can be received, e.g. by the control system 111. The training set is indicative of a set of object data (or object metadata) associated with a respective task, e.g. each object data is labeled with a respective task. The training set can be obtained from one or more sources, for example. The training set can be retrieved by the control system 111 from the database 133, for example.

[0092] The training set can comprise entries, wherein each entry comprises a respective value of a set of metadata associated with a task. For example, a set of object parameters can be provided, e.g. par1, par2 and par3 can be provided. The training set can comprise pairs of values of the three object parameters and a related task definition {(par1_value, par2_value, par3_value); task}, e.g. the task can be provided as a label of the respective values of the set of object parameters. The task can be a visual or an auditory (sound) task, for example.

[0093] The number of the set of parameters can determine the number of nodes of an input layer of the DNN. Each parameter of the set of parameters can be associated with a respective node of the DNN, for example.

[0094] In step 403, the DNN can be trained using the received training set. The DNN can comprise a set of weights, e.g. from the input layer to the first hidden layer, from the first to the second hidden layer, etc. The weights can be initialized by random numbers or values before training of the DNN. The training can be performed to search for optimized parameters (e.g. weights and biases) of the DNN and to minimize a classification error or a residual. The training set is used as input for the feed-forward DNN, for example. This can be calculated by a loss function (cost function) to measure a data loss in the output layer of the DNN. The data loss measures the compatibility between the predicted task and the true label. After obtaining the data loss, the data loss can be minimized by changing the weights and biases of the DNN. This can be performed by backpropagating the loss to each layer and neuron by gradient descent, for example.

[0095] In one example, the training set of step 401 can be continuously augmented with additional data. The training set can be updated, e.g. by adding processed sets of object parameters and associated processing results, as referred to in Figure 2 Additionally or alternatively, the training set can be updated by adding further pairs of object parameters and corresponding tasks. Step 403 can be repeated regularly, e.g. once the training set has been updated. And, in the method of Figure 2 , the retrained DNN can be used instead of the trained DNN, for example.

[0096] Figure 5 is a block diagram illustrating a DNN 500 according to an example of the present subject matter. The DNN 500 can be a fully connected neural network. The DNN 500 may, for example, include an input layer 501 and an output layer 505. The DNN 500 further includes hidden layers 502 and 503. The number of nodes in each hidden layer 502 and 503 may, for example, be user-defined. The number of layers and nodes are for example purposes only, but the number of layers can vary, for example, from 2 to 500. And the layers of the DNN 500 can include different operations, for example, batch normalization, RELU, dropout, convolution, etc.

[0097] The input layer 501 can include a respective node for each parameter in the set of object parameters, for example, if the set of object parameters includes 10 parameters, the input layer can include 10 respective nodes. The output layer may, for example, include a node for each type of task. The output of the DNN 500 can be a score associated with each node of the output layer 505.

[0098] Figure 6 A magnetic resonance imaging system 700 is shown as an example of the medical system 100. The magnetic resonance imaging system 700 includes a magnet 704. The magnet 704 is a superconducting cylindrical type magnet with a bore 706 therein. The use of different types of magnets is also possible; for example, split cylindrical magnets and so-called open magnets or sealed magnets can also be used. A split cylindrical magnet is similar to a standard cylindrical magnet, except that the cryostat has been split into two parts to allow access to the iso-plane of the magnet. Such a magnet can for example be used in conjunction with charged particle beam therapy. An open magnet has two magnet parts, one on top of the other, with a space in between large enough to receive the object 718 to be imaged, the arrangement of the two segments being similar to the arrangement of Helmholtz coils. Inside the cryostat of the cylindrical magnet there is a set of superconducting coils. Within the bore 706 of the cylindrical magnet 704 there is an imaging zone or volume or anatomy 708 in which the magnetic field is strong and uniform enough to perform magnetic resonance imaging.

[0099] Within the bore 706 of the magnet there is also a set of magnetic field gradient coils 710 which are used during acquisition of magnetic resonance data to spatially encode the magnetic spins of a target volume within the imaging volume or examination volume 708 of the magnet 704. The magnetic field gradient coils 710 are connected to a magnetic field gradient coil power supply 712. The magnetic field gradient coils 710 are intended to be representative. Typically, the magnetic field gradient coils 710 incorporate three separate sets of coils for encoding in three orthogonal spatial directions. A magnetic field gradient power supply supplies current to the magnetic field gradient coils. The current supplied to the magnetic field gradient coils 710 is controlled as a function of time and can be ramped or pulsed.

[0100] The MRI system 700 also includes an RF transmit coil 714 at and adjacent to the examination region 708 for the transmission of RF excitation pulses. The RF coil 714 can include, for example, a set of surface coils or other dedicated RF coils. The RF coil 714 can be alternately used for the transmission of RF pulses and for the reception of magnetic resonance signals, e.g., the RF coil 714 can be implemented as an array transmit coil including multiple RF transmit coils. The RF coil 714 is connected to one or more RF amplifiers 715.

[0101] The magnetic field gradient coil power supply 712 and the RF amplifiers 715 are connected to a hardware interface of the control system 111. The memory 107 of the control system 111 can include, for example, control modules. The control modules contain computer executable code that enables the processor 103 to control the operation and functions of the magnetic resonance imaging system 700. The computer executable code also enables the basic operations of the magnetic resonance imaging system 700, e.g., the acquisition of magnetic resonance data.

[0102] As will be appreciated by those skilled in the art, the present application can be embodied as a device, a method or a computer program product. Accordingly, aspects of the present application can take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that can all generally be referred to herein as a "circuit," "module" or "system." Furthermore, aspects of the present application can take the form of a computer program product embodied in one or more computer readable medium(s) having computer executable code embodied thereon.

[0103] Any combination of one or more computer readable medium can be utilized. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A "computer-readable storage medium" as used herein encompasses any tangible storage medium which can store instructions which are executable by a processor of a computing device. The computer-readable storage medium can be referred to as a computer-readable non-transitory storage medium. The computer-readable storage medium can also be referred to as a tangible computer readable medium. In some embodiments, a computer-readable storage medium can also be able to store data which is able to be accessed by the processor of the computing device. Examples of computer- readable storage media include, but are not limited to: a floppy disk, a magnetic hard disk drive, a solid state hard disk, flash memory, a USB thumb drive, Random Access Memory (RAM), Read Only Memory (ROM), an optical disk, a magneto-optical disk, and the register file of the processor. Examples of optical disks include Compact Disks (CDs) and Digital Versatile Disks (DVDs), such as CD-ROM, CD-RW, CD-R, DVD-ROM, DVD-RW, or DVD-R disks. The term computer readable-storage medium also refers to various types of recording media capable of being accessed by the computer device via a network or communication link. For example, a data can be retrieved over a modem, over the internet, or over a local area network. Computer-executable instructions can be embodied in any suitable medium, including, but not limited to, wireless, wired, optical fibers, RF, and so on, or any suitable combination of the foregoing.

[0104] A computer readable signal medium can include a propagated data signal with computer executable instructions embodied therein. Such propagated signals can take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium can be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport program for use by or in connection with an instruction execution system, apparatus, or device.

[0105] "Computer memory" or "memory" is an example of a computer readable storage medium. Computer memory is any memory that a processor can access. "Computer storage" or "storage" is another example of a computer readable storage medium. Computer storage is any non-volatile computer readable storage medium. In some embodiments, computer storage can also be computer memory, or vice versa.

[0106] As used herein, a "processor" encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer processor, or multiple processors or cores. Processing devices can include any processing unit configured or programmed to process application and / or computer-executable instructions. Processing units can include or encompass a single- or multi- core processor, the processor or processing unit of a computer system or other computing device, or a combination thereof. A processor can further be embodied as a combination of processors designed to perform particular tasks.

[0107] Computer-executable instructions can include, for example, instructions configured to be executed by a processor or multiple processors of a computer system. Computer-executable instructions can include programs, subroutines, applications, software objects, modules, components, data structures, or the like. Computer-executable instructions can be stored in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device.

[0108] The computer-executable instructions can be loaded into and executed by a computer, a processor, or a processing unit of a computer system. The computer-executable instructions can be stored in a computer-readable medium, which can be any device or medium that can store or transfer data for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable medium can be, for example, a computer disk, flash memory, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a register. The computer-readable medium can also be a computer network, a hard disk, a floppy disk, a magnetic tape, a USB drive, a CD-ROM, a DVD, or a semiconductor memory such as a programmable logic device (PLD), a programmable array logic (PAL), or a field-programmable gate array (FPGA).

[0109] The computer-executable instructions can be loaded into and executed by a computer, a processor, or a processing unit of a computer system. The computer-executable instructions can be stored in a computer-readable medium, which can be any device or medium that can store or transfer data for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable medium can be, for example, a computer disk, flash memory, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a register. The computer-readable medium can also be a computer network, a hard disk, a floppy disk, a magnetic tape, a USB drive, a CD-ROM, a DVD, or a semiconductor memory such as a programmable logic device (PLD), a programmable array logic (PAL), or a field-programmable gate array (FPGA).

[0110] These computer program instructions can also be stored in a computer- readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions which implement the function / act specified in the flowchart and / or block diagram block or blocks.

[0111] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which run on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0112] A "user interface" as used herein is an interface that allows a user or operator to interact with a computer or computer system. A "user interface" can also be referred to as a "human interface device." A user interface can provide information or data to the operator and / or receive information or data from the operator. A user interface can enable input from an operator to be received by the computer and can provide output from the computer to the user. In other words, the user interface can allow an operator to control or manipulate a computer and the interface can allow the computer to indicate the effects of the operator's control or manipulation. Display of data or information on a display or graphical user interface is an example of providing information to an operator. Receiving data or information from an operator through a keyboard, mouse, trackball, touchpad, pointing stick, graphics tablet, joystick, gamepad, webcam, headset, gearshift, steering wheel, pedals, wired glove, dance pad, remote control, and accelerometer are all examples of user interface components that enable receiving information or data from an operator.

[0113] A "hardware interface" as used herein encompasses an interface that enables a processor of a computer system to interact with or control an external computing device and / or apparatus. A hardware interface can allow the processor to send control signals or instructions to the external computing device and / or apparatus. A hardware interface can also enable the processor to exchange data with the external computing device and / or apparatus. Examples of hardware interfaces include, but are not limited to: a universal serial bus, an IEEE 1394 port, a parallel port, an IEEE 1284 port, a serial port, an RS-232 port, an IEEE-488 port, a Bluetooth connection, a wireless local area network connection, a TCP / IP connection, an Ethernet connection, a control voltage interface, a MIDI interface, an analog input interface, and a digital input interface.

[0114] As used herein, a "display" or "display device" encompasses an output device or user interface suitable for displaying images or data. A display can output visual, audio, and tactile data. Examples of displays include, but are not limited to, computer monitors, television screens, touchscreens, haptic electronic displays, Braille screens, cathode ray tubes (CRTs), storage tubes, bistable displays, e-paper, vector displays, flat panel displays, vacuum fluorescent displays (VF), light-emitting diode (LED) displays, electroluminescent displays (ELD), plasma display panels (PDP), liquid crystal displays (LCD), organic light-emitting diode displays (OLED), projectors, and head-mounted displays.

[0115] While the application has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. The application is not limited to the disclosed embodiments.

[0116] Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed application, from a study of the drawings, the disclosure, and the claims. In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. A single processor or other unit can fulfill the functions of several items recited in the claims. Brackets in claims are not to be construed as limiting the scope of the claims. Particular features set forth in separate claims can be combined and implemented together in a single claim. The reference signs in the claims should not be construed as limiting the scope of the claims.

Claims

1. A medical imaging system (100), comprising: A magnetic resonance imaging (MRI) system (101) configured to acquire functional magnetic resonance imaging (fMRI) data from a subject (718) within an imaging region (708); a memory (107) storing machine-executable instructions; A processor (103) configured to control the medical imaging system (100), wherein execution of the machine-executable instructions causes the processor (103): receiving ( 201 ) a set of predefined object data describing the object, the set of predefined object data comprising values ​​of a set of predefined object parameters, wherein an object parameter in the set of object parameters indicates at least one of age, disease, gender, handedness, or body shape of the subject; A task of receiving (205) a prediction from a trained deep neural network (DNN) in response to inputting (203) the set of object data into the trained DNN; presenting the task to the subject; The MRI system is controlled (207) to acquire fMRI data from the subject in response to the subject performing the predicted task during an acquisition period.

2. The system of claim 1 , wherein the trained DNN is a recurrent neural network (RNN), the set of subject data comprises a set of fMRI images of the subject in a resting state, and wherein The inputting the set of subject data comprises inputting the set of fMRI images to the trained DNN.

3. The system according to claim 1 or 2, wherein the trained DNN is a convolutional neural network (CNN).

4. The system of claim 1 or 2, wherein the trained DNN is configured to output the predicted task associated with setting parameters of the task.

5. The system according to claim 1 or 2, wherein the predicted task is presented to the subject according to setting parameters, wherein Execution of the machine-executable instructions further causes the processor to determine a value of the setting parameter to be a predefined value associated with a task type of the predicted task.

6. The system according to claim 1 or 2, wherein the predicted task is presented to the subject according to setting parameters, wherein Execution of the machine-executable instructions further causes the processor to input the set of object data into another trained DNN and receive setting parameters for the predicted task from the other trained DNN. The system of claim 2 , wherein the fMRI image comprises a 2D or 3D fMRI image.

8. The system according to claim 1 or 2, wherein: Execution of the machine-executable instructions causes the processor to: receiving a training set indicating a set of object data associated with a respective task; The DNN is trained using the training set, thereby generating the trained DNN.

9. The system according to claim 1 or 2, wherein the setting parameter indicates at least one of the following: brightness of the visual stimulus, volume of the auditory stimulus, and duration of the stimulus.

10. A medical imaging method comprising: receiving (201) a collection of object data describing an object; A task of receiving (205) a prediction from a trained deep neural network (DNN) in response to inputting (203) the set of object data into the trained DNN; presenting the task to the subject; An MRI system is controlled (207) to acquire fMRI data from the subject in response to the subject performing the predicted task during an acquisition period.

11. A computer program product comprising machine-executable instructions for execution by a processor, wherein Execution of the machine-executable instructions causes the processor to perform at least part of the method of claim 10 .

Images