Automated configuration of magnetic resonance imaging system

Abstract

A medical system (100, 300) is disclosed herein that includes a memory (110) that stores machine executable instructions (120) and a large language model module (112, 112 '). The large language model module is configured to output a generated pulse sequence setting (126) in response to receiving object metadata (124) as input. The generated pulse sequence settings are configured to control a magnetic resonance imaging system (302) to acquire k-space data (320). The medical system also includes a computing system (104). Execution of the machine executable instructions causes the computing system to receive (200) object metadata and receive (202) the generated pulse sequence settings in response to inputting the object metadata into the large language model module.

Description

Technical Field

[0001] This invention relates to magnetic resonance imaging, and more particularly to the automated configuration of magnetic resonance imaging systems. Background Technology

[0002] As part of the process for generating images of a patient's body, a magnetic resonance imaging (MRI) system or scanner uses a large static magnetic field to align the nuclear spins of atoms. This large static magnetic field is called the B0 field or main magnetic field. By controlling the gradient magnetic field and radio frequency pulses, nuclear spins can be manipulated to generate radio frequency signals, which can be sampled or measured as k-space data. The k-space data can then be reconstructed into an MRI image that images the internal anatomical structures of the object. The timing-related control of the radio frequency signal and radio frequency pulses, as well as the sampling of the k-space data, are described in the pulse sequence settings. Properly configuring an MRI system requires extensive training and experience. Summary of the Invention

[0003] The present invention provides a medical system, a computer program, and a method in the independent claims. Embodiments are given in the dependent claims.

[0004] In one aspect, the present invention provides a medical system including a memory storing machine-executable instructions and a large language model module. The large language model module is configured to output a generated pulse sequence setting in response to receiving object metadata as input. The generated pulse sequence setting is a pulse sequence setting or data that can be converted into a pulse sequence setting. The generated pulse sequence setting is configured to control a magnetic resonance imaging system to acquire k-space data.

[0005] The medical system also includes a computing system. The execution of machine-executable instructions causes the computing system to receive object metadata. Receiving the object metadata may encompass receiving it as input from a user, or receiving it via technical means, such as retrieving it from a storage device or via a network. The execution of the machine-executable instructions also causes the computing system to receive a generated pulse sequence setting in response to inputting the object metadata into the large language model module.

[0006] In another aspect, the present invention provides a method including receiving object metadata. The method further includes receiving a generated pulse sequence setting in response to inputting the object metadata into a large language model module. The large language model module is configured to output the generated pulse sequence setting in response to receiving the object metadata as input. The generated pulse sequence setting is configured to control a magnetic resonance imaging system to acquire k-space data.

[0007] In another aspect, the present invention provides a computer program including machine-executable instructions and a large language model module. The large language model module is configured to output a generated pulse sequence setting in response to receiving object metadata as input. The generated pulse sequence setting is configured to control a magnetic resonance imaging system to acquire k-space data. The execution of the machine-executable instructions causes the computing system to receive the object metadata. The execution of the machine-executable instructions also causes the computing system to receive the generated pulse sequence setting in response to inputting the object metadata into the large language model module. Attached Figure Description

[0008] In the following description, preferred embodiments of the invention will be illustrated by way of example only and with reference to the accompanying drawings, in which:

[0009] Figure 1 An example of a medical system is illustrated.

[0010] Figure 2 The illustration shows the use of Figure 1 A flowchart of the methods used in medical systems.

[0011] Figure 3 This illustration shows another example of a medical system.

[0012] Figure 4 The illustration shows the use of Figure 3 A flowchart of the methods used in medical systems.

[0013] Figure 5 An example of a large language model module is illustrated.

[0014] Figure 6 Another example of a large language model module is illustrated.

[0015] Figure 7 A flowchart illustrating another method is shown. List of reference numerals 100 Medical Systems 102 Computer 104 Computing System 106 Optional hardware interfaces 108 optional user interfaces 110 Memory 120 Machine-executable instructions 122 Large Language Model Modules 122' Large Language Model Module 124 Object Metadata 126. Generated Pulse Sequence Settings 200 Receive object metadata 202 Receive the generated pulse sequence setting in response to inputting object metadata into the large language model module 300 Medical System 302 Magnetic Resonance Imaging System 304 magnet 306 Magnet Chamber 308 Imaging Area 309 Field of view 310 Magnetic Gradient Coil 312 Magnetic Gradient Coil Power Supply 314 RF coil 316 transceiver 318 Objects 320 Object Support 320 k spatial data 322 Magnetic Resonance Imaging Images 400 Acquiring k-space data by using the generated pulse sequence to control the magnetic resonance imaging system. 402 Reconstructing magnetic resonance images from k-space data 500 The largest language model 502 Decision Module 504 The second largest language model 506 Collection of Magnetic Resonance Imaging Protocols 508 Selected Magnetic Resonance Imaging Protocol 600 single large language models 700 Patient Data (Object Metadata) 702 Human MRI Operator Selection of Selected Magnetic Resonance Imaging Protocol 704 Human MRI operators accept or modify the generated pulse sequence settings proposed by the second largest language model. 706. An MRI scan was performed. 708 Radiologists can optionally rate the diagnostic value of the acquired images. 710 Collect ratings and use them for RLHF 712 Use automatic correction (possibly locally) based on the recommended protocol and sequence settings for RLHF to fine-tune the model. Detailed Implementation

[0016] In these figures, similarly numbered elements are equivalent elements or perform the same function. If the functions are equivalent, elements that have been discussed earlier will not necessarily be discussed in later figures.

[0017] An example can be beneficial because it can provide a method for automatically generating pulse sequence settings to control a magnetic resonance imaging (MRI) system. For instance, this allows the MRI system to be used without the presence of a technician.

[0018] Large Language Models (LLMs), such as the architecture behind chatGPT, have demonstrated powerful capabilities in handling flexible natural language queries across various scenarios while recalling detailed information; they are trained using publicly available knowledge from the internet. While generality is an advantage of LLMs, they can be fine-tuned for specific domains and configured to output specific text formats and structured information, rather than free text. For example, LLMs can be used to automate MRI protocol planning with single-state or two-stage configurations. The two-stage configuration is described below.

[0019] A dedicated LLM "agent" (the first major language model) receives object metadata. This object metadata may include patient data (which could include age, weight, etc.) and the reason for the MRI (symptoms, referral physician speculation, ICD-10 code), and recommends appropriate MRI sequences grouped into reasonable protocols (a set of MRI protocols). Once a protocol is selected by clinicians or the decision module, a second LLM agent (the second major language model) pre-configures each sequence based on the patient's age, weight, possible implants, or other relevant information (object metadata), using patient-appropriate settings (generated pulse sequence settings). This can be used for reinforcement learning using human feedback when staff make changes to the recommended protocols or sequence settings, continuously improving recommendations or fine-tuning the model based on site-specific preferences. This software solution can be seamlessly integrated into the user interface of MRI scanners and can be offered as a cloud service or utilizing local model storage devices. This enables remote operation and autonomous operation of MRI systems, directly assisting clinical workflows and reducing staff workload.

[0020] The large language model module used in this paper encompasses software components or executable code that include at least one large language model.

[0021] The Large Language Models (LLMs) used in this paper encompass neural network architectures, typically consisting of Transformers (encoders and decoders) with self-attention layers and residual connections—language models trained on unlabeled text using self-supervised or semi-supervised learning. Typically, LLMs are trained on billions of words. For example, an LLM can be trained in an autoregressive form, where, given a text, the model predicts the next word (token) or multiple words (tokens). Another training mode involves sentences with missing words or tokens, and the LLM predicts the missing words or tokens. Both types of LLMs can be configured for use in so-called prompting scenarios, where a text query or statement is input into the LLM, and the LLM outputs completed sentences or statements. The LLMs described in this paper are configured to run in prompting scenarios. Example LLMs include GPT-3, GPT-4, BERT, LLaMA, etc. LLMs can be trained for specific tasks using reinforcement learning or through reinforcement learning based on human feedback (RLHF). The output of an existing LLM can be fine-tuned. In fine-tuning, a new set of weights can be trained using specific data to connect the last layer of the language model. A common practice is to freeze all weights in the neural network except for the final output layer, so that they only affect the final output and format.

[0022] The pulse sequence settings used in this paper encompass commands, or data that can be converted into such commands, for controlling a magnetic resonance imaging system to acquire k-space data.

[0023] In another example, the execution of machine-executable instructions also enables the computing system to acquire k-space data by using the generated pulse sequence to control the magnetic resonance imaging system. The execution of the machine-executable instructions further enables the computing system to reconstruct a magnetic resonance image based on the k-space data.

[0024] In another example, the medical system also includes the magnetic resonance imaging system.

[0025] In another example, the large language model module includes a first large language model and a second large language model. The first large language model is configured to output a set of magnetic resonance imaging protocols in response to receiving the object metadata as input. The second large language model is configured to output the generated pulse sequence settings in response to receiving a selected magnetic resonance imaging protocol as input. The generated pulse sequence settings are configured to control the magnetic resonance imaging system to acquire k-space data according to the selected magnetic resonance imaging protocol. The execution of machine-executable instructions also causes the computing system to receive the set of magnetic resonance imaging protocols in response to inputting the object metadata into the first large language model.

[0026] The execution of machine-executable instructions also enables the computing system to provide the set of magnetic resonance imaging protocols using the medical system. This may take different forms in different examples. In some cases, the set of magnetic resonance imaging protocols may be provided to other software or artificial intelligence components. In other examples, providing the set of magnetic resonance imaging protocols may encompass making these protocols available on a display or user interface for selection.

[0027] For example, models such as GPT-2, GPT-3, BERT, or LLaMA can be used to implement the first major language model. The first major language model can be further trained through fine-tuning as described above to provide the set of magnetic resonance imaging protocols. In response to the training object metadata, the output can be trained to output a set of text strings, each text string describing a magnetic resonance imaging protocol. For example, the set of text strings can be limited to a dictionary or a finite set of text strings that can be combined together.

[0028] For example, models such as GPT-2, GPT-3, BERT, or LLaMA can be used to implement a second major language model. The second major language model can be further trained through fine-tuning as described above to provide pulse sequence settings generated in response to receiving the selected MRI protocol (i.e., a selected set of MRI sequences). In some examples, the second major language model also receives raw object metadata. During fine-tuning, the output can be restricted to be displayed in an appropriate form as a timing diagram (which can be easily converted into commands for controlling the MRI system) or directly as commands for controlling the MRI system. For the fine-tuning process, the training data includes experimental MRI protocols (input during training) and baseline ground truth pulse sequence settings. For example, training can be performed using historical pairs of experimental MRI protocols and baseline ground truth pulse sequence settings.

[0029] The execution of machine-executable instructions also causes the computing system to receive a selected magnetic resonance imaging protocol in response to providing the set of magnetic resonance imaging protocols. The set of magnetic resonance imaging protocols includes the selected magnetic resonance imaging protocol. As described above, there are multiple ways to select a magnetic resonance imaging protocol.

[0030] The execution of machine-executable instructions also causes the computational system to receive the generated pulse sequence settings in response to inputting a selected MRI protocol into a second large language model. This example can be beneficial because it may provide a more transparent way to generate pulse sequence settings using a large language model. Breaking the process down into several steps makes it more controllable and less likely to produce erroneous results.

[0031] In another example, the large language model module also includes a decision module. The decision module is configured to output a selected magnetic resonance imaging (MRI) protocol in response to receiving a set of MRI protocols as input. The set of MRI protocols is provided by inputting it into the decision module. The selected MRI protocol is received as the output of the decision module in response to receiving the set of MRI protocols. In some cases, the decision module may also receive object metadata as input, and both can be used to determine the selected MRI protocol.

[0032] In some examples, the decision module can compare the set of magnetic resonance imaging protocols with a list or set of hardware constraints. For example, the decision module may incorporate hardware limitations, i.e., which MR hardware is available in the field. A particular type of magnetic resonance hardware may or may not be able to perform a particular magnetic resonance imaging protocol.

[0033] In some examples, the medical system may also include sensors for providing sensor data. These sensors may include devices such as microphones, cameras, and / or 3D cameras. Sensor data can provide information such as the patient's geometry or location. The patient's geometry or location may help in selecting an appropriate magnetic resonance imaging protocol. Such sensor data may also be provided to a second language model to provide geometric settings in the generated pulse sequence.

[0034] In another example, the decision module is implemented as a neural network. The decision module can be trained using training data, which includes a training set of magnetic resonance imaging (MRI) protocols and a benchmark ground truth MRI protocol selected from each MRI protocol training set. Deep learning, for example, can be used.

[0035] In another example, the decision module is implemented as a recurrent neural network. The decision module can be trained using training data, which includes a training set of magnetic resonance imaging protocols and a benchmark ground truth magnetic resonance imaging protocol selected from each magnetic resonance imaging protocol training set.

[0036] In another example, the decision module is implemented as a convolutional neural network. The decision module can be trained using training data for the magnetic resonance imaging protocol, which includes a training set and benchmark ground truth values.

[0037] In another example, the decision module is implemented as a decision tree. The decision module can be configured or trained using a previous set of magnetic resonance imaging protocols and a selected magnetic resonance imaging protocol from each of these previous sets.

[0038] In another example, the decision-making module is implemented as an expert system. The expert system can be configured by adding logic that can be used to select the chosen magnetic resonance imaging protocol.

[0039] In another example, the decision module is implemented as a graph database. The graph database may include nodes and links between nodes. The nodes and links can be used to encode the set of magnetic resonance imaging protocols, where object metadata and possibly other sensor data can be used to select a chosen magnetic resonance imaging protocol.

[0040] The aforementioned artificial intelligence-related methods can be trained by collecting the magnetic resonance imaging protocols selected by physicians when specifying or selecting specific pulse sequence settings and using supervised learning.

[0041] In another example, providing the set of magnetic resonance imaging protocols includes displaying them on a user interface. The selected magnetic resonance imaging protocol is received via a protocol selection process through the user interface.

[0042] In another example, the execution of machine-executable instructions also enables the computing system to use reinforcement learning to train the first large language model using the selected magnetic resonance imaging protocol. This selection can be used to train the first large language module when the decision module or a user chooses the selected magnetic resonance imaging protocol via a user interface.

[0043] In another example, the members of the set of magnetic resonance imaging protocols are described in text form, or described as sentences or phrases.

[0044] In another example, the members of the set of pulse sequence protocols are constructed from a library of pulse sequence components. The library of pulse sequence components can cover complete pulse sequences or portions of pulse sequences that can be combined. This example can be beneficial because it limits how the set of magnetic resonance imaging protocols can be constructed. For example, this can prevent the generation of incorrect magnetic resonance imaging protocols.

[0045] In another example, the execution of machine-executable instructions also enables the computational system to use supervised learning and / or reinforcement learning to train a first-large language model using radiology case reports. Using radiology case reports can be beneficial because these reports may not be routinely available, and large language models trained using the internet have no opportunity to be trained using radiology case reports. Radiology case reports may also indicate preferences for specific locations and / or physicians. Training a first-large language model using these radiology case reports could be advantageous for customization for specific physicians or locations.

[0046] In another example, the large language model module comprises a single large language model. In this example, there is only one large language model, and it is capable of outputting the generated pulse sequence settings in response to receiving object metadata.

[0047] For example, a single large language model can be implemented using models such as GPT-2, GPT-3, BERT, or LLaMA. The large language model can be further trained through fine-tuning as described above to provide the set of magnetic resonance imaging protocols. In response to the training object metadata, the LLM can be trained to directly output the generated pulse sequence settings. During fine-tuning, the output can be restricted to be displayed in an appropriate form as a time series diagram (which can be easily converted into commands for controlling the magnetic resonance imaging system) or directly as commands for controlling the magnetic resonance imaging system. For the fine-tuning process, the training data includes test object metadata and baseline ground truth pulse sequence settings. For example, training can be performed using a history of test object metadata and baseline ground truth pulse sequence settings.

[0048] In another example, the execution of the machine-executable instructions also causes the computing system to display at least a portion of the generated pulse sequence settings on a user interface. The execution of the machine-executable instructions also causes the computing system to receive a modification command from the user interface in response to displaying at least a portion of the generated pulse sequence settings on the user interface. The execution of the machine-executable instructions further causes the computing system to use the modification command to change the generated pulse sequence settings. This example can be beneficial because it may provide a method for editing the generated pulse sequence settings. This modification process may also help in further training large language model modules. If there is only a single large language model, then the data can be used to train that large language model. If this is an example where there are two large language models, then the data can be used to train a second large language model.

[0049] In another example, the execution of machine-executable instructions also enables the computing system to use reinforcement learning to train a large language model by modifying commands.

[0050] In another example, the object metadata includes the object's height.

[0051] In another example, the object metadata includes object weight.

[0052] In another example, the object metadata includes the object's gender or social gender.

[0053] In another example, the object metadata includes the object's medical records.

[0054] In another example, the object metadata includes object diagnostic codes.

[0055] In another example, the object metadata includes text generated by a physician or healthcare provider.

[0056] In another example, the object metadata includes the text generated by the object.

[0057] In another example, the object metadata includes (recorded and / or transcribed) spoken language expressing the object's symptoms or health problems.

[0058] In another example, the object metadata includes (recorded and / or transcribed) spoken language by healthcare professionals regarding medical questions to be answered by imaging examinations.

[0059] Figure 1 An example of a medical system 100 is illustrated. The medical system 100 is shown as including a computer 102, which includes a computing system 104. Computer 102 may represent one or more computers or computer systems located at one or more locations. Similarly, computing system 104 may also represent one or more computing systems or cores located at one or more locations. Computing system 104 is shown communicating with an optional hardware interface 106. If the medical system 100 includes other components, the hardware interface 106 can be used to communicate with and / or control those additional components. Computing system 104 is also shown communicating with an optional user interface 108, which allows an operator or user to control and interact with the medical system 100.

[0060] The computing system 104 is also shown to include memory 110. Memory 110 is intended to represent various types of volatile and non-volatile memory accessible to the computing system 104. In some examples, memory 110 may be a non-transient storage medium.

[0061] Memory 110 is shown as containing machine-executable instructions 120. Machine-executable instructions 120 are instructions for computing system 104 that enable computing system 104 to perform various data analysis and control tasks. Memory 110 is also shown as containing a large language model module 122. Memory 110 is further shown as containing object metadata 124 and the generated pulse sequence settings 126. The generated pulse sequence settings 126 are received from the large language model module 122 in response to inputting object metadata 124 to the large language model module 122. The generated pulse sequence settings 126 are commands or data that can be converted into commands to control the magnetic resonance imaging system to acquire k-space data.

[0062] Figure 2 The illustrated operation is shown. Figure 1The flowchart of the method for the medical system 100 is as follows. In step 200, object metadata 124 is received. In step 202, the generated pulse sequence setting 126 is received by inputting the object metadata 124 into the large language model module.

[0063] Figure 3 Another example of a medical system 300 is illustrated. Figure 3 The medical system depicted in the text is similar to 300 Figure 1 The medical system 100 includes, in addition to, a magnetic resonance imaging system 302 controlled by a computing system 104.

[0064] The magnetic resonance imaging system 302 includes a magnet 304. Magnet 304 is a superconducting cylindrical magnet with a bore 306 passing through it. Split cylindrical magnets and so-called open magnets can also be used. A split cylindrical magnet is similar to a standard cylindrical magnet, except that the cryostat has been split into two parts to allow access to the isocentric plane of the magnet, thus allowing the magnet to be used, for example, in conjunction with charged particle beam therapy. An open magnet has two magnet sections, one on top of the other, with a space in between large enough to accommodate the object. The arrangement of the two sections is similar to that of a Helmholtz coil. Open magnets are popular because the object is less restricted. An assembly of superconducting coils is located inside the cryostat of the cylindrical magnet.

[0065] Within the bore 306 of the cylindrical magnet 304, an imaging region 308 exists, in which the magnetic field is sufficiently strong and uniform to perform magnetic resonance imaging. A field of view 309 within the imaging region 308 is shown. k-space data was acquired for the field of view 309. The region of interest may be the same as the field of view 309, or it may be a sub-volume of the field of view 309. An object 318 is shown supported by an object support 320, such that at least a portion of the object 318 is within the imaging region 308 and the field of view 309.

[0066] The magnet's bore 306 also contains an assembly of magnetic field gradient coils 310, used to acquire k-space data for spatial encoding of magnetic spins within the imaging region 308 of the magnet 304. The magnetic field gradient coils 310 are connected to a magnetic field gradient coil power supply 312. The magnetic field gradient coils 310 are intended to be representative. Typically, the magnetic field gradient coils 310 comprise three separate sets of coils for spatial encoding in three orthogonal spatial directions. The magnetic field gradient power supply supplies current to the magnetic field gradient coils. The current supplied to the magnetic field gradient coils 310 is time-controlled and can be either slanted or pulsed.

[0067] Adjacent to the imaging region 308 is an RF coil 314, which is used to manipulate the orientation of the magnetic spins within the imaging region 308 and to receive RF transmissions from spins also located within the imaging region 308. The RF antenna may comprise multiple coil elements. The RF antenna may also be referred to as a channel or antenna. The RF coil 314 is connected to an RF transceiver 316. The RF coil 314 and the RF transceiver 316 may be replaced by separate transmit and receive coils, as well as separate transmitters and receivers. It is to be understood that the RF coil 314 and the RF transceiver 316 are representative. The RF coil 314 is intended to also represent a dedicated transmit antenna and a dedicated receive antenna. Similarly, the transceiver 316 may also represent separate transmitters and receivers. The RF coil 314 may also have multiple receive / transmit elements, and the RF transceiver 316 may have multiple receive / transmit channels. The transceiver 316 and the gradient controller 312 are shown as a hardware interface 106 connected to the computer system 102.

[0068] The memory 110 is also shown to contain k-space data 320 acquired by controlling the magnetic resonance imaging system 302 using the generated pulse sequence setting 126. The memory 110 is also shown to contain a magnetic resonance image 322 reconstructed from the k-space data 320.

[0069] Figure 4 The illustrated operation is shown. Figure 3 The flowchart of another method in the medical system 300. The execution of steps 200 and 202 is as follows: Figure 2 Illustration. In step 400, the magnetic resonance imaging system 302 is controlled using the generated pulse sequence setting 126 to acquire k-space data 320. In step 402, the k-space data 320 is reconstructed into a magnetic resonance image 322.

[0070] Figure 5 An exemplary implementation of the large language model module 122 is illustrated. In this example, the large language model module 122 includes a first large language model 500, a decision module 502, and a second large language model 504.

[0071] The first language model 500 receives object metadata 124 as input and outputs a set 506 of magnetic resonance imaging protocols as a response. For example, the set 506 of magnetic resonance imaging protocols may be a list or inventory of different magnetic resonance imaging protocols in text or descriptive form.

[0072] Decision module 502 receives the set 506 of magnetic resonance imaging protocols as input and then outputs a selected magnetic resonance imaging protocol 508. The selected magnetic resonance imaging protocol 508 is one of the magnetic resonance imaging protocols in the set 506. Therefore, the decision module is configured to select from the set 506 of magnetic resonance imaging protocols. In some examples, decision module 502 may also receive object metadata 124 as input. In other examples, the decision module receives sensor data. For example, a camera may provide location data describing the object, or estimate the size and / or weight of the object. The sensor data may be used at least in part to determine which magnetic resonance imaging protocol to select.

[0073] Figure 6 An alternative implementation of the large language model module 122' is illustrated. The large language model module 600 is configured to receive object metadata 124 as input and the generated pulse sequence setting 126 as output.

[0074] Large Language Models (LLMs), as described above, revolutionized chatbots with milestones like ChatGPT. LLMs use encoder and decoder architectures with billions to trillions of parameters, enabling them to learn vast amounts of information in an unsupervised manner. For example, LLMs can be trained using all public content on the internet, thus becoming general-purpose chatbots.

[0075] LLMs operate based on predicting the next word in a dialogue. Once pre-trained on more general data (often unsupervised), LLMs can be fine-tuned using domain-specific data to teach them specific contexts, roles, rules, and response types. A popular approach is to combine supervised learning trained using model dialogues, or reinforcement learning based on human feedback.

[0076] In this example, LLM can be used to automate MRI protocol planning. This involves collecting patient data, including the reason for the MRI examination (symptoms, chief complaint, referring physician's speculation and intentions), and prior information that may come from the patient's medical history / past history. Recommending the appropriate combination of MRI scans for the patient and filling in the settings for each sequence can be handled by one or more LLMs fine-tuned for these specific tasks.

[0077] In a typical radiology setting, when patients arrive at the patient registration desk, they register their information and the reason for their MRI examination (usually on the prescription form from the physician, using a standardized ICD-10 code and / or some free text). When it's a patient's turn, some MRI operators (radiologists, technicians) will select or combine scan cards with a set of appropriate MRI scans based on the patient's indications and personal data (such as implants, age, or weight).

[0078] Currently, the MRI scans received rely heavily on the experience of local radiology staff. Locally modified vendor scan protocol settings vary from clinic to clinic and institution to institution, and each team has its own preferences and experience, resulting in significant differences in imaging across departments and institutions. Automating MRI protocol planning not only benefits clinical workflows and speeds up examination setup, saving valuable radiology staff time and increasing patient throughput, but also coordinates and progressively integrates different MRI sequences and settings used for different clinical problems (derived from symptoms and patient history).

[0079] Because MRI scans require trained personnel to set up and perform, their use is currently limited to hospitals and specialized radiology clinics. The automated method proposed in this invention enables MRI scans to be offered to retail customers, such as in shopping malls or on cruise ships, in the form of voluntary self-service scanning, without the presence of any medical professionals. Here, the customer's complaints, speculations, symptoms, or desires for general examinations, expressed in non-medical language, need to be translated into MRI sequences suitable for automated operation, which is impossible under existing technological conditions.

[0080] Examples may include software tools with a backend incorporating a Large Language Model (LLM) and a frontend interface to the MRI machine's scan planning software, including a user interface for radiology staff to confirm or modify the automatically recommended protocol. The LLM machine learning model is configured to return the appropriate MRI sequence and its detailed settings when queried for a specific patient case (using free text, free language, and / or standardized medical codes), such as specifying patient symptoms / complaints, patient history, referring physician speculation, age, weight, sex, possible past medical history, implants, pacemakers, and other information that may be relevant to selecting the optimal MRI protocol to answer the raised or potentially related clinical questions.

[0081] LLM encompasses a broader range of encoder and decoder architectures that can be trained iteratively in different ways and stages. Unsupervised learning can be used initially to train the LLM using generally available knowledge, particularly from the medical field (e.g., from textbooks, research publications, case reports, and other validated sources of information). Supervised learning can then fine-tune the LLM for specific tasks and specific output semantics and syntax; for example, instead of outputting free text, it might output MRI sequence setup files, just like those natively used by MRI systems (which may need to be adapted to different vendor systems and file formats). During field operations, the LLM can be continuously fine-tuned based on user feedback, which can be sifted and filtered before becoming training data for the model. Furthermore, the model can be progressively updated by continuously incorporating newly released medical information through unsupervised learning, supervised learning, and reinforcement learning.

[0082] Figure 7 A flowchart illustrating another example of the illustrated method is shown. In step 700, patient data (e.g., age, sex, implants, symptoms, and MRI cause) from a patient registry is provided as input. The patient data and other data are examples of object metadata 124. In step 500, a first large language model 500 is configured to output a set of suitable MRI protocols for a set 506 of MRI protocols in response to receiving object metadata 124 as input. In step 702, a human MRI operator selects a chosen MRI protocol 508. Alternatively, the human MRI operator may reject all of these and manually formulate an existing imaging protocol, which can be used for reinforcement learning.

[0083] In step 504, the second large language model is configured to output detailed pre-filled settings for all configuration settings for each sequence of the selected MRI protocol 508. In step 704, the human MRI operator accepts or modifies the settings proposed by the second large language model 504. In step 706, an MRI examination is performed, and k-space data 320 can be acquired and used to reconstruct MRI images 322. In step 708, the radiologist gives an optional rating of the diagnostic value of the MRI, assigning a "thumbs up" or "thumbs down" evaluation. In step 710, the collected ratings will be used for reinforcement learning using human feedback or RLHF to improve future recommendations. In step 712, the MRI protocol and sequence settings recommended for RLHF are corrected to fine-tune the model. To accommodate local preferences, this can be performed locally for specific populations or MRI system installations. Figure 7 The method illustrated in the figure has a lower level of automation than other proposed examples.

[0084] In some examples, the implementation plan can be integrated into clinical workflows, such as Figure 7 As described in [the document]. This should work for existing and older scanners. Adaptation for other manufacturers' MRI systems and software is also possible. The LLM can be tuned (fine-tuned) for a specific system to output MRI settings in the desired format. Hardware limitations for each system can be addressed through an iterative approach, detailed below.

[0085] When the LLM is trained to a level sufficient to handle a limited range of medical conditions and imaging tasks, the system can operate without local human intervention, either under remote supervision or completely autonomously.

[0086] The sample interface can be integrated into MRI protocol planning software. When a new scan card is opened for a patient, the system automatically retrieves the patient information associated with the individual patient (information previously entered during patient registration). The reason for the MRI and the patient's specific circumstances are sent as input to the first LLM:

[0087] First LLM (First Largest Language Model 500): High-Level Scan Card Design Assistant

[0088] This LLM agent provides protocol recommendations with reasonable sequences at the contrast / weighting / sequence type / name granularity.

[0089] Its recommendations are based on patient indications (symptoms, etc., provided in free text, free speech, or including medical expert information (such as codes)) and human experience reflected in LLM training materials.

[0090] MRI operators can choose from recommended protocols, modify them, or design their own protocols. Once a set of MRI sequences is selected, a second machine learning model, incorporated into the LLM architecture, receives these sequences as input.

[0091] Second LLM (Second Largest Language Model 504): Detailed Scan Configuration Assistant

[0092] The LLM generates pre-filled configurations (acquisition type, fat suppression, echo time, etc.) for the selected sequence.

[0093] Pre-population is based on patient data (such as weight, age, implant) and possibly other data (including sensor data, such as camera data).

[0094] The model can be fine-tuned to produce output in "MRI Sequence Setup Language" (generated pulse sequence setup 126) format, which is a file format containing all the information needed to perform a scan. Ideally, it should be in the vendor's native format.

[0095] In another example, the second LLM can be iteratively combined with other analytical and numerical tools, rule enforcement systems, or machine learning tools such as classifiers, decision makers, etc., to iteratively refine the output for a specific query by switching between creative, generative LLM outputs and a rigorous rule-based supervisory system (such as the current computational validation phase), which checks the LLM recommendations for hardware capabilities, patient safety, and regulatory compliance (similar to iteration in generative adversarial networks (GANs). If the supervisory algorithm detects a conflict in the recommended sequence or sequence setup, it can point out this conflict (and make suggestions) to the LLM, and it can attempt to resolve it or recommend alternatives until a satisfactory solution is reached.

[0096] Modifications made by radiology staff can be incorporated into algorithmic suggestions via reinforcement learning that utilizes human feedback (RLHF). This can either improve the overall performance of the model by feeding this feedback data into globally maintained and distributed models, or it can fine-tune the model based on more localized preferences by updating the model with specific changes made by staff only for locally used models.

[0097] In the example, a one-click feedback system can be provided on the MRI system and / or diagnostic / viewing software to implement RLHF. During image review, if an image is helpful for diagnosis or, in particular, useless and time-consuming, the radiologist can click the "thumbs up" or "thumbs down" icon in the corner of the image. This feedback can be collected and incorporated into the training with special weights so that the recommendation algorithm prioritizes high-scoring sequences and settings.

[0098] In yet another example, LLM can be enhanced with different AI models and numerical tools to assist, monitor, and improve recommendation algorithms.

[0099] The trained LLM model can be stored on a local computer at the MRI site or hosted online as a cloud service by a maintenance provider. User interaction with the system does not require a high-bandwidth internet connection, as only relatively little text information is sent to and received from the proposed system at the user interface on the MRI machine. Communication and cloud-hosted models and services can be encrypted, providing access only to specific sites, for example, through public / private key (end-to-end) encryption.

[0100] The example can fully mimic parts of the cognitive, training, and experiential tasks currently handled by radiology staff, but can only be fully achieved after years of training and accumulated experience; namely, selecting and configuring (preferably fast-band) MRI protocols for specific patients to acquire sufficient contrast to answer relevant clinical questions. The system provides recommendations based on a constantly updated vast amount of knowledge, assisting MRI operators in designing personalized scan sequences for patients. The operator's task shifts from manually configuring examinations to supervising, approving, and adjusting recommended settings for each patient. This reduces staff stress and time burden while minimizing scanner downtime and patient wait times during scan setup.

[0101] The example facilitates remote and autonomous MRI operation, enabling scenarios where trained radiology staff can operate and configure scanners locally without requiring them. Currently, radiology staff elsewhere can view suggested scans for specific patient data, approve or modify recommendations, and remotely configure scanners. In future scenarios, as the model improves, approval steps may be omitted in certain situations, such as voluntary self-scanning in MRI scanners on shopping malls or cruise ships.

[0102] It should be understood that one or more of the foregoing examples of the present invention can be combined, as long as the combined examples are not mutually exclusive.

[0103] As those skilled in the art will recognize, several aspects of the invention can be implemented as apparatus, method, or computer program product. Therefore, aspects of the invention can take the form of a completely hardware example, a completely software example (including firmware, resident software, microcode, etc.), or an example combining software and hardware aspects, which can be collectively referred to herein as a “circuit,” “module,” or “system.” Furthermore, aspects of the invention can take the form of a computer program product contained in one or more computer-readable media having computer-executable code contained thereon.

[0104] Any combination of one or more computer-readable media can be used. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. As used herein, "computer-readable storage medium" includes any tangible storage medium that can store instructions executable by the processor or computing system of a computing device. The computer-readable storage medium may be referred to as a "computer-readable non-transient storage medium." The computer-readable storage medium may also be referred to as a tangible computer-readable medium. In some examples, the computer-readable storage medium may also be able to store data accessible by the computing system of the computing device. Examples of computer-readable storage media include, but are not limited to: floppy disks, magnetic hard disk drives, solid-state drives, flash memory, USB thumb drives, random access memory (RAM), read-only memory (ROM), optical discs, magneto-optical discs, and register files of computing systems. Examples of optical discs include compact optical discs (CDs) and digital multi-purpose optical discs (DVDs), such as CD-ROMs, CD-RWs, CD-Rs, DVD-ROMs, DVD-RWs, or DVD-R discs. The term computer-readable storage medium also refers to various types of recording media accessible by the computer device via a network or communication link. For example, data can be retrieved via a modem, via the Internet, or via a local area network. Computer-executable code embodied on a computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, RF, or any suitable combination of the foregoing.

[0105] Computer-readable signal media may include propagating data signals having computer-executable code implemented therein, for example, in baseband or as part of a carrier wave. Such propagating signals may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and is capable of transmitting, propagating, or conveying a program for use by or in connection with an instruction execution system, apparatus, or device.

[0106] "Computer memory" or "memory" is an example of a computer-readable storage medium. Computer memory is any memory that can be directly accessed by a computing system. "Computer storage device" or "storage device" is another example of a computer-readable storage medium. A computer storage device is any non-volatile computer-readable storage medium. In some examples, a computer storage device can also be computer memory, or vice versa.

[0107] As used herein, "computing system" encompasses electronic components capable of executing programs or machine-executable instructions or computer-executable code. References to computing systems that include examples of "computing systems" should be interpreted as potentially including more than one computing system or processing core. A computing system can, for example, be a multi-core processor. A computing system can also refer to a collection of computing systems within a single computer system or distributed across multiple computer systems. The term computing system should also be interpreted as potentially referring to a collection or network of computing devices, each including a processor or multiple computing systems. Machine-executable code or instructions can be executed by multiple computing systems or processors, which may be within the same computing device or even distributed across multiple computing devices.

[0108] Machine-executable instructions or computer-executable code may include instructions or programs that cause a processor or other computing system to perform one aspect of the invention. Computer-executable code for performing operations targeting the aspects of the invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages, and compiled into machine-executable instructions. In some cases, the computer-executable code may be used in the form of a high-level language or in a pre-compiled form in conjunction with an interpreter that dynamically generates machine-executable instructions. In other cases, the machine-executable instructions or computer-executable code may be in the form of programming against a programmable gate array.

[0109] The computer-executable code can run as a standalone software package entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0110] Various aspects of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments or examples of the invention. It should be understood that each block or portion of a block in a flowchart, illustration, and / or block diagram can be implemented, where applicable, by computer program instructions in the form of computer-executable code. It should also be understood that combinations of blocks in different flowcharts, illustrations, and / or block diagrams can be combined when not mutually exclusive. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, executed via the computer's computing system or other programmable data processing apparatus, create units for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0111] These machine-executable instructions or computer program instructions may also be stored in a computer-readable medium that is capable of directing a computer, other programmable data processing apparatus or other device to operate in a particular manner, such that the instructions stored in the computer-readable medium produce an article of writing comprising instructions that implement the functions / actions specified in flowcharts and / or one or more block diagrams.

[0112] The machine-executable instructions or computer program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions running on the computer or other programmable apparatus provide for implementing the functions / actions specified in the flowchart and / or one or more block diagram boxes.

[0113] As used herein, a "user interface" is an interface that allows a user or operator to interact with a computer or computer system. A "user interface" can also be referred to as a "human-machine interface device." A user interface can provide information or data to an operator and / or receive information or data from an operator. A user interface enables input from an operator to be received by the computer and can provide output from the computer to the user. In other words, a user interface allows an operator to control or manipulate a computer, and the interface allows the computer to indicate the effects of the operator's control or manipulation. The display of data or information on a monitor or graphical user interface is an example of providing information to an operator. Receiving data via a keyboard, mouse, trackball, touchpad, pointing stick, graphics tablet, joystick, game controller, webcam, headset, pedal, wired gloves, remote control, and accelerometer are all examples of user interface components that implement the receiving of information or data from an operator.

[0114] As used herein, "hardware interface" encompasses an interface that enables a computer system to interact with or control external computing devices and / or apparatuses. A hardware interface allows the computing system to send control signals or instructions to external computing devices and / or apparatuses. A hardware interface also enables the computing system to exchange data with external computing devices and / or apparatuses. Examples of hardware interfaces include, but are not limited to: Universal Serial Bus (USB), IEEE 1394 port, parallel port, IEEE 1284 port, serial port, RS-232 port, IEEE-488 port, Bluetooth connectivity, wireless LAN connectivity, TCP / IP connectivity, Ethernet connectivity, control voltage interface, MIDI interface, analog input interface, and digital input interface.

[0115] As used herein, "display" or "display device" encompasses an output device or user interface suitable for displaying images or data. A display may output visual, audio, and / or tactile data. Examples of displays include, but are not limited to: computer monitors, television screens, touchscreens, tactile electronic displays, Braille screens, etc.

[0116] Cathode ray tubes (CRTs), storage tubes, bistable displays, electronic paper, vector displays, flat panel displays, vacuum fluorescent displays (VFs), light-emitting diode (LED) displays, electroluminescent displays (ELDs), plasma display panels (PDPs), liquid crystal displays (LCDs), organic light-emitting diode (OLED) displays, projectors, and head-mounted displays.

[0117] In this paper, K-space data is defined as the measurement of radio frequency signals emitted by atomic spins, recorded by the antenna of a magnetic resonance imaging (MRI) device during a magnetic resonance imaging (MRI) scan. MRI data is an example of tomographic medical image data.

[0118] Magnetic resonance imaging (MRI) images, or MR images, are defined herein as reconstructed two-dimensional or three-dimensional visualizations of anatomical data contained within magnetic resonance imaging data. Such visualizations can be performed using a computer.

[0119] Although the invention has been illustrated and described in detail in the accompanying drawings and the foregoing description, such illustrations and descriptions should be considered illustrative or exemplary, not restrictive. The invention is not limited to the disclosed embodiments or examples.

[0120] Those skilled in the art, through studying the accompanying drawings, disclosure, and claims, can understand and implement other variations of the disclosed embodiments or examples when practicing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the words "a" or "an" do not exclude a plurality. A single processor or other unit can perform the functions of several items recited in the claims. Although specific measures are recited in dissimilar dependent claims, this does not imply that combinations of these measures cannot be advantageously used. Computer programs can be stored / distributed on suitable media such as optical storage media or solid-state media provided with or as part of other hardware, but can also be distributed in other forms such as via the Internet or other wired or wireless telecommunications systems. No reference numerals in the claims should be construed as limiting the scope.

Claims

1. A medical system (100, 300) comprising: a memory (110) storing: machine executable instructions (120), a large language model module (112, 112'), wherein the large language model module is configured to output a generated pulse sequence setup (126) in response to receiving subject metadata (124) as input, wherein the generated pulse sequence setup is configured to control a magnetic resonance imaging system (302) to acquire k-space data (320); a computing system (104), wherein execution of the machine executable instructions causes the computing system to: receive (200) the subject metadata; and receive (202) the generated pulse sequence setup in response to inputting the subject metadata into the large language model module.

2. The medical system of claim 1, wherein, execution of the machine executable instructions further causes the computing system to: acquire (400) the k-space data by controlling the magnetic resonance imaging system with the generated pulse sequence setup, and reconstruct (402) a magnetic resonance image (322) from the k-space data.

3. The medical system of claim 1 or 2, wherein, the large language model module comprises a first large language model (500) and a second large language model (504), wherein the first large language model is configured to output a set of recommended magnetic resonance imaging protocols (506) in response to receiving the subject metadata as input for selection by a clinical staff or a decision model, wherein the second large language model is configured to output the generated pulse sequence setup in response to receiving a selected magnetic resonance imaging protocol as input, wherein the generated pulse sequence setup is configured to control the magnetic resonance imaging system to acquire k-space data according to the selected magnetic resonance imaging protocol, wherein execution of the machine executable instructions further causes the computing system to: receive a set of magnetic resonance imaging protocols in response to inputting the subject metadata into the first large language model; provide the set of magnetic resonance imaging protocols using the medical system; receive the selected magnetic resonance imaging protocol in response to providing the set of magnetic resonance imaging protocols, wherein the set of magnetic resonance imaging protocols includes the selected magnetic resonance imaging protocol; and receive the generated pulse sequence setup in response to inputting the selected magnetic resonance imaging protocol into the second large language model.

4. The medical system of claim 1, 2, or 3, wherein, the large language model module further comprises a decision module (502), wherein the decision module is configured to output the selected magnetic resonance imaging protocol in response to receiving the set of magnetic resonance imaging protocols as input, wherein the set of magnetic resonance imaging protocols is provided by inputting the set of magnetic resonance imaging protocols into the decision module, wherein the selected magnetic resonance imaging protocol is received as an output of the decision module.

5. The medical system of claim 4, wherein, the decision module is implemented as any one of the following: a neural network, a recurrent neural network, a convolutional neural network, a decision tree, an expert system, and a graph database.

6. The medical system of claim 3, wherein, Providing the set of magnetic resonance imaging protocols includes displaying the set of magnetic resonance imaging protocols on a user interface, and wherein the selected magnetic resonance imaging protocol is received via a selection of the protocol from the user interface.

7. The medical system of claim 4, 5 or 6, when dependent on claim 3, wherein, Execution of the machine executable instructions further causes the computing system to train the first large language module using reinforcement learning with the selected magnetic resonance imaging protocol.

8. The medical system of any one of claims 3 to 6, wherein, Members of the set of magnetic resonance imaging protocols are described in textual fashion.

9. The medical system of any of claims 3 to 8, wherein, Members of the set of magnetic resonance imaging protocols are constructed from a library of pulse sequence components.

10. The medical system of any one of claims 3 to 9, when dependent on claim 3, wherein, Execution of the machine executable instructions further causes the computing system to train the first large language model using supervised learning and / or reinforcement learning with radiology case reports.

11. The medical system of claim 1 or 2, wherein, The large language model module includes a single large language model (600).

12. The medical system of any one of the preceding claims, wherein, Execution of the machine executable instructions further causes the computing system to: display at least part of the generated pulse sequence settings on a user interface; receive a modification command from the user interface in response to displaying the at least part of the generated pulse sequence settings on the user interface; modify the generated pulse sequence settings using the modification command.

13. The medical system of claim 12, wherein, Execution of the machine executable instructions further causes the computing system to train the large language model module using reinforcement learning with the modification command.

14. A method comprising: receiving (200) subject metadata (124); and receiving (202) generated pulse sequence settings (126) in response to inputting the subject metadata into a large language model module (112, 112'), wherein the large language model module is configured to output the generated pulse sequence settings in response to receiving the subject metadata as input, wherein the generated pulse sequence settings are configured to control a magnetic resonance imaging system (302) to acquire k-space data (320). The large language model module is configured to output generated pulse sequence settings (126) in response to receiving subject metadata (124) as input, wherein the generated pulse sequence settings are configured to control a magnetic resonance imaging system (302) to acquire k-space data (320), wherein execution of the machine executable instructions causes a computing system to:

15. A computer program comprising machine executable instructions (120) and a large language model module (112, 112'), wherein, receive (200) subject metadata; and receive (202) the generated pulse sequence settings in response to inputting the subject metadata into the large language model module. ​

Images