Identification of Medical Image Protocols Based on Radiation Data and Metadata

Abstract

Provide an improved protocol for a medical scanner. 【Solution means】A computer-implemented method learns a model of an imaging protocol using a plurality of input examination data sets created by performing a plurality of imaging examinations on at least one patient with at least one scanner. The model learns the imaging protocol by capturing common features across the plurality of input examination data sets. The method re-groups examination data sets having common features under common protocol tags within the plurality of input examination data sets, and learning the model includes generating a plurality of protocol tags. The model is updated as needed based on new input examination data sets.

Description

Technical Field

[0001] The present invention relates to the identification of medical image protocols based on radiation data and metadata.

Background Art

[0002] Medical image scanners (magnetic resonance imaging (MRI) scanners, computed tomography (CT) scanners, positron emission tomography (PET) scanners, ultrasonic scanners, X-ray scanners, etc.) acquire two-dimensional or three-dimensional images of the body. Such images are often used for disease detection, diagnosis, and treatment monitoring.

[0003] Such a scanner images a patient in a process called "imaging acquisition" or simply "acquisition", resulting in one or more images (also referred to herein as "acquisition data"). Each such acquisition is accompanied by a corresponding set of technical parameters specific to the imaging modality used during the acquisition (e.g., MRI or CT). The values of these technical parameters (e.g., echo time and repetition time for MRI, kVp and mAs for CT) define various settings and factors that affect the quality and characteristics of the final image, including which specific tissue characteristics are being evaluated during the acquisition, and as a result, lead to different views (e.g., T1-weighted, T2-weighted) of the imaged tissue. During an "examination" (also referred to herein as an "imaging examination"), a patient undergoes one or more acquisitions so that different, generally complementary, views of the tissue are evaluated. As used herein, the term "imaging protocol" refers to a plurality of acquisition data sets collected during a particular examination. As used herein, the term "examination data set" refers to the following related to a particular examination: (1) a plurality of acquisition data sets representing the acquisitions performed during the particular examination corresponding to the examination data set (i.e., the imaging protocol), and (2) (optionally) one or more non-technical parameters related to the examination, and their associated values.

[0004] Radiologists typically construct an ideal protocol based on the radiologist's diagnostic requirements within certain constraints such as time, patient safety (e.g., SAR or radiation dose), and patient tolerance / satisfaction. Such an ideal protocol is designed to study each clinical indication (e.g., diagnostic problems such as brain tumors, brain multiple sclerosis, pre-brain surgery planning, etc.). Such an ideal protocol varies depending on patient attributes (e.g., age, BMI, etc.). Each ideal protocol is referred to as a "parent protocol".

[0005] Unfortunately, some scanners have technical limitations and may not be able to perform the acquisitions specified in the parent protocol. For example, a particular parent protocol may include acquisitions with parameters and values that a particular scanner cannot implement. As a result, to perform the acquisition on that scanner, it is necessary to modify the protocol for that scanner by changing the parameter values or applying other parameters to that scanner. As a result, it becomes a modified version of the parent protocol, which is referred to as a "child protocol" in this specification. A child protocol is created on the fly, for example, by changing the parameters within the scanner, or is pre-stored in the scanner to shorten the preparation time and improve the reproducibility and uniformity of the images generated by the scanner when performing scans using those child protocols. The resulting child protocol may or may not be stored in the scanner for future use.

[0006] When a scanner operator images a patient, the operator selects a child protocol from the scanner's template list. If no custom child protocol is stored in the scanner, the child protocol selected by the scanner operator is simply the default set of acquisition parameters and corresponding values provided by the scanner manufacturer, from which the scanner operator changes the parameters on the fly to match the expected child protocol.

[0007] There may be cases where the scanner operator needs or desires to modify the child protocol. Examples of modifications that the scanner operator can make to the child protocol to generate an imaging protocol include changing the value of one or more parameters within the child protocol, replacing a parameter within the child protocol with a different parameter within the imaging protocol, and the like. The child protocol is modified to generate an imaging protocol for any one or more of the following reasons.

[0008] · It may be necessary or desirable to change the value of one or more parameters of the child protocol to accommodate the anatomical characteristics of the patient (for example, increasing the number of slices if the patient is larger than assumed in the child protocol). · In cases where the patient is moving and the image becomes un-diagnosable, it may be necessary or desirable to repeat the collection one or more times, resulting in an imaging protocol that includes more collections than the child protocol. · It may be necessary or desirable to change the order of collection in the child protocol to prioritize one image over another, resulting in an imaging protocol in which the collection is performed in an order different from the child protocol. · It may be necessary or desirable to add a new collection, for example, when there is suspicion of abnormal tissue and the new collection can help confirm or rule it out.

[0009] By imaging the patient, the scanner operator obtains an imaging protocol that may be different from the child protocol (and the parent protocol). As the above explanation shows, each imaging protocol is based on the corresponding child protocol, and may be the same as or different from its corresponding child protocol. Similarly, each child protocol is based on the corresponding parent protocol, and may be the same as or different from its corresponding parent protocol. As can be seen from this, each imaging protocol may be the same as or different from the parent protocol of its child protocol (i.e., the "grandparent protocol" of the imaging protocol).

[0010] Such diversity of protocols and their interrelationships complicate management and can cause various problems. Therefore, what is needed is to improve the technology for managing the protocols of medical scanners. SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION

[0011] An object of the present invention is to provide an improved protocol for a medical scanner. MEANS FOR SOLVING THE PROBLEMS

[0012] A computer-implemented method learns a model of an imaging protocol using a plurality of input examination data sets created by performing a plurality of imaging examinations of at least one patient on at least one scanner. The model learns the imaging protocol by capturing common features across the plurality of input examination data sets. The method re-groups examination data sets having common features under a common protocol tag within the plurality of input examination data sets, and learning of the model includes generating a plurality of protocol tags. The model is updated as needed based on new input examination data sets.

[0013] Other features and advantages of various aspects and embodiments of the present invention will become apparent from the following description. ADVANTAGES OF THE INVENTION

[0014] The method according to the present invention can provide an improved protocol for a medical scanner. The analysis of available examination data sets can be repeatedly performed, and the model (and the relationships it represents) can be automatically updated over time. BRIEF DESCRIPTION OF THE DRAWINGS

[0015]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiments for Carrying Out the Invention

[0016] A computer-implemented method learns a model of an imaging protocol using a plurality of input examination data sets created by performing a plurality of imaging examinations on at least one patient with at least one scanner. The model learns the imaging protocol by capturing common features across the plurality of input examination data sets. The method re-groups examination data sets having common features under a common protocol tag within the plurality of input examination data sets, and learning the model includes generating a plurality of protocol tags. The model is updated as needed based on new input examination data sets.

[0017] FIG. 10 is a configuration diagram of a system 1000 for learning a model of an imaging protocol based on a plurality of input examination data sets according to an embodiment of the present invention. FIG. 11 is a flowchart of a method 1100 executed by the system 1000 of FIG. 10 according to an embodiment of the present invention.

[0018] System 1000 includes a patient 1002 and a scanner 1004. System 1000 performs a plurality of imaging examinations on patient 1002 using scanner 1004, thereby generating a plurality of input examination data sets 1012a-N (FIG. 11, operation 1102). As described above, in this specification, the terms "imaging examination" and "examination" are used synonymously, which means that the term "examination" refers to an imaging examination. Although only one patient 1002 is shown in FIG. 10 for ease of illustration, the plurality of examinations may be performed on one or more patients. For example, some examinations may be performed on a first patient and other examinations may be performed on a second patient. Similarly, although only a single scanner 1004 is shown in FIG. 10 for ease of illustration, the plurality of examinations may be performed using one or more scanners. For example, some examinations may be performed using a first scanner and other examinations may be performed using a second scanner.

[0019] Each of the plurality of input inspection data sets 1012a to N includes a plurality of corresponding collection data sets. For the sake of explanation, only the collection data sets 1014a to N within the inspection data set 1012a are shown in FIG. 10. However, it should be understood that the other inspection data sets 1012b to N each include a corresponding plurality of collection data sets. Each collection data set A included in each of the plurality of collection data sets includes a plurality of values corresponding to a plurality of technical parameters used to perform the collection that generated the collection data set A.

[0020] The plurality of input inspection data sets 1012a to N may include various additional data, for example, not only image data and its technical parameters. Examples of such additional data include any one or a plurality of the following arbitrary combinations.

[0021] · Among the DICOM (Digital Imaging and Communications in Medicine) data related to any of the data of the plurality of input inspection data sets 1012a to N, data not yet included in the image data such as DICOM data and its technical parameters. · Information obtained from a scheduling system (not shown) used to generate and / or store one or more schedules related to the inspections used to generate the inspection data sets 1012a to N, for example, demographic information, whether the patient 1002 was sedated during collection, the disease state of the patient 1002, whether the patient 1002 is an inpatient or an outpatient, the body mass index of the patient 1002, the urgency of the inspection, and whether the collection was performed using a contrast agent, etc., one or more of the M non-technical parameters disclosed herein.

[0022] System 1000 also includes a learning engine 1020 that receives a plurality of input inspection data sets 1012a - N as inputs (FIG. 11, operation 1104). The learning engine 1020 learns a model 1022 of an imaging protocol based on the plurality of input inspection data sets 1012a - N (FIG. 11, operation 1106). The learning engine 1020 performs the learning in operation 1106 using, for example, supervised learning and / or unsupervised learning.

[0023] The model 1022 can capture common features, for example, across at least a portion of the plurality of input inspection data sets 1012a - N. The model 1022, for example, re - groups inspection data sets having common features among the plurality of input inspection data sets 1012a - N under a common protocol tag, and the learning of the model 1022 in operation 1106 includes generating a plurality of protocol tags 1028.

[0024] Each of the plurality of protocol tags 1028 describes, for example, a corresponding set of common features within the plurality of input inspection data sets 1012a - N, and each of the inspection data sets within the set of inspection data sets re - grouped under the protocol tag has the corresponding set of common features described by that protocol tag. The plurality of protocol tags 1028 includes, for example, a plurality of embeddings (e.g., a plurality of fixed - size embeddings). In this case, the corresponding set of common features is determined by a clustering algorithm that groups embeddings within a predefined proximity. The protocol tag is also regarded as a "label" used elsewhere in this specification.

[0025] The generation of protocol tag 1028 is shown as part of method 1100 of FIG. 11, which also includes learning model 1022. However, in reality, protocol tag 1028 does not necessarily have to be generated as part of the same process (or by the same entity) that learns model 1022. For example, one process may learn model 1022, and another process may use that model to generate protocol tag 1028. As a specific example, one entity may learn model 1022, and another entity (that did not learn model 1022) may use that model 1022 to generate protocol tag 1028.

[0026] Furthermore, FIGS. 10 and 11 show protocol tag 1028 being generated based on model 1022. More generally, however, protocol tag 1028 can be generated in various ways based on input inspection data sets 1012a - N and / or new input inspection data set 1026. For example, even if protocol tag 1028 is generated based on model 1022 as described above, in such a case, protocol tag 1028 is indirectly generated based on input inspection data sets 1012a - N and / or new input inspection data set 1026.

[0027] System 1000 may also include a plurality of new input inspection data sets 1026. Although not shown in detail in FIG. 10, the plurality of new input inspection data sets 1026 have any of the features disclosed herein in relation to the plurality of input inspection data sets 1012a - N. For example, each inspection data set within the plurality of new input inspection data sets 1026 includes a plurality of collection data sets of the types disclosed herein. The content of the plurality of new input inspection data sets 1026 (e.g., images, technical parameters and parameter values, and non - technical parameters and parameter values) is the same as or different from the content of the plurality of input inspection data sets 1012a - N. The plurality of new input inspection data sets 1026 are generated, for example, after operation 1106 is executed to generate at least an initial version of model 1022.

[0028] The learning engine 1020 uses the model 1022 and the plurality of new input inspection data sets 1026 to generate a plurality of protocol tags 1028 (FIG. 11, operation 1108). Each of the plurality of protocol tags 1028 describes a corresponding set of common features within the plurality of new input inspection data sets 1026.

[0029] Each tag T within the plurality of protocol tags 1028 describes a corresponding set of inspection data sets within the plurality of input inspection data sets 1012a - N, and the corresponding set of inspection data sets includes a plurality of collection data sets that share a corresponding set of common features within the plurality of input inspection data sets 1012a - N.

[0030] A plurality of new input inspection datasets 1026 are shown as a single plurality of new input inspection datasets 1026 in FIG. 10 for ease of illustration, but in reality, the plurality of new input inspection datasets 1026 may include multiple new input inspection datasets. The functions disclosed herein as being performed on the plurality of new input inspection datasets 1026 (e.g., generating protocol tags 1028 based on the plurality of new input inspection datasets 1026) are performed on any subset of the plurality of new input inspection datasets 1026. As an example, the plurality of new input inspection datasets 1026 includes a first new input inspection dataset at a first point in time (e.g., as a result of performing a first set of imaging inspections using scanner 1004), and system 1000 processes the first new input inspection dataset to perform certain functions such as generating a first set of protocol tags within protocol tags 1028. At a later second point in time, the plurality of new input inspection datasets 1026 includes a second plurality of new input inspection datasets (e.g., as a result of performing additional imaging inspections using scanner 1004), and system 1000 processes the second plurality of new input inspection datasets to perform certain functions as follows: (1) generating a corresponding set of protocol tags within protocol tags 1028, or (2) updating model 1022 (using any of the techniques disclosed herein for learning model 1022) to generate an updated version of model 1022 (and, optionally, generating a corresponding set of protocol tags within protocol tags 1028 as a byproduct of the update to model 1022). As these examples illustrate, the plurality of new input inspection datasets 1026 change over time, and the various functions disclosed herein process some or all of the plurality of new input inspection datasets in the state at any given point in time.

[0031] The learning executed by the learning engine 1020 in operation 1106 to learn the model 1022 includes the following steps: (1) a step of learning a first set of protocol tags from a plurality of input inspection data sets 1012a to N; and (2) a step of learning a second set of protocol tags from the plurality of input inspection data sets 1012a to N and the first set of protocol tags, where the second set of protocol tags describes a corresponding set of common features of a corresponding plurality of protocol tags within the first set of protocol tags. The plurality of protocol tags 1028 includes the first set and the second set of protocol tags. Step (2) includes learning what is described herein as a "child protocol" from a "parent protocol", or vice versa. Step (2) is repeated any number of times in relation to the sets of first and second protocol tags, such as protocol tags learned in previous instances of step (2), for example, presenting new information that suggests new insights regarding the previously learned set of protocol tags. Such a learning process is repeated at any depth, for example, until all possible unique common sets are separated.

[0032] To learn the model 1022, the learning executed by the learning engine 1020 in operation 1106 includes starting from N = 1: (1) a step of learning the Nth set of protocol tags from a plurality of inspection data sets and the set(s) of protocol tags previously learned at N ≧ 1. Here, the Nth set of protocol tags describes a corresponding set of common features of a corresponding plurality of (N - 1)-level protocol tags. (2) A step of determining whether an end criterion is satisfied. (3) If the end criterion is satisfied, a step of ending the learning. (4) If the end criterion is not satisfied: (4)(a) increment N, and (4)(b) return to (1) above.

[0033] The learning executed by the learning engine 1020 in operation 1106 to learn a plurality of protocol tags 1028 includes learning a classifier or clustering algorithm for identifying the characteristics of a plurality of input inspection data sets 1012a-N based on the plurality of input inspection data sets 1012a-N, and learning the plurality of protocol tags 1028 includes using a classifier or clustering algorithm to learn the plurality of protocol tags 1028.

[0034] Also, the method 1100 identifies, for each of the plurality of protocol tags 1028, the corresponding organ of interest, thereby identifying the plurality of organs of interest corresponding to the plurality of protocol tags 1028. The method 1100 identifies a label for each of the plurality of protocol tags 1028. Some or all of the input inspection data sets 1012a-N are labeled, and identifying the label associated with each of the plurality of protocol tags 1028 includes identifying the label based on the labeled inspection data sets within the plurality of input inspection data sets 1012a-N. The plurality of organs of interest corresponding to the plurality of protocol tags are identified, for example, by applying learning to a plurality of images such as some or all of the images included in the plurality of input inspection data sets 1012a-N.

[0035] The method 1100 (e.g., the learning engine 1020) also generates, for each input inspection data set within the plurality of input inspection data sets 1012a-N (e.g., before generating the model 1022 in operation 1106), a corresponding graph. Generating such a graph includes, for example, the following processing for each input inspection data set within the plurality of input inspection data sets 1012a-N:

[0036] · For each of the plurality of nodes in the graph corresponding to the plurality of collection data sets within the input inspection data set, storing information regarding the collection corresponding to the node. · For each pair of nodes in the corresponding graph, generating and storing an edge in the graph that represents information about the relationship between the pair of nodes.

[0037] As a result of such a process, a plurality of graphs corresponding to a plurality of input inspection datasets 1012a to N are generated.

[0038] For each node corresponding to a collection, various types of information are stored, such as information regarding some or all of the technical parameters of the collection and / or information regarding some or all of the non-technical parameters of the collection. Information regarding the parameters includes, for example, the identifier of the parameter and / or the value of the parameter.

[0039] For each edge, various types of information are stored, such as the distance between the pair of nodes connected by the edge and / or the distance or similarity between the pair of nodes connected by the edge.

[0040] Various types of information are stored in the graph as one or more global graph features. For example, in the graph representing an inspection, information representing the non-technical parameters of the inspection is stored as a global graph feature of the graph.

[0041] When such a plurality of graphs are generated, the learning of the model 1022 in operation 1106 includes performing learning based on the corresponding plurality of graphs and generating corresponding embeddings representing a plurality of protocol tags 1028.

[0042] Regardless of whether method 1100 generates multiple graphs, after method 1100 learns the model in operation 1106, method 1100 (e.g., learning engine 1020) learns an updated version of model 1022 (not shown in FIG. 10) based on some or all of the multiple new input inspection datasets 1026. For example, system 1000 generates multiple new input inspection datasets 1026 by performing multiple new image inspections on at least one patient (e.g., patient 1002) with at least one scanner (e.g., scanner 1004), and learning engine 1020 learns an updated version of model 1022 based on the multiple new input inspection datasets 1026 generated as a result of performing such multiple new image inspections. As described above, this is merely an example of how multiple new input inspection datasets 1026 are generated and / or updated as needed and how the multiple new input inspection datasets 1026 are processed. The inspection datasets within the multiple new input inspection datasets 1026 are used both to generate protocol tags 1028 and to learn an updated version of model 1022.

[0043] In embodiments where system 1000 and method 1100 generate corresponding multiple embeddings, system 1000 and method 1100 generate a graph corresponding to the corresponding multiple embeddings for the corresponding multiple embeddings. Generating such a graph includes, for example, · For each node of the multiple nodes in the graph corresponding to the multiple embeddings, storing information about the embedding corresponding to that node, and · For each pair of nodes in the corresponding graph, generating and storing in the corresponding graph an edge representing information about the relationship between the pair of embeddings.

[0044] As will be described in more detail elsewhere in this specification, embodiments of the present invention generate a first-level graph representing an inspection data set (each node of the graph includes information regarding a collection data set within the inspection data set corresponding to the graph), and generate a second-level graph representing a set of inspection data sets (each node of the graph includes information (e.g., embedding) regarding an inspection data set within the set of inspection data sets corresponding to the graph), thereby generating and storing graphs at increasingly higher levels. This process continues to generate even higher-level graphs such that lower-level individual graphs correspond to higher-level nodes.

[0045] The learning of model 1022 in operation 1106 further includes performing learning on the corresponding graph generated to generate a plurality of high-level embeddings.

[0046] In embodiments including a plurality of embeddings, method 1100 further includes generating at least one synthetic examination data set based on the plurality of embeddings, where the plurality of input examination data sets do not include the synthetic examination data set. This is used, for example, to generate versions of imaging protocols for different scanners or patient attributes, to average a plurality of imaging protocols, to propose an imaging protocol less affected by patient motion, or to generate an equivalent protocol with faster acquisition.

[0047] There are numerous uses for model 1022. For example, model 1022 is used as follows:

[0048] · To detect deviations in the imaging protocol on the scanner and to early warn the user to prevent "protocol creep" (e.g., an imaging protocol that deviates significantly from the corresponding parent / child protocol) in response to the detection; · Provide an overview of the child and parent imaging protocols to facilitate protocol harmonization between scanner models and scanner vendors; and · Evaluate the statistical values (e.g., average protocol time, variability, etc.) of each imaging protocol and slice the data to obtain detailed insights regarding each imaging protocol (average time per scanner, average time per patient attribute, average time per operator attribute, etc.).

[0049] More generally, embodiments of the present invention provide important building blocks required for many applications built on model 1022 by generating a model 1022 that associates the parent protocol, child protocol, and imaging protocol within an examination data set.

[0050] Now that various embodiments of the present invention have been described at a high level, specific embodiments of the present invention will now be described in more detail.

[0051] As described above, "acquisition" refers to using a scanner to image a patient, thereby generating one or more images (also referred to as acquisition data). The term "acquisition data set" is used herein to refer to the following related to a particular acquisition: (1) a set of K technical parameters (and their values) used to perform the acquisition; (2) (optionally) a set of descriptors calculated from the data obtained by performing the acquisition (e.g., organ labeling or machine learning-based descriptors calculated from the data); and (3) (optionally) one or more non-technical parameters related to the acquisition and their associated values. Each of the multiple acquisitions is associated with its own corresponding set of acquisition descriptors.

[0052] Referring to FIG. 1, an example of a collected data set 100 according to an embodiment of the present invention is illustrated. In the specific example of FIG. 1, the collected data set 100 for MRI collection includes any one or more of the following parameters (including technical parameters and, optionally, one or more non-technical parameters) in addition to other parameters.

[0053] · Echo time (TE) · Repetition time (TR) · Inversion time · Number of slices (N slice ) · Resolution in the x, y, and z dimensions (res x , res x , res x ) · Flip angle · Phase encoding direction · Number of averaging · Number of phase encoding steps · Percentage of phase field of view · Percentage of sampling · Pixel bandwidth · Sequence name · Orientation matrix · Number of volumes · Diffusion sensitization parameter · Coil · With / without contrast agent

[0054] As a specific example, the collected data set for CT collection includes one or more of the following parameters (which may be in addition to any of the above parameters):

[0055] · kVp · mA · Rotation time · mAs · Pitch · Effective mAs · Reconstruction kernel · Scan field of view · Image thickness · CTDI · DLP

[0056] The specific parameters shown in FIG. 1 are merely examples and do not limit the present invention. Instead, any particular collection dataset can include any parameters in any combination, including parameters not shown in FIG. 1. For example, the parameters of the collection dataset may include parameters (and corresponding values) derived from HL7 and / or DICOM pixel data and metadata. Examples of descriptors derived from pixel data include the SNR of the image, a list of organs within the image, the presence of a contrast agent detected within the image, the type of MR weighting within the image (t1 weighting, t2 weighting, proton density weighting), or any other feature extracted from the pixel data, but these are merely examples. Further, the number of parameters K included in the collection dataset can have any value (i.e., the collection dataset can include any number of parameters). Each parameter of the collection dataset has a corresponding value, and that value changes over time. Each collection dataset represents a point within a K-dimensional space defined by the values of the K parameters of the collection dataset. References herein to generating, storing, or otherwise processing a "parameter", such as a parameter of a collection dataset or an examination dataset, include generating, storing, or otherwise processing the value of that parameter.

[0057] As used herein, the term "examination" refers to a set of N collections performed on a patient during a particular imaging session. As used herein, the term "examination dataset" refers to the following in relation to a particular examination: (1) a plurality of collection datasets representing the collections performed in the particular examination corresponding to the examination dataset, and (2) (optionally) one or more non-technical parameters related to the examination, and their associated values. Each of the plurality of examinations is associated with an examination dataset corresponding to itself.

[0058] Examples of non-technical parameters included in the inspection data set include constraints such as radiation dose reference levels (e.g., CTDI, DLP), specific absorption rate (SAR), maximum contrast agent dosage, etc. (i.e., maximum values for patient safety). Other examples of non-technical parameters included in the inspection data set include parameters related to the patient such as the patient's age and body mass index (BMI), whether the patient is sedated during collection, the patient's disease state, whether the patient is an in-patient or an outpatient, the urgency of the inspection, and whether the collection was performed using a contrast agent.

[0059] For example, the collections within an inspection are ordered. The collection data sets within the inspection data set are ordered (e.g., in the same order as the collections within the corresponding inspection). The order of the collections within an inspection represents the intended order and / or the actual order in which the collections are performed.

[0060] Referring to FIG. 2, an example of an inspection data set 200 according to an embodiment of the present invention is illustrated. As shown in FIG. 2, the inspection data set 200 includes N collection data sets, where N is an arbitrary number. The inspection data set 200 includes M non-technical parameters related to the inspection. The inspection data set 200 represents a point in a (K×N + M)-dimensional space defined by the values of the parameters included in the N collection data sets of the inspection data set 200.

[0061] Referring to FIG. 5, a system 500 for performing multiple collections in an inspection according to an embodiment of the present invention is shown. The system 500 includes a patient 502 and a scanner 504. As an example, the scanner 504 is shown as including a plurality of sub-protocols 506. In practice, even if the sub-protocols 506 are not stored in the scanner 504, the system 500 uses the sub-protocols 506. For example, the sub-protocols 506 are generated on the spot by the operator of the scanner. A user (not shown) adds a modification 508 to the sub-protocols 506 to generate an imaging protocol 510 when imaging the patient.

[0062] As part of the examination of patient 502, scanner 504 performs a plurality of acquisitions on patient 502. In each acquisition, scanner 504 images patient 502 and generates corresponding acquisition data (i.e., an image). FIG. 5 shows the resulting examination data set 512, which includes N acquisition data sets 514a - N corresponding to N acquisitions within the examination.

[0063] As used herein, the term "type of acquisition" refers to an acquisition data set having specific parameter values. For example, even if two different acquisition data sets have the same parameters, one or more of those parameters may have different values in the two acquisition data sets. In this case, the two acquisition data sets represent two different types of acquisitions. Note the following.

[0064] · Depending on the clinical condition, different types of acquisitions may be required or may benefit from different types of acquisitions to provide the best insight for each condition. · Using the same acquisition technique, acquisitions can be performed for different clinical conditions, but different technical parameters are used, for example, for the purpose of evaluating different tissue characteristics. · It may be desirable or necessary to repeat an acquisition, for example, if the patient moves and the image becomes unclear. · For example, it may be desirable or necessary to add a new acquisition, such as when there is suspicion of abnormal tissue and the new acquisition can help confirm or rule it out.

[0065] The imaging center or the radiology department typically constructs an ideal protocol (referred to herein as the "parent protocol") that describes a consensus on the need for imaging (i.e., a list of the collection data sets) for each imaging modality (e.g., MRI, CT) and for each organ. Each parent protocol (and general protocol) is defined by a list of the collection data sets. The consensus on the need for imaging varies depending on patient attributes (age, BMI, etc.). The following are non-limiting examples of parent protocols:

[0066] · Brain MR - tumor without contrast agent · Brain MR - pediatric tumor without contrast agent · Brain MR - demyelination without contrast agent · Brain MR - stroke with and without contrast agent · Knee MR · Shoulder MR · Brain CT · Brain CT - high speed · Knee CT

[0067] Referring to FIG. 3, an example of a parent protocol 300 according to an embodiment of the present invention is shown. As shown in FIG. 3, the parent protocol 300 includes N collection data sets, where N is an arbitrary number. The collection data sets within the parent protocol are, for example, ordered. The order of collection within the parent protocol represents the order in which the collection is intended and / or the order in which it is actually performed. When applying an imaging protocol derived from the parent protocol, the collection may be performed in an order different from the order specified in the parent protocol.

[0068] As described above, for various reasons, it may not be possible, practical, or desirable to implement the parent protocol on a particular scanner without modification. For example, a scanner may have technical characteristics or limitations that prevent it from performing imaging according to a particular parent protocol. As a result, it may be necessary or desirable to modify the parent protocol and perform the scan using the modified parent protocol. Such a modified parent protocol is referred to herein as a "child protocol."

[0069] The parent protocol is modified in various ways, such as one or more of the following methods, to generate a child protocol:

[0070] · Change the value of a parameter of the parent protocol from a first value to a second value, and generate a child protocol in which the parameter has the second value. · Remove a parameter from the parent protocol and generate a child protocol without the parameter. · Add a parameter not included in the parent protocol to the parent protocol to generate a child protocol that includes the added parameter. · Change the order of two or more collections in the parent protocol and generate a child protocol in which the collections are performed in an order different from that of the parent protocol.

[0071] The child protocol is often stored in the scanner (like a template) to shorten the preparation time and improve the reproducibility and uniformity of imaging. The child protocol stored in the scanner includes, for example, each collection dataset of the child protocol, each collection dataset includes one or more parameters, and each parameter may or may not include a value. The collection datasets of the child protocol stored in the scanner may or may not be ordered.

[0072] A unique name can be stored in association with each child protocol in the scanner to facilitate the display and selection of the child protocol. The scanner typically displays the names of the child protocols stored in the scanner and provides a user interface that allows the scanner operator to select, add, delete, and modify the child protocols. Before imaging a patient, the scanner operator can use such a scanner user interface to select a specific child protocol stored in the scanner.

[0073] However, it may be desirable or necessary for a scanner operator to make one or more modifications to the child protocol to generate a modified protocol (which is an example of what is referred to herein as an "imaged protocol" and which becomes the examination data set). Such modifications can include, for example, any of the modifications described above in connection with modifying the parent protocol to generate the child protocol. Examples of reasons for modifying the child protocol to generate an imaged protocol include accommodating the anatomical features of a patient, having to repeat one or more acquisitions within the child protocol, changing the acquisition order to prioritize some images over others for a radiologist, and the like.

[0074] FIG. 4 is a diagram showing the relationship of a parent protocol, its child protocols, and its imaged protocol (shown as an examination in FIG. 4) according to an embodiment of the present invention. As shown in FIG. 4, one parent protocol has a plurality of child protocols. Each of these child protocols has one or more of its own imaged protocols. Each of the child protocols corresponds to an individual scanner. As can be seen from this, the imaged protocols of all the child protocols are associated with and stored in the scanner associated with the child protocol.

[0075] Note that in existing systems, the relationship between each parent protocol and its child protocols and imaged protocols (represented in FIG. 4 by the lines connecting the parent protocol to its child protocols and the lines connecting the child protocols to their imaged protocols) is not stored in any scanner or elsewhere. As will be explained in more detail below, one advantage of embodiments of the present invention is that these relationships can be used to automatically identify, store, and update them over time.

[0076] More generally, embodiments of the present invention use one or more inspection data sets (e.g., input inspection data sets 1012a - N and / or new input inspection data set 1026) representing inspections actually performed using an imaging protocol to automatically generate a model (e.g., model 1022) representing the relationship (such as the relationship illustrated in FIG. 4) between a parent protocol, its child protocols, and the imaging protocol. Embodiments of the present invention repeatedly perform such analysis based on newly available inspection data sets and automatically update the model (and the relationships it represents) repeatedly over time.

[0077] One advantage of embodiments of the present invention is that by working in the reverse direction from an imaging protocol to an expression of a parent protocol, information about the parent protocol can be made explicit even if there was no previous explicit expression of that parent protocol. Embodiments of the present invention generate a human-readable output representing the resulting parent protocol, thereby facilitating understanding of the parent protocol and its descendant child and imaging protocols.

[0078] Embodiments of the present invention can generate such a model, for example, by the methods disclosed above in connection with FIGS. 10 and 11. In some embodiments, model generation includes, for example, any of the following. In the following, reference to performing a function related to inspection or collection includes performing such a function using appropriate data (e.g., inspection data sets and / or collection data sets).

[0079] In embodiments of the present invention, a metric is used to compare tests. Such a metric measures how far apart two tests are from each other. One of the challenges in developing such a metric is that the two tests may include different numbers of collections. If each test is considered as a separate point in a large-dimensional space, the metric must measure the distance between points in spaces of different dimensions. In embodiments of the present invention, before calculating the distance between the embedded representations, the embeddings of the two test datasets are calculated and those test datasets are converted into representations of a fixed size. Embodiments of the present invention alternatively use a custom distance metric designed to handle tests with different numbers of collections, as described below.

[0080] First, consider the distance d(a1, a2) between two collections a1 and a2. Assume that both collections a1 and a2 have the same number K of parameters. In embodiments of the present invention, for the distance d(a1, a2), for example, any of the following is used:

[0081] · Simple L1 or L2 norm; · Relative distance between each element of a1 and a2 to account for differences in scale of each element; · More complex metrics derived from, for example, a K-dimensional embedding learned from the data.

[0082] In embodiments of the present invention, the distance D(e1, e2) between two tests e1 and e2 is calculated in various ways as follows using the distance d(a1, a2). Let there be N 1 tests included in test e1 and N 2 collections included in test e2, where N 1 and N 2 may or may not be equal to each other.

[0083] e1[a 1,I within (a 1 ..a N1 )] for each collection a 1,I , embodiments of the present invention calculate, for a 1,I and e2(a 2,1..a 2,N2 ) Evaluate the minimum distance between all collections of e1. This evaluation is repeated for each collection of e1, and the resulting minimum distances are aggregated (e.g., summed). That is,

[0084] TIFF2025517107000002.tif9167

[0085] [0047b] Also, it is possible to aggregate with non-technical parameters of e1 and e2. If d^NT(d1, d2) is the distance between non-technical parameters,

[0086] TIFF2025517107000003.tif10162

[0087] To make the metric symmetric and ensure D(e1, e2) = D(e2, e1), the following metrics are used:

[0088] TIFF2025517107000004.tif9167

[0089] As a result of testing embodiments of the present invention, it was shown that the mismatches in the acquisition descriptor do not all have the same weight within the metric. For example, mismatches in the TR (repetition time) are often not very important, while mismatches in the contrast are often of great importance. For example, it is important not to associate an image without contrast with an image with contrast. As a result, it may be useful to assign a relatively high weight to the contrast parameter and a relatively low weight to the TR parameter in the distance metric. In embodiments of the present invention, such weights are automatically assigned in the following manner.

[0090] In embodiments of the present invention, as follows, a weighted sum is used to adjust the importance (weight) of different features (parameters):

[0091] TIFF2025517107000005.tif17144

[0092] One approach that can be used in embodiments of the present invention is to estimate the weights using implicit clustering metrics that can measure clustering performance without knowing the true labels. The goal is to learn the weights to adjust the inspection metrics (i.e., to focus on important parameters) and optimize several implicit clustering metrics that describe the clustering performance (e.g., weights indicating cluster separability, silhouette coefficient, Calinski-Harabasz coefficient, etc.). In other words, the goal is to find the weights w to maximize. TIFF2025517107000006.tif9145

[0093] One way to do this is to use an optimization algorithm that does not require an explicit formulation of the derivative, such as the Bound Optimization By Quadratic Approximation (BOBYQA) algorithm of the NLopt library. That is, use BOBYQA as follows:

[0094] TIFF2025517107000007.tif26156

[0095] One approach that can be used in embodiments of the present invention is to estimate the weights using a manually labeled dataset and explicit clustering metrics. Assuming the true labels are known, the goal is to learn the weights (i.e., adjust the inspection metrics) that lead to clustering as close as possible to the correct data (e.g., Rand index - accuracy or mutual information). One example of this is to use an optimization algorithm that does not require an explicit formulation of the derivative, such as BOBYQA. That is, use BOBYQA as follows:

[0096] TIFF2025517107000008.tif35152

[0097] Once such an inspection distance metric is developed, it can be used to identify a child protocol and / or a parent protocol from a set of inspections in any of the following ways. The imaging protocol can be considered an observation of an unknown child protocol (forward model), and then embodiments of the present invention can invert the forward model to recover the unknown underlying child protocol.

[0098] Unsupervised learning is used to learn a set of child protocols based on an inspection dataset using a selected inspection distance metric. For example, any of various clustering algorithms can be used to identify groups of inspections that are similar to each other with respect to the distance metric between inspections, using the selected inspection distance metric and the set of inspections.

[0099] Alternatively, for example, supervised learning is used to learn a set of child protocols based on a set of inspections using a selected inspection distance metric. For example, if each of the inspection datasets is labeled with a protocol name, supervised learning uses the selected inspection distance metric to train a classifier that automatically recognizes the child protocols.

[0100] Alternatively, for example, semi-supervised learning is used to learn a set of child protocols based on a set of inspections using a selected inspection distance metric. For example, unsupervised clustering is used to generate an initial set of child protocols. Next, the user manually corrects the resulting labels, and the manually corrected labels are used to retrain and improve the classifier, thereby improving the classifier. This process can be repeated any number of times to continuously improve the classifier.

[0101] Alternatively, for example, self-supervised learning is used to learn a set of child protocols based on a set of inspections using a selected inspection distance metric. For example, reinforcement learning is used to learn a set of child protocols based on a set of inspections using a selected inspection distance metric.

[0102] Regardless of the method used, the result is a set of child protocols corresponding to the tests used to learn the child protocols.

[0103] It is desirable to update the model (e.g., classifier) developed above over time. For example, the parent protocol, child protocols, and imaging protocol can evolve over time. As a result, if the model does not adapt over time to reflect such evolution, the model will become increasingly inaccurate over time. A new imaging protocol introduced after model generation may initially be considered a deviation from the model, but using semi-supervised learning or self-supervised learning, new child protocols are learned from the new imaging protocol.

[0104] In the above description, how embodiments of the present invention are used to learn child protocols based on an imaging protocol has been described. Embodiments of the present invention are used to learn a parent protocol based on the child protocol and / or imaging protocol. In some embodiments of the present invention, the child protocol is first learned based on the imaging protocol by any of the methods disclosed above, and then one or more parent protocols are learned based on the obtained child protocol (optionally also based on the imaging protocol). This is an example of learning a first set of protocol tags and learning a second set of protocol tags based on the first set of protocol tags, as disclosed elsewhere in this specification.

[0105] Embodiments of the present invention cluster a plurality of parent protocols using any of the techniques disclosed above in relation to child protocols. For example, a child protocol (after being learned by any of the methods disclosed above) is treated as an observation of an unknown parent protocol, and then, in embodiments of the present invention, clustering is used among representative instances of each child protocol to group child protocols that are close to each other. Each resulting cluster corresponds to a different parent protocol, all child protocols within a particular cluster are children of the same parent protocol, and any two child protocols belonging to different clusters are children of different parent protocols.

[0106] In embodiments of the present invention, the name of a protocol (such as a parent protocol, a child protocol, or an imaging protocol) is identified or generated as follows:

[0107] · The organ of interest (e.g., MR-brain / MR-neck / MR-abdomen) is identified by labeling the organ using deep learning and / or other techniques for some or all of the images during the examination. · Whether or not there was contrast is determined by using deep learning and / or other techniques for some or all of the images during the examination and / or from metadata (e.g., DICOM and / or metadata from a scheduling system). · A database of known labeled protocols (obtainable from multiple facilities) is used to identify the most likely protocol name.

[0108] Embodiments of the present invention evaluate a data set of examinations associated with each child protocol (i.e., within each cluster) to estimate the non-parametric statistical distribution of the examinations associated with each child protocol. Thereby, embodiments of the present invention detect deviations from the normal variability in the protocol. Embodiments of the present invention generate a distribution of the normal variability within the protocol. Embodiments of the present invention utilize that distribution to identify collections and examinations that deviate from the range of normal variability.

[0109] Embodiments of the present invention also identify similar child protocols and their associated parent protocols, thereby enabling the same protocol to be identified and compared between scanners. In other words, embodiments of the present invention identify multiple different child protocols on multiple scanners and determine that all of those child protocols are children of the same parent protocol. Once this is done, embodiments of the present invention can identify differences between different child protocols belonging to the same parent protocol and reconcile the child protocols of the same parent protocol between scanners.

[0110] Different protocols each have their own unique challenges and complexities. For example, when considering the effectiveness of imaging (the ratio of active scan time), in an examination using a contrast agent, it is normal for the effectiveness to be low because the patient needs to leave the scanner once and then return to the scanner again. Some protocols are inherently more difficult to execute. To "compare apples to apples", embodiments of the present invention individually measure statistics for each pair of child / parent protocols and compare the instances of those child / parent protocols (i.e., the imaging protocols that are descendants of the child / parent protocols) to their respective child / parent protocols.

[0111] Once the child and parent protocols are learned, embodiments of the present invention calculate, for each inspection, a heatmap representing the deviation of parameters (i.e., out-of-distribution parameters) from the child protocol and / or the parent protocol. Such heatmaps are generated at the inspection level and / or the collection level. Such heatmaps are generated for primary (direct) parameters (e.g., DICOM metadata or RIS) and / or derived (computed) parameters (e.g., period or repetition). In such a heatmap, each parameter is represented in a graphical representation (e.g., a circle), and the area of the graphical representation is a function (e.g., equal or proportional) according to the percentile of the value of the parameter in the statistical distribution previously calculated for the corresponding child protocol or parent protocol. Embodiments of the present invention generate a visual output representing the heatmap and provide it to the user to facilitate understanding and analysis.

[0112] More generally, such a heatmap represents factors (e.g., parameters of collection in an inspection or the number of collections) that contributed to the protocol being classified as a deviation, and for each factor, can take any form as long as it assigns a value as a function representing the degree to which that factor contributed to the protocol being classified as a deviation. A graphical heatmap where each factor is represented as a shape (e.g., a circle) and its area shows as a function the degree of contribution of that factor to the protocol being classified as a deviation is just one example of this. Another example is a rank list where multiple factors are listed in ascending or descending order of the degree to which each factor contributed to the protocol being classified as a deviation.

[0113] Consider the distribution of the degree to which factors contribute to a protocol being classified as a deviation. Regardless of the form the heatmap takes, the order of the factors within the heatmap is a function of the order of the factors within the distribution. For example, the size of the shapes representing the factors within a graphical heatmap is ordered as a function of the order of the factors within the distribution (e.g., such that the size decreases or increases). As another example, the order of the factors within a rank list may be a function of the order of the factors within the distribution (e.g., the same order or the reverse order).

[0114] Other embodiments of the present invention include techniques for converting a variable-sized vector representing each test dataset into a fixed-sized representation by calculating an embedding. The embedding can be used to perform various functions such as calculating distances between tests, separating tests, labeling tests, predicting values from tests, generating new tests, and the like.

[0115] One way to create an embedding is to use graph learning. Generally, a graph G is composed of a set of nodes (V) and a set of edges (E) between those nodes, and is represented as G = (V, E). As is well known to those skilled in the art, graphs are represented in various ways. For example, the edges within a graph can be either directed or undirected.

[0116] Graphs are particularly useful and have unique characteristics in their ability to represent unstructured complex data and systems. Also, for example, they enable the description of relationships between entities such as social networks (nodes = a person, edges = whether these people are "friends"), chemical compounds, drug interactions, knowledge concepts, interconnected devices, and the like.

[0117] The size and topology of a graph can be arbitrarily set. However, one problem is that there is no fixed node order. This makes it difficult to apply conventional machine learning concepts to graphs.

[0118] Each node and each edge in the graph has one or more corresponding features associated therewith. This is useful for encoding information and / or representing relationships. As will be described in more detail below, in embodiments of the present invention, node / edge features are used by graph learning techniques and important properties of the features can be internalized.

[0119] As described above, different inspections (and different protocols) involve different numbers of collections. As a result, different inspection datasets and protocols are of various sizes and, in most learning tasks, become "unstructured" data for which graph learning techniques are more suitable than conventional machine learning techniques.

[0120] Embodiments of the present invention include techniques for representing an inspection or protocol as a graph and learning from such graph data. More specifically, in embodiments of the present invention, an inspection or protocol can be represented as a graph, and each collection becomes a node of the graph. The nodes have "node features" and are used to assign information regarding the corresponding collection to each node. For example, a feature vector representing a collection (see FIG. 1) is an example of such a node feature and is assigned to the node corresponding to the collection. In the example shown in FIG. 8, graph 8 representing an inspection or protocol including five collections includes five nodes representing those collections, and the feature vector of each collection is assigned to the corresponding node.

[0121] Embodiments of the present invention encode relationships between collections on edges in a graph. For example, an edge between two nodes representing two corresponding collections encodes the relationship between those two collections. Examples of such relationships include the distance (or similarity) between two collections (for various methods of calculating such distances, see the above description). Multiple edges encode multiple such relationships (e.g., distances) between the collections corresponding to the nodes connected by the edges. In the example of graph 800 in FIG. 8, the edges connect the following node pairs: (1) nodes representing collection numbers 1 and 2, (2) nodes representing collection numbers 1 and 5, (3) nodes representing collection numbers 3 and 4, (4) nodes representing collection numbers 3 and 5, (5) nodes representing collection numbers 4 and 5. These specific edges are shown as examples for illustrative purposes only.

[0122] In embodiments of the present invention, for a graph including such nodes and edges, the functions disclosed herein are performed, or the graph is first binarized before performing such functions. Such binarization can be performed, for example, by applying a threshold to all edges and masking (e.g., setting the given information to zero) edges for which the given information (e.g., distance) does not meet the threshold (e.g., does not exceed the threshold). The purpose of such edge masking is to prevent information from being shared between these nodes during the graph convolution operation involved in the learning of a graph convolutional network, which is part of model selection.

[0123] In an embodiment of the present invention, one or more "edge features" are assigned to any edge in order to encode information regarding the relationship(s) represented by the edge. For example, the edge feature encodes a label associated with the relationship represented by the edge. As a specific example, the edge feature can be used to identify that two collections connected by the edge are a repetition of collection by movement, a repetition of collection by another artifact, or the same type of collection (e.g., a high-speed version and a low-speed version of the same collection).

[0124] In an embodiment of the present invention, one or more "graph features" are also assigned to the entire graph. Examples of such graph features include information regarding the examination represented by the graph, such as the above-described M non-technical parameters (e.g., patient age, body mass index, sedated / non-sedated). Embeddings calculated from text by natural language processing (NLP) input and / or NLP techniques can also be used as graph features. This includes the text description of the protocol, the text description of each collection, or vector embeddings representing word / sentence embeddings.

[0125] Referring to FIG. 6, a flowchart of a method 600 performed by one embodiment of the present invention to generate a graph of the type described above based on a set of collections in an examination or protocol is shown. Method 600 uses any of the techniques disclosed herein to generate a feature vector for each of the N collections in the examination or protocol based on imaging parameters and / or other data sources (FIG. 6, operation 602). Method 600 adds the feature vector of each obtained collection to the corresponding node in the graph (FIG. 6, operation 604).

[0126] For each pair of the feature vectors of the nodes, method 600 calculates an index (e.g., distance or similarity) based on the pair of feature vectors such as cosine similarity, L1 norm, or L2 norm, and generates a graph adjacency matrix in which each cell at positions i, j contains the index for the pair of nodes i, j (Figure 6, operation 606). The adjacency matrix of the graph is an example of the representation of the graph.

[0127] Method 600 defines the edges of the graph using the adjacency matrix (Figure 6, operation 608) and optionally creates a binary graph by first thresholding the adjacency matrix. Method 600 assigns edge features to the edges to encode the information between the pairs of the collection (Figure 6, operation 610). Method 600 assigns any of the non-technical inspection level descriptors disclosed herein to the graph as "graph features" (Figure 6, operation 612).

[0128] As described above, it is difficult to apply conventional machine learning to unstructured data. Therefore, in the embodiments of the present invention, inspections and protocols are represented as graphs, and graph learning is utilized to analyze this type of data. "Graph learning" means applying machine learning to graphs. Graph learning can be applied to perform various functions such as node classification, prediction of relationships between nodes (i.e., the existence of edges between nodes), and embedding of the graph into another representation that reveals relevant characteristics of the graph, and this other representation is used to perform functions such as graph classification and prediction. Although details will be described later, in graph learning, the graph is mapped to a manifold, and an embedding of the graph is generated such that similar graphs are embedded close to each other.

[0129] When using conventional techniques to extract valuable information from graph data, a common approach is to first manually design features. Another approach is to automatically learn features from the data. Graph learning automatically generates representative vectors, hereinafter referred to as "embeddings" in this specification, that contain meaningful information. For example, in an embodiment, embeddings corresponding to individual nodes within a graph and / or embeddings corresponding to the entire graph (or any subgraph thereof) are generated. As this implies, an embedding representing a particular unit (e.g., a node, subgraph, or graph) may contain information derived from that particular unit, may not contain all the information contained in that particular unit, or may contain information contained in neighboring units of that particular unit. One advantage of mapping data into an embedding space is that the similarity between data is reflected in the newly learned manifold. As a result, graphs, subgraphs, and nodes with similar characteristics will have embeddings that are close to each other in the space.

[0130] Embodiments of the present invention can generate embeddings in any of a variety of ways. For example, embodiments of the present invention can generate embeddings using unsupervised learning or supervised learning. The choice of learning method can be made, for example, based on the specific downstream task to be performed using the embeddings. For example, if the downstream task depends on performing a specific classification, a supervised learning method can be used to generate the embeddings. Alternatively, for example, if the downstream application discovers patterns or correlations between data points, an unsupervised learning method can also be used to generate the embeddings.

[0131] Regardless of the type of learning method used to generate the embeddings, a network such as an encoder is used, which can aggregate information from the connected nodes into a single vector to generate the embeddings. Embedding vectors are obtained for each node in the graph, and those vectors are transformed into graph embeddings using any of a variety of pooling strategies.

[0132] When using an unsupervised learning method to generate embeddings, the graph neural network uses a decoder-type network that attempts to reconstruct the graph's adjacency matrix from the output of the encoder, i.e., the embeddings. In such cases, the embeddings are optimized by minimizing the loss between the original graph and the reconstructed graph. To ensure that the embedding vectors hold information specific to each node, a loss function that quantifies the similarity of the nodes between the original space and the reconstructed space is used.

[0133] When using a supervised learning method to generate embeddings, the decoder network is replaced with a neural network that converts the embedding vectors into a specific output, e.g., a target vector representing a meaningful label / class. The embeddings are optimized by minimizing the loss between the predicted output and the target output.

[0134] It is also possible to combine and use both unsupervised and supervised learning methods. When a hybrid learning method is used to generate embeddings, the generated embeddings are fed in parallel to the decoder network and the neural network, or a set of neural networks. The embeddings are optimized by combining the loss between the predicted output and the target output (by the supervised approach) and the loss between the original space and the reconstructed space (by the unsupervised approach).

[0135] Embodiments of the present invention use the generated embeddings in combination with embeddings obtained from natural language processing methods for further downstream processing tasks. Referring to FIG. 6, when inspection-level protocol text is available, embodiments of the present invention use a sequence-to-sequence model or a transformer model to generate new text-level embeddings. These embeddings are used in combination with the graph embeddings for downstream tasks.

[0136] When embodiments of the present invention generate one or more embeddings, these embeddings representing an examination or protocol are used by one or more downstream applications to perform various functions such as:

[0137] · Use embeddings for the separation task. Such tasks are related to using machine learning algorithms to separate embeddings for the purpose of distinguishing / identifying distinct protocols or sub - protocols (e.g., "learning a facility's protocol from data"). Such separation is performed, for example, at the examination / protocol level or at the scanner level. · Use embeddings for the classification / labeling task. Such tasks are related to assigning (or predicting) specific descriptors to the embeddings (e.g., predicting a class from an embedding such as prediction of a body part, CPT codes for insurance purposes, protocol names (e.g., mapping to different dictionaries, such as Radlex), involved collections, MRI scanners used, medical departments that requested the protocol, determination of whether the protocol meets the "appropriateness criteria" of the American College of Radiology, or recommendations of specific scanners for specific protocols and indications, etc.). · Use embeddings for the regression task. Such tasks are related to predicting specific values from the embeddings, and examples include the duration of a protocol, room utilization efficiency, slot utilization efficiency, protocol efficiency, required patient preparation time, radiologist processing time, time taken by a radiologist to read an examination, time from order to examination, or patient age, etc. · Use embeddings for the generation / recommendation task. Such tasks correspond to the inversion of the embedding process for the purpose of generating recommendation items or generating new protocols or examinations, or the combination of multiple tasks. Examples include generating equivalent protocols for different scanners, generating alternative protocols for outlier cases (e.g., newborns, obese patients, patients with implants), standardizing protocols across all scanners, generating high - speed / low - speed versions of protocols, combining protocols, or generating protocol names, etc.

[0138] Examples of separation tasks include the following:

[0139] · Protocol identification with a specific scanner. In an embodiment of the present invention, a clustering algorithm is used for embedding in an unsupervised manner to automatically learn groups of inspections / protocols (also referred to herein as scanner imaging protocols (or sub-protocols)) having similar characteristics (occupying the same embedding space). · Protocol identification across multiple scanners. In an embodiment of the present invention, the embeddings are compared between scanners to learn the parent protocol. · Out-of-distribution and new protocols. Using specific clusters (correlated with the parent and sub-protocols described above) resulting from the embedding distribution of the inspections, various algorithms are used to determine to what extent a new given inspection can be confident that it belongs to a certain cluster. This is used, for example, to determine inspections that may be inappropriately labeled and / or to identify protocol deviations (e.g., an inspection technician has changed some parameters). This is also used, for example, to identify that a new protocol has been created. For example, if the determination of "out-of-distribution" instances within a cluster increases, this may be the result of a new protocol being created and needs to be identified. · Protocol unification across multiple scanners. Having both a parent protocol and a sub-protocol allows the differences between scanners in the same protocol to be evaluated, which helps in protocol unification.

[0140] Examples of classification / labeling tasks include the following:

[0141] · Inspection / protocol name. This includes associating a name with an inspection / protocol based on its characteristics. · Assign / predict CPT codes to the inspection / protocol. This includes assigning a CPT code to an inspection / protocol based on the characteristics of the inspection / protocol. · Add descriptors. This includes identifying and adding specific descriptors that uniquely identify the clusters, such as anatomical descriptors and contrast agent descriptors.

[0142] The regression task can construct a model that predicts continuous values by combining the embeddings representing the protocol / examination with a regression model, and examples include any one or more of the following: protocol / examination time, protocol / examination preparation time, radiologist's protocol examination reading time, image quality, and diagnostic values.

[0143] The generation / recommendation task is used to reverse the process of encoding graph information into embeddings and generate the (approximate) content of the information that was initially encoded into the embeddings. For example, at the node level, the generation / recommendation task generates specific collection parameters that have been encoded. Such information is propagated to the graph level and then used to generate the entire protocol for each examination. Examples of generation / recommendation tasks include the following:

[0144] · Standard protocol - collection and generation of its parameters. This includes generating the corresponding graph from the embeddings. · Generate a unified version of the same protocol on another scanner. · Smart recommendations. By combining various outputs from the tasks described above, the learning pipeline is specialized to make specific recommendations. If the embeddings are trained to aggregate valuable information at the examination / protocol level, they serve as a starting point for appropriate dimensionality reduction to perform simulations and determine the optimal recommendations for specific tasks.

[0145] For example, as shown in FIG. 9, embodiments of the present invention include techniques for representing a set of examinations or a set of protocols as a graph, and techniques for learning from such graph data, such as solving the problem of learning from child protocols to parent protocols or learning across different hospitals.

[0146] For example, FIG. 7 is a flowchart of a method 700 for converting a set of tests or protocols into graph data according to an embodiment of the present invention. More specifically, embodiments of the present invention can represent each protocol (or test) in the set as a corresponding node in the graph. More specifically, method 700 first generates an embedding vector for each of the K protocols or tests using any of the techniques disclosed herein (FIG. 7, operation 702). Method 700 also creates a graph with N nodes and assigns each embedding vector to the corresponding node in the graph (FIG. 7, operation 704). In this embodiment, the embedding vector is an example of a "node feature" and provides information about the protocol (e.g., each test) corresponding to each node.

[0147] Embodiments of the present invention encode the relationships between protocols / tests at the edges in the graph. For example, an edge between two nodes representing two corresponding protocols can encode the relationship between those two protocols. Examples of such relationships include the distance (or similarity) between two protocols (the distance between the embedding vectors set as nodes).

[0148] For example, method 700 calculates an index for each pair of node embedding vectors based on the pair of embedding vectors (FIG. 7, operation 706). Method 700 generates a graph adjacency matrix, where each cell at position i,j in this graph adjacency matrix contains the index corresponding to the pair of nodes i,j. The adjacency matrix of the graph is an example of a representation of the graph.

[0149] Method 700 defines the edges of a graph using an adjacency matrix (Figure 7, operation 708) and optionally creates a binary graph by first thresholding the adjacency matrix. Method 700 assigns one or more "edge features" to any edge (Figure 7, operation 710) in order to encode information about the relationship(s) represented by the edge(s). For example, an edge feature encodes a label associated with the relationship represented by the edge. As a specific example, edge features can be used to identify whether two protocols / tests connected by an edge are from the same body part (e.g., nerve, chest, lower limb) or from the same facility (these are just two examples).

[0150] Method 700 assigns one or more other high-level descriptors as "graph features" to the entire graph (Figure 7, operation 712). Examples of such graph features include information about the set of protocols / tests represented in the graph, such as a facility or a body part (these are just two examples).

[0151] Figure 9 shows an example of a system 900 in which each of a plurality of graphs represents a corresponding test. For illustration purposes, on the left side of Figure 9, a plurality of low-level graphs (test instances) for which embedding vectors are calculated are shown, and on the right side of Figure 9, one of the plurality of graphs for encoding a set of tests is shown. System 900 creates embeddings for a plurality of tests based on the plurality of graphs using any of the techniques disclosed herein.

[0152] In some embodiments, the techniques described herein include a method executed by at least one computer processor that executes computer program instructions stored on at least one non-transitory computer-readable medium, the method including: (A) receiving a plurality of input examination data sets created by performing a plurality of imaging examinations on at least one patient with at least one scanner, each of the plurality of input examination data sets including a plurality of collection data sets, each collection data set A in the plurality of collection data sets including a corresponding plurality of values of a plurality of technical parameters used to perform the collection that generated collection data set A; and (B) learning a model of an imaging protocol based on the plurality of input examination data sets.

[0153] The model is capable of capturing common features across the plurality of input examination data sets. The model re-groups examination data sets having common features within the plurality of input examination data sets under a common protocol tag, and learning the model includes generating a plurality of protocol tags.

[0154] The method may further include: (C) receiving a plurality of new input examination data sets; and (D) using the model to generate a plurality of protocol tags, each of the plurality of protocol tags describing a corresponding set of common features within the plurality of new input examination data sets.

[0155] Learning may include supervised learning and / or unsupervised learning.

[0156] Each tag T within the plurality of protocol tags describes a corresponding set of examination data sets within the plurality of input examination data sets, and the set of examination data sets corresponding to tag T includes a plurality of collection data sets that share a corresponding set of common features within the plurality of input examination data sets.

[0157] Learning may include the following: (B)(1) learning a first set of protocol tags from a plurality of input inspection data sets; and (B)(2) learning a second set of protocol tags from the plurality of input inspection data sets and the first set of protocol tags, wherein the second set of protocol tags describes a corresponding set of common features of a corresponding plurality of protocol tags within the first set of protocol tags.

[0158] When N = 1, learning may include the following: (B)(1) learning an Nth set of protocol tags from a plurality of input inspection data sets and a set (or sets) of protocol tags previously learned at N ≧ 1, wherein the Nth set of protocol tags describes a corresponding set of common features of a corresponding plurality of (N - 1)-level protocol tags; (B)(2) determining whether an end criterion is met; (B)(3) if the end criterion is met, ending the learning; (B)(4) if the end criterion is not met, (B)(4)(a) incrementing N, and (B)(4)(b) returning to (B)(1) above.

[0159] Operation (B) includes learning a classifier or clustering algorithm for identifying characteristics of protocol tags based on a plurality of input inspection data sets, and learning of the model includes learning a plurality of protocol tags using the classifier or clustering algorithm.

[0160] The method may further include the following. (C) For each of the plurality of protocol tags, identifying a corresponding organ of interest, thereby identifying a plurality of organs of interest corresponding to the plurality of protocol tags; and (D) for each of the plurality of protocol tags, identifying a label.

[0161] Operation (C) includes identifying a plurality of organs of interest corresponding to a plurality of protocol tags by applying learning to a plurality of images within the plurality of input inspection data sets.

[0162] Operation (D) includes identifying the label associated with each of the plurality of protocol tags based on a set of labeled inspection data sets.

[0163] The plurality of protocol tags include a plurality of fixed-size embeddings.

[0164] Operation (A) may include the following: (A)(1) For each input inspection data set in the plurality of input inspection data sets, generating a corresponding graph, including the following: (A)(1)(a) For each of the plurality of nodes in the graph corresponding to the plurality of collection data sets within the input inspection data set, saving information regarding the collection corresponding to the node, and (A)(1)(b) For each pair of nodes in the corresponding graph, generating and saving an edge within the graph representing information regarding the relationship between the pair of nodes, thereby generating a plurality of graphs corresponding to the plurality of input inspection data sets.

[0165] Operation (B) includes performing learning based on the corresponding plurality of graphs and generating corresponding plurality of embeddings representing the plurality of protocol tags.

[0166] After performing (A) and (B), the method may further include: (C) receiving a plurality of new input inspection data sets created by performing a plurality of new imaging inspections on at least one patient with at least one scanner, and (D) learning an updated version of the model of the imaging protocol based on the plurality of new input inspection data sets.

[0167] The method may further include the following. (C) For a corresponding plurality of embeddings, generating a graph corresponding to the corresponding plurality of embeddings, including: (C)(1)(a) For each node among a plurality of nodes in the graph corresponding to the plurality of embeddings, storing information regarding the embedding corresponding to the node, and (C)(1)(b) For each pair of nodes in the corresponding graph, generating and storing in the corresponding graph an edge representing information regarding the relationship between the pair of embeddings.

[0168] Operation (B) can further include performing learning on the corresponding graph generated in (C) to generate a plurality of high-level embeddings.

[0169] The method may further include the following. (C) Generating at least one synthetic test dataset based on a plurality of embeddings, wherein the plurality of input test datasets do not include the synthetic test dataset.

[0170] In some embodiments, the technology described herein includes a system including at least one non-transitory computer-readable medium storing computer program instructions, the computer program instructions being executable by at least one computer processor to perform a method, the method including: (A) receiving a plurality of input test datasets created by performing a plurality of imaging examinations on at least one patient with at least one scanner, each of the plurality of input test datasets including a plurality of collection datasets, each collection dataset A in the plurality of collection datasets including a corresponding plurality of values of a plurality of technical parameters used to perform the collection that generated collection dataset A, and (B) learning a model of an imaging protocol based on the plurality of input test datasets.

[0171] The above describes the present invention with respect to specific embodiments. However, the above embodiments are provided only as examples and do not limit or define the scope of the present invention. Various other embodiments, including but not limited to the following, are within the scope of the present invention. For example, the elements and components described herein can be divided into additional components or combined into fewer components to perform the same function.

[0172] Any of the functions disclosed herein can be implemented using means for performing those functions. Such means include, but are not limited to, any of the components disclosed herein, such as computer-related components described below.

[0173] The above-described technology can be implemented, for example, in hardware, one or more computer programs substantially stored on one or more computer-readable media, firmware, or any combination thereof. The above technology can be implemented in one or more computer programs that are executed (or executable by a programmable computer) on a programmable computer that includes any of the following number of any combinations: a processor, a storage medium readable and / or writable by the processor (including, for example, volatile and non-volatile memories and / or storage elements), an input device, and an output device. The program code can be applied to data input using the input device, perform the described functions, and generate output using the output device.

[0174] Embodiments of the present invention include features that can only be implemented or realized by using one or more computers, computer processors, and / or other elements of a computer system. Such functions are impossible or not practical to implement mentally and / or manually. For example, embodiments of the present invention can apply deep learning to learn child protocols and parent protocols. Such functions are inherently based on computer technology and cannot be performed mentally or manually.

[0175] In this specification, claims that affirmatively require a computer, processor, memory, or similar computer-related element are intended to require such an element, and should not be construed as if such an element does not exist in or is not required by such a claim. Such claims are not intended to cover methods and / or systems lacking the described computer-related element and should not be so construed. For example, a method claim in this specification that describes that the claimed method is performed by a computer, processor, memory, and / or similar computer-related element is intended to include only methods performed by the described computer-related element(s) and should be so construed only. Such a method claim should not be construed to include, for example, a method performed mentally or manually (e.g., using a pencil and paper). Similarly, in this specification, a product claim that describes that the claimed product includes a computer, processor, memory, and / or similar computer-related element is intended to include only products that include the described computer-related element(s) and should be so construed only. Such a product claim should not be construed to include, for example, a product that does not include the described computer-related element(s).

[0176] Each computer program within the scope of the present invention below can be implemented in any programming language such as assembly language, machine language, high-level procedural programming language, or object-oriented programming language. The programming language may be, for example, a compiled or interpreted programming language.

[0177] Such computer programs can be implemented in a computer program product specifically embodied in a machine-readable storage device for execution by a computer processor. The steps of the method of the present invention can be executed by one or more computer processors executing a program embodied on a computer-readable medium, operating on inputs and generating outputs to perform the functions of the present invention. Suitable processors include, by way of example, general-purpose and special-purpose microprocessors. Generally, a processor receives (reads) instructions and data from memory (such as read-only memory and random access memory), and writes (stores) instructions and data to memory. Storage devices suitable for specifically embodying computer program instructions and data include, for example, semiconductor memory devices such as EPROM, EEPROM, flash memory devices, magnetic disks such as internal hard disks and removable disks, magneto-optical disks, and all forms of non-volatile memory such as CD-ROM. Any of the above may be supplemented or incorporated by a specially designed ASIC (application-specific integrated circuit) or FPGA (field-programmable gate array). Computers generally can also receive (read) programs and data from non-transitory computer-readable storage media such as internal disks (not shown) and removable disks, or write (store) programs and data to non-transitory computer-readable storage media. These elements can also be found in conventional desktop computers and workstations, or other computers suitable for executing the computer programs for implementing the methods described herein, and can be used in combination with digital printing engines, marking engines, display monitors, or other raster output devices that can generate color or grayscale pixels on paper, film, display screens, or other output media.

[0178] The data disclosed in this specification can be implemented, for example, in one or more data structures specifically stored on a non-transitory computer-readable medium. In embodiments of the present invention, such data can be stored in such data structure(s) and read from such data structure(s).

[0179] Any step or operation disclosed herein as being performed by or executable by a computer or other machine is automatically performed by the computer or other machine, whether or not explicitly disclosed herein. Steps or operations that are automatically performed are performed only by a computer or other machine without human intervention. Steps or operations that are automatically performed operate, for example, based only on input received from a computer or other machine rather than from a human. Steps or operations that are automatically performed are initiated, for example, by a signal received from a computer or other machine rather than from a human. Steps or operations that are automatically performed provide output to, for example, a computer or other machine rather than to a human.

[0180] The terms "A or B", "at least one of A or / and B", "at least one of A and B", "at least one of A or B", or "one or more of A or / and B" as used in various embodiments of the present disclosure include any combination of all the words listed therewith. For example, "A or B", "at least one of A and B" or "at least one of A or B" may mean: (1) including at least one A, (2) including at least one B, (3) including either A or B, or (4) including both at least one A and at least one B.

Description of Reference Numerals

[0181] 500, 1000 systems 502, 1002 patients 504 and 1004 Scanners 1020 Learning Engine 1022 Model 1026 New Input Inspection Dataset 1028 Protocol Tag

Claims

1. A method executed by at least one computer processor that executes computer program instructions stored on at least one non-transitory computer-readable medium, the method comprising: (A) receiving a plurality of input examination data sets created by performing a plurality of imaging examinations on at least one patient with at least one scanner, each of the plurality of input examination data sets including a plurality of acquisition data sets, each acquisition data set A within the plurality of acquisition data sets including a corresponding plurality of values of a plurality of technical parameters used to perform an acquisition that generated the acquisition data set A; and (B) learning a model of an imaging protocol based on the plurality of input examination data sets.

2. The method according to claim 1, wherein the model captures common features across the plurality of input examination data sets.

3. The method according to claim 2, wherein the model re-groups examination data sets having common features within the plurality of input examination data sets under a common protocol tag, and learning of the model includes generating a plurality of protocol tags.

4. (C) receiving a plurality of new input examination data sets; and (D) using the model to generate a plurality of protocol tags, each of the plurality of protocol tags describing a corresponding set of common features within the plurality of new input examination data sets.

5. The method according to any one of claims 1 to 3, wherein the learning includes supervised learning.

6. The method according to any one of claims 1 to 3, wherein the learning includes unsupervised learning.

7. Each tag T within the plurality of protocol tags describes a corresponding set of examination data sets within the plurality of input examination data sets, and the set of examination data sets corresponding to tag T includes a plurality of acquisition data sets sharing a corresponding set of common features within the plurality of input examination data sets.

8. The learning is (B)(1) Learning a first set of protocol tags from the plurality of input inspection data sets; (B)(2) Learning a second set of protocol tags from the plurality of input inspection data sets and the first set of protocol tags, wherein the second set of protocol tags describes a corresponding set of common features of the corresponding plurality of protocol tags within the first set of protocol tags; The method according to any one of claims 1 to 3, characterized by including. (Claim 9) (B)(1) When N = 1, (B)(1) Learning an Nth set of protocol tags from a plurality of input inspection data sets and a set of protocol tags previously learned at N ≧ 1, wherein the Nth set of protocol tags describes a corresponding set of common features of the corresponding plurality of (N - 1) - level protocol tags; (B)(2) Determining whether an end criterion is satisfied; (B)(3) When the end criterion is satisfied, ending the learning; (B)(4) When the end criterion is not satisfied, (B)(4)(a) Incrementing N; (B)(4)(b) Returning to (B)(1) above; The method according to any one of claims 1 to 3, characterized by including. (Claim 10) (B) includes learning a classifier or clustering algorithm for identifying characteristics of protocol tags based on the plurality of input inspection data sets, (B) The method according to any one of claims 1 to 3, characterized in that the step of learning the model includes learning the plurality of protocol tags using the classifier or clustering algorithm. (Claim 11) (C) For each of the plurality of protocol tags, identifying a corresponding organ of interest, thereby identifying a plurality of organs of interest corresponding to the plurality of protocol tags; (D) For each of the plurality of protocol tags, further including the step of identifying a label; The method according to claim 3, characterized by including. (Claim 12) (C) The method according to claim 11, characterized in that it includes the step of identifying a plurality of organs of interest corresponding to the plurality of protocol tags by applying learning to a plurality of images in the plurality of input inspection data sets. (Claim 13) The method according to claim 11, wherein (D) includes identifying a label associated with each of the plurality of protocol tags based on a set of labeled inspection data sets.

14. The method according to claim 3, wherein the plurality of protocol tags include a plurality of fixed-size embeddings.

15. The (A) is (A)(1) For each input inspection data set in the plurality of input inspection data sets, generating a corresponding graph, comprising: (A)(1)(a) For each of the plurality of nodes in the graph corresponding to the plurality of collection data sets within the input inspection data set, storing information regarding the collection corresponding to the node; (A)(1)(b) For each pair of nodes in the corresponding graph, generating and storing an edge within the graph that represents information regarding the relationship between the pair of nodes; Thereby generating a plurality of graphs corresponding to the plurality of input inspection data sets, the method according to claim 1.

16. The method according to claim 15, wherein (B) includes performing learning based on the corresponding plurality of graphs and generating corresponding plurality of embeddings representing a plurality of protocol tags.

17. After performing the steps of (A) and (B), (C) receiving a plurality of new input inspection data sets created by performing a plurality of new imaging inspections on at least one patient with at least one scanner; (D) learning an updated version of the model of the imaging protocol based on the plurality of new input inspection data sets; the method according to any one of claims 1 or 16, further comprising.

18. (C) further includes generating a graph corresponding to the corresponding plurality of embeddings, and (C) is (C)(1)(a) For each node among the plurality of nodes in the graph corresponding to the plurality of embeddings, storing information regarding the embedding corresponding to the node; (C)(1)(b)For each pair of nodes in the corresponding graph, generating and storing in the corresponding graph an edge representing information regarding the relationship between the pairs before embedding, the method according to claim 16, characterized by including this step.

19. Said (B) The method according to claim 18, characterized by including the step of performing learning on the corresponding graph generated in said (C) to generate a plurality of high-level embeddings.

20. (C)Based on the plurality of embeddings, generating at least one synthetic test dataset, wherein the plurality of input test datasets do not include the synthetic test dataset, the method according to claim 14, further characterized by including this step.

21. A system including at least one non-transitory computer-readable medium storing computer program instructions, the computer program instructions being executable by at least one computer processor to perform a method, the method (A)Receiving a plurality of input test datasets created by performing a plurality of imaging tests on at least one patient with at least one scanner, Each of the plurality of input test datasets includes a plurality of collection datasets, Each collection dataset A of the plurality of collection datasets includes corresponding plurality of values of a plurality of technical parameters used to perform the collection that generated the collection dataset A, (B)Based on the plurality of input test datasets, learning a model of an imaging protocol, the system characterized by including this.

Images