System and method for medical imaging protocol name standardization

Abstract

A system and method for medical imaging protocol name standardization includes generating a synthetic training dataset from medical criteria in a public document using knowledge extraction, the synthetic training data set includes a plurality of combinations of possible medical imaging protocol names for respective ones of the plurality of standard protocol codes for each standard medical imaging protocol. The system and method also include generating a lightweight text classification model (24) from the synthetic training dataset using machine learning. The system and method also include receiving medical imaging protocol names using a lightweight text classification model (24) and outputting a list of most probable protocol codes based on the medical imaging protocol names.

Description

BACKGROUND

[0001] The subject matter disclosed herein relates to systems and methods for medical imaging protocol name standardization.

[0002] When processing data from imaging exams, usually in Digital Imaging and Communications in Medicine (DICOM) form, the protocol name is used to route the exam to the corresponding treatment. Specifically, the protocol name helps the technician to match the clinical imaging protocol with the order. Only a specific sequence will match a single protocol. A protocol is a precise instruction that defines how a medical image set should be acquired to maximize the diagnostic quality, deliver consistent scan quality, and provide efficient and effective radiology service delivery. One example is dose monitoring, where a radiation dose threshold is given for some exam categories, and any exam must be mapped to the correct exam category via its hospital-specific protocol name. In some cases, this mapping is done completely manually.

[0003] The biggest obstacle in processing medical imaging text is the number and variety of abbreviations. Large Language Models (LLMs) are technically the best tool for processing this kind of text. However, the best LLMs available today that are both multilingual and medical imaging aware are not open source, and they will be open source in the future. On the other hand, medical imaging text contains Protected Health Information (PHI). Therefore, healthcare products cannot send such data to third parties without a very special contract term agreed with the hospital / customer. Imaging scanners use protocols to scan patients. Many hospital organizations maintain their own set of protocols for specific scenarios and operations. However, these protocols need to be maintained for each scanner model, as they are not compatible between vendors (e.g., original equipment manufacturers), and often not compatible between the same scanner model series. Protocol compatibility can be defined as the ability to use a protocol from one scanner to scan in another scanner to achieve similar results without the need to manually modify the protocol. The individual modifications are not considered, as it can depend on the personal preferences of the one who dictates the scan. Without proper scanner protocol management software, creating new protocols outside the scanner can be a challenge. Likewise, driving common results across protocols is a challenge, as it cannot be done outside the protocol management software. Therefore, there is a need to cross-transport protocols across various vendors to have consistency in radiology, for which there is currently no solution in the field. SUMMARY

[0004] The following presents a summary of certain embodiments disclosed herein. It should be appreciated that this brief summary is provided merely for purposes of summarizing some particular embodiments and that it is not meant to limit the scope of the disclosure. Indeed, the present disclosure can encompass a wide range of aspects not mentioned below.

[0005] In one embodiment, a computer-implemented method for standardizing medical imaging protocol names is provided. The computer-implemented method includes generating a synthetic training dataset from medical standards in public documents via a processing system comprising one or more processors, utilizing knowledge extraction. This training dataset includes, for a given language and a given imaging modality, multiple combinations of possible medical imaging protocol names for corresponding standard protocol codes from a plurality of standard protocol codes for each standard medical imaging protocol. The computer-implemented method also includes generating a lightweight text classification model from the synthetic training dataset via the processing system using machine learning. The computer-implemented method further includes receiving medical imaging protocol names via the processing system using the lightweight text classification model and outputting a list of the most likely protocol codes based on those names.

[0006] In another embodiment, a system is provided. The system includes memory that encodes processor-executable routines. The system also includes a processing system comprising one or more processors configured to access the memory and execute the processor-executable routines, wherein the processor-executable routines, when executed by the processing system, cause the processing system to perform actions. These actions include generating a synthetic training dataset from medical standards in public documents using knowledge extraction, wherein for a given language and a given imaging modality, the synthetic training dataset includes multiple combinations of possible medical imaging protocol names for corresponding standard protocol codes from a plurality of standard protocol codes for each standard medical imaging protocol. The actions also include generating a lightweight text classification model from the synthetic training dataset using machine learning. Furthermore, the actions include using the lightweight text classification model to receive medical imaging protocol names and, based on the medical imaging protocol names, outputting a list of the most probable protocol codes.

[0007] In yet another embodiment, a non-transitory computer-readable medium is provided, comprising processor-executable code that, when executed by a processing system comprising one or more processors, causes the processing system to perform actions. These actions include generating a synthetic training dataset from medical standards in public documents using knowledge extraction, wherein, for a given language and a given imaging modality, the synthetic training dataset includes multiple combinations of possible medical imaging protocol names for a given standard protocol code from a plurality of standard protocol codes for each standard medical imaging protocol. The actions also include generating a lightweight text classification model from the synthetic training dataset using machine learning. Furthermore, the actions include using the lightweight text classification model to receive medical imaging protocol names and, based on those names, outputting a list of the most probable protocol codes. Attached Figure Description

[0008] These and other features, aspects, and advantages of the invention will be better understood when reading the following detailed description with reference to the accompanying drawings, in which the same reference numerals denote the same parts throughout the drawings, wherein:

[0009] Figure 1 This is a schematic diagram of a system configured for medical imaging protocol name standardization according to various aspects of this disclosure;

[0010] Figure 2 This is a flowchart of a method for optimizing the standardization of medical imaging protocol names according to various aspects of this disclosure;

[0011] Figure 3 This is a flowchart of a method for generating a synthetic training dataset according to various aspects of this disclosure;

[0012] Figure 4 This is a flowchart of a method for constructing a lightweight text classification model according to various aspects of this disclosure;

[0013] Figure 5 This is a flowchart of a method for utilizing a lightweight text classification model according to various aspects of this disclosure;

[0014] Figure 6 This is an example of a graphical user interface showing the generated training data on a display according to various aspects of this disclosure;

[0015] Figure 7 This is a schematic diagram of a process for utilizing a lightweight text classification model (e.g., a logistic regression model) according to various aspects of this disclosure;

[0016] Figure 8 This is an example of a graphical user interface that displays generated n-tuples (e.g., character word slices) on a display according to various aspects of this disclosure;

[0017] Figure 9 This is an example of a graphical user interface on a display for outputting a lightweight text classification model (e.g., for Spanish) according to various aspects of this disclosure; and

[0018] Figure 10 This is an example of a graphical user interface on a display for outputting a lightweight text classification model (e.g., for French) according to various aspects of this disclosure. Detailed Implementation

[0019] One or more specific implementations will be described below. To provide a concise description of these implementations, not all features of the actual implementation will be described in this specification. It should be understood that in the development of any such actual implementation, as in any engineering or design project, many implementation-specific decisions must be made to achieve the developer's specific objectives, such as complying with system-related and business-related constraints that may differ from implementation to implementation. Furthermore, it should be understood that such development efforts may be complex and time-consuming, but will in any case remain routine tasks of design, fabrication, and manufacturing for those skilled in the art who benefit from this disclosure.

[0020] When describing elements of various embodiments of the subject matter of this invention, the articles “a,” “an,” “the,” and “described” are intended to indicate the presence of one or more elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that additional elements may exist in addition to those listed. Furthermore, any numerical examples in the following discussion are intended to be non-limiting, and therefore the additional numerical values, ranges, and percentages are within the scope of the disclosed embodiments.

[0021] Some general information is provided to give general background to the various aspects of this disclosure and to facilitate understanding and interpretation of certain technical concepts described herein.

[0022] As used herein, the terms processor, processing system, or processing unit refer to any type of processing unit capable of performing the required computations required for various implementation schemes, such as single-core or multi-core: CPU, accelerated processing unit (APU), graphics processing unit, DSP, FPGA, ASIC, or combinations thereof.

[0023] As used herein, the term "computing system" refers to an electronic computing device, such as, but not limited to, a single computer, virtual machine, virtual container, host, server, laptop computer, and / or mobile device, or multiple electronic computing devices working together to perform functions described as being performed on or by a computing system. As used herein, the terms "application," "application module" (or "module"), "engine," or "program" or "plugin" refer to one or more sets of computer software instructions (e.g., computer programs and / or scripts) that can be executed by one or more processors of a computing system to provide a specific function. Computer software instructions can be written in any suitable programming language, such as C, C++, C#, Fortran, Perl, MATLAB, SAS, SPSS, Python, JavaScript, and JAVA. Such computer software instructions can include standalone applications with data input and data display aspects (e.g., modules). Alternatively, the disclosed computer software instructions can be classes instantiated as distributed objects. The disclosed computer software instructions can also be component software, such as JAVABEANS or ENTERPRISE JAVABEANS. Furthermore, the disclosed applications or engines can be implemented in computer software, computer hardware, or a combination thereof.

[0024] As used herein, the terms “automatic” and “automatically” refer to actions performed by a computing device or computing system (e.g., within one or more computing devices) without human intervention. For example, an automatically executed function may be performed by a computing device or system solely based on data stored on and / or received by that computing device or system, even without prompting from a human user. By way of a non-limiting example only, a computing device or system may make decisions and / or initiate other functions solely based on decisions made by the computing device or system, regardless of any other input relating to the decision.

[0025] The deep learning (DL) methods discussed in this paper can be based on artificial neural networks and therefore may encompass one or more of the following: deep neural networks, fully interconnected networks, convolutional neural networks (CNNs), transformer-based networks, unfolded neural networks, perceptrons, encoders / decoders, recurrent networks, wavelet filter banks, u-nets, generative adversarial networks (GANs), dense neural networks, or other neural network architectures. Neural networks may include shortcuts, activations, batch normalization layers, and / or other features. These techniques are referred to as DL techniques in this paper, although the term may also be used specifically with reference to the use of deep neural networks, which are neural networks with multiple layers.

[0026] As discussed in this paper, deep learning (DL) techniques (also known as deep machine learning, hierarchical learning, or deep structured learning) are a branch of machine learning techniques that employ mathematical representations of data and artificial neural networks used to learn and process such representations. For example, DL methods can be characterized as using one or more algorithms to extract or model a class of highly abstract concepts from data of interest. This can be accomplished using one or more processing layers, where each layer typically corresponds to a different level of abstraction and may therefore take or utilize different aspects of the initial data or the output of the previous layer (i.e., the hierarchical or cascaded structure of the layers) as the target of the process or algorithm for a given layer. In the context of image processing or reconstruction, this can be characterized as different layers corresponding to different feature levels or resolutions in the data. Generally, the processing from one representation space to the next level of representation space can be viewed as a “stage” of a process. Each stage of the process can be performed by a single neural network or by different parts of a larger neural network.

[0027] This disclosure provides systems and methods for standardizing medical protocol names. Specifically, the system and methods are capable of mapping imaging protocol names in free text to standardized forms for the most common languages, with good tolerance for medical abbreviations. The disclosed system and methods provide a short list of candidates for protocol standardization. Through transitivity, the solution can automatically provide a short list of which protocol names should be mapped to which examination category when defining all mappings between medical standards and examination categories.

[0028] The disclosed implementation enables the utilization of state-of-the-art large language models without any privacy concerns. Furthermore, the disclosed implementation supports a large number of languages ​​with limited human involvement because the required medical knowledge (and related variants in flux) is extracted from large models without privacy issues. The disclosed implementation enables the generation of data from English standards in various languages ​​without explicit translation. The disclosed implementation reduces the costs typically associated with manual mapping. The disclosed implementation provides a classifier that is nearly as accurate as large language models (LLMs) using existing techniques. The disclosed implementation also offers a much faster and cheaper option than using large LLMs because the classification model is relatively lightweight and has millisecond inference times. The disclosed implementation also allows the training set to be parameterized to desired conditions (e.g., abbreviations).

[0029] Figure 1This is a schematic diagram of a system 10 (e.g., a medical imaging protocol name standardization system) configured for medical imaging protocol name standardization (e.g., for scanning or radiology protocols used in medical imaging scanners). The scanning protocol considers the imaging modality, the purpose of the scan, the anatomical region of interest to be imaged, and scanning parameters (e.g., acquisition parameters). As depicted, system 10 includes (e.g., implemented in a computing device) a protocol name standardization device 12. The protocol name standardization device 12 may be located on the medical imaging system or may be remotely positioned relative to any medical imaging system. The protocol name standardization device 12 is configured to map medical imaging protocol names in free text to standardized forms for most common languages, while providing good tolerance for medical abbreviations. The protocol name standardization device 12 is configured to generate synthetic data using knowledge extraction. The protocol name standardization device 12 is also configured to use the generated synthetic data to generate a lightweight text classification model (e.g., via standard machine learning algorithms). The protocol name standardization device 12 is further configured to map medical imaging protocol names to standardized forms using the lightweight text classification model.

[0030] The protocol-standardized device 12 includes one or more processors forming a processing system 14, which are configured to execute machine-readable instructions stored in non-transitory memory 16. The processors of the processing system 14 may be single-core or multi-core, and the programs executing on them may be configured for parallel or distributed processing. In some embodiments, the processing system 14 may optionally include individual components distributed across two or more devices that may be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of the processing system 14 may be virtualized and executed by remotely accessible networked computing devices configured in a cloud computing configuration.

[0031] The protocol name standardization device 12 also includes a non-transitory memory 16. The non-transitory memory 16 stores a data generation module 18. The data generation module 18 is configured to generate one or more synthetic training datasets from medical standards in public documents (e.g., public documents in English, such as the RadLex standard in radiology) using knowledge extraction. Specifically, the data generation module can generate multiple datasets for a given language and a given imaging modality (e.g., computed tomography, magnetic resonance imaging, etc.). For a given language and a given imaging modality, the training datasets include multiple combinations of possible medical imaging protocol names for the corresponding standard protocol codes from multiple standard protocol codes for each standard medical imaging protocol.

[0032] Data generation module 18 is configured to access and utilize one or more LLMs. LLMs provide radiological context and offer attentional benefits (i.e., sequences of words / abbreviations together have meaning). In some embodiments, the LLM is closed-source (e.g., GPT-3.5, GPT-4, and GPT-4o). In some embodiments, the LLM is open-source (e.g., the Llama family, etc.). In some embodiments, the LLM is medical language specific. Data generation module 18 is configured to receive one or more prompts or user input when utilizing one or more LLMs. Prompts may include the target language, target modality, the number of protocol name variants to be generated, and / or other information. In some embodiments, R may be used for scripting when utilizing an LLM. Data generation module 18 is configured to obtain a corresponding information block from a public document relating to each of a plurality of standard protocol codes, wherein the public document is in English. Data generation module 18 is configured to generate, for each corresponding information block of each standard protocol code, an extended and explicit text description (i.e., the entire protocol description) in English using an LLM (via prompts from user input). Variants and frequent abbreviations can be requested when prompting the LLM. Furthermore, inference parameters of the LLM (e.g., temperature and / or repetition avoidance parameters) can be adjusted in the generated synthetic training dataset. Data generation module 18 is configured to generate a set of medical imaging protocol names (the entire protocol name) in a given language with multiple variants and multiple abbreviations using a multilingual medical large language model (LMM) based on each extended and explicit text description. In some implementations, the given language is also English. In some implementations, the given language is different from English. Data generation module 18 is configured to generate multiple sets of medical imaging protocol names in a given language with multiple variants and multiple abbreviations. Data generation module 18 is configured to append each imaging protocol name in the medical imaging protocol name set to its corresponding standard protocol code of origin to define the source-target line for the synthetic training dataset in the given language. In some implementations, synthetic training datasets are generated for a given imaging modality in multiple different languages. Because the chosen medical criteria are public (by definition) and PHI-free, privacy is not violated in the generation of the training dataset.

[0033] In the process utilized by data generation module 18, medical abbreviations are always part of / embedded in a very meaningful context. Therefore, the process ensures that abbreviations are generated based on (explicitly generated) extended descriptions of the protocol, rather than simply abbreviating from a few words. Thus, even abbreviations not necessarily found in the LLM training set can be generated. Furthermore, data is generated modally, ensuring that each imaging modality has its own set of medical imaging concepts. Additionally, the training dataset can be made as large as needed and also balanced. Balance means that each class is represented equally well in the training dataset. Both the size and balance of the dataset contribute to the accuracy of predictions made by the classification model generated from the dataset.

[0034] Non-transitory memory 16 may also store classification model generation module 20. Non-transitory memory 16 may also store one or more machine learning algorithms 22 (e.g., standard machine learning algorithms). Classification model generation module 20 is configured to utilize machine learning (ML) algorithms 22 to generate one or more lightweight text classification models 24 from one or more synthetic training datasets. In some implementations, Python may be used for model building or generation. Compared to large LLMs, lightweight text classification models reduce computational complexity, memory usage, and power consumption. Classification model generation module 20 is configured to utilize machine learning to generate corresponding lightweight text classification models 24 from synthetic training datasets for different combinations of language and imaging modalities (i.e., lightweight text classification models for combinations of a single imaging modality and a single language). Each lightweight text classification model 24 is configured to receive a medical imaging protocol name (e.g., in a given or target language) and output a list of the most likely protocol codes based on the medical imaging protocol name. In addition to the most likely protocol codes, lightweight text classification model 24 may also output the coast name of each possible protocol code. In some implementations, the lightweight text classification model 24 is configured to utilize logistic regression when determining a list of the most likely protocol codes based on the medical imaging protocol name. In some implementations, the lightweight text classification model 24 is configured to determine the most likely protocol based on the medical imaging protocol name. In some implementations, the lightweight text classification model is configured to compute and output a corresponding confidence score for each protocol code in the list of the most likely protocol codes. The confidence score can be a numerical score or multiple symbols (e.g., stars). In some implementations, the language of the received medical imaging protocol name is detected. Based on the detected language, a corresponding lightweight text classification model 24 is selected from the appropriate models for use.

[0035] In some embodiments, the nontransitory memory 16 may include components located at two or more devices that can be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of the nontransitory memory 16 may include remotely accessible networked storage devices configured in a cloud computing configuration.

[0036] User input device 26 may include one or more of the following: a touchscreen, keyboard, mouse, touchpad, motion-sensing camera, or other device configured to enable a user to interact with protocol name normalization device 12. In one example, user input device 26 may enable a user to input medical imaging protocol names into a lightweight text classification model 24. In another example, user input device 26 may enable a user to input prompts into prompt data generation module 18. These prompts may include the target language, target imaging modality, the target number of protocol name variants to be generated, and / or other information (e.g., expected variability via different parameter values, addition of self-managed steps, etc.). Prompts may also relate to how to analyze / expand the input text. Prompts may also probe for typographical errors (e.g., when they occur in the actual protocol name). Prompts may relate to adding more precise instructions. In some embodiments, prompts are zero-shot prompts. In some embodiments, prompts are few-shot prompts.

[0037] Display device 28 may include one or more display devices utilizing virtually any type of technology. In some embodiments, display device 28 may include a computer monitor and may display the most likely protocol code, associated short name, and confidence score. Display device 28 may be combined with processing system 14, nontransitory memory 16, and / or user input device 26 in a shared housing, or it may be a peripheral display device and may include a monitor, touchscreen, projector, or other display device known in the art that enables a user to view data and / or interact with various data stored in nontransitory memory 16.

[0038] Processing system 14 is configured to generate a synthetic training dataset from medical standards in public documents using knowledge extraction (e.g., automatic knowledge extraction), wherein, for a given language and a given imaging modality, the synthetic training dataset includes multiple combinations of possible medical imaging protocol names for corresponding standard protocol codes from a plurality of standard protocol codes for each standard medical imaging protocol. Processing system 14 is also configured to generate a lightweight text classification model from the synthetic training dataset using machine learning. Processing system 14 is further configured to receive medical imaging protocol names using the lightweight text classification model and output a list of the most probable protocol codes based on the medical imaging protocol names.

[0039] Processing system 14 is configured to obtain a corresponding information block from a public document, each of a plurality of standard protocol codes, in English. Processing system 14 is also configured to generate a synthetic training dataset by generating a large language model for each corresponding information block for each standard protocol code, and an extended and explicit text description in English for the standard medical imaging protocol. Processing system 14 is further configured to generate a synthetic training dataset by generating a set of medical imaging protocol names in a given language with multiple variants and abbreviations using a large multilingual medical model based on each extended and explicit text description. Processing system 14 is further configured to generate a synthetic training dataset by generating multiple sets of medical imaging protocol names in a given language with multiple variants and abbreviations. Processing system 14 is further configured to generate a synthetic training dataset by appending each imaging protocol name in the medical imaging protocol name set to its corresponding standard protocol code of origin to define the source-target line of the synthetic training dataset for the given language. The processing system 14 is also configured to use machine learning to generate corresponding lightweight text classification models for different combinations of language and imaging modalities from a synthetic training dataset; detect the language of the received medical imaging protocol name; and select a corresponding lightweight text classification model from the corresponding lightweight text classification models for use based on the detected language.

[0040] Figure 2 This is a flowchart of a method for optimizing a protocol used for standardizing medical imaging protocol names. One or more steps of method 30 can be performed by... Figure 1 The protocol name is standardized and executed by one or more components of device 12.

[0041] Method 30 includes generating a synthetic training dataset (box 32) from medical standards in public documents (e.g., public documents in English, such as the RadLex standard in radiology) using knowledge extraction. For a given language and a given imaging modality, the synthetic training dataset includes multiple combinations of possible medical imaging protocol names for corresponding standard protocol codes from a plurality of standard protocol codes for each standard medical imaging protocol. In some embodiments, multiple synthetic training datasets may be generated for the same combination of a given language and a given imaging modality. In some embodiments, one or more training datasets are generated for different combinations of different languages ​​and different imaging modalities. In some embodiments, the given language may be English. In some embodiments, the given language may be a language different from English.

[0042] Method 30 also includes generating a lightweight text classification model from a synthetic training dataset using machine learning (box 34). In some implementations, the lightweight text classification model can be built to predict multiple labels (e.g., up to 1300 for RadLex CT). In some implementations, the lightweight text classification model can be built for multiple languages. If some illusions sometimes occur in the training dataset, the collective aspects of the dataset are automatically mitigated. In some implementations, the generation of the lightweight text classification model includes fine-tuning a small LLM. In some implementations, standard machine learning algorithms (e.g., logistic regression) can be utilized when building the model.

[0043] Method 30 further includes utilizing a lightweight text classification model to receive medical imaging protocol names and output a list of the most likely protocol codes based on those names (box 36). Compared to large LLMs, the lightweight text classification model reduces computational complexity, memory usage, and power consumption. The lightweight text classification model is configured to receive medical imaging protocol names (e.g., in a given or target language) and output a list of the most likely protocol codes based on those names. In addition to the most likely protocol codes, the lightweight text classification model may also output a coast name for each possible protocol code. In some embodiments, the lightweight text classification model is configured to utilize logistic regression when determining the list of the most likely protocol codes based on the medical imaging protocol names. In some embodiments, the lightweight text classification model is configured to determine the most likely protocol based on the medical imaging protocol names. In some embodiments, the lightweight text classification model is configured to compute and output a corresponding confidence score for each protocol code in the list of the most likely protocol codes. The confidence score can be a numerical score or multiple symbols (e.g., stars).

[0044] Figure 3 This is a flowchart of method 38 for generating a synthetic training dataset. One or more steps of method 38 can be... Figure 1 The protocol name is standardized and executed by one or more components of device 12.

[0045] Method 38 includes obtaining a corresponding information block relating to each of a plurality of standard protocol codes from a public document (e.g., RadLex) in which the public document is in English (box 40). Method 38 includes accessing one or more LLMs (box 42). LLMs provide radiological context and provide attentional benefits (i.e., the sequence of words / abbreviations together has meaning). In some embodiments, the LLM is closed-source (e.g., GPT-3.5, GPT-4, and GPT-4o). In some embodiments, the LLM is open-source (e.g., the Llama family, etc.). In some embodiments, the LLM is medical language specific.

[0046] Method 38 includes receiving one or more prompts (e.g., user input) for generating a synthetic training dataset using one or more LLMs (Box 44). The prompts may include the target language, target imaging modality, number of targets for the protocol name variants to be generated, and / or other information (e.g., expected variability via different parameter values, addition of self-managed steps, etc.). The prompts may also relate to how to analyze / expand the input text. The prompts may also probe for typographical errors (e.g., when they occur in the actual protocol names). The prompts may relate to adding more precise instructions. In some implementations, the prompts are zero-shot prompts. In some implementations, the prompts are few-shot prompts.

[0047] Method 38 further includes generating (e.g., inferring) an extended and explicit text description (e.g., a full text description) in English for each corresponding information block for each standard protocol code (e.g., for a given or target imaging modality) using LLM (box 46). Method 38 also includes generating a set of medical imaging protocol names in a given or target language with multiple variations and multiple abbreviations for a given imaging modality using multilingual medical LLM based on each extended and explicit text description (box 48). In some embodiments, the given language may be English. In some embodiments, the given language may be a language different from English. Method 38 even includes appending each imaging protocol name in the set of medical imaging protocol names to the corresponding standard protocol code from which it originates to define a source-target row for a synthetic training dataset for a given language (box 50). In some embodiments, boxes 46 through 50 of method 38 may be repeated more than once to generate multiple sets of medical imaging protocol names in a given language with multiple variations and multiple abbreviations. In some implementations, blocks 46 to 50 of method 38 can be performed for multiple different languages ​​for a given image modality. In some implementations, blocks 46 and 48 of method 38 can be performed simultaneously.

[0048] Figure 4 This is a flowchart of method 52 for building a lightweight text classification model. One or more steps of method 52 can be... Figure 1 The protocol name is standardized and executed by one or more components of device 12.

[0049] Method 52 includes receiving or obtaining one or more synthetic training datasets (such as in...) Figure 2 Method 38 (generated in box 54). Method 52 also includes using one or more machine learning algorithms (e.g., standard machine learning algorithms) to build or generate a lightweight text classification model from one or more synthetic training datasets (box 56). An example of a machine learning algorithm used to build a lightweight text classification model is logistic regression. In some implementations, a lightweight text classification model can be built for a single given language and a given imaging modality. In some implementations, a lightweight text classification model can be built for a given imaging modality for multiple different languages. If some illusions sometimes occur in the training dataset, the collective aspects of the dataset automatically mitigate the illusions. In some implementations, the generation of the lightweight text classification model includes fine-tuning a small LLM. Method 52 also includes storing the lightweight text classification model (e.g., in...) Figure 1 The protocol name standardization device 12 is shown in box 58. Method 52 can be used to generate multiple text classification models for different combinations of language and imaging modalities (i.e., for each model, one or more languages ​​are associated with different corresponding imaging modalities).

[0050] Figure 5 This is a flowchart of method 60, which utilizes a lightweight text classification model. One or more steps of method 60 can be... Figure 1 The protocol name is standardized and executed by one or more components of device 12.

[0051] Method 60 includes receiving user input of a medical imaging protocol name (box 62). The input medical imaging protocol name may contain abbreviations and / or typographical errors. In some embodiments, method 60 includes detecting the language of the received or input medical imaging protocol name (box 64). In some embodiments, method 60 further includes selecting, if multiple lightweight text classification models exist, a corresponding lightweight text classification model for use based on the language detected in the medical imaging protocol name (box 66). Method 60 also includes outputting a list of the most probable protocol codes from the lightweight text classification model (e.g., the selected lightweight text classification model) based on the medical imaging protocol name (e.g., for a given medical imaging modality) (box 68). The list of most probable protocol codes may also be accompanied by a short English name for the corresponding protocol code. In some embodiments, the lightweight text classification model calculates and outputs a corresponding confidence score for each protocol code in the list of most probable protocol codes. The confidence score may be a numerical score or multiple symbols (e.g., stars). In some implementations, conformal regression can be applied for initial or raw confidence score calibration purposes before outputting confidence scores. In some implementations, the lightweight text classification model determines the most likely protocol based on the name of the medical imaging protocol. Confidence scores provide trust in the results of the machine learning-based text classification model (which is impossible if LLM is used for classification due to the illusion problem). In some implementations, with more hints, the lightweight text classification model can self-access its own confidence scores (in cases where the lightweight text classification model experiences illusions regarding its own confidence estimates).

[0052] Figure 6 This is an example of a graphical user interface 70 on display 28, which shows generated training data (e.g., synthetic training data). Utilizing... Figure 1The data generation module 18 generates training data. Knowledge extraction is used to generate the training data. The target imaging modality is computed tomography. The medical standard in public documentation is the RadLex standard in radiology. For standard medical imaging protocols, the LLM is prompted to generate an extended and explicit textual description (e.g., the entire description) for each protocol code based on its description in public documentation (i.e., RadLex). Column 72 is the protocol code (i.e., RPID). Column 74 is these extended and explicit textual descriptions (i.e., long _names) for each protocol code (more informative than the code). The LLM (e.g., multilingual medical LLM) is also prompted from this extended and explicit description to list protocol name variants with multiple variations and multiple abbreviations. Both the hints and inferred parameters (e.g., temperature) require frequent variations and frequent abbreviations. Column 76 is the generated protocol name variants (i.e., the generated _names). Since the chosen medical standard is public (by definition) and PHI-free, privacy is not violated in the generation of the training dataset. This process can be repeated multiple times to aggregate a list of many variations.

[0053] Figure 7 This is a schematic diagram of process 78 for utilizing a lightweight text classification model 80 (e.g., a logistic regression model). The lightweight text classification model 80 is constructed using one or more standard machine learning algorithms (particularly logistic regression). As depicted, process 78 includes inputting a medical imaging protocol name 82 (i.e., a protocol name string) into the lightweight text classification model 80. Process 78 includes vectorizing or discretizing the input medical imaging protocol name 82 into a vectorized protocol name 84 with an n-tuple vocabulary 86 (i.e., a sequence of a given number of adjacent letters in a specific order). This provides robustness to various abbreviations. Various different types of n-tuples can be utilized. For example, character word fragments can be utilized. Process 78 also includes using logistic regression 87 on the vectorized protocol name 84 to perform and output a multi-label classification 88. In some implementations, other standard machine learning algorithms (e.g., support vector machines, Naive Bayes, etc.) can be utilized. Process 78 includes applying elements 90 of the standard (e.g., core or complete) to the multi-label classification output 88 to output the best recommendation 92 from the standard (i.e., a list of the most likely protocol codes based on the medical imaging protocol name). In some embodiments, the list of most likely protocol codes may be accompanied by corresponding confidence scores based on the probability of the prediction (combined with conformal prediction). In some embodiments, the most likely protocol codes are output. In some embodiments, multiple small or lightweight text classification models may exist. In this case, the combination of imaging modality and language is handled collectively by method experts (i.e., a small or lightweight text classification model is built for each combination).

[0054] Figure 8This is an example of a graphical user interface 94 on display 28, which shows the generated n-tuples. The following n-tuples are generated using character fragments. Figure 8 In the example, (e.g., by LLM) "abdomen / pelvis" is generated as an example of code (C). Words of 3 to 6 letters are selected. In some implementations, other selections are possible. The graphical user interface 94 depicts the list generated using character word pieces. Each sequence of these small letter sequences acts as a signature for abdomen / pelvis, and thus as a signature for code (C).

[0055] Figure 9 This is an example of a graphical user interface 96 on display 28 used for outputting a lightweight text classification model. The graphical user interface 96 can be used with... Figure 9 The depicted versions differ. The graphical user interface 96 includes a target imaging modality drop-down field 98 for selecting the target imaging modality. As depicted, the selected target imaging modality is computed tomography (CT). The graphical user interface 96 also includes a language drop-down field 100 for selecting the language of the input medical imaging protocol name. As depicted, the selected language is Spanish. In some embodiments, the language of the received medical imaging protocol name is detected. The graphical user interface 96 also includes a field 102 for entering the medical imaging protocol name. As depicted, the provided medical imaging protocol name is in Spanish, and typographical errors are also included. The graphical user interface 96 also includes a button 104 for submitting the target imaging modality, the language of the entered medical imaging protocol name, and the medical imaging protocol name. The bottom portion of the graphical user interface includes a result 106 from the submitted target imaging modality, the language of the entered medical imaging protocol name, and the medical imaging protocol name. Specifically, the result 106 is a list of the most likely protocol codes 108 (in the medical standard RadLex) based on the medical imaging protocol name. Each corresponding protocol code 108 is associated with a short name 110 having an English abbreviation (i.e., a short name). Each corresponding protocol code 108 is also associated with a confidence score 112. As depicted, multiple symbols (e.g., stars) represent confidence scores, with a higher number of stars associated with a higher confidence score. In some embodiments, the confidence score may be a numerical score. Figure 9 The implementation scheme demonstrates multilingual aspects, as the RadLex standard documentation is written only in English. Furthermore, Figure 9 The implementation scheme demonstrates tolerance for typographical errors, as these errors occur frequently in the protocol name.

[0056] Figure 10 This is an example of a graphical user interface 114 on display 28 used for outputting a lightweight text classification model. The graphical user interface 114 can be used with...Figure 10 The depicted versions differ. The graphical user interface 114 includes a language dropdown field 116 for selecting the language of the input medical imaging protocol name. As shown, the selected language is French. In some implementations, the language of the received medical imaging protocol name is detected. The graphical user interface 114 also includes a field 118 for inputting the medical imaging protocol name. As depicted, the provided medical imaging protocol name is French. The graphical user interface 114 also includes a button 120 for submitting the target imaging modality, the language of the input medical imaging protocol name, and the medical imaging protocol name. The bottom portion of the graphical user interface includes results 122 from the submitted target imaging modality, the language of the input medical imaging protocol name, and the medical imaging protocol name. Specifically, results 122 are a list of the most likely protocol codes 124 (in the medical standard RadLex) based on the medical imaging protocol name. Each corresponding protocol code 124 is associated with a name 126 having an English abbreviation (i.e., a long name). Each corresponding protocol code 124 is also associated with a confidence score 128. As depicted, multiple symbols (e.g., stars) represent confidence scores, with a higher number of stars associated with a higher confidence score. In some implementations, the confidence score may be a numerical score. “Abd ss” is the common protocol name in French. However, “ss” is an abbreviation of “sans” (i.e., “without”) in French, and “sans” itself is an abbreviation of “sans contraste” (i.e., “without contrast agent”).

[0057] The technical effects of the disclosed implementation include enabling the utilization of optimal large-scale language models without any privacy concerns. Furthermore, the technical effects of the disclosed implementation include supporting a large number of languages ​​with limited human involvement, as the required medical knowledge (and related variants in variation) is extracted from large-scale models without privacy concerns. The technical effects of the disclosed implementation include enabling the generation of data from English standards in various languages ​​without explicit translation. The technical effects of the disclosed implementation include reducing the costs typically associated with manual mapping. The technical effects of the disclosed implementation include providing classifiers that are nearly as accurate as those using existing LLM techniques. The technical effects of the disclosed implementation include providing a much faster and cheaper option than using large-scale LLMs, as the classification model is relatively lightweight and has millisecond inference times. The technical effects of the disclosed implementation include enabling the training set to be parameterized to the desired conditions (e.g., abbreviations).

[0058] Referring to the technology presented herein and protected by the claims, and applying it to physical objects and concrete examples of practical nature, which explicitly improves the present art, and therefore is not abstract, intangible, or purely theoretical. Furthermore, if any claim appended to the end of this specification contains one or more elements designated as “means for [performing]…” or “steps for [performing]…”, such elements are intended to be interpreted according to 35U.SC112(f). However, for any claim containing elements designated in any other manner, such elements are not intended to be interpreted according to 35U.SC112(f).

[0059] This disclosure also provides support for a computer-implemented method for standardizing medical imaging protocol names, the method comprising: generating a synthetic training dataset from medical standards in a public document via a processing system including one or more processors using knowledge extraction (e.g., automatic knowledge extraction), wherein, for a given language and a given imaging modality, the synthetic training dataset includes multiple combinations of possible medical imaging protocol names for corresponding standard protocol codes among a plurality of standard protocol codes for each standard medical imaging protocol; generating a lightweight text classification model from the synthetic training dataset via the processing system using machine learning; and receiving medical imaging protocol names via the processing system using the lightweight text classification model, and outputting a list of the most probable protocol codes based on the medical imaging protocol names. In a first example of the method, the method further includes obtaining a corresponding information block related to each of the plurality of standard protocol codes from a public document via the processing system, wherein the public document is in the English language. In a second example of the method (optionally including the first example), generating the synthetic training dataset comprises: for each corresponding information block for each standard protocol code, generating an extended and explicit text description of the standard medical imaging protocol in English using a large language model. In a third example of the method (optionally including one or both of the first and second examples), generating the synthetic training dataset includes: generating a set of medical imaging protocol names in the given language with multiple variants and multiple abbreviations using a large multilingual medical model, based on each extended and explicit text description. In a fourth example of the method (optionally including one or more or each of the first to third examples), the given language is also English. In a fifth example (optionally including one or more or each of the first to fourth examples), the given language is different from the given language. In a sixth example (optionally including one or more or each of the first to fifth examples), generating the synthetic training dataset includes: generating multiple sets of medical imaging protocol names in the given language with multiple variants and multiple abbreviations. In a seventh example (optionally including one or more or each of the first to sixth examples), the method further includes generating the synthetic training dataset by appending each imaging protocol name in the set of medical imaging protocol names to the corresponding standard protocol code from which it originates, to define the source-target line of the synthetic training dataset for the given language. In the eighth example (optionally including one or more or each of the first through seventh examples), a synthetic training dataset is generated for the given imaging modality and for multiple different languages. In the ninth example (optionally including one or more of the first through eighth examples), a synthetic training dataset is generated for multiple different imaging modalities and for multiple different languages.In the tenth example (optionally including one or more of the first to ninth examples), the method further includes generating corresponding lightweight text classification models for different combinations of language and imaging modalities from the synthetic training dataset via the processing system using machine learning; detecting the language of the received medical imaging protocol name via the processing system; and selecting a corresponding lightweight text classification model from the corresponding lightweight text classification models for use based on the detected language via the processing system. In the eleventh example (optionally including one or more of the first to tenth examples), the lightweight text classification model utilizes logistic regression when determining the list of most likely protocol codes based on the medical imaging protocol name. In the twelfth example (optionally including one or more of the first to eleventh examples), the lightweight text classification model determines the most likely protocol based on the medical imaging protocol name. In the thirteenth example (optionally including one or more of the first to twelfth examples), the lightweight text classification model calculates and outputs a corresponding confidence score for each protocol code in the list of most likely protocol codes.

[0060] This disclosure also provides support for a system comprising: a memory encoding processor-executable routines; and a processing system including one or more processors and configured to access the memory and execute the processor-executable routines, wherein the processor-executable routines, when executed by the processing system, cause the processing system to: generate a synthetic training dataset from medical standards in a public document using knowledge extraction, wherein, for a given language and a given imaging modality, the synthetic training dataset includes multiple combinations of possible medical imaging protocol names for corresponding standard protocol codes from a plurality of standard protocol codes for each standard medical imaging protocol; generate a lightweight text classification model from the synthetic training dataset using machine learning; and utilize the lightweight text classification model to receive medical imaging protocol names and output a list of the most probable protocol codes based on the medical imaging protocol names. In a first example of the system, the processor-executable routines, when executed by the processing system, cause the processing system to obtain a corresponding information block from a public document relating to each of the plurality of standard protocol codes, wherein the public document is in English. In a second example of the system (optionally including the first example), generating the synthetic training dataset includes: for each corresponding information block of each standard protocol code, generating an extended and explicit text description in English for the standard medical imaging protocol using a large language model. In a third example of the system (optionally including one or both of the first and second examples), generating the synthetic training dataset includes: based on each extended and explicit text description, generating a set of medical imaging protocol names in the given language with multiple variations and multiple abbreviations using a multilingual medical large language model. In a fourth example of the system (optionally including one or more, or each of, the first to third examples), generating the synthetic training dataset includes: generating multiple sets of medical imaging protocol names in the given language with multiple variations and multiple abbreviations.

[0061] This disclosure also provides support for a non-transitory computer-readable medium including processor-executable code that, when executed by a processing system comprising one or more processors, causes the processing system to: generate a synthetic training dataset from medical standards in public documents using knowledge extraction, wherein, for a given language and a given imaging modality, the synthetic training dataset includes multiple combinations of possible medical imaging protocol names for a given standard protocol code from a plurality of standard protocol codes for each standard medical imaging protocol; generate a lightweight text classification model from the synthetic training dataset using machine learning; and use the lightweight text classification model to receive medical imaging protocol names and output a list of the most likely protocol codes based on the medical imaging protocol names.

[0062] This written description uses examples to disclose the subject matter of the invention, including best practices, and also enables those skilled in the art to practice the subject matter, including making and using any device or system and performing any included methods. The patent scope of this subject matter is defined by the claims and may include other examples that would occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that are not indistinguishable from the literal language of the claims, or if they include equivalent structural elements that have minor differences from the literal language of the claims.

Claims

1. A system comprising: Memory (16) that encodes processor-executable routines; and A processing system (14) comprising one or more processors and configured to access the memory (16) and execute processor-executable routines, wherein the processor-executable routines, when executed by the processing system (14), cause the processing system (14) to: A synthetic training dataset is generated from medical standards in public documents using knowledge extraction, wherein, for a given language and a given imaging modality, the synthetic training dataset includes multiple combinations of possible medical imaging protocol names for the corresponding standard protocol codes from a plurality of standard protocol codes for each standard medical imaging protocol. A lightweight text classification model was generated from the synthetic training dataset using machine learning (24); as well as The lightweight text classification model (24) is used to receive the medical imaging protocol name and output a list of the most likely protocol codes based on the medical imaging protocol name.

2. The system of claim 1, wherein the processor executable routine, when executed by the processing system (14), causes the processing system (14) to obtain from the public document a corresponding information block relating to each of the plurality of standard protocol codes, wherein the public document is in English.

3. The system according to claim 2, wherein generating the synthetic training dataset comprises: For each corresponding information block of each standard protocol code, an extended and explicit text description in English is generated using a large language model for the standard medical imaging protocol.

4. The system of claim 3, wherein generating the synthetic training dataset comprises: Based on each extended and explicit text description, a set of medical imaging protocol names in the given language is generated using a multilingual medical large language model, with multiple variants and multiple abbreviations.

5. The system of claim 4, wherein generating the synthetic training dataset comprises: Generate multiple sets of medical imaging protocol names in the given language, with multiple variations and multiple abbreviations.

6. A computer-implemented method for standardizing medical imaging protocol names, the computer-implemented method comprising: A synthetic training dataset is generated from medical standards in public documents via a processing system (14) including one or more processors, using knowledge extraction, wherein for a given language and a given imaging modality, the synthetic training dataset includes multiple combinations of possible medical imaging protocol names of the corresponding standard protocol codes from multiple standard protocol codes for each standard medical imaging protocol. A lightweight text classification model (24) is generated from the synthetic training dataset using machine learning via the processing system (14); as well as The processing system (14) uses the lightweight text classification model (24) to receive medical imaging protocol names and outputs a list of the most likely protocol codes based on the medical imaging protocol names.

7. The computer-implemented method of claim 6, further comprising obtaining, via the processing system (14) from the public document, a corresponding information block relating to each of the plurality of standard protocol codes, wherein the public document is in English.

8. The computer-implemented method of claim 7, wherein generating the synthetic training dataset comprises: For each corresponding information block of each standard protocol code, an extended and explicit text description in English is generated using a large language model for the standard medical imaging protocol.

9. The computer-implemented method of claim 8, wherein generating the synthetic training dataset comprises: Based on each extended and explicit text description, a set of medical imaging protocol names in the given language is generated using a multilingual medical large language model, with multiple variants and multiple abbreviations.

10. The computer-implemented method of claim 9, wherein the given language is also English.

11. The computer-implemented method of claim 9, wherein the given language is different from a given language.

12. The computer-implemented method of claim 9, wherein generating the synthetic training dataset comprises: Generate multiple sets of medical imaging protocol names in the given language, with multiple variations and multiple abbreviations.

13. The computer-implemented method of claim 9, wherein generating the synthetic training dataset comprises: Each imaging protocol name in the medical imaging protocol name set is appended to the corresponding standard protocol code from which it originates to define the source-target row for the synthetic training dataset for the given language.

14. The computer-implemented method of claim 6, wherein the synthetic training dataset is generated for the given imaging modality and for multiple different languages.

15. The computer-implemented method of claim 6, wherein the synthetic training dataset is generated for multiple different imaging modalities and multiple different languages.

Images