Systems and methods for smart medical editors

Abstract

The invention relates to a system and method for an intelligent medical editor. Various systems and methods for an intelligent medical editor are provided. The intelligent medical editor may edit medical materials, such as electronic medical records, medical reports, and treatment plans. An intelligent medical editor may provide editing recommendations and suggestion additions to medical materials to a user of the editor. Editing recommendations and suggestion addition may be related to contradictions, imperfections, clarity, and clinical guidelines.

Description

Technical Field

[0001] The subject matter disclosed herein relates generally to healthcare technology, and more particularly to editing-assisted technology for medical reports and records. Background Art

[0002] Medical reports, documents and records are an essential part of healthcare around the world. These can be radiology reports, lists of chronic health conditions, lists of medications, imaging results, diagnostic reports, department reports, clinical notes, staff action orders, etc. The information in medical reports, documents and records is used to make important decisions that affect human health.

[0003] Currently, many negative issues affect the editing and updating of such medical materials. There may be ambiguity in meaning and word usage rather than clarity. Ambiguity may include poor wording, or different types of wording used by medical professionals in different regions or languages ​​for the same question. For a given patient, there may be inconsistencies within a record or between different records rather than correctness. Incomplete materials or information may cause problems rather than full and complete data. Information may be missing rather than captured. For example, a radiologist may not find a lesion in a medical image when reviewing a computed tomography (CT) scan. In addition, inconsistencies in information between medical materials and medical standards or guidelines may lead to legal or medical problems. Of course, information may simply be wrong rather than correct. Human errors may occur when writing medical reports and editing medical records. Spelling errors may change the meaning of information and affect patient care, such as affecting a patient's medication dosage. Accurate information that helps clinical decision-making is important. Systems and methods are needed to improve such issues within medical materials.

[0004] Not only can medical materials themselves be problematic, but they also take a long time and a lot of manpower to generate, edit, and correct. Clinicians, doctors, physicians, nurses, and other healthcare professionals are paid well but have a limited amount of time. They can often best use that time serving patients rather than updating medical reports, documents, and records. Systems and methods are needed to speed up the time it takes users to edit and update medical materials and save money for the stakeholders involved.

[0005] Today, medical professionals must also deal with many computers and information technology (IT) systems during the patient journey. During the patient journey, medical professionals may need to use multiple tools on different platforms and manually work with IT systems, including electronic medical records (EMRs) and electronic health records (EHRs). These varying IT systems may have gaps in compatibility and increase the burden on medical professionals, especially in situations where professionals need to provide comprehensive multimodal patient analysis. Systems and methods are needed to unify the compatibility and usability of medical material editing systems.

[0006] Ensuring the accuracy, completeness, and clarity of EMRs is critical to improving clinical outcomes. Inaccurate and incomplete EMRs and other medical materials can adversely affect the speed and quality of all subsequent clinical decisions, such as diagnostic / treatment decisions, clinical trial selection, reference guidelines, and case studies.

[0007] Medical materials (such as the reports, records, and documents mentioned above) may also have problems with personalization bias. Reports may be too specific to a department or individual, rather than standard reports for the hospital or industry. Systems and methods are needed to help generate medical materials that meet industry standards. Summary of the invention

[0008] This summary introduces concepts that are described in more detail in the detailed description. It should not be used to identify essential features of the claimed subject matter, nor should it be used to limit the scope of the claimed subject matter.

[0009] In one aspect, medical systems and methods are presented, which may include: one or more processors; memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: when a user writes a free-text medical report, identifying multiple text entities in the report in real time; suggesting editing recommendations related to text entries to the user, wherein the editing recommendations are generated by one or more artificial intelligence modules; and wherein the editing recommendations involve possible errors or improvements based on data including patient medical history, clinical care guidelines, and medical ontology. The medical system and method may also include retraining the artificial intelligence module using the medical report and the user selections related to the editing recommendations as training data.

[0010] The errors or improvements may include: clinical inaccuracies where the report includes nonexistent or incorrect findings regarding the patient's anatomy; a treatment plan that is incompatible with the clinical care guidelines; information in the report regarding prior medical appointments or previously incomplete medical examinations; detected differences between the report and the relevant summary reporting standards associated with the report type; and suggested links to source data from the patient's EMR or patient images stored in a PACS.

[0011] The editorial recommendation may be presented on the user interface via a text overlay near the text entry in the report.The editorial recommendation may be presented on the user interface via a chatbot powered by generative artificial intelligence and a large language model.

[0012] In one aspect, a medical system and method are presented, which may include a device having one or more processors and a memory, including the following steps: identifying multiple text entries in a free-text medical report; generating suggestions using an intelligent module; providing suggested content related to the patient and the medical report type on a user interface; providing suggestions related to the text entry and other report content to the user when the user edits the report; providing warnings about possible errors based on at least one of the patient's medical history, clinical care guidelines, or a medical ontology; and updating the medical report based on a user-selected selection of the suggested content. The intelligent module may be at least one of a clinical information extraction module, an enriched information coverage module, a contradiction and incompleteness detection module, an edit recommendation module, a language module fine-tuning module, and a natural language understanding update module. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Numerous aspects, embodiments, objects and advantages of the present invention will become apparent upon consideration of the following detailed description in conjunction with the accompanying drawings, in which like reference numerals refer to like parts throughout, and in which:

[0014] Figure 1 The stages in a patient journey are shown according to one embodiment.

[0015] Figure 2A A hardware device for use in a healthcare environment is shown according to one embodiment.

[0016] Figure 2B A block diagram of a computed tomography hardware and software system is shown according to one embodiment.

[0017] Figure 3 A block diagram of an intelligent medical editor according to one embodiment is shown.

[0018] Figure 4 A flow chart of an intelligent medical editor according to one embodiment is shown.

[0019] Figure 5 A natural language processing (NLP) model update module according to one embodiment is shown.

[0020] Figure 6 A natural language generation (NLG) module with a natural word prediction (NWP) module is shown according to one embodiment.

[0021] Figure 7 A block diagram supporting naturalistic report generation according to one embodiment is shown.

[0022] Fig. 8A A local model for language fine-tuning is shown according to one embodiment.

[0023] Figure 8B Details of a language tuning module are shown according to one embodiment.

[0024] Fig. 9 A clinical information extraction module according to one embodiment is shown.

[0025] Fig.10 An editorial recommendation module according to one embodiment is shown.

[0026] Fig.11 A contradiction and incompleteness detection module according to one embodiment is shown.

[0027] Fig.12 A clinical care guideline deviation process is shown according to one embodiment.

[0028] Fig.13 A user interface with suggested edits to a digital document is shown according to one embodiment.

[0029] Fig.14 Details of a rich information overlay module are shown according to one embodiment.

[0030] Fig.15 A comparison between a standard report and a report supporting a suggestion is shown according to one embodiment.

[0031] Fig.16 A user interface with suggested edits to a digital document is shown according to one embodiment.

[0032] Fig.17 A user interface with medical corrections in a digital document is shown according to one embodiment.

[0033] Fig.18 A block diagram related to editorial recommendations utilizing predictive and generative models is shown according to one embodiment.

[0034] Fig.19 Flowcharts and block diagrams related to patient profiling are shown according to one embodiment.

[0035] Fig. 20 A flow chart for updating a medical report based on selected suggested edits is shown according to one embodiment.

[0036] Fig.21 A schematic block diagram of a sample computing environment is shown according to one embodiment.

[0037] Fig. 22 A schematic block diagram of another sample computing environment is shown according to one embodiment.

[0038] Fig.23 A schematic block diagram of another sample computing environment is shown according to one embodiment.

[0039] Fig.24 A schematic block diagram of another sample computing environment is shown according to one embodiment.

[0040] Fig.25 A schematic block diagram of another sample computing environment is shown according to one embodiment.

[0041] Fig.26 A schematic diagram illustrating a suitable operating environment is shown according to one embodiment. DETAILED DESCRIPTION

[0042] In the following detailed description, reference is made to the accompanying drawings forming a part thereof, and wherein a practicable specific example is shown by way of illustration. These examples are described in sufficient detail to enable those skilled in the art to practice this theme, and it should be understood that other examples may be utilized, and logical, mechanical, electrical and other changes may be made without departing from the scope of the disclosed theme. Therefore, the purpose of providing the following detailed description is to describe exemplary embodiments, rather than being regarded as limiting the scope of the theme described in the present disclosure. Certain features from the different aspects described below may be combined to form new aspects of the theme discussed below.

[0043] When introducing elements of various embodiments of the present disclosure, the words "a," "an," "the," and "said" are intended to mean that there are one or more of the elements. The terms "comprising," "including," and "having" are intended to be inclusive and mean that there may be additional elements besides the listed elements.

[0044] In one embodiment, systems and methods are proposed for EMR editing assistance throughout the patient journey and numerous downstream analyses utilizing, but not limited to, standardized EMRs, clinician interactions, and multimodal patient medical histories. This solves multiple problems caused by varying EMR reporting standards by providing the correctness, completeness, and clarity of the EMR. While clinicians benefit from AI assistance, feedback from the interactions is used to improve the AI ​​assistance itself. These improvements are made in a joint manner without compromising patient privacy.

[0045] The systems and methods of this article produce many technical effects, including identifying edit candidates using masks and causal large language models (LLMs), factual contradiction and incompleteness detection using patient data models, clinical guidelines, and medical ontologies, incompleteness detection and report structure recommendations using summary report standards, cross-reference recommendations, providing standard tools for writing medical reports, providing suggestions for alternative words using medical standard vocabulary, and linking using patient data models and clinical guidelines, and realigning reports to different templates using large language models (LLMs). Further technical effects produced by the systems and methods of this article include automatic identification of missing body parts (due to biological sex or past surgery to remove body parts), such as a male patient diagnosed with ovarian cancer, and medical guidance compatibility detection. For example, if a treatment decision is made that is not in any of the treatment options recommended by the care guide, a warning may pop up.

[0046] Medical text processing is an important application of natural language processing (NLP), and supervised machine learning is used to process medical text to extract useful information. Supervised machine learning techniques require annotated text to learn from the data. All useful words need to be labeled with the most appropriate tag among all possible class tags. The annotated text needs to be in a format that can represent word-token relationships and other representations based on NLP tasks. Some common formats include XMI (XML Metadata Interchange) and CoNLL. The CoNLL format is a text file with one word per line and sentences separated by blank lines.

[0047] The time and redundancy required can be reduced with the help of smart annotation and editing tools, which help annotate reports while writing them by providing tag suggestions for each word. This requires an artificial intelligence (AI)-enabled report writing tool that needs to be trained with limited data. The tool helps us get annotated data quickly. The model can be adjusted at the end of each report by comparing the predictions with the actual annotation choices of the physician, thereby improving the model's adaptability to a specific physician or report type.

[0048] In one embodiment, the intelligent medical editor can be very influential because multiple modules can be integrated into the tool and intelligence can be provided based on the domain. This will help medical health care professionals quickly obtain annotated data to perform NLP tasks in different fields. In one embodiment, the intelligent medical editor also helps to form reports in a standard form and make a wide variety of reports similar. This is conducive to training models with less data.

[0049] Figure 1 The stages in a patient journey are shown according to one embodiment. Figure 1The patient journey 100 is included, which has the following steps: information collection 102, information integration and interpretation 104, working diagnosis 106, treatment not required 108, treatment planning 110, information collection 112, information integration and interpretation 114, treatment administration 116, information collection 118, information integration and interpretation 120, repeat process 122, treatment conclusion 124 and follow-up 126. The patient journey 100 also has patient stages of pre-diagnosis 130, pre-treatment 132, during treatment 143 and post-treatment 146.

[0050] The initial stage of the patient journey is pre-diagnosis 130. This includes initial information collection 102 before or during the initial medical appointment. The information can be data, observations, measurements, images, test results, medical opinions, evidence, or other information collected by the patient or the initial consultation medical provider. At this stage, the patient seeks care for their symptoms from a provider (usually a primary care physician or other general practitioner). In the later stages of the patient journey, a specialist may be consulted. Sometimes patients will directly seek a specialist to evaluate their condition, which may or may not shorten their journey, depending on their choice of an appropriate specialist. Information collection 102 may come from a variety of information collection modes, such as clinical history and interview, physical examination, diagnostic testing, specialist consultation, etc.

[0051] The steps in the patient journey are enabled through medical reports and records such as EMRs, radiology information systems (RIS), radiology reports, and other medical types of reports and records. The information may be integrated into the EMR in step 104, which may include evidence collected, reasoning and thought processes, conclusions, prescriptions for further testing, prescriptions for treatment, treatment plan thought processes, treatment progress notes, treatment response notes, and follow-up notes.

[0052] The information integration and interpretation steps 104, 114, and 120 may be implemented by an intelligent medical editor, as described throughout this document. The information may be analyzed by an intelligent medical editor, and then suggestions for integration and interpretation may be presented to accelerate and improve the information integration and interpretation steps 104, 114, and 120. Such an intelligent medical editor may provide automatic creation of reports as well as automatic suggestions and edit recommendations.

[0053] During the initial consultation with the medical provider, an initial working diagnosis 106 is developed and can be placed into the intelligent medical editor. Sometimes treatment may not be needed because the medical problem can resolve on its own and the patient journey ends with no treatment needed 108. Other times, more information is collected at information collection 102. And many times, initial treatment is planned at treatment plan 110. Treatment plan 110 can be developed jointly by the initial medical provider, nurses, specialists, and the patient. The intelligent medical editor can recommend treatment based on medical guidelines.

[0054] During treatment planning 110, the medical provider continues to perform information collection 112 using a variety of different types of information and medical devices. And the collected information feeds into information integration and interpretation 114. In one embodiment, this is done using a computing device running an intelligent medical editor. These steps can be completed in the pre-treatment stage 132 of the patient journey.

[0055] The treatment phase 134 of the patient journey may include treatment administration 116. Treatment administration 116 may include many types of treatments based on symptoms and underlying medical issues, such as watchful waiting, chemotherapy or other drug therapy, radiation therapy, immunotherapy, vaccine therapy, stem cell transplant, blood transfusion, palliative care, physical therapy, and other medical treatments. The health system may provide both on-site care and follow-up care (prescriptions, physical therapy, counseling, or recommended lifestyle changes) for the patient.

[0056] The ongoing treatment phase 134 of the patient journey also includes further information collection 118, information integration and interpretation 120, and possible repetition of the process 122. This process may be repeated as needed, as this phase of the diagnostic journey may involve one or more evaluations by specialists. For example, a patient presenting with a fever of unknown origin may be referred by his or her primary care physician to a rheumatologist and an infectious disease specialist, either simultaneously or sequentially. This phase may be expedited in inpatient settings that offer multispecialty consultations, particularly in academic medical centers.

[0057] The post-treatment phase 136 of the patient journey can include treatment conclusion 124 and follow-up 126. Treatment conclusion 124 occurs when a medical condition is at a point where treatment is paused or completed. The patient manages his or her care between clinical visits; at the same time, the health system facilitates interactions between the patient and the physician to enable the patient to cope with symptoms and maintain good health. This patient follow-up and retention phase is a critical part of the patient journey that is often forgotten after the patient receives treatment. It is necessary to obtain feedback through follow-up 126 so that the system can gather insights into the patient experience and satisfaction. The patient's input can then be used to improve the patient journey mapping, which ultimately helps improve the patient experience, quality of care, and process efficiency.

[0058] Figure 2A A hardware device for use in a healthcare environment is shown according to one embodiment. Figure 2B A block diagram of a computed tomography hardware and software system according to one embodiment is shown. Such a machine can be used as a medical imaging device during the patient journey.

[0059] Figure 2A and Figure 2BA computed tomography (CT) imaging system 10 including a gantry 12 is shown. The gantry 12 has a rotating member 13. An x-ray source 14 projects an x-ray beam 16 through a pre-patient collimator 15 toward a detector assembly 18 on the opposite side of the rotating member 13. The x-ray source 14 may also be referred to as an x-ray tube or an x-ray generating assembly. The x-ray source 14 is a transmitting assembly. The rotating member 13 may be attached to a fixed structure of the gantry 12 using a main bearing. The detector assembly 18 is formed by a plurality of detectors 20 and a data acquisition system (DAS) 22, and may include a post-patient collimator. The plurality of detectors 20 senses the projected x-rays transmitted through a subject 24, and the DAS 22 converts the data into a digital signal for subsequent processing. Each detector 20 generates an analog or digital electrical signal representing the intensity of the irradiated x-ray beam and, therefore, the intensity of the attenuated beam when transmitted through the subject 24. During scanning to acquire x-ray projection data, the rotating member 13 and the components mounted thereon may rotate about a center of rotation.

[0060] The rotation of the rotating member 13 and the operation of the x-ray source 14 are controlled by a control mechanism 26 of the CT system 10. The control mechanism 26 may include an x-ray controller 28 and a generator 30 that provide power and timing signals to the x-ray source 14, and a gantry motor controller 32 that controls the rotation speed and position of the rotating member 13. An image reconstructor 34 receives sampled and digitized x-ray data from the DAS 22 and performs high-speed image reconstruction. The reconstructed image is output to a computer 36, which stores the image in a computer storage device 38.

[0061] The computer 36 also receives commands and scanning parameters from the operator via an operator console 40, which has some form of operator interface, such as a keyboard, mouse, touch-sensitive controller, voice-activated controller, or any other suitable input device. A display 42 allows the operator to observe reconstructed images and other data from the computer 36. The computer 36 uses the commands and parameters provided by the operator to provide control signals and information to the DAS 22, the x-ray controller 28, and the gantry motor controller 32. In addition, the computer 36 operates a table motor controller 44, which controls a motorized table 46 to position the subject 24 and the gantry 12. In particular, the table 46 moves the subject 24 in whole or in part through a gantry opening 48 or portal. A coordinate system 50 defines a patient or Z-axis 52 along which the subject 24 moves into and out of the opening 48, a gantry circumference or X-axis 54 along which the detector assembly 18 passes, and a Y-axis 56 along a direction from the focal point of the x-ray tube 14 to the detector assembly 18.

[0062] Such imaging systems provide data, images, and information for the information collection phase and interpretation in the patient journey. In one embodiment, the computer 36 may have an intelligent medical editor that provides the CT images along with the radiology report that is edited using a system such as that disclosed herein. The radiology report may then be further integrated into a larger EMR using a different instance or the same intelligent medical editor.

[0063] Figure 3 A block diagram of an intelligent medical editor according to one embodiment is shown. Figure 3 It includes a medical system 300, an EMR editor 302, a clinical information extraction module 304, a rich information coverage module 306, an editing recommendation module 308, a usage and feedback log module 310, a contradiction and incompleteness module 312, a language model fine-tuning module 314, an NLU model update module 316, a medical data structure 318, an NLU model 320, a medical ontology 322, a patient profile and past Dx / Tx data 324, a clinical guideline 326, a summary report standard 328, a clinical language model 330, a usage log 332, and a user profile and preference 334. In different embodiments, the medical system 300 can be an intelligent medical editor, or various parts of the medical system 300 can be intelligent medical editors. Each module can be in the intelligent medical editor software or in other connected software programs. These modules can be separate modules within a larger program, or they can be separate programs linked together, such as through process calls. Figure 3 Functional components of a medical system 300 including an intelligent medical editor are indicated.

[0064] The EMR editor 302 is an example editor for electronic medical records. In other embodiments, the editor can edit reports, records, and other medical materials. The EMR editor 302 provides a user interface that interacts with the user, presenting information to the user and receiving information from the user. The EMR editor 302 is connected to a data storage system in which EMR records are stored for updating. The EMR editor 302 can send text data and structured data to a clinical information extraction module 304. The EMR editor 302 can receive enriched information from an enriched information overlay module 306. The EMR editor 302 can receive edit recommendations from an edit recommendation module 308. The EMR editor 302 can receive user decisions and feedback from a usage and feedback log module. The EMR editor can also send information to these modules based on user interactions. For example, if the edit recommendation module 308 provides an edit suggestion and the user selects the edit suggestion, the response to the recommendation may be returned to the edit recommendation module 308 for implementation of the change and input into the usage log 332 via the usage and feedback log module 310 to better adaptively learn user behavior (both individually for a specific user and generally for all users who make similar suggestions on the EMR editor 302).

[0065] The EMR editor 302 can provide an initial EMR record or template report that is pre-populated with data and then overlaid with suggestions and editing recommendations. And as the user interacts with the EMR record or report, the system can continuously adjust its suggestions and information in real time. This provides real-time assistance to medical professionals in creating and updating medical materials. In addition to suggestions and editing recommendations, the EMR editor can also provide annotations of information, text, images, and other data objects while preparing the report. This helps reduce the time it takes for doctors to annotate the report after it is completed.

[0066] At least Figures 13 to 17 User interface examples are shown in . These user interface examples show examples of medical editing suggestions and recommendations that the intelligent medical editor can provide in various embodiments. All of these provide users with a concise list of improved edits. In some embodiments, the user can interact with the suggestion (such as a mouse click, a finger touch, a right click, a finger swipe, or other input depending on the computer hardware and software involved) to understand the reason for providing the suggestion. In these cases, the software can provide medical reasoning, ontology, grammar or other reasoning for making the suggestion so that the user can further trust and understand the suggestion.

[0067] Fig.15 A comparison between a standard report and a report supporting a suggestion is shown according to one embodiment. Fig.15An example of an initial medical report 1502 input to an intelligent medical editor such as an annotation editor 1506 is included. The annotation editor 1506 provides a user interface screen 1504 that includes the initial medical report 1502, and also provides five edit suggestions for the user to consider. The user can interact with the suggested edits in the user interface screen 1504 to change and potentially improve the initial medical report 1502.

[0068] return Figure 3 , the clinical information extraction module 304 can receive text data and structured data from the EMR editor 302. The clinical information extraction module 304 can receive the medical data structure 318 and the NLU module 320. The clinical information extraction module 304 can place the information into stored data, including into the medical data structure 318.

[0069] The clinical information extraction module 304 acquires data and information from a myriad of sources, some of which are healthcare related and some of which may not be healthcare related. This is an important part of the system as it helps the intelligent medical editor to be context aware. Information may be provided to and from the NLU model update module 316, medical data structures 318, NLU models 320, medical ontologies 322, patient profiles and past Dx / Tx data 324, clinical guidelines 326, summary reporting standards 328, clinical language models 330, usage logs 332, and user profiles and preferences 334. The context aware intelligent medical editor is better able to provide rich information and editing recommendations to the user.

[0070] Other data sources may include the following. Medical ontology helps identify medical vocabulary and medical reasoning. If medical vocabulary is identified, the system can standardize the mentioned content and provide definitions and materials on the screen. And based on user behavior, the system can learn and make our model more accurate. The patient's patient profile and past EMR records, images and test results allow the editing recommendations to be personalized for the patient's situation. Clinical guidelines (both global guidelines and hospital- or practitioner-specific guidelines) help customize editing recommendations based on standard clinical best practices. Past patient EMRs (doctor, department, hospital-specific) help customize editing recommendations based on personal / department style and treatment methods. User profiles and user activity usage logs allow editing recommendations to be personalized based on personal style and preferences. Current (even if only partially populated) EMR (structured and unstructured) data allows editing recommendations to be customized based on the current context and structure of the report being typed. Language models (with the support of rich text) can ensure grammatical and logical consistency and help generate editing candidates. Summary report standards 328 such as template suggestions can help identify mandatory items.

[0071] The clinical information extraction module 304 can determine and output a statement identifier, subject, predicate, object, source text segment, text start position, confidence score, and result statement for a given input (e.g., a text string). The confidence score can come from an artificial intelligence or machine learning algorithm that provides the analysis and provides a confidence level that the metadata assigned or determined is accurate. In some embodiments, the additional tag information can be metadata.

[0072] Fig. 9 Further details about the clinical information extraction module 900 in an embodiment are shown. These features may be in the clinical information extraction module 304. Fig. 9 Includes EHR 902, entity recognition 904, assertion recognition 906, relationship recognition 908, and ontology linking 910. The clinical information extraction module 900 obtains the electronic health record text in the EHR 902. Then, pre-existing neural large language models can be created from large clinical text datasets. These models can be fine-tuned using refined, oncology-specific (or other medical expertise), specially annotated and enhanced text data, and then mapped to oncology-specific (as an example) lexicons, taxonomies, and ontologies.

[0073] In entity recognition 904, the text is analyzed to identify entities that can be classified in one of a variety of ways. In the example shown, a medical condition ("tumor") is found in the text along with a location ("location") and a body part ("body part"). This then allows the intelligent medical editor system to view other classifications and related databases for "tumor," "location," and "body part," in this example. In addition, the system can analyze the information more clearly and understand the information based on the entity types it recognizes.

[0074] In assertion identification 906, a subset of text with an identified type (in this example, "tumor") is analyzed and the text indicates that the patient has such a tumor. Therefore, the tumor is present in the patient, and the clinical information extraction module 900 asserts that the "tumor" is "present".

[0075] In relationship identification 908, relationships between different parts of the text that have been identified as entities are determined. Time, location, with which patient, by which doctor, which medical specialty, which medications and dosages, and other relationships can be determined by relationship identification 908. Fig. 9 In the example above, “tumor” was found at “location” and “body part”. Therefore, nodular neoplastic spread (“tumor”) was found at “location” and “in” the mesorectal fat (“body part”) between the 8 and 10 o’clock positions. The relationship between the data and the classification types is very helpful in providing useful context to the editorial recommendation module.

[0076] Additionally, certain entity types related to medical dictionaries, taxonomies, and / or ontologies may be further associated with related content in ontology links 910. These may be medical codes, proper medical names, or other agreed upon terms or expressions. Fig. 9 In the example of , tumor spread is linked to the standard SNOWMED CT code and reference number 38541300, and location is linked to RADLEX RID6028. SNOMED CT is one of a set of specified standards for the electronic exchange of clinical health information in U.S. federal government systems. RADLEX is related to the radiology dictionary. RadLex contains individual terms and concepts related to medical imaging that can be used in a variety of applications, including radiology reporting, data mining, and research. The RadLex Playbook combines RadLex terms using a consistent syntax to provide a set of standard exam names and codes. By illustrating an embodiment of Fig. 9 In the exemplary steps, an intelligent medical editor using a clinical information extraction module can gain context about the information entering the system and build some basic understanding, which is a good support in determining editing suggestions and annotation support.

[0077] return Figure 3 , the enriched information coverage module 306 obtains text data, structured data, and medical data structures 318. In addition, the enriched information coverage module can obtain the output of other modules (such as the clinical information extraction module 304). The enriched information coverage module 306 obtains input data and finds additional relevant information within the system or from outside the system (for example, a medical dictionary on the Internet, a medical guide in a hospital intranet, or other external systems). The enriched information coverage module 306 can then provide additional tags and deeper information (if requested by the user) about the content of the medical report (enriched information). Such tags (also included in the enriched information) can be entity tags for words and phrases, chapter tags, relationships or associations between entities, identification negations, weak assertions, assumptions and conditions, identification time assertions, and ontology concept identifications of entities. And this enriched information can be provided to the EMR editor 302 and displayed to the user for interaction. Fig.14 An example is shown of how the rich information coverage module can have a positive impact on the intelligent medical editor.

[0078] Fig.14 Details of a rich information overlay module are shown according to one embodiment. Fig.14The invention includes enriched coverage output 400, imaging test tags 1402, entity tags 1404, assertions 1406, time assertions 1408, mapping to medical ontology 1410, section title tags 1412, body part tags 1420, body part tags 1422, body part tags 1424, body part tags 1426, imaging findings tags 1430, imaging findings with body probability pointers 1432, lateral tags 1436, structure of reference tags with body direction tags 1440, structure of reference tags 1442, and structure of reference tags 1444. The imaging test tags 1402 and Fig.14 The medical report shown is associated with the CT examination text.

[0079] The imaging test label 1402 is an example of rich information overlaid on the report text on the display screen. The imaging test label 1402 includes an entity label 1404 (imaging test), an assertion 1406 (n=none), and a time assertion 1408 (←). Therefore, the imaging test label 1402 shows that the text is broken down into categories that can be linked and understood with other information. This also helps to click on each one to get more information. As described above, the rich information overlay module can find and / or receive additional information from internal and global sources, which can be provided to the user as more information if the label is interacted with. The imaging test label 1402 indicates in an alternative manner that a CT-type imaging test has not been performed previously. And the associated CPT code for this CT examination will be CT70498. Current Procedural Terminology (CPT) codes provide a unified language for doctors and healthcare professionals to encode medical services and procedures. In one embodiment, a user can click on a CPT code to get a list of other CPT codes and / or more information about the details of the code. This information can be provided by a rich information overlay module (such as rich information overlay module 306).

[0080] Chapter marking can be performed throughout data records, medical information, medical records, and medical reports. For example, Fig.14The section of the medical report shown is the "Findings" section. Therefore, the clinical information extraction module has identified the word "Findings" as a section title, and the enriched information overlay module has displayed a section title tag 1412. Individual body parts are also labeled in the report, such as body part tags 1420, body part tags 1422, body part tags 1424, and body part tags 1426. With the discovery of body parts, more information can be generated based on the number of times the body part appears, the vitality of the body part, the relationship of the body part to other body parts listed in the report, and other factors, thereby providing additional context for the report and helping to provide better editorial recommendations. Imaging discovery tags 1430 and imaging findings with body probability pointers 1432 are all findings of imaging scans performed by medical equipment. And the second imaging discovery has a pointer to a possible body part associated with the imaging discovery. This helps the system to link different medical tests with other areas of the context for the test (such as results and / or related body parts). External tags 1436 provide information about the location and direction of the structure or body part in the report. The structure of reference tags with body direction tag 1440, the structure of reference tag 1442, and the structure of reference tag 1444 provide additional information about the location and location of the body part referenced in the report. For example, the reference structure with body part direction tag 1440 indicates that it is the lower part of the rectum, with body part tag 1424. When performing machine learning or artificial intelligence analysis as part of the intelligent medical editor, the relationship and / or association between the marked entities may be very important. These provide additional rich information for both the user and the intelligent medical editor. The rich information from the rich information overlay module is provided to both the user and other modules in the intelligent medical editor.

[0081] return Figure 3 The editing recommendation module 308 is an important component of the intelligent medical editor. The editing recommendation module 308 provides editing recommendations to the EMR editor 302 for display on the display screen and for user interaction using computer input devices. Examples of editing recommendations are described in detail throughout the specification and drawings, including references to Figures 13 to 17 The editing recommendation module 308 obtains information from the entire system, such as obtaining information from the contradiction and incompleteness detection module 312 , the language model fine-tuning module 314 , and the user profile and preference 334 .

[0082] The editing recommendation module 308 can provide editing recommendations in real time when the user edits medical materials (such as medical reports or medical records). Therefore, the editing recommendation module 308 is an element that helps the intelligent medical editor to suggest tags / annotations while the health care professional is writing the report. An example of editing recommendations is related to the FHIR standard. The FHIR (Fast Healthcare Interoperability Resources) standard defines how health care information can be exchanged between different computer systems, regardless of how the information is stored in these systems. If the clinical information extraction module 304 identifies that some parts of the material may be related to some fields of the FHIR or report, the editing recommendation module can automatically fill in or automatically fill in the FHIR concept. The encoded FHIR concepts can be parsed from the report, and the editing recommendation module 308 can automatically fill in the FHIR data of the report, which is a much-needed feature for improving the work efficiency of doctors and bringing huge benefits to downstream applications.

[0083] Fig.10 Details of an editorial recommendation module according to one embodiment are shown. Fig.10 The editorial recommendation module includes an editing recommendation module 1000, a word / sentence / template completion module 1002, a template detection module 1004, a sentence rewording module 1006, an ambiguity reduction module 1008, a contradiction elimination module 1010, an integrity assurance module 1012, and an editing interpretation module 1014. The editing recommendation module has many types or categories of editing recommendations. These editing recommendations may be related to many types of problems discussed throughout this article, including but not limited to ambiguity, contradiction, incompleteness, rewording problems, and automatic completion of words, templates, ideas, phrases, paragraphs, or sentences.

[0084] The word / sentence / template completion module 1002 is enabled by a large language model (LLM), a neural network, artificial intelligence, generative artificial intelligence, a rule engine, and / or machine learning. In one embodiment, this can be done by a language model fine-tuning module 314. The LLM has acquired specific knowledge about the grammar, semantics, and ontology inherent in human language. The LLM that helps identify edit candidates can be a mask (MLM) or a causal LLM. Causal language modeling is the task of predicting a token following a sequence of tokens.

[0085] In addition, based on the previous medical history reported in the hospital or the usage log 332 of the user or department, patterns can be identified and / or clinical guidelines can be followed. The word / sentence / template completion module 1002 suggests text and other outputs that the user may like and / or want the rest of the word, sentence or template to be included in. This can provide useful automatic completion suggestions for partially typed words, partially typed sentences, partially typed templates, and various report fields such as in EMR. The system can detect the type of report and related templates through the template detection module 1004. In some medical environments, there may be a library of related templates that the template detection module 1004 can know and help pre-fill.

[0086] The sentence rewording module 1006 analyzes the sentences and can suggest rewording improvements. This can be based on best practices learned from analyzing other reports, based on correct medical wording, or based on language issues. Typical issues that the sentence rewriting module 1006 addresses include lack of conciseness, lack of clarity, reformatting changes, need to be consistent with different templates or formats, incorrect use of correct medical terminology, etc. In addition, if the language of the report is not the user's first language, the sentence rewriting module 1006 can provide helpful suggestions for the language of the report.

[0087] The ambiguity reduction module 1008 analyzes the report and provides suggestions to the edit recommendation module 1000 to help improve clarity and reduce ambiguity. The ambiguity reduction module 1008 can resolve issues such as the presence of unknown words, incomplete or ambiguous sentences, pronoun problems, anaphora, ellipsis, relative time references, polysemous words, acronyms, and abbreviations. The detection and remediation of such issues can be performed in submodules or subroutines. Fig.10 An ambiguity reduction module 1008 is shown, which includes sub-modules of an unknown word handler, a polysemous word handler, and an anaphora handler.

[0088] The inconsistency resolution module 1010 analyzes the report and provides suggestions to the edit recommendation module 1000 to resolve the inconsistency. The inconsistency may include statements that contradict other parts of the report, statements that contradict patient data, statements that contradict medical ontology, statements that contradict clinical guidelines, format inconsistencies within the document, and formatting inconsistencies with summary standards. For example, if the patient is male and the report shows that the patient is female, or the report shows that the patient has ovarian cancer, there may be a contradiction. Sometimes the contradiction may be caused by an error (such as a typographical error). One part of the report may show that the patient weighs 176 pounds, while another part of the report may show that the patient weighs 17 pounds. The inconsistency resolution module 1010 may notice the contradiction and also suggest an edit displayed on the screen, that the second weight (17) should be updated, potentially updated to 176.

[0089] The completeness assurance module 1012 analyzes the report and provides suggestions to the edit recommendation module 1000 to assist in completeness. Incomplete reports may cause medical problems and waste time. Incompleteness issues addressed by the completeness assurance module may include lack of cross-reference links (e.g., links to previous reports, images, or medical reference data sources), incomplete complaints, diagnoses, or treatment histories, broader concepts replacing more refined concepts, and missing aspects compared to reporting standards. Many medical professionals face time pressures and may not be able to fully complete medical orders, reports, and records. By providing specific edits and suggested text, they may feel less burdened when completing words, thereby improving patient care and speeding up medical workflow. One suggestion that can be output on the user's screen is to automatically follow up with another hospital or another doctor. If the patient's medical record is incomplete, the system can automatically generate a communication to an entity that may have the required information. Then, an edit recommendation can be displayed on the screen, such as "Contact Dr. Faber to obtain a copy of the imaging scan?" If the user clicks on the recommendation, a draft email or other communication type that has been completed and is available for editing and / or sending can be displayed on the screen. This may help the completeness of the record without burdening the medical professional. The completeness assurance module 1012 may include submodules for concept elaboration (adding more text when a concept is not fully elaborated), cross-reference linker (providing links if multiple parts of a report can be cross-referenced for better understanding), and consistency enforcement.

[0090] The edit explanation module 1014 provides additional information and context about why a certain edit was recommended. A user using the intelligent medical editor may not understand the reason for the recommendation or how the recommendation was generated. By clicking on the recommendation (or automatically provided without user input prompt), the intelligent medical editor can provide additional information provided by the edit explanation module 1014. Such explanations can explain why the cross-reference was made and can provide links to relevant patient data models and clinical guidelines. Fig.10 The modules shown in are not an exhaustive list, and other features described throughout this document may be provided by the editorial recommendation module 1000.

[0091] Return to Figure 3, the use and feedback log module 310 is crucial for processing the user's request and training the system for future editing suggestions. How each user interacts with the system is received from the EMR editor 302 as user decisions and feedback. The use and feedback log module 310 records user decisions and feedback, and provides them to the use log 332 and / or user profile and preference 334. The explicit request from the user and the user's personal use mode (click, touch, visual contact and any other type of input) can be recorded. The user may receive a pop-up window or other screen or voice feedback, asking how the experience is or recommending improvement suggestions for a specific editing. When the user does not take the recommended editing, these are especially valuable. The editing suggestion may have different types of feedback, such as there is an error, missing, or the data provided is not fine or too fine.

[0092] In one embodiment, the inconsistency and incompleteness module 312 can be its own module, such as Figure 3 As shown, or in one embodiment, it can also be part of the editor recommendation module, such as Fig.10 See the details of the inconsistency resolution module 1010 and the integrity assurance module 1012 mentioned above. The inconsistency and incompleteness module 312 can detect differences between the content being written and the content available from the patient medical record, clinical care, and medical ontologies.

[0093] Fig.17 A user interface with medical corrections in a digital document is shown according to one embodiment. The inconsistency and incompleteness module identifies inconsistencies 1704 with the medical report 1702 introduced in the user interface 1700 of the intelligent medical editor. A visual icon may be displayed. A pop-up window 1706 or sidebar window may be displayed to explain the detected inconsistency or error. And suggestions may be made to the user for automatic correction. Fig.17 In the example above, the intelligent medical editor might suggest changing the word “male” to “female” in the report. Fig.17 Also shown are toggles associated with features of the intelligent medical editor. These may include a domain knowledge option 1708, an enable automatic annotation option, and a protocol suggestion option. Thus, the user may control the editing recommendations and may adjust the features based on their preferences. Additionally, the inconsistency and incompleteness module may detect that the word "male" in the report is incorrectly spelled as "mail" and automatically correct or provide a prompt with the editing suggestion to the user.

[0094] Fig.11 A contradiction and incompleteness detection module according to one embodiment is shown. Such a module can use patient data models, clinical guidelines, and medical ontologies to perform fact contradiction and incompleteness detection. Figure 3 and Fig.10In addition to the discussion in Fig.11 Additional technical details related to this module are also given. Fig.11 It includes a contradiction and incompleteness detection module 1100, a knowledge graph comparison module 1102, a knowledge graph creation module 1104, and a guide consistency detection module 1106 (about Fig.12 further discussed), formatting normalization module 1108, standard incompatibility detection module 1110, medical terminology mention normalization module 1112, and cross-reference lookup module 1114.

[0095] The knowledge graph comparison module 1102 can obtain the output of the clinical information extraction module and includes many fields, such as statement ID, subject, predicate, object, source text segment, text starting position, confidence score and result statement. The knowledge graph in the system can be compared with the knowledge graph of the specific record being edited to find contradictions and incompleteness. Such knowledge graphs can be created by the knowledge graph creation module 1104, which can obtain data about the patient (such as including statement ID, date, subject, predicate, object pointer to source information, patient journey stage, confidence score and structured data of result statement). In one embodiment, the patient-specific knowledge graph can also be considered as a patient data model. Semantic networks can be used to generate and compare knowledge graphs. Semantic networks are knowledge structures that describe how concepts are related to each other and explain how they are interconnected. Semantic networks use artificial intelligence (AI) programming to mine data, connect concepts and draw attention to relationships: using a CPU to tag parts of speech in text; using a CPU to identify known words in the text; and using the CPU to create a semantic network, the semantic network including at least one of the identified known words and at least one relationship to at least one semantic concept associated with at least one of the identified known words; and wherein the output is related to the probability of a specific occurrence.

[0096] Summary report standards such as summary report standards 328 may be used by the formatting standardization module 1108 to provide incompleteness detection and report structure recommendations. The medical term mention standardization module 1112 reviews the use of medical terms and provides suggested edits to align the common usage, spelling, and abbreviations of medical terms in the report. The standards incompatibility detection module 1110 compares the report to medical industry standards such as billing codes, dental guidelines, medical rules, and hospital requirements. The cross-reference lookup module 1114 detects whether cross-references should exist between information items and provides edit suggestions to add cross-references.

[0097] return Figure 3, the language model fine-tuning module 314 collaborates with the clinical language model 330, industry data / standards, and usage logs 332 to update and fine-tune the language model. This can be an LLM as discussed herein, and can include generative artificial intelligence capabilities to continuously update and fine-tune the language model. The language model fine-tuning module 314 provides the specific language used in the editing recommendation to the editing recommendation module 308 based on the type of report, the users involved, the context of the text, and other factors such as medical ontology 332, medical data structure 318, patient profile and past Dx / Tx data 324, clinical guidelines 326, summary report standards 328, and user profiles and preferences 334. The language model fine-tuning module 314 allows the intelligent medical editor system to realign the report to a different template using a large language model. In addition, the language model fine-tuning module 314 can be connected to the NLU model update module 316 and share bidirectional information with it, because both modules process natural language.

[0098] Medical data structure 318 may have many types, depending on the medical EMR company, IT vendor, government regulations for medical data, which fields may be required for certain disease types, varying report record types depending on medical specialties, database requirements, and other factors.

[0099] The NLU model update module 316 receives the usage log 332 and the joint NLU model. The NLU model update module 316 provides the updated NLU model to the NLU model 320. These NLU (natural language understanding) models may also include NLP (natural language processing), NLG (natural language generation) and NWP (natural text processing) models. These will be combined Figure 5 , Figure 6 , Fig. 8A , Figure 8B The NLU model 320 is provided throughout the intelligent medical editor because many parts of the system require natural language processing, generation and understanding, such as the clinical information extraction module 304, the rich information coverage module 306, the edit recommendation module 308, the usage and feedback log module 310, the contradiction and incompleteness detection module 312 and the language model fine-tuning module 314.

[0100] Patient Profile and Past Dx / Tx Data 324 is a collection of all patient data across the system. The more accurate the data about the patient, the more insights and recommendations the intelligent medical editor can make. Patient profile data may come from EMRs, medical reports, medical imaging scans, medical notes, patient communications, etc. Patient profile data may include their allergies, problems, medical history, and medications. Fig.19 More discussion was given about patient data existing in hospital settings and the use of patient profile data to determine insights and recommendations.

[0101] User profiles and preferences 334 store and analyze information about past, current, and future users of the EMR editor 302. Each user may have a different role (such as a nurse, primary care physician, radiologist), different abilities to use software technology, different knowledge of medical terminology, different knowledge of medical standards, etc. User profiles and preferences 334 obtain information from the usage and feedback log module 310, and the output may be provided to any module, but specifically to the edit recommendation module 308. Edits associated with a report may sometimes differ depending on the users involved. For example, one user always uses abbreviations, while another user always enters full words. Or if a user has a pattern of using a certain drug to treat the same patient's disease, it may be used as a recommendation in the future. Therefore, pattern recognition and artificial intelligence learning are very helpful for user profiles and preferences 334, and the module may have such technology built in.

[0102] Figure 4 A flow chart of an intelligent medical editor according to one embodiment is shown. Diagram 400 includes EMR editor 402, user 404, clinical incorrectness detection 406, clinical warning 408, natural language understanding 410, natural language generation and natural text processing (NWP) - NLG 412, medical ontology 414, cause-effect pairs 416, patient history 418, clinical care guidelines 420, and vocabulary (vocab) standardization 422.

[0103] EMR editor 402 interacts with user 404 through a computer input and output system. User 404 uses EMR editor 402 as an intelligent medical editor to generate, edit, and improve medical materials such as radiology reports, electronic medical records, and prescription orders. Clinical inaccuracy detection 406 and clinical warning 408 can be modules that receive drug information from input modules such as medical ontology 414, cause-effect pairs 416, patient history 418, clinical care guidelines 420. Figure 4 In an embodiment of the present invention, an edit suggestion module may be included in the EMR editor 402. The EMR editor 402 may receive warnings and incorrectness detections and determine edit suggestions with the aid of language interactions enabled in the system. The EMR editor 402 provides the user with displayed edit suggestions for the user to select and interact with, for example, Fig.13 shown.

[0104] In various embodiments, clinical incorrectness detection 406 and clinical warning 408 may have analysis code with software algorithms and / or artificial intelligence models. Clinical incorrectness detection 406 uses medical ontology 414 and cause-effect pairs 416 to find patterns and relationships between information in patient history 418 and the medical condition that the patient is evaluating. Then, clinical incorrectness detection 406 detects whether there is clinical incorrectness and medically possible conditions in the current information in the report based on cause-effect pairs 416 and patient history 418. Patient history 418 can receive information signals about the patient in the form of images and / or text. An example of detecting clinical incorrectness is that the patient has no left leg, but the imaging report shows a left knee fracture. Cause-effect pairs 416 will detect that a body part was removed in a past operation (reason), and therefore the current report cannot be based on the body part. Or the patient has an ovarian cyst, but the patient has no ovaries. Discovering such incorrectness can save lives, save time, and maintain the reputation of medical professionals.

[0105] Clinical warning 408 can provide detected warnings to EMR editor 402 for display on the user display. In various embodiments, such warnings can also provide suggested edits and / or changes. If a treatment decision is made that is not in any treatment option recommended by the nursing guide, a warning may pop up. Such guidelines may include the National Comprehensive Cancer Network (NCCN) cancer guidelines. By browsing the patient history guide, possible treatment options can be found.

[0106] Language interaction is achieved through NLG 412, NLU 410, and vocabulary standardization 422. This is discussed elsewhere in Figure 3 , Figure 5 , Figure 6 , Fig. 8A and Figure 8BFurther discussion. NLG 412 generates natural language and includes a next word prediction module for suggesting the next word when writing a report. NLG 412 can perform content determination, document structuring, aggregation, vocabulary selection, reference expression generation and / or implementation actions. NLU 310 takes input reports and patient medical histories and generates a natural language understanding of the content. NLU includes rearranging unstructured data so that the module can "understand" and analyze the data. Therefore, this natural language understanding can be compared with the medical ontology 414 and clinical care guidelines 420 as examples. The medical ontology may include tables such as preferred names (such as cisplatin / epirubicin), synonyms (such as cisplatin / epirubicin, cisplatin-e therapeutic implants, CDDP-eTI and CDDP / EPI), medical ontology IDs (such as medical institutions or organizations' websites and links to specific pages) and codes (such as C11555). Vocabulary standardization can obtain medical ontology information, including synonyms. All mentioned medical phrases (including abbreviations or synonyms) will be mapped to the standard index of the ontology. For example: Non-small cell lung cancer - NSCLC-C2926. This helps standardize reports regardless of writing style. In addition, if a user editing a report calls the same thing three different ways (such as by abbreviations or shorthand or slang), vocabulary standardization 412 can suggest consistent naming throughout the record or report related to that thing.

[0107] Figure 5 A natural language processing (NLP) model update module according to one embodiment is shown. Figure 5 It includes a networked system 500, a basic NLP model 502, a remote site 504, a model fine-tuning 506, a new model 508, and a model aggregation 510. The basic NLP model 502 is an NLP model that has been trained on sufficiently general tasks for downstream applications. The goal of the networked system 500 is to provide the benefits of long-term improvements in NLP through a large amount of manually annotated data. This can also help improve downstream clinical applications.

[0108] After the basic NLP model 502 is provided to the medical remote site 504, the medical staff at the remote site will use the model in their intelligent medical editor. Through the process of selecting editing suggestions, corrections, reacting to information, and receiving feedback, the model is fine-tuned for that specific remote site. Then, the fine-tuned model 506 is a new model 508 that can be provided back to the creator of the NLP model 502. Using the model aggregation 510 technology, the basic NLP model 502 can be improved for the entire system, not just a single remote site 504. Model aggregation 510 obtains a new model 508 based on the fine-tuned models 506 from multiple remote sites 504. In one embodiment, the NLP model 502 can be a large language model.

[0109] The networked system 500 in the figure shows how NLP models can be improved through crowdsourced feedback and improvements. This allows clinicians (oncologists, nurses, and primary care physicians) to participate in technology crowdsourcing. It is critical to ensure that the performance of AI and NLP models is sufficient to meet clinical standards before integrating them with key clinical systems such as EMRs or other medical reports. Large, diverse, and authentic manual feedback is critical to validating and continuously improving AI and NLP models. Two obstacles that hinder the collection of such feedback are: the high time and cost required for high-quality manual feedback; and data privacy requirements that lead to administrative and data anonymization overhead. The system in this article provides (1) creating a low-invasive way for clinicians to provide in-situ feedback to AI and NLP, and to improve AI and NLP models by creating a joint approach. This eliminates the need to share patient data outside organizational / legal boundaries, thereby enabling widespread crowdsourcing of training / validation data for such models. Therefore, improvements can be made in a joint manner without compromising patient privacy. These benefits and features are also related to Fig. 8A and Figure 8B Related to the system shown in .

[0110] That is, each remote site 504 may still have a model 506 fine-tuned specifically for its site, its guidelines, its personnel, jurisdictional cultural or legal requirements, and the site's associated tastes. The system can tailor the use of the intelligent medical editor to the preferences of a specific user, a specific department, and a specific location.

[0111] Figure 6 A natural language generation (NLG) module with a natural word prediction (NWP) module is shown according to one embodiment. Figure 6 Included are natural language system 600, NWP module 602, NLU module 604, EMR editor 606, user behavior 608, and downstream tasks 610. Downstream tasks 610 can be numerous in a medical environment and include using such outputs in an intelligent medical editor such as EMR editor 606. Downstream tasks may include summarization of previous reports and highlighting important information.

[0112] The task of the NWP module 602 is to predict upcoming words (or provide multiple suggestions) based on a string of input text, images, or other input content. The predicted future words are provided to downstream tasks 610. Figure 6 In the simplified example of , the NWP module 602 takes the input "the boy sits on..." and predicts that the next word may be "chair", "sofa" or "bench".

[0113] The EMR editor 606 can provide input to the natural language processing system 600 from the content of the EMR or other medical materials. In addition, the current user behavior 608 provides real-time input to the natural language system 600. The user behavior 608 may affect the NLU module 604 and the NWP module 602, because each user may have a different language style. Since different styles that may include education levels and compliance with appropriate terminology, the natural language system 600 may be able to combine the user's style and provide it to other users in editing recommendations. For example, if there is a radiology supervisor who does an excellent job or wants employees to work consistently, the system can use the NLU module 604 to understand the user behavior 608 of key users, and then provide the department staff with relevant editing recommendations in their medical materials. In addition, if government officials or regulatory agencies expect medical reports and materials to be written in certain standard ways, this can also be fed into the NLU module 604 and NWP 602 to improve editing recommendations for users. In one embodiment, when editing recommendations come from a specific source or are based on specific inputs, they can be color-coded or marked in a specific way. For example, if the editorial recommendation has three options, the user-customized editorial recommendation might take one color or mark, the department head editorial recommendation might take a second color or mark, and the government editorial recommendation might take a third color or mark. Providing users with a little context and information about the source of certain editorial recommendations in the smart medical editor might help them adopt and become proficient in using such a tool. It might also help shape behavior.

[0114] Figure 7 A block diagram supporting naturalistic report generation according to one embodiment is shown. Figure 7 Includes system 700, standard report generation 702, medical information 704, patient EHR DB (database) 706, disease DB 708, text parser 710, language system 712, generative artificial intelligence large language model service 714 (GAI LLM service), word prediction 716, backend application (application) 718, medical dictionary 720, user interface 722, security layer 724, patient medical history 726, records from other hospitals 728, family history 730, and patient physical activity 732.

[0115] The intelligent medical editor can use the report generation capability of a system similar to system 700, which includes standard report generation 702. Standard reports can be made for different types of diagnoses and treatments. Standard report generation 702 can obtain information from various sources from a text parser, such as patient medical history 726, user input through user interface 722, back-end application 718, and medical information 704. Standard report generation obtains a report template or a previous report example and generates a new report. Then, it pre-fills the areas it knows are relevant (such as patient names and other factual data in the record). In the intelligent medical editor described throughout the specification, it can also provide editing recommendations and suggestions for other parts of the report that are not fully understood or may require medical professionals to select. Therefore, users can interact with reports that have been completed for most of the content. And standard report generation 706 generates reports with pre-filled information and editing recommendations based on previous user reports, other reports, medical standards, and third-party reports. This allows the use of best practices. In addition, a lot of time can be saved by such a system. The text parser 710 can use the GAI and LLM services 714 and word prediction 716 to better provide factual information for pre-populating standard report generation 702 and improving editorial recommendations. The text parser 710 can directly access the medical dictionary 720 to help it understand obscure or uncommon medical terms.

[0116] Patient history 726 may include internal information and external information. Internal information may include patient EHR records from patient EHR DB 706. Patient history 726 may also include records from other hospitals 728, family history 730, and patient physical activity 732. These may come from other devices such as smart phones, smart watches, activity trackers, medical devices in the home or medical environment, family history in family members' EHR records, and other locations.

[0117] Back-end application 718 may be a specific software application for a disease, country, hospital, care pathway, or other target group. Back-end application 718 may be provided by a specific vendor and may help provide information related to a specific report, such as government required reports that some vendors excel at generating and / or pre-populating. This allows for more efficient and accurate report generation. Additionally, back-end application 718 may be an application related to natural language processing or interaction to help text parsers better handle certain types of information. Security authorization layer 724 allows for appropriate security and authorization to prevent the disclosure of important user and patient information.

[0118] Fig. 8A A local model for language fine-tuning is shown according to one embodiment. Fig. 8AThe system includes a fine-tuning system 800, a language fine-tuning module 802, an editing option lookup module 804, a clinical language model 806, a hospital-1 810, a radiology report 812, a pathology report 814, a consultation note 816, a hospital-specific model 820, and a hospital-specific recommendation 822. The hospital can provide specific recommendations 822 to its users through the editing option lookup module 804 (also referred to as the editing recommendation module) of the intelligent medical editor. The hospital-specific recommendation 822 may come from hospital guidelines or rules. These recommendations can also be based on best practices detected from the medical materials of hospital-1 810, such as radiology reports 812, pathology reports 814, and / or consultation notes 816. Therefore, a hospital-specific model 820 can be generated for language and editing recommendations. The clinical model 806 can provide a publicly available language model trained on a large corpus of medical documents, and then the language model fine-tuning module 802 and / or the editing option lookup module 804 can be specialized to perform hospital-specific downstream tasks.

[0119] The language model fine-tuning module 802 receives a clinical language model 806 and medical materials specific to hospital-1 810, such as radiology reports 812, pathology reports 814, and / or consultation notes 816. Thus, the language model fine-tuning module 802 can use hospital-specific reports to fine-tune its language model, thereby allowing hospital-specific recommendations to be adopted. For example, hospital-1 may be located in a place where certain terms or languages ​​are used that are different from other hospitals, and certain dialects are more common. Another example is that a certain hospital requires a specific radiology report section in its report, while other hospitals do not (in one example, insurance or government billing). As another example, the model may need to be fine-tuned to adapt to a specific application to achieve the desired results (such as clinical word classification, relationship assertion, word / sentence classification, etc.). The language model fine-tuning module 802 can provide a hospital-specific model to the edit option search module 804, thereby allowing the edit option search module 804 to suggest the edit that is most suitable for the hospital in question. This can also be done at a more specific level, such as by department or by individual user. These aspects of the systems and methods may assist with EMR and reporting tasks such as document aggregation, similar documents, and finding key information to pre-populate medical reports.

[0120] Figure 8B Details of a language tuning module are shown according to one embodiment. Figure 8BIncludes a fine-tuning ecosystem 850, radiology reports 852 (including radiology reports 812 from hospital-1 and other hospitals), pathology reports 854 (including pathology reports 814 from hospital-1 and other hospitals), consultation notes 856 (including consultation notes 816 from hospital-1 and other hospitals), language fine-tuning module 802, editing option lookup module 804, clinical language model 806 and report / test type recommendation 860. Figure 8B The value of distributed and / or crowdsourced information to help improve language models and editorial recommendations is shown. This allows clinicians, oncologists, nurses, assistants, physicians, and others in the medical system to provide useful feedback to the system for improving language models and editorial recommendations in the intelligent medical editor. In addition, the models can be fine-tuned not only based on the requirements of the system users or hospitals or governments, but also based on the patients. For example, fine-tuning may be based on the patient's timeline medical history, disease type, treatment type, device use, drug dosage, and other factors in the patient's journey. To effectively help with recommendations, multiple models can be aggregated and recommendations can be made based on specific criteria / domain-specific rules.

[0121] exist Figure 8B In the example, a fine-tuning ecosystem 850 exists to collect similar types of medical materials from different medical institutions (e.g., hospital-1, hospital-2, and hospital-3). With this information, the language model fine-tuning module can create models based on trends across geographies, medical systems, and medical systems of different sizes, all while developing models specific to the type of medical material involved. Therefore, the language model and editorial recommendations can be specific and predictive for the report / test type, as shown in the report / test type recommendation 860. For example, the terminology in the oncology care pathway may be different from the cardiology care pathway.

[0122] Fig.12 A clinical care guideline deviation process is shown according to one embodiment. Fig.12 It also addresses how the clinical guidelines 326 may interact in the system, including interacting with the inconsistency and incompleteness detection module 312 . Fig.12 Included are clinical guideline workflow 1200, clinical guideline 1202, and deviation 1204. Clinical guideline 1202 in this example is a clinical assessment pre-treatment assessment that may suggest initial treatment. Certain tests may be performed on the patient, and based on the results of the tests, certain treatments may be recommended and / or required. If a user selects a non-standard treatment, medical test, and / or medication in clinical guideline 326, this may be flagged as a deviation 1204. The intelligent medical editor may display a deviation alert and suggest an alternative recommendation. Fig.13An example of this is the contradiction notification 1322. The recommendation might be "Guidelines have suggested this initial treatment." Additionally, if different countries have different guidelines, the intelligent medical editor can provide the correct guidelines for that location / hospital and also provide recommendations for other hospitals / other hospitals around the world.

[0123] Such systems improve care and reduce the multiple decisions that can lead to healthcare disparities. As a result, such systems save costs and provide better care to patients. Unnecessary deviations in care lead to suboptimal care outcomes, increased costs, and compromised quality of life. Clinical standardization remains the top opportunity for CFOs to save costs while maintaining quality. Cancer guidelines and pathways guide physicians through data and flowcharts to provide clinically optimal and cost-effective care. The high complexity of cancer diseases and the multiple treatment possibilities make adherence to this treatment regimen very challenging and can lead to widespread deviations. Fig.12 The example of a lung cancer guideline shown in illustrates the numerous decision points and data required to traverse the guideline to a relevant point.

[0124] If the user approves a deviation from the guideline, the system can save the deviation for reporting and analysis. And then in the future, the system can check whether the current deviation is consistent with past deviations for the same type of patient / condition, and provide a percentage confidence of the previous deviation. The intelligent medical editor can also suggest which deviations were approved in the past on similar patients. In addition, the system can provide statistics related to the deviation, such as how much deviation there is for the clinician. And in some cases, if the entire medical field reaches a consensus, the guidelines can be changed based on the collective deviation in favor of health care.

[0125] Fig.13 A user interface with suggested edits to a digital document is shown according to one embodiment. Fig.13 Included are a medical editor 1300, a user interface 1302, event date recommendations 1304, internal conflict recommendations 1306, date recommendations 1308, missing information recommendations 1310, formatting recommendations 1312, spelling recommendations 1314, acronym expansion recommendations 1316, image source recommendations 1318, ontology code link recommendations 1320, inconsistency notifications 1322, and diagnosis recommendations 1324. The user interface 1302 shows a medical report that already has some content. The content may be typed, pre-filled by the intelligent medical editor, or a combination of both. The different sections of the report have titles that are displayed in capital letters on the left side of the report. A system (such as the system of FIG. 2 ) reviews the material in the report and can provide alerts, notifications, and recommendations to help improve the content of the report.

[0126] These alerts, notifications, and recommendations can be provided in different ways. Fig.13Alerts, notifications, and recommendations are shown overlaid on the relevant sections of the report. Alternatively, these may be small alert bubbles or icons that you click to get more information. Alternatively, these may be located in a sidebar or top bar of a user interface window. Alternatively, these may be implemented through interactive chatbots or conversational interactions. These may be supported by generative artificial intelligence and large language models. There may be other ways of alerting and interacting, such as vibration, touch, voice and other audio inputs. Alerts, notifications, and recommendations may also have feedback buttons or icons. This allows users to point out errors, problems, suggestions, or positive feedback on alerts, notifications, or recommendations. The system may use this to improve its language models and edit recommendations.

[0127] The event date recommendation 1304 is an example of adding additional information to the report that helps provide complete details. In this example, the medical editor 1300 recommends adding "August 23, 2022" to the report, referring to the date the patient presented with the medical complaint. This improves accuracy by using the event date recommendation 1304 of the patient's past data. This can be done through the patient profile, such as reference Fig.19 As discussed further above. As an example, the medical editor may detect references to past events in the report based on looking at past tense words, tracking patient journeys, and / or language models of certain fields in the EMR. If more information about the past event is available, the editorial recommendation module may make recommendations similar to the event date recommendation 1304. It may be suggested to provide more information about past visits, and both short and long recommendations may be provided. And based on the user's interactions (most frequently selected), the intelligent medical editor may prioritize the recommendations that the user likes the most. The date recommendation 1308 also suggests adding dates related to recent medical examinations (in this case, mammography).

[0128] The internal conflict recommendation 1306 may use the rest of the report or other patient profile information to detect conflicts. In this example, the report on the fourth line states "left breast." Therefore, the third line reads "right breast," and the intelligent medical editor (e.g., via the inconsistency and incompleteness detection module 312) may detect the internal inconsistency and ask the user for clarification. If the user clicks on the internal conflict recommendation 1306 (in this case, "left?"), the intelligent medical editor may automatically change the word "right" to "left." Another example of an internal conflict is an incorrect date listed in the report, such as a date associated with an event date recommendation.

[0129] Missing Information Recommendations 1310 may provide recommendations for additions to medical material. This would be useful information that could be added to the report but is not currently present. Fig.13In the example of , the STAGE field of the medical report only says "Stage 1." Staging refers to the extent of the cancer, such as how large the tumor is and whether it has spread. Missing information recommendation 1310 suggests showing "TNM staging": T1, N0, and M0. T refers to the size and extent of the main tumor. The main tumor is often called the primary tumor. N refers to the number of nearby lymph nodes that have cancer. M refers to whether the cancer has metastasized. This means that the cancer has spread from the primary tumor to other parts of the body. Therefore, with the recommended missing information added, the report is more accurate, clear, and helpful for oncologists reviewing the cancer.

[0130] Formatting recommendations 1312 highlight ways to improve the formatting of medical reports. For example, certain report types require or recommend certain formats. In other examples, the language model may detect that the report has a custom format, and the edit recommendation module may recommend edits related to maintaining the custom format. Fig.13 In the example of , all other report titles are in uppercase letters, so the medical editor 1300 recommends changing "Recommendations" to "RECOMMENDATIONS". This improves readability by recommending formatting changes. Related modules can also be used to recommend other types of formatting, spelling, and grammar edits. Certain types of spelling may be specific to medical ontologies and medical lexicography. Spelling recommendations 1314 point out spelling errors for specific drugs (not in common dictionaries) and provide corrected spelling recommendations. It is useful to have specialized medical dictionaries, guidelines, and codes. In addition, certain types of UB-04 and CPT codes may exist in EMR records, and the intelligent medical editor may detect issues of inaccuracy, incorrect misnaming. The unified billing form (called UB-04 or CMS1450) is a key element in the healthcare billing process. In addition, if the report has a summary report standard, the system can identify the parts of the report that do not follow the summary report standard. Summary reports present information as discrete data elements and associated responses.

[0131] Acronym expansion recommendations 1316 help standardize communication in medical reports. When a user uses an uncommon acronym (or an acronym that is better suited as a full word), the editorial recommendation module can suggest stating the acronym in full. These may come from a medical dictionary or medical ontology. This can be helpful and more precise if a patient or other non-medical professional will be reviewing the report or record, and they may not know all the acronyms that a medical professional may use. Fig.13 In the example of , the report has the phrase "UOQ of left breast." Acronym expansion recommendation 1316 recommends changing "UOQ" to "Upper Outer Quadrant."

[0132] Image source recommendations 1318 may provide additional source information about the content. This could be an internet content source with a hyperlink, or it could be a source location in an imaging database such as PACS - Picture Archiving and Communication Service, or a source location in a patient profile or medical record. By adding links or other source information, the person reading the report is able to better understand the details of the situation and make better decisions in a faster manner. Fig.13 In the example of , a mammographic image of the left breast shows a mass in the upper outer quadrant. Image source recommendations 1318 recommend adding the exact series and images related to mammography. Image source recommendations 1318 may also have clickable links for the viewer to see the exact images. This improves completeness by using the patient's past radiological image data for image source recommendations.

[0133] Ontology Code Link Recommendation 1320 Use medical ontologies to improve accuracy in medical reports. Fig.13 In the example of , the medical editor 1300 recommends adding ICD codes 147.9 and 174.4 to the report. The International Classification of Diseases (ICD) is a system used by physicians to classify and code all diagnoses, symptoms, and claim procedures. For example, the ICD-9 code 174.9 for malignant neoplasm of unspecified site of breast (female) is a medical category listed by the WHO within that range. This is important for adding Fig.13 It is valuable to have all the correct diagnostic terms and codes in the report because it is in a specific section of the diagnostic report. Therefore, it is valuable to have all the correct diagnostic terms and codes.

[0134] Notice of Inconsistency 1322 states that the recommended treatment is inconsistent with clinical guidelines. Fig.12 Further discussion and accuracy can be improved by using clinical guidelines to identify inconsistencies with current medical knowledge. Fig.13 In the example of , the medical editor 1300 notes that the recommended treatment conflicts with the NCCN (National Comprehensive Cancer Network) BINV-1 guidelines for breast cancer treatment and recommends adjustments to the recommended treatment plan. In some embodiments, if the user ignores this suggestion, the system may request more information about the medical professional's reasoning to record why they deviated from the clinical guidelines. In addition, the diagnostic recommendation 1324 also recommends editing the tests customized for the patient to add a liver function panel (LFT) test based on clinical guidelines. Time can be saved by completing all relevant medical tests at the right time.

[0135] Fig.16 A user interface with suggested edits to a digital document is shown according to one embodiment. Fig.16An intelligent medical editor 1600 is included that includes a user interface for interacting with a user on a computing device. This could be a personal computer, smartphone, tablet, medical imaging device screen, or other input / output device. Fig.16 A medical record 1602 including information about a patient is shown. For example, the clinical information extraction module 304 determines that "lungs" is a body part 104 and that a medication name is a medication that can be compared by the inconsistency and incompleteness detection module 312 to verify that it is a correct medication name 1608. This is done through a medical spelling correction pop-up recommendation and an annotation pop-up, which is similar to Fig.13 The recommendation above is different from that shown in . Therefore, Fig.16 An intelligent medical editor is shown, which focuses on specific domain knowledge in the medical field and supports automatic annotations and suggestions.

[0136] Fig.18 A block diagram related to editorial recommendations utilizing predictive and generative models is shown according to one embodiment. Fig.18 Components that may be in the edit recommendation module 1800, edit recommendations 1802, current EMR report and details 1804, medical input 1806, fine-tuning module 1808, patient data 1810, prompt selection 1812, predictive model 1814, generative model 1816, base module 1820, and edit option lookup module 1822 are shown. Inputs to the edit recommendation module 1800 include medical input 1806, which provides a baseline for the base module, medical ontologies, medical dictionaries, and other medical inputs. Fig.18 In the example of , medical input 1806 includes MeSH tree structure data for medical conditions. The Medical Subject Headings (MeSH) dictionary is a controlled and hierarchically organized vocabulary produced by the National Library of Medicine. It is used for indexing, cataloging and searching biomedical and health-related information. Input to the edit recommendation module 1800 also includes patient data from EMR reports and details 1804 and patient data 1810 including timeline metadata (such as internally hosted data from within the hospital system).

[0137] Prompt selection 1812 selects the most relevant information from multiple options that are appropriate for the data currently being reported. Fine-tuning module 1808 receives the clinical language prediction model (performing tasks such as classification and extraction), fine-tunes it based on feedback and usage information within the system, and provides the fine-tuned model to base module 1820. Fine-tuning module 1808 selects task-specific models based on the current activities of the intelligent medical editor and its users. In various examples, this may mean that fine-tuning module 1808 selects models specific to hospitals, diseases, or patients. The fine-tuning process helps the model access external knowledge sources and improve its performance.

[0138] The base module 1820 obtains the fine-tuned model from the fine-tuning module 1808, the selected information and data from the prompt selection 1812, the clinical language generative model 1816, metadata, ontologies, clinicians, and other system inputs. The base module 1820 determines the factual information in the content it generates. This reduces the problem of illusions and inaccuracies and helps to minimize the randomness that may come from the use of GAI. The clinical language generative model 1816 is a generative artificial intelligence (GAI), a generative language model (GLM), a base model, and a large language model (LLM), which perform tasks such as summary generation, text generation, and conversational agents. Artificial intelligence (AI) and generative language models (GLM) provide important opportunities for enhancing medical education, including providing realistic simulations, digital patients, personalized feedback, evaluation methods, and eliminating language barriers. The clinical language generative model 1816 has been specifically trained in the clinical field.

[0139] The editing options lookup module 1822 receives the fine-tuned model and related outputs from the base module 1820 to predict options for editing the medical record and output editing recommendations. The editing recommendations 1802 include providing recommendations in real time while typing the report, suggesting corrections, and providing alternative text. Figures 13 to 17 An example of editing recommendations provided to a user of an intelligent medical editor is shown.

[0140] Fig.19 Flowcharts and block diagrams related to patient profiling according to one embodiment are shown. Understanding information about the patient is important for an intelligent medical editor. Fig.19 Included are a patient profiling system 1900, a patient profiler 1902, a similar patient lookup 1904, an editor 1906, a future use evaluation 1908, a fine-tuning model selection module 1910, patient data 1912, and past patient data 1914. The patient profiling system 1900 facilitates the collection and understanding of information about a patient.

[0141] Patient profiler 1902 builds context-aware patient profile based on patient data of the entire medical system. This includes patient EMR data and past TX (treatment / therapy) and DX (diagnosis) related to past patient data 1914. The patient data input into the patient profiler also includes patient data 1912 from the entire medical system, including family history, past medical history, examination information, genomic information, medication, social history and physical activity. The patient profile may include information such as significant health changes that occur when the patient is at home, visiting a doctor or hospital, surgical procedures and any related complications, medication, immunization (such as flu shot or tetanus shot) and other related health information. In addition, the user's preferences when using medical system technology, such as medical system applications on their smart phones, smart medical editors, or other computing devices they interact with. In addition, patient data may include informal information that can be input via voice, video, personal wearable sensors and other newer types of input (comments, opinions, location data and spoken language from smart phones, security cameras, etc.) and observations from friends and other third parties (such as whether the patient takes medication). The patient profiler 1902 is updated periodically to maintain the patient's context and medical history.

[0142] Intelligent medical editor systems and methods can use patient profiles for editing recommendations, inconsistency detection, and other aspects discussed throughout. Editor 1906 can use patient profile information and information related to similar patients (such as through similar patient lookup 1904), similar reports, and similar diseases to provide suggestions and editing recommendations to users of editor 1906. As part of future use evaluation 1908, patient profiles and similar patient information can be absorbed (ingested) into modules throughout the system, such as fine-tuning model selection module 1910.

[0143] Fig. 20 A flow chart for updating a medical report based on selected suggested edits according to one embodiment is shown. Medical process 2000 may include one or more of six steps that may be performed by an intelligent medical editor. In step 2002, the system may identify a plurality of textual entries in a free-text medical report. In step 2004, the system may generate suggestions using an intelligent module, as described in detail throughout this document and at least in connection with Figure 3 Related Examples. In step 2006, the system may provide suggested content related to the patient and medical report type. In step 2008, as the user interacts with the report, the system may provide suggestions to the user (e.g., editorial recommendations) related to text entries and other report content. Figures 13 to 17Such an example is shown as an example in various embodiments. In step 2010, the system can provide warnings about possible errors (user or system generated errors) based on patient medical history, clinical care guidelines, medical ontologies, and other inputs and models. In step 2012, the system can update the medical report based on the selected selection of suggested content. In addition, the system can update its learning module (a learning module that allows feedback input based on a rule engine, artificial intelligence, generative artificial intelligence, machine learning, deep learning, etc.).

[0144] Intelligent Medical Editor systems and methods allow for correct, timely, and industry-best medical materials. Intelligent Medical Editor systems and methods can leverage current government guidance and industry trends to improve healthcare. Intelligent Medical Editor systems and methods can: provide a single solution to handle multi-modal time interval data sources; use patient history records and by-product modules to provide processed data that can be used for other downstream solutions; provide insights from clinician interactions to better assist in treatment; and leverage a large amount of patient history to provide similar patient profiles for hospitalized patients. Overall, patients benefit from prior cases and customized care, clinicians receive better assistance and save more time to focus on treatment, and hospitals benefit by obtaining one EMR solution instead of using multiple tools to handle different data, reporting standards, etc. All of this is done while maintaining patient privacy.

[0145] Intelligent medical editor systems and methods help ensure the correctness, completeness, and clarity of EMRs and other medical records, which improves clinical outcomes. Inaccurate and incomplete EMRs can adversely affect the speed and quality of all subsequent clinical decisions, such as diagnostic / treatment decisions, clinical trial selection, reference guidelines, and case studies. Intelligent medical editor systems and methods help by providing live AI assistance that provides clinicians with editorial feedback and improvement suggestions as they create and update EMRs.

[0146] Intelligent Medical Editor systems and methods ensure that the performance of AI models is sufficient to meet clinical standards, which is critical before integrating AI models with key clinical systems such as EMRs. Large, diverse, and authentic manual feedback is essential for validating and continuously improving AI models. Two barriers to collecting such feedback are: (1) the high time and cost required for high-quality manual feedback; and (2) data privacy requirements that lead to administrative and data anonymization overhead. Intelligent Medical Editor systems and methods help (1) create a low-invasive way for clinicians to provide live feedback to AI, and (2) create a federated approach to improving AI models that eliminates the need to share patient data beyond organizational / legal boundaries, thereby enabling broad crowdsourcing of training / validation data for AI models.

[0147] The intelligent medical editor system and method help users by accurately pre-annotating the report before the clinician opens it, and present the pre-annotation to the clinician in an easy-to-understand and easy-to-modify view. Both pre-annotations (including pre-filled information) and annotations modified / created by clinicians help to enrich the report with coding concepts in structure. This has the following clinical benefits, bringing much-needed standardization to clinical reports. Because other doctors' reports do not follow the summary reporting standards required for oncology, other users may find it difficult to quickly read other doctors' reports. The real-time automatic correction and suggestions provided by the NLP subsystem help to achieve this. The intelligent medical editor system and method are also useful because they automatically fill in concepts related to medical standards (including FHIR, UB-04, etc.) mentioned throughout the process. If the coded concepts can be parsed directly from the report through integration with the EMR system, the system and method in this article can automatically fill in such data, which is a much-needed feature for improving the work efficiency of doctors and bringing huge benefits to downstream applications.

[0148] The systems and processes described below may be embodied in hardware, such as a single integrated circuit (IC) chip, multiple ICs, application specific integrated circuits (ASICs), etc. In addition, the order in which some or all of the program blocks appear in each program should not be considered limiting. On the contrary, it should be understood that some process blocks can be performed in various orders, not all of which can be explicitly shown in the present disclosure.

[0149] The illustrated aspects of the present disclosure may also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0150] In addition, it should be understood that the various assemblies described in this specification may include circuits, which may include assemblies and circuit elements with appropriate values ​​to realize the embodiments of this theme innovation. In addition, it should be understood that many different assemblies may be realized on one or more integrated circuit (IC) chips. For example, in one embodiment, a group of assemblies may be realized in a single IC chip. In other embodiments, one or more corresponding assemblies may be manufactured or realized on a separate IC chip.

[0151] refer to Fig.21, illustrates a schematic block diagram of a computing environment 2100 according to the present disclosure, wherein a theme system, method and computer-readable medium can be deployed. The computing environment 2100 includes one or more clients 2102 (e.g., laptop computers, smart phones, PDAs, media players, computers, portable electronic devices, tablet computers, etc.). The client 2102 can be hardware and / or software (e.g., threads, processes, computing devices). The computing environment 2100 also includes one or more servers 2104. The server 2104 can also be hardware or hardware combined with software (e.g., threads, processes, computing devices). For example, the server 2104 can accommodate threads to perform transformations by adopting various aspects of the present disclosure. In various embodiments, one or more theme front-end components can be deployed as the hardware and / or software of the client 2102, and one or more theme back-end components can be deployed as the hardware and / or software of the server 2104. A possible communication between the client 2102 and the server 2104 can be in the form of data packets transmitted between two or more computer processes, wherein the data packets can include video data. For example, the data packets can include metadata, for example, associated context information. The computing environment 2100 includes a communication framework 2106 (eg, a global communication network such as the Internet, or a mobile network) that can be used to facilitate communications between the client 2102 and the server 2104 .

[0152] Communication may be facilitated by wired (including fiber optic) and / or wireless technologies. Client 2102 includes or is operably connected to one or more client data repositories 2108 that can be used to store information local to client 2102 (e.g., associated context information). Similarly, server 2104 may operably include or be operably connected to one or more server data repositories 2110 that can be used to store information local to server 2104.

[0153] In one embodiment, the client 2102 can transmit the encoded file to the server 2104 in accordance with the disclosed subject matter. The server 2104 can store the file, decode the file, or transmit the file to another client 2102. It should be appreciated that the client 2102 can also transfer an uncompressed file to the server 2104, and the server 2104 can compress the file in accordance with the disclosed subject matter. Likewise, the server 2104 can encode video information and transmit the information to one or more clients 2102 via the communication framework 2106.

[0154] Fig. 22A schematic block diagram of another example computing environment 2200 according to the present disclosure is illustrated, in which the subject systems, methods, and computer-readable media may be deployed. The computing environment 2200 includes a cloud deployment architecture consisting of one or more clients 2202, which may be communicatively coupled to a system cloud 2204 via a network (e.g., the Internet). The system cloud 2204 may include a cloud load balancer, one or more application containers, one or more cloud service containers, a cloud data repository, and a cloud network that communicatively couples one or more cloud components to the cloud data repository. According to the cloud deployment architecture, the client 2202 may include one or more client devices (e.g., mobile devices, laptop computers, desktop computers, etc.), which may include or use suitable applications (e.g., native mobile applications, web-based applications, thin / fat client applications, etc.) to access and use one or more features and functions of the subject native / reconstructed medical imaging system deployed in the system cloud 2204. In various specific implementations, one or more components of the system 100 may be distributed between the client 2202 and the system cloud 2204.

[0155] Fig.23 A schematic block diagram of another example computing environment 2300 according to the present disclosure is illustrated, in which the subject systems, methods, and computer-readable media may be deployed. The computing environment 2300 includes a virtualized enterprise deployment consisting of one or more clients 2202, which may be communicatively coupled to a remote data center 2302 via a network (e.g., the Internet). The remote data center 2302 may include an application server subnet 2304, which may provide a load balancer, one or more application containers, one or more virtualized servers, and one or more rack servers. The data center 2302 may also include one or more data repositories, which may be communicatively coupled to the application server subnet 2304 via a data center network. According to the virtualized enterprise deployment, the client 2202 may include one or more client devices (e.g., mobile devices, laptops, desktop computers, etc.), which may include or use suitable applications (e.g., native mobile applications, web-based applications, thin / fat client applications, etc.) to access and use one or more features and functions of the subject native / reconstructed medical imaging system deployed in the system data center 2302 and the application server subnet 2304. In various implementations, one or more components of the system 100 may be distributed between the client 2202 and the application server subnet 2304 , and one or more data repositories may be provided remotely at the data center 2302 .

[0156] Fig.24A schematic block diagram of another example computing environment 2400 according to the present disclosure is illustrated, in which the subject systems, methods, and computer-readable media may be deployed. The computing environment 2400 includes a local enterprise deployment consisting of one or more clients 2202, which may be communicatively coupled to an application server subnet 2404 via a network (e.g., the Internet). According to this embodiment, the application server subnet 2404 may be provided at an enterprise building 2402 (e.g., as opposed to a remote data center 2302). The application server subnet 2404 may include a load balancer, one or more application containers, and one or more servers. The application server subnet 2404 may be communicatively coupled to one or more data repositories provided at the enterprise building 2402 via an enterprise network. Similar to cloud and virtualized enterprise deployments, clients 2202 may include one or more client devices (e.g., mobile devices, laptops, desktop computers, etc.) that may include or use suitable applications (e.g., native mobile applications, web-based applications, thin / fat client applications, etc.) to access and use one or more features and functions of the subject native / reconstructed medical imaging system (e.g., system 100, etc.) deployed at an enterprise premises 2402 and in an application server subnet 2404. In various implementations, one or more components of the system herein may be distributed between clients 2202 and an application server subnet 2404, and one or more data repositories may be provided at the enterprise premises 2402.

[0157] Fig.25 A schematic block diagram of another example computing environment according to the present disclosure is illustrated in which the subject systems, methods, and computer-readable media may be deployed. The computing environment includes a local device deployment in which all components of the systems herein are provided at a single client device 2502. With this implementation, the client device 2502 may include a web-based application that may be communicatively coupled to one or more application containers via loopback. The one or more application containers may be communicatively coupled to one or more databases and / or one or more local file systems via loopback.

[0158] refer to Fig.26 , a suitable environment 2600 for implementing various aspects of the claimed subject matter includes a computer 2602. The computer 2602 includes a processing unit 2604, a system memory 2606, a codec 2605, and a system bus 2608. The system bus 2608 couples system components including, but not limited to, the system memory 2606 to the processing unit 2604. The processing unit 2604 can be any of a variety of available processors. Dual microprocessors and other multi-processor architectures can also be used as the processing unit 2604.

[0159] The system bus 2608 can be any of several types of bus structures including a memory bus or memory controller, a peripheral bus or external bus, and / or a local bus using various available bus architectures including, but not limited to, Industry Standard Architecture (ISA), Micro Channel Architecture (MSA), Extended ISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB), Peripheral Component Interconnect (PCI), card bus, Universal Serial Bus (USB), Advanced Graphics Port (AGP), Personal Computer Memory Card International Association bus (PCMCIA), FireWire (IEEE 22104), and Small Computer System Interface (SCSI).

[0160] System memory 2606 includes volatile memory 2610 and non-volatile memory 2612. Basic input / output system (BIOS) (including basic routines such as transferring information between elements within computer 2602 during startup) is stored in non-volatile memory 2612. In addition, according to the innovation of the present invention, codec 2605 may include at least one of an encoder or a decoder, wherein at least one of the encoder or the decoder may be composed of hardware, a combination of hardware and software, or software. Although codec 2605 is shown as a separate component, codec 2605 may be included in non-volatile memory 2612. By way of example and not limitation, non-volatile memory 2612 may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory 2210 includes random access memory (RAM), which acts as an external cache memory. According to aspects of the present invention, volatile memory can store write operation retry logic, etc. By way of example and not limitation, RAM can be provided in a variety of forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), and enhanced SDRAM (ESDRAM).

[0161] The computer 2602 may also include removable / non-removable, volatile / nonvolatile computer storage media. Fig.19For example, a disk storage device 2614 is illustrated. The disk storage device 2614 includes, but is not limited to, devices such as a disk drive, a solid state disk (SSD), a floppy disk drive, a tape drive, a Zip drive, a flash memory card, or a memory stick. In addition, the disk storage device 2614 may include a separate storage medium or a storage medium combined with other storage media, including but not limited to an optical drive, such as a compact disk ROM device (CD-ROM), a CD recordable drive (CD-R drive), a CD rewritable drive (CD-RW drive), or a digital versatile magneto-optical disk ROM drive (DVD-ROM). In order to facilitate connecting the disk storage device 2614 to the system bus 2608, a removable / non-removable interface, such as an interface 2616, is generally used.

[0162] It should be understood that Fig.26 Software is described that acts as an intermediary between a user and the basic computer resources of the computer system 2602 in a suitable operating environment 2600. Such software includes an operating system 2618. The operating system 2618, which may be stored on disk storage 2614, is used to control and allocate the resources of the computer system 2602. Application programs 2620 utilize the management of resources by the operating system 2618 through program modules 2624, and program data 2626 stored in system memory 2606 or on disk storage 2614, such as a startup / shutdown transaction table. It should be appreciated that the claimed subject matter may be implemented with various operating systems or combinations of operating systems.

[0163] The user inputs commands or information into the computer 2602 through input devices 2628. Input devices 2628 include, but are not limited to, pointing devices such as mice, trackballs, styluses, touchpads, keyboards, microphones, joysticks, gamepads, satellite dishes, scanners, TV tuner cards, digital cameras, digital video cameras, webcams, microphones, etc. These and other input devices are connected to the processing unit 2604 through the system bus 2608 via interface ports 2630. Interface ports 2630 include, for example, serial ports, parallel ports, game ports, and universal serial buses (USB). Output devices 2636 use some of the same types of ports as input devices. Thus, for example, a USB port can be used to provide input to the computer 2602 and output information from the computer 2602 to output devices 2636. Output adapters 2634 are provided to illustrate that there are some output devices 2636 such as monitors, speakers, and printers, as well as other output devices 2636 that require special adapters. By way of illustration and not limitation, output adapters 2634 include video and sound cards that provide a means of connection between output devices 2636 and system bus 2608. It should be noted that other devices and / or systems of devices provide both input and output capabilities, such as remote computer 2638.

[0164] Computer 2602 can work in a networked environment using logical connections to one or more remote computers, such as remote computer 2638. Remote computer 2638 can be a personal computer, server, router, network PC, workstation, microprocessor-based device, peer device, smart phone, tablet computer or other network node, and typically includes many elements described relative to computer 2602. For the purpose of simplicity, only memory storage device 2640 is shown for remote computer 2638. Remote computer 2638 is logically connected to computer 2602 through network interface 2642, and then connected via communication connection 2644. Network interface 2642 includes wired and / or wireless communication networks, such as local area networks (LANs) and wide area networks (WANs) and cellular networks. LAN technologies include fiber distributed data interface (FDDI), copper distributed data interface (CDDI), Ethernet, token ring, etc. WAN technologies include, but are not limited to, point-to-point links, circuit switching networks such as integrated services digital networks (ISDN) and variations thereon, packet switching networks, and digital subscriber lines (DSL).

[0165] The communication connection 2644 refers to the hardware / software used to connect the network interface 2642 to the bus 2608. Although the communication connection 2644 is shown internal to the computer 2602 for clarity of illustration, the communication connection can also be external to the computer 2602. For exemplary purposes only, the hardware / software required to connect to the network interface 2642 includes internal technology and external technology, such as modems, including conventional telephone grade modems, cable modems and DSL modems, ISDN adapters, as well as wired and wireless Ethernet cards, hubs and routers.

[0166] The content described above includes examples of embodiments of the present invention. Of course, it is impossible to describe every conceivable combination of components or methods for the purpose of describing the claimed subject matter, but it should be understood that many further combinations and permutations of the innovation of the present invention are possible. Therefore, the claimed subject matter is intended to cover all these changes, modifications and variations that fall within the spirit and scope of the appended claims. In addition, the above description of the illustrated embodiments of the present disclosure (including the content described in the abstract) is not intended to be exhaustive or to limit the disclosed embodiments to the disclosed precise form. Although specific embodiments and examples are described in the present disclosure for illustrative purposes, various modifications can be made within the scope of various embodiments and examples as can be appreciated by those skilled in the relevant art.

[0167] Specifically and with respect to the various functions performed by the above-described components, devices, circuits, systems, etc., unless otherwise specified, the terms used to describe these components are intended to correspond to any component that performs the specified function of the described component (e.g., a functional equivalent), even if not structurally equivalent to the disclosed structure, which performs the functions of the exemplary aspects of the claimed subject matter illustrated in this disclosure. In this regard, it will also be recognized that the innovation includes systems as well as computer-readable storage media having computer-executable instructions for performing the actions and / or events of the various methods of the claimed subject matter.

[0168] The aforementioned systems / circuits / modules have been described with respect to the interactions between several components / frames. It should be understood that such systems / circuits and components / frames may include those components or specified sub-components, some specified components or sub-components, and / or additional components, and conform to the aforementioned various permutations and combinations. Sub-components may also be implemented as communicatively coupled to other components except those included in the parent component (layered). In addition, it should be noted that one or more components may be combined into a single component providing aggregate functionality, or divided into several separate sub-components, and any one or more intermediate layers (such as a management layer) may be provided to communicatively couple to these sub-components to provide integrated functionality. Any component described in the present disclosure may also interact with one or more other components not specifically described in the present disclosure but known to those skilled in the art.

[0169] Furthermore, while a particular feature of the subject innovation may be disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of the other implementations as may be desired and advantageous for any given or particular application. Furthermore, where the terms "comprises," "including," "having," "comprising," variations thereof and other similar words are used in the detailed description or claims, these terms are intended to be inclusive, similar to the term "comprising," as an open transition word that does not exclude any additional or other elements.

[0170] As used in this application, the terms "component", "system", etc. are generally intended to refer to a computer-related entity, which is hardware (e.g., circuitry), a combination of hardware and software, software, or an entity associated with an operating machine having one or more specific functions. For example, a component can be, but is not limited to, a process running on a processor (e.g., a digital signal processor), a processor, an object, an executable file, a thread of execution, a program, and / or a computer. By way of example, both an application running on a controller and the controller can be a component. One or more components can reside within a process and / or a thread of execution, and a component can be located on one computer and / or distributed between two or more computers. In addition, a "device" can be in the form of specially designed hardware; generalized hardware that is specialized by executing software thereon, which enables the hardware to perform specific functions; software stored on a computer-readable storage medium; software transmitted on a computer-readable transmission medium; or a combination thereof.

[0171] In addition, the words "example" or "exemplary" are used in this disclosure to indicate use as an example, instance or illustration. Any aspect or design described as "exemplary" in this disclosure is not necessarily to be construed as being preferred or advantageous over other aspects or designs. On the contrary, the use of the words "example" or "exemplary" is intended to present concepts in a specific way. As used in this patent application, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X employs A or B" is intended to mean any natural inclusive replacement. That is, if X employs A; X employs B; or X employs both A and B, "X employs A or B" is satisfied in any of the foregoing cases. In addition, the articles "one" and "an" used in this application and the appended claims should generally be interpreted as referring to "one or more" unless otherwise specified or clear from the context to refer to the singular form.

[0172] The computing device generally includes various media, which may include computer-readable storage media and / or communication media, wherein the use of these two terms in this specification is different from each other, as described below. Computer-readable storage media can be any available storage media that can be accessed by a computer, which generally has a non-transient property, and may include volatile and non-volatile media, removable and non-removable media. By way of example and not limitation, computer-readable storage media can be implemented in conjunction with any method or technology for storing information such as computer-readable instructions, program modules, structured data or unstructured data. Computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, cassette, tape, magnetic disk storage or other magnetic storage device, or other tangible and / or non-transient media that can be used to store the required information. Computer-readable storage media can be accessed by one or more local or remote computing devices, for example, via access requests, queries or other data retrieval protocols to implement various operations on information stored by the medium.

[0173] According to the above exemplary system, the method that can be implemented according to the subject matter will be better understood with reference to the flowchart of each accompanying drawing. In order to simplify the description, these methods are described and shown as a series of actions. However, the actions according to the present disclosure can occur in various orders and / or occur simultaneously, and occur together with other actions not presented and described in the present disclosure. In addition, according to certain aspects of the present disclosure, all the actions shown may not be required to implement these methods. In addition, those skilled in the art will understand and appreciate that the method can alternatively be represented as a series of interrelated states by a state diagram or an event. In addition, it should be understood that the method disclosed in the present disclosure can be stored on an article of manufacture, so as to facilitate the transmission and transfer of this method to a computing device. As used in the present disclosure, the term "article of manufacture" is intended to cover a computer program that can be accessed from any computer-readable device or storage medium.

[0174] It should be understood that the above description is intended to be illustrative and not restrictive. For example, the above embodiments (and / or aspects thereof) can be used in combination with each other. In addition, without departing from the scope of the present invention, many modifications can be made to adapt specific situations or materials to the teachings of the various embodiments of the present invention. Although the size and type of the material described herein are intended to limit the parameters of the various embodiments of the present invention, these embodiments are by no means restrictive but exemplary embodiments. After reviewing the above description, many other embodiments will be apparent to those skilled in the art. Therefore, the scope of the various embodiments of the present invention should be determined with reference to the full scope of the equivalents given by the attached claims and these claims.

[0175] In the following claims, the terms "including" and "in which" are used as the plain-Chinese equivalents of the respective terms "comprising" and "wherein." Moreover, in the following claims, the terms "first," "second," and "third," etc. are used merely as labels and are not intended to impose numerical requirements on their objects. Moreover, the following claim limitations are not written in means-plus-function format and are not intended to be interpreted based on 35 U.S.C. §112, sixth paragraph, unless and until such claim limitations expressly use the phrase "means for..." followed by a recitation of functionality without other structure.

[0176] This written description uses examples to disclose various embodiments of the invention, including the best mode, and also enables those skilled in the art to practice the various embodiments of the invention, including making and using any device or system and performing any included method. The patent scope of the various embodiments of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. If such other examples have structural elements that are not different from the literal language of the claims, or if the examples include equivalent structural elements that are slightly different from the literal language of the claims, such other examples are intended to fall within the scope of the claims. The embodiments of the present disclosure shown in the drawings and described above are merely exemplary embodiments and are not intended to limit the scope of the appended claims, including any equivalents included within the scope of the claims. Various modifications are possible and will be obvious to those skilled in the art. It is intended that any combination of non-mutually exclusive features described herein is within the scope of the invention. That is, the features of the embodiments may be combined with any appropriate aspect described above, and the optional features of any one aspect may be combined with any other appropriate aspect. Similarly, the features listed in the dependent claims may be combined with non-mutually exclusive features of other dependent claims, especially when the dependent claims are dependent on the same independent claim. In some jurisdictions requiring single claim dependencies, it may be practice to use such dependencies, but this should not be taken to mean that the features in the dependent claims are mutually exclusive.

Claims

1. A medical method, comprising: When a user writes a free-text medical report, identifying multiple text entities in the report in real time; suggesting to the user edit recommendations related to the text entry, wherein the edit recommendations are generated by one or more artificial intelligence modules; as well as The editorial recommendations involve possible errors or improvements based on data including patient history, clinical care guidelines, and medical ontologies.

2. The medical method according to claim 1, further comprising: The artificial intelligence module is retrained using the medical report and the user selections related to the editorial recommendations as training data.

3. The medical method according to claim 1, wherein: The errors or modifications include clinical inaccuracies, wherein the report includes non-existent or incorrect findings regarding the patient's anatomy.

4. The medical method according to claim 1, wherein: The error or modification includes a treatment plan that is incompatible with the clinical care guideline.

5. The medical method according to claim 1, wherein: The errors or improvements include information in the report regarding a previous medical appointment or a previously incomplete medical examination.

6. The medical method according to claim 1, wherein: The error or improvement includes a detected difference between the report and an associated summary report standard associated with the report type.

7. The medical method according to claim 1, wherein: The error or improvement includes a suggested link to source data from the patient EMR or patient images stored in a PACS.

8. The medical method according to claim 1, wherein: Editorial recommendations are presented on a user interface via a text overlay adjacent to the text entry in the report.

9. The medical method according to claim 1, wherein: Editor recommendations are presented on the user interface via a chatbot powered by generative artificial intelligence and a large language model.

10. A medical system, comprising: one or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions, the instructions comprising the medical method according to any one of claims 1 to 9.

11. A non-transitory computer-readable medium having a computer program stored thereon, the computer program having at least one code segment, the at least one code segment being executable by a machine to enable the machine to perform the steps of the medical method according to any one of claims 1 to 9.

Images