Consistent finding item categorization

Abstract

A system determines a finding item template based on a current report's characteristics. The system parses and categorizes text from prior reports into finding items using the template's criteria, displaying categorized phrases in a user interface. This enhances consistency and speed of reporting, allowing users to edit finding items and apply AI feature detection to images.

Description

CONSISTENT FINDING ITEM CATEGORIZATIONBACKGROUND

[0001] In the field of medical imaging, interpretation and reporting of imaging exams are tasks performed by radiologists and other medical professionals. These reports often contain detailed descriptions of findings, which are important for diagnosing and monitoring patient conditions. However, the current process of generating these reports can be cumbersome and inconsistent, as the process typically involves manually reviewing and comparing multiple prior reports, each potentially organized differently. This lack of standardization can lead to inefficiencies and errors, making it difficult for healthcare providers to track the progression or regression of specific medical conditions over time.

[0002] In addition, increasingly, imaging reports are informed by information derived from web-services, sometimes using artificial intelligence, that analyze medical images from the current and / or prior exams, analyze other aspects of the medical record, or incorporate published guidelines or scientific findings. Therefore, the report of a medical imaging exam may need to receive input from one or more sources, while providing the user a consistent method of seeing these inputs in an organized manner so that they can be accepted, rejected, or edited in a manner to enables efficient production of a new report conforming to the desired organization of the reading physician.SUMMARY

[0003] The systems, methods, and devices described herein each have several aspects, no single one of which is solely responsible for its desirable attributes. Without limiting the scope of this disclosure, several non-limiting features will now be described briefly.

[0004] Existing systems for medical imaging reporting often fail to provide a streamlined method for organizing and categorizing findings from prior reports or other sources in a consistent manner. Today, when a reading physician is creating a report on a current exam, he or she typically must read various documents such as the reports of prior exam, forms completed by technologists, or results of various Alalgorithms that have assessed the current exam or other parts of the medical record. The physician may copy and paste such results into the current exam or may create a new report taking these inputs into account. This process is inherently inefficient and error-prone.

[0005] In the embodiments described herein, when reading physicians interpret medical imaging exams, they have the option to automatically present content from relevant prior reports (from “prior exams”) or other sources of relevant information reorganized into finding items that match the current report template. For example, the prior report of a chest radiograph may be one or more paragraphs of text describing the findings, without any categorization or organization into finding items. A current report template associated with a current medical imaging exam may instead include sections for the following finding items: Lungs / Pleura, Mediastinum, Cardiac, Bones, Tubes / Lines, Other. The systems and methods discussed herein use speech organization (as previously described in U.S. Patent Application No. 18 / 493,672, titled “Speech Organization in Medical Imaging Examination Reports,” which is hereby incorporated by reference in its entirety and for all purposes and attached hereto as Appendix A) to automatically categorize text of the prior report and present it reorganized by the finding items that match the current report template.

[0006] This processing may occur as a result of a reading physician’s preference and may leverage rules to determine which information is relevant. For example, rules may determine which and how many prior exams are most relevant for extraction of information reported previously. When reading a Chest radiograph, a rule may determine that previously reported findings should be extracted from the two most recent radiographs of the chest, rather than from a prior MRI of the shoulder. A rule or Al prompt may determine which data is extracted from prior reports and other sources based on the chronicity of the findings, indications of the current examination, patient demographics, or other such criteria. The extraction of such information may occur ahead of the time of need based on pre-caching rules or other workflow triggers. For example, re-categorization of one or more prior exams into finding items associated with a current exam may occur as soon as the current exam is accessible so that the re-categorization is already available when a user (e.g., a radiologist or referring physician) accesses the current exam hours or days later. Therefore, when the current exam is displayed, the prior report content can be presented as a series of finding items alongside the medical images for the current and / or prior exams. Usingthe innovations discussed herein, the reading physician can then independently indicate for each finding item whether it is resolved, improved, unchanged, or worse. The reading physician can also drag any finding item over the desired position on a current image to automatically label the image in reference to the finding item.

[0007] As a result, reporting is made faster, less-error prone, and more consistent. In essence, the systems and methods discussed herein may take information from one or more sources, which may be determined by rules and preferences, and organizes this information in a consistent manner that is tailored for the report template for the exam being actively reported.

[0008] A “finding item” is defined herein as one of a list of categories included in the findings of an imaging report template. For example, in a CT of the Abdomen report, the Finding section of the report may include finding items such as LIVER, SPLEEN, PANCREAS, KIDNEYS, BOWEL, RETROPERITONEUM AND OTHER. As discussed herein, a user may select a finding item using one of several interaction methods to cause the system to present relevant information from prior reports or other sources relevant to that finding item. For example, suppose the reading physician is interested in understanding what information is known about the liver from prior imaging reports or current / prior Al analyses of the images or other information sources. Previously, the physician would have needed to read through these multiple sources of information. However, the systems and methods discussed herein allow the physician to quickly select the desired finding item in the current report template and via various actions, request the desired information from prior reports or other sources. When the results are presented, they are compiled, organized, and the sources may be cited, and information presented such that it can be easily accepted, rejected, or edited. For example, if reading a chest radiograph, any appropriate user can select a finding item such as lungs to see an organized collection of information extracted from prior imaging examinations or other sources relevant to the lungs. The physician may then incorporate this information as desired in the current report.

[0009] The systems and methods discussed herein may provide some or all of the technical features / advantages noted below:• Pre-processing of prior reports or information from other sources to present such information organized based on the finding items that match the current exam.• Ability to page back to see how a finding item was described in prior reports or in information from other sources based on using speech organization to parse text into matching finding items.• User interface that integrates viewing of report content and medical images• Dragging finding item text into an image to auto-label the image in reference to the finding item.• Automated categorization of report content using artificial intelligence models, enhancing accuracy and reducing manual effort.• Real-time synchronization of report updates across multiple devices and platforms, ensuring consistent access to the latest information.• Customizable templates for finding items, providing customized reporting based on specific clinical needs or user preferences.• Enhanced search functionality to quickly locate specific findings or historical data within a large volume of reports.• Integration with existing electronic health record (EHR) systems to streamline data flow and reduce redundancy.• Advanced analytics to track changes in patient conditions over time, providing valuable insights for clinical decision-making.

[0010] The following description includes discussion of various processes and components that may perform artificial intelligence (“Al”) processing or functionality. Al generally refers to the field of creating computer systems that can perform tasks that typically require human intelligence. This includes understanding natural language, recognizing objects in images, making decisions, and solving complex problems. Al systems can be built using various techniques, like neural networks, rulebased systems, or decision trees, for example. Neural networks learn from vast amounts of data and can improve their performance over time. Neural networks may be particularly effective in tasks that involve pattern recognition, such as image recognition, speech recognition, or Natural Language Processing.

[0011] Natural Language Processing (NLP) is an area of artificial intelligence (Al) that focuses on teaching computers to understand, interpret, and generate human language. By combining techniques from computer science, machine learning, and / or linguistics, NLP allows for more intuitive and user-friendly communication withcomputers. NLP may perform a variety of functions, such as sentiment analysis, which determines the emotional tone of text; machine translation, which automatically translates text from one language or format to another; entity recognition, which identifies and categorizes things like people, organizations, or locations within text; text summarization, which creates a summary of a piece of text; speech recognition, which converts spoken language into written text; question-answering, which provides accurate and relevant answers to user queries, and / or other related functions. Natural Language Understanding (NLU), as used herein, is a type of NLP that focuses on the comprehension aspect of human language. NLU may attempt to better understand the meaning and context of the text, including idioms, metaphors, and other linguistic nuances.

[0012] A Language Model is any algorithm, rule, model, and / or other programmatic instructions that can predict the probability of a sequence of words. A language model may, given a starting text string (e.g., one or more words), predict the next word in the sequence. A language model may calculate the probability of different word combinations based on the patterns learned during training (based on a set of text data from books, articles, websites, audio files, etc.). A language model may generate many combinations of one or more next words (and / or sentences) that are coherent and contextually relevant. Thus, a language model can be an advanced artificial intelligence algorithm that has been trained to understand, generate, and manipulate language. A language model can be useful for natural language processing, including receiving natural language prompts and providing natural language responses based on the text on which the model is trained. A language model may include an n-gram, exponential, positional, neural network, and / or other type of model.

[0013] A Large Language Model (“LLM”) is any type of language model that has been trained on a larger data set and has a larger number of training parameters compared to a regular language model. An LLM can understand more intricate patterns and generate text that is more coherent and contextually relevant due to its extensive training. Thus, an LLM may perform well on a wide range of topics and tasks. An LLM may comprise a NN trained using self-supervised learning. An LLM may be of any type, including a Question Answer (“QA”) LLM that may be optimized for generating answers from a context, a multimodel LLM / model, and / or the like. An LLM (and / or other models of the present disclosure), may include, for example, attention-based and / or transformer architecture or functionality. LLMs can be extremely useful for natural language processing, including receiving natural language prompts and providing natural language responses based on the text on which the model is trained.

[0014] As used herein, references to specific uses and / or implementations of Al, NLP, NLU, or LLM should be interpreted to include any other implementations, including any of those discussed above. For example, references to NLP herein should be interpreted to include NLU also.

[0015] A system of one or more computers can be configured to perform the below example operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 illustrates an example computing system configured to perform various operations related to medical imaging report processing.

[0017] Figure 2 is a flowchart depicting an example process for categorizing prior exam information for use with a current exam.

[0018] Figure 3 is a block diagram illustrating an example of how a speech organization module applies a selected finding item template to multiple related exams.

[0019] Figure 4 presents an example user interface that displays a current exam and a prior exam side-by-side.

[0020] Figure 5 (including Figures 5A, 5B, 5C, and 5D) are block diagrams illustrating example implementations of a speech organization module.

[0021] Figure 6 is an example user interface that includes a summary panel displaying impressions and summaries of findings.

[0022] Figure 7 is an example user interface showing a finding item summary generated in response to user selection of a finding item.DETAILED DESCRIPTION

[0023] Embodiments of the invention will now be described with reference to the accompanying figures, wherein like numerals refer to like elements throughout.The terminology used in the description presented herein is not intended to be interpreted in any limited or restrictive manner, simply because it is being utilized in conjunction with certain specific embodiments. Furthermore, embodiments of the invention may include several novel features, no single one of which is solely responsible for its desirable attributes or which is essential to practicing the inventions herein described.

[0024] Although certain preferred embodiments and examples are disclosed below, inventive subject matter extends beyond the specifically disclosed embodiments to other alternative embodiments and / or uses and to modifications and equivalents thereof. Thus, the scope of the claims appended hereto is not limited by any of the particular embodiments described below. For example, in any method or process disclosed herein, the acts or operations of the method or process may be performed in any suitable sequence and are not necessarily limited to any particular disclosed sequence. Various operations may be described as multiple discrete operations in turn, in a manner that may be helpful in understanding certain embodiments; however, the order of description should not be construed to imply that these operations are order dependent. Additionally, the structures, systems, and / or devices described herein may be embodied as integrated components or as separate components. For purposes of comparing various embodiments, certain aspects and advantages of these embodiments are described. Not necessarily all such aspects or advantages are achieved by any particular embodiment. Thus, for example, various embodiments may be carried out in a manner that achieves or optimizes one advantage or group of advantages as taught herein without necessarily achieving other aspects or advantages as may also be taught or suggested herein.

[0025] The systems and methods discussed herein may be performed by various computing systems, which are referred to herein generally as a viewing device or computing system (such as computing system 150 of Figure 1 ). A computing system may include, for example, a picture archiving and communication system (“FACs”) or other computing system configured to display images, such as computed tomography (“CT”), magnetic resonance imaging (“MRI”), ultrasound (“US”), radiography (“XR”), positron emission tomography (“PET”), nuclear medicine (“NM”), fluoroscopy (“FL”), photographs, and / or any other type of image. Any of the computer processing discussed herein, such as application of artificial intelligence (“Al”) and / or development or updating of Al algorithms, may be performed at the computing systemand / or at one or more backend or cloud devices, such as one or more servers. Thus, even if a particular computerized processes is described herein as being performed by a particular computing system (e.g., a PACS or sever), the processes may be performed partially or fully by other devices.

[0026] A challenge in clinical healthcare is that current medical imaging reports make it difficult for imaging specialists, clinicians, and patients to track the progression or regression of disease relative to a specific finding item (e.g., a specific anatomical region orfeature). For example, a patient may undergo various ultrasound, MRI, CT, and PET imaging exams to assess a liver tumor over time. Each imaging report (of the imaging exams) may describe many different observations and conclusions in different ways. For example, a particular characteristic of patient anatomy may be described with reference to a first finding item (e.g., a particular section heading) in an exam report for imaging in a first modality, while description of the same characteristic of patient anatomy may be described with reference to a second finding item in an exam report for imaging in a second modality. Sorting through these complex reports to determine how a particular lesion, for example, is doing may be difficult and time-consuming. Additionally, the description of a particular anatomical feature, e.g., the liver, may appear inconsistent in different parts of each report.

[0027] Furthermore, when reporting a medical imaging exam, time is wasted dictating text to describe a finding item that was already previously described, such as in a prior related exam.

[0028] To address these challenges, described herein are computerized systems and methods that provide a way for a reading physician to quickly view prior reports re-organized by the finding items that are desired in the current report. With the information in the prior reports so organized, the reading physician can quickly mark each finding item as resolved, improved, unchanged, worse, and / or dictate new text to be associated with each finding item.

[0029] In addition, an Al algorithm may assess medical images to determine whether a finding item shows a finding vs. no finding. The systems and methods discussed herein provide an indication, via a user interface, of whether Al has detected finding vs. no finding for each finding item that is assessed by Al. With the report content of each of the current and prior reports categorized into the same finding items,features detected by an Al across the multiple exams may be compared to better track changes in the particular finding items.Example System

[0030] Figure 1 illustrates an example computing system 150 (also referred to herein as a “computing device 150” or “system 150”). The computing system 150 may take various forms. In one embodiment, the computing system 150 may be a computer workstation having modules 151 , such as software, firmware, and / or hardware modules. In other embodiments, modules 151 may reside on another computing device, such as a web server, and the user directly interacts with a second computing device that is connected to the web server via a computer network.

[0031] In various embodiments, the computing system 150 comprises one or more of a server, a desktop computer, a workstation, a laptop computer, a mobile computer, a Smartphone, a tablet computer, a cell phone, a personal digital assistant, a gaming system, a kiosk, any other device that utilizes a graphical user interface, including office equipment, automobiles, industrial equipment, and / or a television, for example. In one embodiment, for example, the computing system 150 comprises a tablet computer that provides a user interface responsive to contact with a human hand / finger or stylus.

[0032] The computing system 150 may run an off-the-shelf operating system 154 such as a Windows, Linux, MacOS, Android, iOS, orother. The computing system 150 may also run a more specialized operating system which may be designed for the specific tasks performed by the computing system 150.

[0033] The computing system 150 may include one or more hardware computing processors 152. The computer processors 152 may include central processing units (CPUs) and may further include dedicated processors such as graphics processor chips, or other specialized processors. The processors generally are used to execute computer instructions based on the software modules 151 to cause the computing device to perform operations as specified by the modules 151 .

[0034] The various software modules 151 (or simply “modules 151”) may be provided on a computer readable medium, such as a compact disc, digital video disc, flash drive, or any other tangible medium. Such software code may be stored, partially or fully, on a memory device of the executing computing device for execution by the computing device. The application modules may include, by way of example,components, such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. For example, modules may include software code written in a programming language, such as, for example, Java, JavaScript, ActionScript, Visual Basic, HTML, C, C++, or C#. While “modules” are generally discussed herein with reference to software, any modules may alternatively be represented in hardware or firmware. Generally, the modules described herein refer to logical modules that may be combined with other modules or divided into submodules despite their physical organization or storage.

[0035] The computing system 150 may also include memory 153. The memory 153 may include volatile data storage such as RAM or SDRAM. The memory 153 may also include more permanent forms of storage such as a hard disk drive, a flash disk, flash memory, a solid state drive, or some other type of non-volatile storage.

[0036] The computing system 150 may also include or be interfaced to one or more display devices 155 that provide information to the users. A display device 155 may provide for the presentation of GUIs, application software data, and multimedia presentations, for example. Display devices 155 may include a video display, such as one or more high-resolution computer monitors, or a display device integrated into or attached to a laptop computer, handheld computer, Smartphone, computer tablet device, or medical scanner. In other embodiments, the display device 155 may include an LCD, OLED, or other thin screen display surface, a monitor, television, projector, a display integrated into wearable glasses, such as a virtual reality or augmented reality headset, or any other device that visually depicts user interfaces and data to viewers.

[0037] The computing system 150 may also include or be interfaced to one or more input devices 156 which receive input from users, such as a keyboard, trackball, mouse, 3D mouse, drawing tablet, joystick, game controller, touch screen (e.g., capacitive or resistive touch screen), touchpad, accelerometer, video camera and / or microphone.

[0038] The computing system 150 may also include one or more interfaces 157 which allow information exchange between computing system 150 and other computers and input / output devices using systems such as Ethernet, Wi-Fi, Bluetooth, as well as other wired and wireless data communications techniques.

[0039] The modules of the computing system 150 may be connected using a standard based bus system. The functionality provided for in the components and modules of computing system 150 may be combined into fewer components and modules or further separated into additional components and modules.

[0040] In the example of Figure 1 , the computing system 150 is connected to a computer network 160, which allows communications with various other devices, both local and remote. The computer network 160 may take various forms. It may be a wired network or a wireless network, or it may be some combination of both. The computer network 160 may be a single computer network, or it may be a combination or collection of different networks and network protocols. For example, the computer network 160 may include one or more local area networks (LAN), wide area networks (WAN), personal area networks (PAN), cellular or data networks, and / or the Internet.

[0041] Various devices and subsystems may be connected to the network 160. For example, one or more medical imaging device that generate images associated with a patient in various formats, such as Computed Tomography (“CT”), magnetic resonance imaging (“MRI”), Ultrasound (“US”), (X-Ray) (“XR”), Positron emission tomography (“PET”), Nuclear Medicine (“NM”), Fluoroscopy (“FL”), photographs, illustrations and / or any other type of image. These devices may be used to acquire such medical images 186 from patients, and may share the acquired images with other devices on the network 160. Medical images may be stored in any format, such as an open source format or a proprietary format. A common format for image storage in the PACS system is the Digital Imaging and Communications in Medicine (DICOM) format.

[0042] In the example of Figure 1 , a report and image storage 190 stores various modalities of medical images 186 and medical reports 188. In an example implementation, when a new medical image acquired via medical imaging equipment, is stored in the medical images 186 an image segmentation and analysis process is initiated. In some embodiments, the computing system 150 may be notified of the available image and initiate automatic segmentation and / or analysis of the medical image.Example Speech Organization (or “Text Organization”)

[0043] In the example of Figure 1 , the computing system 150 is configured to execute a Speech Organization module 175. In some embodiments, the SpeechOrganization module 175 is stored partially or fully in the software modules 151 of the system 150. In some implementations, the Speech Organization module 175 may be stored remote from the computing system 150, such as on another device that is accessible via a local or wide area network (e.g., via network 160). For example, the speech organization module 175 may operate in the cloud (e.g., on a server connected to the Internet), in communication with the computing system 150, which displays the various user interfaces to the user and interacts with user. In such an embodiment, any number of user devices may communicate with and make use of the speech organization module 175 with little or no software installation required on their mobile computing device (e.g., a browser-based user interface may be implemented in some examples). In an example implementation, the Speech Organization module 175 parses input text (accessed in a prior exam or received via voice or other input device) and associates the text with appropriate finding items of a medical report. As discussed in further detail below, in some embodiments the speech organization module 175 is configured to parse and reorganize content of a prior medical exam, such as a dictated report of a prior medical exam, into finding items that correlate with finding items in a current exam.

[0044] Figure 2 is a flowchart illustrating an example process of categorizing (or re-categorizing) prior exam information for use in conjunction with a current exam, such as to better identify changes and / or trends in patient condition. Depending on the implementation, the method may include fewer or additional blocks and / or the blocks may be performed in an order different than illustrated.

[0045] Beginning at block 202, a current exam is accessed. For example, a medical imaging exam of a particular modality (e.g., ultrasound, MRI, CT, PET, etc.) may be made accessible to the computing system. For example, a user of the system 150 may select an imaging exam for review, e.g., a newly received imaging exam that the user is tasked with reviewing and generating a report on. In some embodiments, the method execute without a current exam.

[0046] Next, at block 204 one or more related exams (also generally referred to herein as “prior exams”) are identified, either automatically by the computing system and / or by selection of the user. In general, a related exam is any prior medical information that may be relevant to analysis or understanding of a patient’s medical condition, such as a patient’s medical history, current status or diagnosis, and / or any other information that may be useful in comparative analysis. For example, a relatedexam may be compared to a current exam, and / or other related exams, to identify changes, improvements, deteriorations, etc. in particular patient conditions. Related exams may include all exams of a particular patient within a particular time period, such as within a prior six months, year, or other time period. Related exams may be identified based on metadata criteria. For example, related exams for a particular current exam may be limited to prior exams of the patient associated with a particular anatomical area, modality, date range, clinical indication, and / or any other criteria. In some embodiments, the computing system 150 may identify related exams based on relevancy rules and / or exam characteristics.

[0047] At block 206, a finding item template associated with the current exam is automatically or manually selected. In some implementations, a finding item template is automatically selected based on characteristics of the current exam, such as an exam type. In some implementations a finding item template is part of, or is determined based on, a report template (e.g., an outline of a report with finding items associated with a particular exam type and / or attribute, and possibly with default report text for some or all of the finding items). For example, in one implementation a report template associated with a current exam may be selected and then finding item criteria for each of the finding items in the report template are used to identify finding item text to be associated with respective finding items, as discussed below. The set of finding item criteria (and related finding items) associated with a report may be considered a finding item template, even if the finding item criteria are not stored in a single file or associated in other manners.

[0048] In some embodiments, a finding items template is generated in response to selection (either automatic or manual) of a report template for a particular medical imaging exam. For example, each finding item in a selected report template may be identified. Then, for each finding item, finding item criteria for that finding item is determined, such as based on a lookup in a table or other data structure. The finding item criteria associated with each finding item in the report template may then be stored as a finding item template.

[0049] In some embodiments, each exam type is associated with a finding item template. In some examples, an exam type may be associated with multiple finding item templates and / or multiple exam types may be associated with a same finding item template. In some examples, the finding item(s) associated with an exam may be stored data linked to an exam and not stored in a single document or file. Insome examples, a finding item template is generated to include certain finding items. For example, a finding item template may be generated based on various attributes related to the exam (such as body region, patient age, use of contrast material, imaging modality or combination thereof) to include only those finding items that are relevant to those attributes. The attributes related to the example may also be used when a report is generated. While the exact embodiment may vary as described above, an exam is associated with a listing (or identifiers) of finding items that can come from one or more templates linked to the exam or from data linked to the exam. For example, finding items associated with a chest x-ray may include LUNGS, HEART, MEDIASTINUM, PLEURA, BONES, TUBES / LINES, and UPPER ABDOMEN. Such finding items may be associated with anatomical regions or other classifications. The finding items may vary depending on a user’s preferences or attributes of the user. Each finding item may be associated with default report content. For example, the LUNG finding items may be associated with “No pneumonia, mass, or other abnormality.”

[0050] Each finding item may be associated with a “finding item criteria,” which is usable to identify finding items that should be associated with the particular finding item (e.g., in addition to the default report content for that finding item or changes to the default report content). In some embodiments, finding item criteria may include a matching algorithm configured to identify text that is associated with that particular finding item. A matching algorithm may include character strings, keyword terms, searching rules, and / or artificial intelligence algorithms configured to identify text matching the concept of the finding item. For example, a matching algorithm for a lungs finding item may include letter combinations (e.g., word parts, words, phrases, etc. with regular expressions) that identify certain terms or combinations of terms in text. In some embodiments, a matching algorithm for identifying text associated with a particular finding item may be updated in response to manual association of text to a finding item. A “finding item template” as used herein may include a template for an actual medical report, e.g., including fields for portions of an actual medical report, such as title, medical practitioner information, patient information, default finding item text, etc., along with formatting parameters for the medical report. However, a “finding item template” may also refer to a set of data associated with the report, such as data associating finding items with associated finding item criteria. Thus, a finding item template may be a list, table, text file, database, and / or any other data format.

[0051] With one or more related exams identified (at block 204) and a finding item template selected (at block 206), the method begins a process (blocks 210-222) that may be repeated for each related exam. In general, this process re-organizes text of the related exam into the finding items of the selected templated. The finding items for each of the current and related exams may then more easily and effectively be compared and contrasted to identify changes in patient condition.

[0052] At block 210, for the currently selected related exam, text associated with the related exam is accessed. For example, the related exam text may include medical report information, such as information dictated by a radiologist in a text blob (e.g., uncategorized information) or organized in some manner (e.g., in sections of a report). The related exam text may include additional information, such as patient information, exam details, clinical history and indications, annotations on images, etc.

[0053] In some embodiments, the related exam text is parsed into “phrases” of one or more words that identify a concept, and the phrases will then be processed by a categorization model for a possible association with a finding item in the selected finding item template. In some embodiments, categorization of phrases may be performed by matching of a character string to determine if additional characters or words should be grouped into a phrase. In some embodiments, the parsing is performed based on presence of a delimiter, such as a period, comma, or semicolon, that separates phrases from one another. In some embodiments, phrases may be determined based on a language model, such as a large language model, that is provided with input text and instructed to parse the input text into phrases. In other embodiments, input text may be parsed into phrases using other logic, such as a natural language processing library that provides sentence tokenization capabilities, part of speech tagging, topic modeling (e.g., Latent Dirichlet Allocation), n-gram analysis, and / or any other suitable algorithm.

[0054] Moving to block 212, a categorization model is applied to phrases of the related exam text to determine associated finding items in the finding item template. For example, the categorization model may include a matching algorithm associated with each finding item in the selected template. In some implementations, the categorization model may only use matching algorithms associated with the specific finding items in the finding item template, rather than to all of the possible finding items that could be included in report, such as tens, hundreds, thousands, or more finding items. In this way, the categorization model may more quickly andaccurately associate text (e.g., phrases within the text) with the correct and relevant finding item.

[0055] In some embodiments, the matching algorithm for each of the finding items is applied to the phrases of text and outputs a confidence level that the phrases are associated with a particular finding item. For example, a finding items template with six finding items may be associated with six matching algorithms that include six sets of character strings that may be deterministically identified within text. In another example, some or all of the matching algorithms of finding items in the template may be evaluated with reference to a phrase in the related exam text and a finding item with the highest confidence level may be associated with the phrase.

[0056] Next, at block 214, the system determines whether a finding item associated with the phrase has been identified, such as at a threshold confidence level. In some examples, a highest confidence level for a finding item may cause selection of that finding item as the associated finding item. The example confidence levels below represent example confidence levels for each of seven finding items that are calculated based on a particular text, such as may be extracted from the related exam. In this example, the highest confidence level is associated with the Lungs, so in a system where the highest confidence level is determined as the matching finding item, the particular text would be associated with the lungs finding item. In other examples, the confidence level must be greater than a predetermined threshold for an input text to be assigned to the finding item. For example, if the threshold confidence level was 90%, the confidence levels in the table below would be inconclusive since the highest confidence level was 85% for the lungs finding item. In another example, if a highest finding item confidence level is near another finding item confidence level, e.g., within a predetermined minimum, such as 5 percentage points, the categorization model may return an inconclusive output indicating that the text has not been associated with a particular finding item. In other examples, the selection of a finding item based on a categorization model may be performed in other manners, such as some combination of the rules noted above.

[0057] If at block 214 an associated finding item has been identified, the method continues to block 216 where the phrase from the related exam text is associated with the finding item. This association may be immediately displayed in a user interface, such as in association with other text associated with the same finding item from the current exam and / or other related exams. In some embodiments, the association of the text with a finding item may be automatically added to a report, e.g., indicating findings from a prior exam.

[0058] If, however, at block 214, an associated finding item is not identified (e.g., if a confidence level of an association between the phrase from the related exam and each of the finding items in the template is less than a threshold), the method continues to block 218 where additional rules, e.g, deterministic rules, may be evaluated to identify a corresponding finding item to which the phrase should be assigned. One example of a rule may indicate that input text that is not automatically matched to a finding item by a matching algorithm is associated with the last matched finding item. Thus, a phrase that is not matched with a particular finding item at block 212 may be automatically assigned to a previously matched finding item. Another example rule may indicate that input text that is not automatically matched to a finding item may cause an alert or prompt to be provided to the user to allow the user to select the appropriate finding item. Another example rule may indicate that if the unmatched text is the first text (e.g., first text received for a particular report), the system associates the text with the OTHER finding item. In some examples, a drop-down menu and / or other selection interface may be provided to the user, such as displaying only a few of the finding items having the highest confidence levels. In other examples, other rules, such as may be organized in a hierarchical manner, may be generated by the system and / or the user to allow for more comprehensive association of input text to finding items. In some examples, to create a new finding item that is not in the template, the text that is dictated or reported may preceded a colon punctuation. Forexample, if BRAIN: appears, the system will associate the next dictated sentence or phrase to a new finding item called BRAIN.

[0059] Once a rule has identified the appropriate finding item for the phrase of the related exam text, the method continues to block 220 where the phrase is associated with the finding item according to the rule. This process (blocks 210-220) may be repeated for multiple phrases of text from the related exam, such as until all of the related exam text is organized into finding items in the finding items template.

[0060] Next, at block 222 the system determines whether additional related exams were identified. If so, the method returns to block 210 where phrases of text in the related exam are categorized according to the same finding item template.

[0061] At block 224, finding item information from the current and / or related exams may be displayed for viewing by the user.

[0062] Figure 3 is a block diagram illustrating how speech organization may be used to apply a selected finding item template to multiple related exams. In this example, two prior reports 510A and 510B are associated with a current exam (e.g., automatically selected by the system based on related exam rules and / or manually by a user) and are accessed by a speech organization module 575 which is configured to reorganize text of the prior reports 510A and 510B each according to the same finding item template 520. In some embodiments, the finding item template 520 is selected based on characteristics of the current exam, reader preferences, system preferences, or any other characteristic. In some embodiments, the prior reports 510A and 510B include finding items different than in finding item template 520 and / or are not organized into finding items (e.g., the report may include a free-form blurb of text). As shown, the speech organization module 575 applies the finding item template 520 to each of the prior reports 510, e.g., using a process similar to that discussed above with reference to Figure 2, and outputs categorized finding items 535A and 535B. Advantageously, no matter what format the exam reports 510A and 510B are in, the speech organization module 575 outputs finding items categorized based on the same finding item template 520, such as based on the same finding item criteria that are used to categorize text of the reports into particular finding items. The re-categorized finding items 535A and 535B may then be made available to a display module 550 for display to a viewer.Example User Interface

[0063] Figure 4 is an example user interface that displays a current exam and a prior exam side-by-side, along with information from the two exams categorized according to a finding items template. In this example, a current exam pane 410 shows an image of the current exam of the patient John Doe while a prior exam pane 420 shows an image of a prior exam of the patient. In some embodiments, additional related exams may be included in the user interface. The modality of the current exam and prior exam may be the same or may be different. For example, the current exam may be an X-ray, while the prior exam may be an MRI, CT, PET scan, or some other imaging modality. Similarly, format and organization (if any) of the exam report for the current and prior exams may be different. For example, a report for the prior exam may include a first set of finding items while a report format of the current exam may include a second set of different finding items (with little or no overlap with the finding items of the first set). Accordingly, the speech organization methodology discussed above with reference to Figure 2 advantageously reorganizes or categorizes textual information from the prior exam to correspond to the finding item template of the current exam.

[0064] In the example of Figure 4, a finding items bar 430 at the bottom of screen can be shown (or not) per user preference or user action. In this example, the finding items include Lungs and Pleura, Cardiac, Mediastinum, Bones, Tubes / Lines, and Other. A scroll bar (not shown) may be present to navigate to other finding items. As noted above, the prior exam report (associated with the prior exam image shown in 420) has been reorganized into finding items based on a finding item template selected based on a type of the current exam. The finding items boxes 432A-432F may include finding item text from one or both of the current and prior exams. For example, the user interface may include an option to allow the user to select which categorized finding items to include in the finding items bar 430. In this example, the reader can check boxes for particular finding items to indicate changes, e.g., changes in condition of the patient with reference to that particular finding item. The text may then be updated.

[0065] In the example of Figure 4, finding items 432A and 432C are shown with purple text to indicate that those finding items were found to be abnormal on the current exam using Al. In other embodiments, other visual indicators of Al findings or differences between prior and current exams may be included. In this example, eachfinding item box 432 includes a microphone icon that allows the reader to dictate into a selected item or field within an item. In addition, the reader may use speech organization to dictate text that the system will parse into the appropriate finding items.

[0066] In some implementations, the reader can drag a finding item (e.g., one of the finding item boxes 432) over an image to place a label on the image that references that finding item. In one implementation, if the reader right clicks and drags a finding item into a viewport, the text of the finding item is displayed. The user can then page within the viewport to see how that finding item is described in prior reports. The user optionally can show only the finding items that were previously abnormal, can show only those that are currently abnormal per Al. The currently active finding item may expand automatically so that more text is visible, then shrink when not in use.Example Speech Organization

[0067] Figure 5 (including Figures 5A, 5B, 5C, and 5D) illustrate example implementations of a speech organization module 575, which may provide the same or similar functionality as speech organization module 175 (Figure 1 ), such as in the context of a computing system or cloud computing environment. In the examples of Figure 5, an example input text (e.g., text from a related exam report) is accessed by the speech organization module 575, which applies a finding item template and outputs a report fragment including phrases from the input text that are categorized into the appropriate finding items (e.g., categories) within the report.

[0068] In the example of Figure 5A, the input text 510A includes a text blurb, or free-form text, such as may be included in a prior exam or may be provided by a user of the system. In this example, the input text 510A does not include associations with finding items. The finding item template 530A has been selected by a user to perform categorization of input text into finding items. For example, a user may select a finding item template that includes only those finding items that are of importance for a particular project. Similarly, a user may select finding item criteria for particular finding items, such as to customize portions of the input text (generally referred to as “phrases” of the input text) that may be associated with respective finding items. In some embodiments, a finding item template is associated with an individual, a group of individuals, an organization, or one or more computing systems, such that a finding item template is automatically selected as a default based on these factors.

[0069] In the example of Figure 5A, a finding item template 530A includes two finding items: lungs and heart, each with associated finding item criteria. In this example, the finding item criteria are character strings, but in other embodiments other types of finding item criteria may be used. As shown in Figure 5A, as the finding item template 530A is applied to the input text 510A, a report fragment 550A is provided as an output, with phrases from the input text that matched the lungs finding item criteria associated with the lungs finding item and phrases within the input text matching the heart finding item criteria associated with the heart finding item. In this example, not all of the input text 510A has been categorized as related to one of the finding items in the template A. In some embodiments, additional input text that is not categorized based on the template may be categorized into an “Other” category or the like.

[0070] Figure 5B illustrates the same input text 510A being accessed by the speech organization module 575, but now in the evaluation of finding item template 530B (rather than finding item template 530A as in Figure 5A). In this example, finding item template 530B includes two finding items: bones and heart, each with associated finding item criteria, which are character strings in this example. When the speech organization module 575 evaluates the report text 510A with reference to the finding item template 530B, the report fragment 550B includes phrases from the input text matching finding item criteria for the bones finding item associated with bones and phrases matching finding item criteria associated with the heart finding item associated with the heart. Other portions of the report text that did not match finding item criteria for bones or heart are not included in the report fragment (although in other embodiments the nonmatching text and / or phrases may be included in and “other” category or the like).

[0071] Figure 5C illustrates an example where a medical report is processed by the speech organization module 575 based on the same finding item template 530A that was used in Figure 5A. Here, though, the input text 510C already includes finding items and associated finding item text. In some embodiments, the speech organization module 575 is configured to disregard finding item associations of text in the input text 510C, such that certain phrases of the input text 510C may be associated with different finding items in the report fragment 550C and the input text 510C. For example, in some embodiments a finding item template may include a higher-level finding item than what is included in an input text. For example, a finding item template may include a finding item for “Chest” that generally includes finding items that might be related tomultiple other finding items in other reports, such as “lungs”, “heart”, and “mediastinum.” Thus, through the use of the speech organization module 575 and a user-selected template (template 530A in the example of Figure 5C), an existing medical report may be smartly re-organized to provide the appropriate finding item text for each of the finding items in the template.Example Universal Dialogue System

[0072] In some implementations, the speech organization module 575 provides a “universal dialogue” function configured to receive inputs from multiple sources and compile the information according to a selected template. As discussed above, the speech organization module 575 may process a text blurb from a radiologist and / or findings from a prior report to generate findings according to a particular template. For example, a radiologist might provide a text blurb describing a "3 by 2 centimeter mass in the left lung" that needs to be incorporated into the report, and the speech organization module 575 may transform the text blurb into finding item text associated with a corresponding finding item of a report. In this example, a text blurb is only one example of a data source from which the speech organization module 575 may receive information for processing into findings.

[0073] Figure 5D is a block diagram illustrating an example of a speech organization module 575D configured to receive inputs 559 from multiple sources and to organize information from those inputs according to a finding item template 572. In this example, the inputs include one or more prior report findings 560 (e.g., multiple reports associated with related exams for the same patient), one or more artificial intelligence analyses of reports 566 (the current report and / or prior reports associated with previous related exams), one or more technologist forms 568, and / or one or more other data sources 562. As discussed further herein, any type of input may be accessed by the speech organization module 575D to identify information matching the finding item template 572, such as parsing out information from the various inputs to generate finding item text or relevant finding items indicated in the template 572. Advantageously, the finding item text 574 that is output from the speech organization module 575D may provide a more robust and / or accurate set of finding item text based on the multiple inputs, rather than a single input from a radiologist text blurb, for example. Each of the other speech organization modules discussed herein, such aswith reference to Figures 3, 5A - 5C may also be configured to access additional inputs beyond what is shown in those specific figures.

[0074] The data sources 559 may include various Al algorithms that analyze imaging data to detect anomalies such as tumors or fractures. For example, an Al algorithm might identify a "pituitary mass" in an MRI scan, and this finding would be integrated into the report (e.g., as finding item text associated with a corresponding finding item, according to the selected finding item template). Another input may be from technologist forms, which may be filled out by medical technologists during imaging procedures, providing additional data such as patient positioning or technical parameters used during the scan. Another input may be patient documents, such as those processed via optical character recognition (OCR), which may provide information such as medical history or consent forms. Another input may be previous reports, which allow the system to pull over findings from past exams for comparison, such as chronic "atherosclerosis of the aorta" noted in earlier chest X-rays. Another input may be imaging data from exams like chest X-rays, CT scans, and MRIs, which may serve as foundational inputs, with the system analyzing these images to extract relevant findings. For example, a CT scan might reveal "fluid level in the left ancillary sinus," which may be documented in a report. Other input sources may include, for example, laboratory test results (e.g., blood tests, biopsy results, genetic testing), electronic health records (EHRs) (e.g., medical history, medications, allergies), clinical notes (e.g., physician or specialist consultations), pathology reports (e.g., tumor type and grade), vital signs and monitoring data (e.g., heart rate, blood pressure, oxygen saturation), surgical reports (e.g., procedure details, complications), radiation therapy records (e.g., dosage, target areas), patient-reported outcomes (e.g., symptoms, quality of life), wearable device data (e.g., activity levels, sleep patterns), and / or any other data source.

[0075] In some implementations, this universal dialogue features provides a chronological reorganization of information from multiple sources, organized by consistent finding items. By reorganizing medical information by finding items, rather than by exam, for example, radiologists can easily monitor changes in specific conditions or anatomical areas. For example, the system can extract and organize descriptions of the lung and pleura (e.g., a finding item) from previous exams, Al algorithms, EHR data, etc. and provide only the information related to that finding itemto the user. This focused information may provide a clearer and more easily understandable timeline of developments or resolutions in the finding item category.

[0076] Figure 6 is another example user interface comprising the user interface of Figure 4 with a summary panel 610 on the right side of the user interface. In this example, the summary panel 610 includes an impressions pane 620 configured to display impressions, which are generally an interpretation of the most significant findings. In some embodiments, the impressions are generated automatically by the module 575, such as based on finding items that are extracted from inputs from one or more sources, such as may be associated with the current exam and one or more previous exams of the patient.

[0077] In some embodiments, a summary of a particular finding item may be generated to include finding item text extracted from multiple inputs. For example, the user may select a particular finding item to trigger generation of a compiled chronically- organized list of results for the selected finding items. In one embodiment, for example, the user may drag a finding (at the bottom or right of the user interface in Figure 7) to the main image viewing area, such as over one of the image panes 410, 420. This action may instruct the system to “Use speech pre-organization to extract results from prior exams based on relevancy rules and display those results in a convenient manner.” For example, the user may click and drag the Lungs and Pleura finding item 432a to trigger generation of a summary of finding item text associated with the Lungs and Pleura finding item from all available inputs.

[0078] Figure 7 is an example user interface showing an example of a finding item summary 710 that has been generated in response to user selection of the Lungs and Pleura finding item 432a as discussed above. This example finding item summary 710 includes finding item text associated with the Lungs and Pluera finding item category compiled based on inputs from multiple sources associated with multiples exams. The finding item text for the selected finding item (e.g., Lungs and Pleura in the example of Figure 7) may be generated in response to selection of the finding item and / or may be pre-generated by the speech organization module 575. For example, certain inputs may be pre-processed by the speech organization module 575 as they are received, and stored in associated with selected finding item template. For example, if a user selects a particular finding item template for use with exams having certain characteristics, the finding item text generated according to that finding item template may be generated and stored for later access by the user. For example,when an exam is opened, any available sources are accessed to extract information and generate finding item text according to the finding item template. This finding item text may then be saved in association with the exam, for example, so it is quickly available to the user when a later exam for the same patient is received.Example Implementations

[0079] Examples of the implementations of the present disclosure can be described in view of the following example clauses. The features recited in the below example implementations can be combined with additional features disclosed herein. Furthermore, additional inventive combinations of features are disclosed herein, which are not specifically recited in the below example implementations, and which do not include the same features as the specific implementations below. For sake of brevity, the below example implementations do not identify every inventive aspect of this disclosure. The below example implementations are not intended to identify key features or essential features of any subject matter described herein. Any of the example clauses below, or any features of the example clauses, can be combined with any one or more other example clauses, or features of the example clauses or other features of the present disclosure.

[0080] Clause 1. A computing system comprising: a hardware computer processor; and a non-transitory computer readable medium having software instructions stored thereon, the software instructions executable by the hardware computer processor to cause the computing system to perform operations comprising: accessing a current medical imaging report of a patient; determining a prior medical imaging report associated with the patient; determining, based on one or more characteristics of the current medical imaging report, a finding item template including a plurality of finding items and associated finding item criteria; parsing text of the prior medical imaging report into phrases; categorizing phrases of the text of the prior medical imaging report into finding items based on the finding item criteria of the finding item template; and displaying, in a user interface, at least some of the finding items with the categorized phrases from the prior medical imaging report and categorized phrases from the current medical imaging report.

[0081] Clause 2. The computing system of clause 1 , wherein the user interface includes controls configured to allow a user to independently edit each finding item.

[0082] Clause 3. The computing system of clause 1 , further comprising: applying artificial intelligence (Al) feature detection to medical images of the current medical imaging report; and displaying in the user interface, indications of finding items where the Al feature detection indicated a finding.

[0083] Clause 4. The computing system of clause 1 , wherein the user interface is configured to allow finding item text to be dragged onto an image to label a location on the image.

[0084] Clause 5. The computing system of clause 4, further comprising: receiving a request to display the image to another user; and generating a user interface including the image with the finding item label.

[0085] Clause 6. The computing system of clause 1 , wherein the user interface including controls allowing a user to view a portion of the prior exam text associated with a categorized finding item.

[0086] Clause 7. The computing system of clause 1 , wherein said categorizing phrases includes: for each phrase in parsed text, applying a categorization model configured to evaluate the finding item criteria of the determined finding item template with reference to the phrase, wherein the categorization model provides a matching finding item for each phrase, based on likelihoods of the phrase matching with particular finding items, and not other finding items that are not included in the determined finding item template; and associating the phrase to a matching finding item.

[0087] Clause 8. The computing system of clause 7, wherein the categorization model further includes one or more deterministic rules configured to associate a phrase with a finding item.

[0088] Clause 9. The computing system of clause 1 , wherein each of the finding item criteria includes a character string or rule for identifying phrases that should be associated with the corresponding finding item.

[0089] Clause 10. The computing system of clause 1 , wherein each of the finding item criteria comprises an artificial intelligence model configured to identify phrases conceptually associated with the corresponding finding item.

[0090] Clause 11. The computing system of clause 10, wherein the artificial intelligence model comprises one or more large language model.

[0091] Clause 12. The computing system of clause 7, wherein matching finding items are determined based on confidence levels that the phrase matches respective finding items.

[0092] Clause 13. The computing system of clause 12, wherein a finding item with a highest confidence level is selected as the matching finding item.

[0093] Clause 14. The computing system of clause 8, wherein the deterministic rules are evaluated in response to unsuccessfully identifying a matching finding item based on the finding item criteria.

[0094] Clause 15. The computing system of clause 1 , wherein the prior medical imaging report is parsed into phrases by a large language model.

[0095] Clause 16. A computing system comprising: a hardware computer processor; and a non-transitory computer-readable medium having software instructions stored thereon, the software instructions executable by the hardware computer processor to cause the computing system to perform operations comprising: receiving inputs from multiple data sources associated with a current exam or a prior exam, the data sources including one or more of text blurbs from radiologists, Al algorithm outputs, technologist forms, patient documents, previous reports, imaging data, laboratory test results, electronic health records, clinical notes, pathology reports, vital signs, surgical reports, radiation therapy records, patient-reported outcomes, or wearable device data; analyzing data from the received inputs according to a selected finding item template; generating finding item text based on the analyzed data, wherein the finding item text includes information from multiple inputs; and displaying the finding item text in association with a corresponding finding item in a user interface.

[0096] Clause 17. The computing system of clause 16, wherein the user interface includes a summary panel including impressions of significant findings.

[0097] Clause 18. The computing system of clause 16, wherein the operations further comprise: generating separate finding item text for each of a plurality of exams, each according to the selected finding item template; and organizing the finding item text chronologically based on dates of the respective exams.

[0098] Clause 19. The computing system of clause 16, wherein the user interface includes controls that allow a user to select a particularfinding item, triggering generation of a chronologically-organized list of finding item text for the selected finding item.

[0099] Clause 20. The computing system of clause 19, wherein the controls include a user dragging a finding item to an image viewing area.

[0100] Clause 21 . The computing system of clause 16, wherein the operations further comprise: pre-processing inputs, according to the selecting finding item template, as they are received; and storing the generated finding item text in association with corresponding finding items for later access.

[0101] Clause 22. The computing system of clause 17, further comprising: automatically generating the impressions of significant findings based on one or more of the finding item texts generated from multiple inputs.

[0102] Clause 23. The computing system of clause 16, wherein the user interface includes a control allowing addition of the finding item text to a current report.

[0103] Clause 24. A method for organizing medical information, comprising: receiving inputs from multiple data sources, including text blurbs, Al outputs, and imaging data; compiling the inputs into a structured dialogue interface according to a selected finding item template; organizing the information chronologically by finding items; generating summaries of finding items from multiple inputs; and displaying the information and summaries in a user interface with a summary panel indicating significant findings.Additional Implementation Details and Embodiments

[0104] Various embodiments of the present disclosure may be a system, a method, and / or a computer program product at any possible technical detail level of integration. The computer program product may include a computer readable storage medium (or mediums) having computer readable program instructions thereon for causing processor to carry out aspects of the present disclosure.

[0105] For example, the functionality described herein may be performed as software instructions are executed by, and / or in response to software instructions being executed by, one or more hardware processors and / or any other suitable computing devices. The software instructions and / or other executable code may be read from a computer readable storage medium (or mediums).

[0106] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, andcombinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0107] These computer readable program instructions may be provided to a processor of a general-purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function / act specified in the flowchart(s) and / or block diagram(s) block or blocks.

[0108] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks. For example, the instructions may initially be carried on a magnetic disk or solid-state drive of a remote computer. The remote computer may load the instructions and / or modules into its dynamic memory and send the instructions over a telephone, cable, or optical line using a modem. A modem local to a server computing system may receive the data on the telephone / cable / optical line and use a converter device including the appropriate circuitry to place the data on a bus. The bus may carry the data to a memory, from which a processor may retrieve and execute the instructions. The instructions received by the memory may optionally be stored on a storage device (e.g., a solid-state drive) either before or after execution by the computer processor.

[0109] The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams mayrepresent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. In addition, certain blocks may be omitted in some implementations. The methods and processes described herein are also not limited to any particular sequence, and the blocks or states relating thereto can be performed in other sequences that are appropriate.

[0110] As described above, in various embodiments certain functionality may be accessible by a user through a web-based viewer (such as a web browser), or other suitable software program. In such implementations, the user interface may be generated by a server computing system and transmitted to a web browser of the user (e.g., running on the user’s computing system). Alternatively, data (e.g., user interface data) necessary for generating the user interface may be provided by the server computing system to the browser, where the user interface may be generated (e.g., the user interface data may be executed by a browser accessing a web service and may be configured to render the user interfaces based on the user interface data). The user may then interact with the user interface through the web-browser. User interfaces of certain implementations may be accessible through one or more dedicated software applications. In certain embodiments, one or more of the computing devices and / or systems of the disclosure may include mobile computing devices, and user interfaces may be accessible through such mobile computing devices (for example, smartphones and / or tablets).

[0111] Many variations and modifications may be made to the abovedescribed embodiments, the elements of which are to be understood as being among other acceptable examples. All such modifications and variations are intended to be included herein within the scope of this disclosure. The foregoing description details certain embodiments. It will be appreciated, however, that no matter how detailed the foregoing appears in text, the systems and methods can be practiced in many ways. As is also stated above, it should be noted that the use of particular terminology when describing certain features or aspects of the systems and methods should not be taken to imply that the terminology is being re-defined herein to be restricted to including anyspecific characteristics of the features or aspects of the systems and methods with which that terminology is associated.

[0112] Conditional language, such as, among others, “can,” “could,” “might,” or “may,” unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments include, while other embodiments may not include, certain features, elements, and / or steps. Thus, such conditional language is not generally intended to imply that features, elements and / or steps are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without user input or prompting, whether these features, elements and / or steps are included or are to be performed in any particular embodiment.

[0113] The term “substantially” when used in conjunction with the term “realtime” forms a phrase that will be readily understood by a person of ordinary skill in the art. For example, it is readily understood that such language will include speeds in which no or little delay or waiting is discernible, or where such delay is sufficiently short so as not to be disruptive, irritating, or otherwise vexing to a user.

[0114] Conjunctive language such as the phrase “at least one of X, Y, and Z,” or “at least one of X, Y, or Z,” unless specifically stated otherwise, is to be understood with the context as used in general to convey that an item, term, etc. may be either X, Y, or Z, or a combination thereof. For example, the term “or” is used in its inclusive sense (and not in its exclusive sense) so that when used, for example, to connect a list of elements, the term “or” means one, some, or all of the elements in the list. Thus, such conjunctive language is not generally intended to imply that certain embodiments require at least one of X, at least one of Y, and at least one of Z to each be present.

[0115] The term “a” as used herein should be given an inclusive rather than exclusive interpretation. For example, unless specifically noted, the term “a” should not be understood to mean “exactly one” or “one and only one”; instead, the term “a” means “one or more” or “at least one,” whether used in the claims or elsewhere in the specification and regardless of uses of quantifiers such as “at least one,” “one or more,” or “a plurality” elsewhere in the claims or specification.

[0116] The term “comprising” as used herein should be given an inclusive rather than exclusive interpretation. For example, a general purpose computer comprising one or more processors should not be interpreted as excluding othercomputer components, and may possibly include such components as memory, input / output devices, and / or network interfaces, among others.

[0117] While the above detailed description has shown, described, and pointed out novel features as applied to various embodiments, it may be understood that various omissions, substitutions, and changes in the form and details of the devices or processes illustrated may be made without departing from the spirit of the disclosure. As may be recognized, certain embodiments of the inventions described herein may be embodied within a form that does not provide all of the features and benefits set forth herein, as some features may be used or practiced separately from others. The scope of certain inventions disclosed herein is indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims

WHAT IS CLAIMED IS:1 . A computing system comprising: a hardware computer processor; and a non-transitory computer readable medium having software instructions stored thereon, the software instructions executable by the hardware computer processor to cause the computing system to perform operations comprising: accessing a current medical imaging report of a patient; determining a prior medical imaging report associated with the patient; determining, based on one or more characteristics of the current medical imaging report, a finding item template including a plurality of finding items and associated finding item criteria; parsing text of the prior medical imaging report into phrases; categorizing phrases of the text of the prior medical imaging report into finding items based on the finding item criteria of the finding item template; and displaying, in a user interface, at least some of the finding items with the categorized phrases from the prior medical imaging report and categorized phrases from the current medical imaging report.

2. The computing system of claim 1 , wherein the user interface includes controls configured to allow a user to independently edit each finding item.

3. The computing system of claim 1 , further comprising: applying artificial intelligence (Al) feature detection to medical images of the current medical imaging report; and displaying in the user interface, indications of finding items where the Al feature detection indicated a finding.

4. The computing system of claim 1 , wherein the user interface is configured to allow finding item text to be dragged onto an image to label a location on the image.

5. The computing system of claim 4, further comprising: receiving a request to display the image to another user; and generating a user interface including the image with the finding item label.

6. The computing system of claim 1 , wherein the user interface including controls allowing a user to view a portion of the prior exam text associated with a categorized finding item.

7. The computing system of claim 1 , wherein said categorizing phrases includes: for each phrase in parsed text, applying a categorization model configured to evaluate the finding item criteria of the determined finding item template with reference to the phrase, wherein the categorization model provides a matching finding item for each phrase, based on likelihoods of the phrase matching with particular finding items, and not other finding items that are not included in the determined finding item template; and associating the phrase to a matching finding item.

8. The computing system of claim 7, wherein the categorization model further includes one or more deterministic rules configured to associate a phrase with a finding item.

9. The computing system of claim 1 , wherein each of the finding item criteria includes a character string or rule for identifying phrases that should be associated with the corresponding finding item.

10. The computing system of claim 1 , wherein each of the finding item criteria comprises an artificial intelligence model configured to identify phrases conceptually associated with the corresponding finding item.

11. The computing system of claim 10, wherein the artificial intelligence model comprises one or more large language model.

12. The computing system of claim 7, wherein matching finding items are determined based on confidence levels that the phrase matches respective finding items.

13. The computing system of claim 12, wherein a finding item with a highest confidence level is selected as the matching finding item.

14. The computing system of claim 8, wherein the deterministic rules are evaluated in response to unsuccessfully identifying a matching finding item based on the finding item criteria.

15. The computing system of claim 1 , wherein the prior medical imaging report is parsed into phrases by a large language model.

16. A computing system comprising: a hardware computer processor; anda non-transitory computer-readable medium having software instructions stored thereon, the software instructions executable by the hardware computer processor to cause the computing system to perform operations comprising: receiving inputs from multiple data sources associated with a current exam or a prior exam, the data sources including one or more of text blurbs from radiologists, Al algorithm outputs, technologist forms, patient documents, previous reports, imaging data, laboratory test results, electronic health records, clinical notes, pathology reports, vital signs, surgical reports, radiation therapy records, patient-reported outcomes, or wearable device data; analyzing data from the received inputs according to a selected finding item template; generating finding item text based on the analyzed data, wherein the finding item text includes information from multiple inputs; and displaying the finding item text in association with a corresponding finding item in a user interface.

17. The computing system of claim 16, wherein the user interface includes a summary panel including impressions of significant findings.

18. The computing system of claim 16, wherein the operations further comprise: generating separate finding item text for each of a plurality of exams, each according to the selected finding item template; and organizing the finding item text chronologically based on dates of the respective exams.

19. The computing system of claim 16, wherein the user interface includes controls that allow a user to select a particular finding item, triggering generation of a chronologically-organized list of finding item text for the selected finding item.

20. The computing system of claim 19, wherein the controls include a user dragging a finding item to an image viewing area.

21. The computing system of claim 16, wherein the operations further comprise: pre-processing inputs, according to the selecting finding item template, as they are received; and storing the generated finding item text in association with corresponding finding items for later access.

22. The computing system of claim 17, further comprising: automatically generating the impressions of significant findings based on one or more of the finding item texts generated from multiple inputs.

23. The computing system of claim 16, wherein the user interface includes a control allowing addition of the finding item text to a current report.

24. A method for organizing medical information, comprising: receiving inputs from multiple data sources, including text blurbs, Al outputs, and imaging data; compiling the inputs into a structured dialogue interface according to a selected finding item template; organizing the information chronologically by finding items; generating summaries of finding items from multiple inputs; and displaying the information and summaries in a user interface with a summary panel indicating significant findings.

Images