Medical information system ruleset creation and/or evaluation graphical user interface

[0020] figure 1 An exemplary computing system 102 such as a workstation, computer, or the like is shown. Computing system 102 includes one or more processors 104 and a computer-readable storage medium 106 (physical memory) encoded with or embedded with computer-readable instructions for execution by one or more processors 104 The system 102 is caused to perform various functions. The computing system 102 also includes one or more output devices 108, eg, displays, printers, etc., that may be used to visually present and/or provide information.

[0021] Computing system 102 also includes one or more input devices 110 , eg, a keyboard, keypad, mouse, trackball, digital pen, microphone, etc., that allow a user to interact with computing system 102 . Computing system 102 also includes communication components 112 that allow system 102 to communicate with one or more data repositories 114, such as Picture Archiving and Communication Systems (PACS), and one or more remote devices 116, Radiology Information System (RIS), Hospital Information System (HIS), database, server, computer and/or other repository, such as another computing system, computer and/or other remote device 116 .

[0022] In the illustrated embodiment, storage medium 106 includes at least instructions corresponding to rule-based clinical decision support (CDS) application 100 that, when executed by processor 104, cause computer system 102 to run the CDS application, The application will take and render one or more graphical user interfaces (GUIs). Running the CDS application will facilitate the user to create medical reports based on findings, measurements and/or other reporting elements.

[0023] A suitable GUI presents information within one or more windows. As used herein, a window is a visual area or area of ​​a GUI that presents (or visually outputs) information and/or accepts input or information. One or more windows may be overlapped, graphically placed, and/or moved back and forth (eg, by a mouse, etc.) relative to one or more other windows. Such a window may be independent of or dependent on another window.

[0024] In one example, one or more GUIs will provide interactive tools that will facilitate the creation and/or evaluation of one or more rules for generating a medical report, which will be described in more detail below. As used herein, a rule consists of one or more clauses that include one or more propositions combined using logical operators (eg, AND, OR, NOT, MUTUAL EXCLUSION, etc.) that evaluate to true or false The atomic logic statement. The collection of rules forms a rule set 118 that formalizes one or more relationships between logical reporting elements, such as medical findings, measurements, patient demographics (attributes), and/or other reports element. The rule set 118 may be stored within the storage medium 106 (as shown), the data repository 114 and/or other storage medium. In one embodiment, for example, the GUI allows one or more rule sets 118 to be evaluated using anticipatory and/or retrospective methods to check for inconsistencies and/or contradictory logical report elements in the medical report 122 .

[0025] For a proactive approach, one or more rule sets 118 may be evaluated while the medical report 122 is being created, and the evaluation results 120 may guide the user by indicating inconsistent and/or potentially missing report elements. For retrospective methods, for example, one or more rule sets 118 may be evaluated using previously created medical reports 122 to collect information about the accuracy of those reports and/or obtain information about one or more rule sets 118. Effectiveness Insights. The assessment results 120 and/or the medical report 122 may be stored in the storage medium 106 (as shown), the data repository 114 and/or other storage medium. The above can help improve overall accuracy and ultimately patient outcomes.

[0026] One or more of such GUIs may be used for teaching, research, diagnostics, and/or other tools. For teaching, the evaluation of rules during the creation of medical reports can serve as a valuable teaching tool for clinicians unfamiliar with the field or in a given location. For research, retrospective evaluation of complex rule sets can be employed in the data mining function to explore complex relationships between report data elements for research purposes. For diagnosis, the GUI can be used to develop very fine-grained custom rules that will attempt to diagnose disease states and/or suggest appropriate findings for a certain patient, clinician, and/or institution, eg, to enhance consistent usage.

[0027] The rulesets in storage medium 106, data repository 114, and/or elsewhere may be shared and/or utilized by one or more other computing systems and/or other systems.

[0028] figure 2 An exemplary GUI 202 is shown that allows a user to create and/or edit one or more rule sets.

[0029] The illustrated GUI 202 includes a window (LREW) 206 with one or more logical report elements 1 ,206 2 ,…,206 N (wherein N is an integer greater than or equal to 1) logical reporting area 204, each window presents a set of logical reporting elements. The text will logically report elements window 206 1 ,206 2 ,…,206 N Collectively referred to as the Logical Report Elements window 206 . Examples of suitable logical report elements include, but are not limited to, medical findings, measurements, patient demographics (attributes), and/or other report elements.

[0030] The illustrated GUI 202 also includes a rules creation area 208 that includes one or more ruleset creation windows 210 1 ,…,210 M (wherein, M is an integer greater than or equal to 1), and the windows are collectively referred to as rule creation windows 210 herein. Window 210 may be populated with logical reporting elements presented through logical reporting elements window 206 to create a rule set. In one example, logical reporting elements are dragged out of window 206 by a mouse or the like and placed into one or more rule creation windows 210 .

[0031]The ruleset types window 212 presents one or more user-selectable ruleset types. Examples of rule set types include, but are not limited to, mutually exclusive 214, exclusive 216, implicit 218, and/or other rule set types. Exemplary mutually exclusive ruleset type indicates that the ruleset can contain only a single clause; Exemplary exclusionary ruleset type indicates that if the first clause evaluates to true, then the second clause must be false; Exemplary implicit ruleset type Indication: If the first clause evaluates to true, then the second clause must be true. A clause consists of one or more logical statements that evaluate to true or false.

[0032] The ruleset severity window 220 presents one or more severity options. Examples of severity options include, but are not limited to: Mandatory 222, Recommended 224, Warning Only 226, and/or other severity options. Mandatory severity level indication: If rule set validation fails, then the violation in the resulting medical report must be corrected before finalizing the medical report if it is evaluated in the expected manner (ie, the rule set is always enforced ). Suggested severity level indication: The user may ignore the indication of a rule set validation failure and/or accept the predicted suggestion to correct the ruleset (ie, the ruleset may be explicitly ignored). Warning Severity Level Indication Only: Only present a notification indicating that the ruleset validation failed (ie, only make the user aware of the failure).

[0033] It should be appreciated that in another embodiment, one or more of the illustrated windows may be omitted or presented in another GUI. Additionally, one or more other windows may be presented in GUI 202 .

[0034] image 3 An example of GUI 202 in conjunction with an ultrasound (US) application is provided. It should be understood that this example is non-limiting and that the GUI may be used with other imaging (eg, US, CT, PET, SPECT, MRI, and/or other imaging modalities) and/or non-imaging applications.

[0035] Logical report elements window 304 1 Multiple discovery propositions 306 are included, including the discovery of left ventricle (LV), right ventricle (RV), atrium, mitral valve, tricuspid valve, aorta, lung, blood vessel, pericardium, ICD-9, and others. In the illustrated embodiment, the conclusion proposition 306 includes logical statements indicating that a particular finding should or should not be present within the report.

[0036] Findings can be simple factual statements (eg, "There is a large basal aneurysm") or more complex statements, such as multiple choice findings that allow the user to select one item from a set of choices (eg, "The left ventricle" (mild, moderate, severe) expansion"). For these more complex discoveries, the discovery proposition may be evaluated to be true if the multiple-choice discovery is present in the report with only certain values ​​populated, or, if the The finding is present in the report populated with any of the choices, then the conclusion proposition can be evaluated to be true.

[0037] Logical report elements window 304 2 A number of measurement propositions 308 are included. In the illustrated embodiment, measurement proposition 308 includes logical statements that process a particular anatomical measurement that evaluates to true or false. An example could be "LVIDd<3.0cm". Along with simple numerical comparisons ( , >=, not=), additional examples of measurement propositions involve stating that a measurement is within/outside a particular range, and stating that a measurement is/is not present in the report Inside.

[0038] Logical report elements window 304 3 Include patient/study demographic or attribute propositions 310, eg, attributes such as age, date of birth, blood pressure, etc. In the illustrated embodiment, the demographic proposition 310 includes logical statements that process specific patient demographics (eg, patient age, gender, etc.) found in the report that evaluate to true or false. Similar to discovery and measurement propositions, this statement may involve a numerical value (eg, patient age > 20) or a specific choice of the parameter involved (eg, gender=male).

[0039] It should be appreciated that one or more of findings 306, one or more of measurements 308, and/or one or more of attributes 310 may be default or custom (eg, by a clinician, institutions, etc.).

[0040] The ruleset types window 312 includes an exclusive ruleset type 314 (selected in this example), a mutually exclusive ruleset type 316 and an implicit ruleset type 318 . Based on exclusion ruleset type 314 selection, ruleset creation window 320 1 Include one or more clauses (eg, the window 320 shown 1 single clause in 322). Drop-down box 323 specifies window 320 1 Whether at least one ("arbitrary") or all of the propositions contained in must be true to trigger the rule. Ruleset creation window 320 2 Clause 326 is included that corresponds to a finding that must be false based on clause 320 . In the illustrated embodiment, soft buttons 328 are provided for creating, clearing and/or replacing entries, and a window 330 is provided for adding comments and the like.

[0041] GUI 202 also includes a summary window 332 listing the created rule sets. Highlighting or selecting a specific one of the rules will automatically populate other windows with information corresponding to the selected one of the rules. Various soft buttons 334 allow opening, saving, and running the ruleset, as well as saving the ruleset as a text file (eg, for printing, sharing, etc.), windows 336 and 338 allow naming the ruleset reporting profile, and presenting the rules Set descriptor.

[0042] Rule severity window 340 includes mandatory 342, recommended 344 (selected), and warning only 346 severity level options. As described herein, different actions can be determined with severity levels while rules are being evaluated, allowing users to ignore certain rules while always enforcing other rules. This capability can also take into account the user's experience by employing a role-based approach or other mechanisms.

[0043] Figure 4 GUI 402 is shown presenting the results of one or more expected evaluated rule sets for implicit rule types.

[0044] Window 404 displays the user-selectable evaluated rules that failed the evaluation. In the illustrated embodiment, window 404 displays the identifying indicia corresponding to the rule set violated and the rules within the rule set. The selected rule shown within window 404 (by highlighting) is an implicit rule, as indicated by the presented identification mark (IMPLIES).

[0045] Window 406 presents one or more identified conflicts for the evaluated rule set selected in window 404 . In this example, the conflicts presented indicate that the discovery, measurement and attributes ("LVID", "within [4.2, 5.9]" and "male") are found in the absence of certain implicit discoveries ("LV-0062" and "LC-0061"). Windows 408 and 410 present the rules selected within window 404 . Window 408 presents the findings, measures and attributes present in the current medical report and window 410 presents the implied findings of this rule.

[0046] Window 412 displays comments entered to explain the rule currently selected in window 404 .

[0047] Soft buttons 414 and 416 respectively allow the user to delete one or more of the selected findings, measurements and attributes, or to add one or more unselected implicit findings, in an attempt to resolve the conflict. Once the discovery is removed, the rule set is checked again for violations.

[0048] Other soft buttons 418 allow the user to move back and forth through violations and/or ignore certain rules, eg, rules identified as ignorable such as those not designated as mandatory. In this example, the violation cannot be ignored.

[0049] A log may be maintained that records whenever a medical report is modified in response to a rule violation. For example, if the user wants to add a discovery based on the displayed rule violation, this event will be logged. This log can later be examined to determine how effective the rules are in catching errors, the percentage of time medical reports are modified in response to rule violations, and so on. This in turn enables the development of more efficient rule sets.

[0050] Similar to the other GUIs described herein, the contents of GUI 402 are provided for purposes of explanation and not limitation.

[0051] Figure 5 A GUI 502 is shown that presents the results of one or more expected evaluated rule sets for exclusion rule types.

[0052] Window 504 presents user-selectable evaluated rules that failed the evaluation. In the illustrated embodiment, window 504 presents the identifying indicia corresponding to the rule set violated and the rules within the rule set. The selected rules shown within window 504 (by highlighting) are exclusionary rules, as indicated by the presented identification mark (EXCLUDES).

[0053] Window 506 presents one or more identified conflicts for the evaluated rule selected in window 504 . In this example, the conflicts presented indicate that some of the multiple findings ("MV606.1 and LV104") were selected at the same time. Windows 508 and 510 present the rule set selected in window 504 . Window 508 presents the first discovery ("MV606.1") and window 510 presents the discovery ("LV104") excluded by this rule.

[0054] Window 512 displays comments entered to explain the rules in window 504 .

[0055] Soft buttons 514 and 516, respectively, allow the user to delete one or more selected findings in an attempt to resolve the conflict. Once the discovery is removed, the rule set is checked again for violations.

[0056] Other soft buttons 518 allow the user to move back and forth through violations and/or ignore certain rules, eg, rules identified as ignorable such as those not designated as mandatory. In this example, the violation can be ignored.

[0057] Similar to the other GUIs described herein, the contents of GUI 502 are provided for purposes of explanation and not limitation.

[0058] The GUI 402 and/or 502 can be evaluated using this anticipatory rule as a guide for the user to create a more accurate medical report. For example, this can be done when a medical report is about to be finalized or when required. The GUI shown not only shows which rule sets were violated, but also suggests solutions (e.g. adding/removing specific findings) and, where possible, allows the user to easily perform all the suggested solution.

[0059] Image 6 A GUI 602 is shown presenting the results of one or more retrospectively evaluated rule sets.

[0060] Soft button 604 allows the user to invoke a review assessment. The file output options area 606 allows the user to select one or more file output types for the evaluation results, and to invoke the file output according to the selected type. The assessment status area 608 shows various information such as the number of studies eligible for assessment, the number of assessed studies, and the number of assessed studies among violations. In other embodiments, similar or different inclusions of more or less information may be displayed in area 608 . Results window 610 presents information about any studies that were generated during one or more violations during the rule set evaluation process. Selecting a specific study can invoke the instantiation of another window with further information about the specific ruleset violations found for that study.

[0061] The available rules may be evaluated for each relevant study within the database and statistics collected on how many and which rules were violated, as well as any other relevant information stored within the database (eg, who created the report, etc.). set for evaluation. Various printed and graphical reports of this information can be generated. This information can be used to determine the overall accuracy of medical reports in the database, with the aim of improving said accuracy through training, new protocols and procedures, and/or developing a set of rules with precise accuracy. Making this functionality available interactively during the development of the ruleset allows for a more efficient design of a robust ruleset that will minimize alert fatigue etc.

[0062] Similar to the other GUIs described herein, the contents of GUI 602 are provided for purposes of explanation and not limitation.

[0063] Several non-limiting examples of other suitable functionality that can be incorporated into one or more of the GUIs described herein and/or into other GUIs are given below.

[0064] Alert rule sets can include rule types that alert users (with text messages or other actions) when violated, but do not recommend other modifications to report data. An alert facility can be combined with the above-described types of rules, which can act as a type of electronic help system by acting as a mechanism to further explain the logic behind unusually complex rules while the user is creating a medical report.

[0065] A set of authority data rules may include a set of implicit rules that pre-seed the medical report with appropriate findings based on measurements found in the report. These rules are executed immediately to initially populate the report, after which the user can interactively remove any irrelevant findings from these findings.

[0066] An alert content rule set may state that elements in its single constituent clause should be present in the report. For example, a patient's date of birth or certain measurements may be required to be included in each report, depending on the institution's protocol.

[0067] A multimodal rule set includes rules that can be used across multiple imaging modalities and/or non-imaging modalities. For example, rules can be developed that take data elements from multiple medical reports from different sources and infer conclusions from the aggregated data.

[0068] Other rule sets can also be envisaged here.

[0069] Also recognize that machine learning can be employed to facilitate rule generation. For example, instead of static rules, adaptive systems that "learn" over time can be created by employing various artificial intelligence techniques to create and/or modify the rules themselves. As a simple example, a test of correlations between findings in a medical database can be used to develop rules that imply that certain findings presume that other findings are already reported in the report, based on statistical probability at a given site.

[0070] Appropriate rules also include rules that take into account past data as well as data from current research. An example of such a rule might be a rule that compares the change (ie, trend) of a particular measurement over time to a known threshold. Similarly, rules can be constructed that relate to the development of disease states or pathologies (as described by findings in relevant reports) over time.

[0071] Figure 7 A method for creating a ruleset is shown.

[0072] At 702, an interactive ruleset creation GUI is presented in conjunction with running the CDS application.

[0073] At 704, the GUI is employed to create and save a set of rules that formalize relationships between logical report elements, eg, medical findings, measurements, patient demographics (attributes), and/or other report elements.

[0074] At 706, a textual description of the ruleset can be generated. In another embodiment, operation 706 is omitted.

[0075] Figure 8 A method for prospective evaluation of a rule set is shown.

[0076] At 802, the user interacts with the interactive ruleset creation GUI to create a ruleset for the study of interest.

[0077] At 804, the rule set is evaluated based on the currently initiated study, which in this example is the study of interest.

[0078] At 806, a GUI showing any ruleset violations is displayed.

[0079] At 808, the violation is resolved through the GUI and the ruleset is saved.

[0080] At 810, a medical report is generated for the study of interest based on the set of rules.

[0081] Figure 9 A method for retrospective evaluation of rule sets is shown.

[0082] At 902, a user interacts with the interactive ruleset creation GUI to create a ruleset.

[0083] In 904, the user retrieves data from a database, etc. (eg, figure 1 one of the data repositories 104) to obtain previously created medical reports.

[0084] At 906, the rule set is evaluated based on the medical report.

[0085] At 908, a GUI showing any ruleset violations is displayed.

[0086] It should be appreciated that there is no limitation on the order of operations in the methods described herein. Therefore, other sequences can be envisaged here. Furthermore, one or more operations may be omitted, and/or one or more additional operations may be included.

[0087] The present invention has been described with reference to the preferred embodiments. Modifications and changes may occur to those skilled in the art upon reading and understanding the foregoing detailed description. This means that the present invention should be construed to include all such modifications and variations as fall within the scope of the appended claims or their equivalents.