Expert Report Editor

Abstract

A radiology report editor utilizing a series of templates for different types of reports. The templates provide appropriate statements that can be inserted and modified based on specific observations. The statements are presented to the user in the order in which they typically appear in the report. The statements may contain grammatically interchangeable qualifiers and are presented to the user in an order of historical statistical use. Reports can be based on a selection of a previous report that best matches the metadata of the new report or that best matches the image being reported.

Description

Field of the Invention

[0001] The present invention relates to an editor for generating expert reports based on observation results, particularly for radiology reports, but also applicable to the generation of reports in other fields. Background of the Invention

[0003] The present invention provides a report editor for quickly generating reports which are essentially a list of observations, logically derived conclusions and recommendations. The present invention has been developed for radiology reports and will be discussed in this context, but it should be understood that the present invention is applicable to a wide range of technical fields where reports are comparable (i.e., observations, logically derived conclusions and recommendations).

[0004] Typically, a radiology report includes the patient history, details of the imaging technique used, details of the findings, conclusions, and recommendations. Generating such a report requires the skills of the radiologist and can often take 15 minutes or longer to produce using traditional word processing tools, depending on the radiologist's language skills. Reports for a particular body part and technique are often repetitive, i.e., have the same or similar observations and conclusions, especially for patients with similar demographic characteristics and presenting similar symptoms. However, many observations and their reports contain minor variations from the norm. The similarity of radiology images can also lead to repetitive observations and conclusions.

[0005] Conventional radiology reporting systems are based on dictation and transcription or speech recognition to translate imaging observations and a synthesis of prior knowledge of anatomy, pathology, and imaging diagnosis dictated by the radiologist to compose the final report and image interpretation. This approach relies on the (fallible) memory and (limited) knowledge of the reporting radiologist and the (fallible) transcription process to produce a written report that is intended to add value to the image that could otherwise be interpreted directly by another health practitioner. Limitations of this approach include the imprecision of linguistic expressions that are subject to the vagaries of language, the tendency to interobserver variability, and the inability to easily form accurate input data for any machine learning algorithm that is intended to improve image interpretation without further classification, characterization, or coding.

[0006] Proforma or template-based reporting modules improve upon traditional reporting strategies by reducing the problem of imprecise report expression, but rigid style, limited scope or cumbersome language, complex template selection, and the large number of permutations in which human pathology may manifest when imaged make traditional template-based descriptive systems "out of tune." Traditional templates cannot account for limited knowledge and fallible memory. So-called boilerplate templates have limited or no ability to accommodate the reporting preferences of each radiologist.

[0007] Many radiology practitioners maintain a collection of past reports to use as templates for new reports. Having a complete template for each pathology or combination of pathologies is impractical due to the variability of observations, and still requires considerable time to generate reports. Maintaining such a collection of templates is an onerous task, as corrections or improvements to standard terms or phrases may require revisions to multiple documents.

[0008] Without appropriate additions to the template knowledge base, advances in disease understanding or new treatment strategies involving imaging may be missed once training is complete. Typically, templates do not provide feedback to the user in the form of teaching or training.

[0009] Verbal reports are often susceptible to misinterpretation due to the reporting radiologist's lack of precision. Most metadata available to help formulate reports and opinions is used only to a limited extent by the reporting radiologist based on recalled knowledge.

[0010] It is an object of the present invention to provide a report editor that exploits similarities in radiology data to speed up report generation while facilitating the recording of individual differences, or at least to provide a useful alternative to the public. SUMMARY OF THE INVENTION

[0012] In a first aspect, the present invention comprises an editor for generating a report on an image based on a template, wherein the template comprises a series of statements that are selectively included in the report from a database.

[0013] Preferably, the sentence includes selectable qualifiers that change the meaning of the sentence, and the qualifiers are presented for selection in an order based on their relative probabilities of selection.

[0014] Preferably, when a qualifier is selected, the probability of selection of the qualifier is updated.

[0015] Preferably, the reports are stored in a database and the new report may be based on a selected report from the stored reports and the selection of statements in the new report is copied from the selection of statements in the selected report.

[0016] The user can select a stored report on which to base a new report, and can base a new report on a selection from multiple stored reports.

[0017] Preferably, the user is presented with a selection of stored reports on which the new report is based, the stored reports being presented in order of scores determined by matching metadata of the new report with metadata of the stored reports, or by matching images of the new report with images of the stored reports.

[0018] The statements may be presented in an order that corresponds to the probability that the statement may be selected based on the selection of the statement in the historical report. The probability that a statement will be selected may be based on the selection of other statements, or based on the selection of other pairs of statements.

[0019] It should be noted that any one of the aspects described above may include any features of any other aspect described above, and may appropriately include any features of any embodiment described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The preferred features, embodiments and variations of the present invention can be seen from the following detailed description, which provides sufficient information for enabling those skilled in the art to perform the present invention. The detailed description should not be construed as limiting the scope of the foregoing summary of the present invention in any way. The detailed description will refer to the following multiple drawings.

[0022] Figure 1 A partial screenshot of the editor of the present invention is shown, showing the statement selection pane (middle pane) and the radiology report (right pane) as generated by the present invention.

[0023] Figure 2 The statement selection pane center pane of the editor is shown with a portion expanded to show the statement with expanded qualifiers.

[0024] Detailed Description of the Invention

[0025] The following detailed description of the present invention refers to the accompanying drawings. Wherever possible, the same reference numerals will be used throughout the drawings and the following description to refer to the same or like elements.

[0026] The present invention provides a new reporting system using machine learning algorithms, which is developed to address the limitations of manual and semi-automatic processes of the prior art to improve productivity and reduce the chance of errors. In the actual implementation of developing the system, AI learning algorithms are applied to a very large database of more than 150,000 sentence variants, which are derived from more than 500,000 report studies, which contain fixed stems and modifiers for modifying or limiting the meaning of sentences. This sentence reservoir covers more than 99% of the events in the reported imaging study library. These sentences are grouped in a consistent manner into a library of topic-based report building blocks. These report blocks are appropriately used in one, several or more types of reports. Each sentence or group of sentences is classified and indexed to allow automatic retrieval. The usage pattern is accurately analyzed, and the probability of the sentence being used in combination and the order in which the sentence is used are entered into a mathematical array, which learns and modifies the calculated probability of use through each newly completed report.

[0027] AI based on metadata analysis of patients (age, sex, presenting symptoms, clinical diagnosis, physical examination) and radiologists’ reporting patterns is used to predict the sentences needed for each new imaging study. The prediction algorithm uses a novel prediction strategy to accurately predict rarely used sentence combinations as well as common usage patterns. The less frequently a particular sentence appears, the more likely it is that other sentences in a common report are related to each other. If a sentence has been used in a single instance, the frequency ratio of each other sentence in the report relative to that sentence is 100%.

[0028] Each reported study has a wealth of additional metadata based on codes from the sentence repository items used in the report. Image similarity scores are derived from calculated comparisons of previous images coded using the same sentence repository. This similarity score is used to further refine the probability that a sentence was used in the specific study being reported.

[0029] With trainees using predicted sentence patterns and modifying them to produce report drafts, experienced radiologists can complete report analysis faster, easily modifying selected sentences because they are presented in a standardized order with standardized wording, all accessed by users through an interface based on standard web-based technology and cloud-based storage of sentence metadata. The experienced reporter's modifications are stored in their entirety to allow trainees to receive complete feedback on their drafts, so that a comprehensive learning event log is available for each report attempted by the trainee. Trainees quickly improve to achieve 90-100% accuracy in the generation of report drafts.

[0030] The web interface is interactive, presenting sentence draft data in a variety of formats and with extensive navigation aids to facilitate rapid access and modification of selected sentences with limited keystrokes and very limited mouse clicks for most operations. Sentence selection is continually modified as the reporting radiologist selects or rejects sentences to optimize report accuracy and minimize time to completion.

[0031] Anonymous metadata from each completed report is added to a sentence metadata database and used by a machine learning algorithm to better predict the content of similar reports in the future.

[0032] Additional sentences can be added to the sentence groups, and new sentence groups can be added and combined with existing sentence reservoir elements to expand the range of styles and imaging studies covered by the reporting system.

[0033] The system continuously learns from and improves sentence predictions based on previous usage patterns. Because the novel selection algorithm uses co-occurrence frequencies, rather than the total frequency of occurrence of sentences, rare or unique usage patterns are accurately recalled as more common or very common usage patterns.

[0034] A preferred embodiment of the present invention provides a radiology report editor that utilizes a series of templates for different types of reports. The templates provide appropriate statements that can be inserted and in many cases modified based on the specific observations. These statements are presented to the user in the order in which they typically appear in the report, allowing the user to scroll down the list of statements and select the relevant statement. Some statements can be modified by selecting alternative qualifiers, such as small / medium / large, which are selected from a drop-down list with default options, where the default options of the drop-down list are the most likely choices based on the selections made in previous reports. Each statement in the report is expressed in the same way to make the report more readable and allow machine analysis. Reports can also be based on previous reports or selected based on comparisons of report metadata or images. Preferably, each new report adds to a corpus of historical reports with statistical observations, allowing each new report to utilize previous reports.

[0035] Before describing the present invention, the structure of the reports generated by the system will be described.The present invention will first be described from a functional perspective by describing the interaction of a user with the system to generate reports.

[0036] Although radiology reports can take many forms, describing a wide variety of medical conditions and a variety of imaging techniques, the format of the reports can be standardized to allow machine generation and editing. For the present invention, the following has been selected: Figure 1 The report format on the right includes the Title, Comments, History, Technical, and Narrative sections of the findings. The Title section briefly identifies the type of study; the Comments section provides a summary of the key findings and conclusions that can be drawn from the key findings; the History section provides the patient's medical history; the Technical section describes the imaging performed; and, in addition, the Narrative section of the findings is a series of statements detailing the observations made based on the study. The present invention assists radiologists in generating the various sections of the report. The Title, History, and Technical sections are easily generated from the report metadata and will not be discussed in detail. The focus of the present invention is on populating the Narrative section and Comments section of the findings, which include a series of standardized statements selected and modified by the radiologist.

[0037] When a report is generated, the user is presented with a series of standardized statements that can be included in the report, such as "there is some evidence of degradation". Often, one or more phrases in a statement can be replaced by other grammatically interchangeable phrases. In this case, the phrase "some" is interchangeable with any phrase selected from "no / some / weak / strong". The various statements presented to the user can be included or not included in the report as required and moved within the report. The use of selectable statements minimizes or even eliminates typing, speeding up report generation, particularly for people with disabilities or poor English language skills. Statements have metadata, such as default selections. Selected statements in the narrative section of the findings can be marked as key observations so that these selected statements will also appear in the comments section.

[0038] To start a new report, metadata for the report is entered into the editor either from a compatible management program or manually by the user using selection of conventional GUI components. Metadata includes: patient demographic details such as age, gender; patient medical history; study details; i.e., image type and target area, such as chest X-ray; and administrative information such as date and clinician details.

[0039] Once the metadata has been entered, the templates can be loaded. A template is available for each type of study, such as chest x-ray, and contains the appropriate statements for that type of study.

[0040] In addition to generating new reports using templates, new reports can be based on previous reports. Users can also browse previous reports, search for reports by keyword, or search for reports based on metadata. Once an initial selection of candidate reports is identified, images from those reports can be compared with images from the current report to refine the selection.

[0041] To search for reports based on metadata, the user selects metadata criteria to base the search on and assigns weights to each criterion. For example, when starting to report on knee x-rays for sports injuries, the user is most likely to choose to search based on study type and patient age, and assign a higher weight to the study type. Metadata used to match previous reports includes study type, age, and gender. Weights may also be assigned to the age of previous reports, as newer reports in the system tend to be more accurate because the reports take advantage of the strengths of the system, which improve over time. A score is generated for each report based on the degree of metadata match and the weights assigned to the metadata. A list of reports is presented to the user in the order of the scores generated.

[0042] When searching based on image matching, an image matching algorithm is used to compare the images of the current study with images of previous comparable studies. The algorithm does not need to identify any elements of the images, it only needs to give a score for the degree of match. The image matching score can be used alone, or in conjunction with the metadata matching score. The selected scoring scheme is then used to determine the order in which reports are presented to the user to choose from.

[0043] The candidate reports are presented in a list format with selected metadata and scores for the user to browse. After selecting a candidate report from the list, the contents of the report are displayed to the user. The user can then select all or only a selection of statements in the report to include in the new report. Statements from multiple previous reports may be included in the new report. This is particularly useful in cases where multiple pathologies are present.

[0044] Once a template, a previous report, or a combination of statements from multiple previous reports is selected, the editor presents the user with a 3-pane interface. Typically, the left pane contains search and navigation help, the middle pane provides a list of statements that can be selected as report content, and the right pane displays a preview of the report formed by the selected statements. The middle and right panes are displayed in Figure 1 middle.

[0045] When a template is loaded to generate a report, several statements are selected by default and populate the report. Typically, these correspond to normal forms. To add statements to a report, the user first selects the section of the report to be edited to be able to display the statements available for selection. Typically, related statements are grouped together into subsections. In Figure 1 , the Comments section is selected, showing the first subsection unexpanded and the second subsection expanded. Clicking an unexpanded subsection expands it and unexpands the other subsections. Clicking statements in a subsection adds them to the corresponding section of the report.

[0046] When a report is being edited and the user selects a statement in the report, the Template pane automatically updates to show the section and subsections of the template that the statement came from. This facilitates adding statements from the same section and is especially useful when revising a report or creating a report based on a previous report.

[0047] Figure 2The editor is shown with the selected statement, "There are advanced osteoarthritic changes as described" including the qualifier "advanced". The qualifier is highlighted for user identification. Upon placing focus on a qualifier, a drop-down list of alternative qualifiers is presented for selection, with the alternative qualifiers presented in the order corresponding to the probability that they will be selected. Once a qualifier is selected, the statement is updated in the editor portion and the report portion. The probability of selection of a qualifier is stored in the template and updated each time a qualifier is selected. The accuracy of the qualifier presented will be improved each time the template is used.

[0048] While the report is being generated, a background process analyzes the selected statements for consistency and redundancy. This is done by comparing the text and metadata of the selected statements to each other and to the metadata of the report. For example, if a statement in a subsection is selected, and that statement is marked as a default statement in its metadata (which is usually the normal case), then further selected statements from the same subsection (which are not marked as default statements in their metadata) will be highlighted as potential errors.

[0049] The editor also provides logins for different users and tracks each user's activity. The editor interface includes a count of changes made to the report by each user and can highlight each user's additions, deletions, or modifications on a user-by-user basis. Typically, reports will be reviewed by senior radiologists, and the number of changes they need to make provides a good measure of the accuracy of the initial editor. When reviewing the report after it has been reviewed, the initial editor can review the changes made and learn from the reviewing radiologist.

[0050] The editor has been discussed from the user's perspective, but the editor can also be used in an administrative mode where selectable statements and alternative phrases can be edited. Typically, if no suitable statement is available, a supervisor will add a new statement to satisfy the current report and to make it available for future reports. Users can also add free text to various parts of the report if desired.

[0051] The standardized statements presented to the user for selection are presented in order of their probability of selection, which probability is derived from a prediction strategy based on historical reports. The prediction strategy consists of three different elements: first, a database that illustrates the relative frequencies with which statements appear in common reports; second, some means for providing some initial statements from which predictions can be generated; and third, a method for determining, for each statement in the vector comprising the available statements, the cumulative chance that each statement also appears in combination with the selected statement based on a plurality of different relative frequencies.

[0052] The first element of the prediction strategy is a database or frequency array that describes the relative frequencies of statements, which includes the chance that if a report contains one statement, then the report also contains another statement, that is, the value (j, k) represents the probability that k appears in a report that also contains j. Therefore, it is a square array of size equal to the number of unique statements, or a vector of length equal to the number of statements, where each element is a vector of the same length.

[0053] To create the frequency array, find the frequency vector for each statement. For each statement j, return all reports containing j. For every other statement k, the frequency corresponding to k is the ratio of the number of times k occurs in the reports returned to the number of reports returned. For example, if j appears in 10 reports and k appears in 2 of them, then (j, k) = 0.2, regardless of how many times k appears in all reports. This can be extended to statement pairs, i.e., follow the above process but replace j with an i, j pair. This provides greater robustness between the associations found, at the expense of a larger database and computational requirements.

[0054] The second element of the prediction strategy is to use initial or seed sentences in order to generate predictions. This is done by matching keywords in the sentences to classify the sentences according to anatomical features or pathologies. This is a simple approach and the resulting classification list is incomplete and inaccurate, but is sufficient for the purpose. The average prediction accuracy is found by using these classified sentences as input and checking the resulting predictions against the reports in which the input sentences appeared. This is the average proportion of those sentences in the reports that were also found in the predictions.

[0055] Using the valid values ​​for the two selected classification types, an array is created to give the predictions that have been shown to be of greatest use. First, the 20 highest ranked statements for each classification type are found. The first three elements of each type are inserted as the first six elements of the new vector, and when there are fewer than three statements in a classification type, the corresponding vector element is empty. The remaining elements of the vector are filled in turn with the most predictive statements from either classification type.

[0056] The use of pairwise matrices shows potential for generating seed sentences, as sentence pairs corresponding to the chosen categorical input are easily established, and this can also be extended to other forms of metadata such as age, history, gender, or any other information that can be maintained along with the sentences.

[0057] The third element of the prediction strategy is to determine the probability of each statement also occurring in combination with the selected statements. For each statement, the prediction corresponds to the statement with the highest frequency ratio. If all reports containing the selected statement contain the second statement, then the frequency ratio for the second statement should be 1, and that statement is returned as the prediction. For n desired predictions, the statements corresponding to the n highest prediction ratios are returned. When multiple statements are taken as the basis for generating predictions, a vector formed by averaging the corresponding prediction ratios is created, and the prediction is returned from the highest average ratio.

[0058] This approach is most useful when the values ​​used as the basis for prediction have a relatively low overall frequency of occurrence in the corpus of reports, because the less frequently a particular statement appears, the more likely it is that other statements in the common report are related in some way. If a statement has been used in a single instance, then the frequency ratio of every other statement in the report relative to that statement is 1, and if that statement is used as input, then the output of the prediction will be that aforementioned report. Conversely, if a statement that appears in almost all reports is used as the only input, then the prediction returned will essentially be a list of the most common statements.

[0059] The preferred embodiment of the present invention includes a client application that generates a user interface and provides various services such as proofreading, and a server application for storing and retrieving reports and templates from a database. The implementation of the client and server applications can take various forms well known in the field of computer science. More relevant to the present invention than the programming details is the structure of the reports and templates and the associated metadata.

[0060] The reader will now appreciate the invention which provides a report editor that facilitates rapid report generation. In most cases, the user can generate a report without typing anything, and as the user is prompted with a large number of available statements, they are also likely to produce a more accurate report than would otherwise be the case. As the system accumulates reports over time, it will be able to provide better matching reports as a basis for new reports, presenting statement qualifiers in the order most likely to be selected.

[0061] Further advantages and improvements may be better implemented with respect to the present invention without departing from the scope of the present invention. Although the present invention has been shown and described in what are considered to be the most practical and preferred embodiments, it will be appreciated that deviations may be made within the scope of the present invention, which is not limited to the details disclosed herein, but will be based on the full scope of the claims, thereby including any and all equivalent devices and means. Any discussion of the prior art throughout the specification should not be considered as an admission that such prior art is well known or forms part of the common general knowledge in the field.

[0062] In this specification and claims (if any), the word "comprising" and its derivatives (including "comprises" and "comprise") include each of the integers stated but do not exclude the inclusion of one or more additional integers.

Claims

1. A report editor comprising: database; wherein a report on the images is generated according to a template; wherein the template comprises a series of statements from the database that are selectively included in the report; wherein the database stores a plurality of reports including historical reports; wherein the generated new report is based on a selected report from the plurality of reports stored in the database; wherein the selection of the series of statements in the new report is copied from a selection of statements in the selected report or from a plurality of stored reports; wherein the series of statements are presented in an order corresponding to the probability of any particular statement being selected; wherein the probability that the particular statement is selected is based on a selection of a first statement and a frequency of selection of a second statement when the first statement has been selected in the history report; or based on a selection of a pair of statements and a frequency of selection of the second statement when the pair of statements are selected in the history report; wherein a series of sentences selected in the new report are edited, and the edited sentences are stored in the database; wherein the edited statement is included in a completed report, and the completed report is stored in the database; wherein the stored edited statements and the stored completed reports update the database and are used as subsequent templates for subsequent new reports; and wherein each statement in the series of statements includes optional qualifiers that change the meaning of the statement; wherein the qualifiers are presented for selection in order based on associated probabilities of selection; wherein the selection probabilities of the qualifiers are stored in the template and updated when a previous qualifier is selected; or Therein, anonymous metadata from each completed report is added to a statement metadata database in the database and used to generate the subsequent new report, thereby improving the accuracy of the subsequent new report relative to previously generated reports and the historical reports.

2. The report editor of claim 1, wherein the reports stored in the database are presented in order of scores, the scores being determined by matching metadata of the new report with metadata of the stored reports.

3. The report editor of claim 1, wherein the reports stored in the database are presented in order of scores, the scores being determined by matching the image of the new report with the images of the stored reports. 4 . The report editor of claim 1 , wherein the probability that a statement will be selected is based on a frequency of selection of the statement in the historical report.

5. The report editor of claim 1, wherein a medical report on a radiology image is generated from a radiology report template, wherein: The series of statements selectively included in the medical report includes statements from a database of stored radiology reports.

6. The report editor of claim 1, wherein the subsequent template for the subsequent new report is generated from a series of statements stored in the database, the series of statements including edited statements from a stored completed report.

7. The report editor according to claim 1, further comprising: Weighting is given to previously reported years, generating a score for each report stored in the database based on the degree of metadata matching and the weight given to the metadata; and A list of reports is presented in order of the scores generated.

8. The report editor according to claim 1, wherein: Each time the template is used to generate a subsequent new report, the accuracy with which the template presents the qualifier increases.

Images