Automatic generation of a structured medical impression in a medical findings report

Abstract

A computer-implemented method and system, a computer program product, data carrier and a data carrier-signal for automatically generating a structured Impression-section of a medical findings report are introduced. Patient-specific information is accessed and its semantic meaning is determined using a large language model, LLM. By means of the LLM it is further determined whether a database comprising medical reporting elements, MREs, contains a matching MRE. A matching MRE is present if its semantic meaning matches with the determined semantic meaning of the piece of information and if a rule for automatically generating a structured entry in the Impression-section of the medical findings report is defined for it. In case a matching MRE can be determined, an entry in the Impression-section according to the rule defined for the matching MRE is generated, and in case no matching MRE can be determined, a free text summary of the information is generated in the Impression-section.

Description

[0001] Automatic generation of a structured medical impression in a medical findings report

[0002] TECHNICAL FIELD

[0003] The present disclosure relates to a method, system, computer program product and non-volatile storage medium for automatically generating a validated and actionable Impression-section of a medical report.

[0004] BACKGROUND

[0005] In a clinical context, doctors usually record findings, such as radiological, histological or laboratory findings, in form of written reports. Such medical (findings) reports document the examination or investigations of a patient and play an important role in communication between medical professionals. Medical reports represent medicolegal documents and, therefore, have to be validated, i.e., it has to be approved and signed by a medical professional.

[0006] Medical findings reports usually comprise at least a Findings-section and an Impressionsection. In the Findings-section, the individual findings are listed together with any related parameters. Thus, the individual findings are comprehensively described.

[0007] The Impression-section of the radiological report represents a summary of the most important pathological findings reported. The Impression-section is seen as a “conclusion” of the report, usually no additional information is added except for recommendations. Whereas in the Findings-section findings are comprehensively described, in the Impression-section they are only mentioned together with the most important characteristics (the ones that would make a difference in patient management and therapeutic decisions). Additionally, further information from other sections than the Impression-section (e.g., “Procedure”, “Clinical Information” or “Comparison”) as well as information from previous reports may be used for creating the impression if they are deemed relevant for the impression. The impression, or Impressionsection, can therefore contain conclusions, assessments, plans and / or recommendations. The Impression-section is currently still mostly dictated or typed manually by the radiologist at the end of the report based on already reported findings.

[0008] Ideally, the Impression-section contains actionable data, which means data is machine- interpretable, such is minable and can be further processed automatically, e.g., to automatically prompt a decision. This can, e.g., be accomplished by standardization and / or coding of medical reports.

[0009] To standardize medical reports structured reports have been proposed. In general, a report is considered to be structured if at least some of the elements of the report can be uniquely identified, such that ideally a meaning can be automatically associated with the entry. A structured report comprises a data structure, e.g., a hierarchical data structure like a decision tree. Thus, structured reporting assures consistency between reports, increases data accessibility and quality, and decreases the risk of omission. Furthermore, structured reporting has the potential to create actionable data in clinical routine.

[0010] The basic idea of structuring is already reflected in the organization of the report into systematic sections. The organization of a report according to the MMRT standard (five top-level elements: procedure, clinical information, comparison, findings and impression) is an example of such a basic structure. A very high degree of structuring is provided by so-called synoptic reports, which format an electronic report in the form of discrete data fields. That means each type of information has a specific place and format in the report. Such reports are considered to be fully structured, because each element of the report is uniquely identifiable. Fully structured / synoptic reports standardize the way data is collected, transmitted, stored, retrieved, and shared between clinical information systems and are actionable, i.e., comprise actionable data. In practice, the degree of structure will, vary and depend on standards of the institution and potentially also the personal preference of the reporting user. Thus, while, often the basic, five-top-level element structure will be present the structure will range from free-text only to fully structured / synoptic covering any hybrid form in between.

[0011] A common way to implement structured reports uses so called reporting templates, that provide a medical report with standardized structures and terminology that can be completed by a user. Such templates make it possible to provide customized structures and terminology for specific diagnostic contexts. Templates can therefore depend, for example, on the type of examination, a clinical question, an imaging modality and / or a body part or area. For example, a radiology findings report for an examination of the lungs requires different aspects to be addressed and different parameters to be determined than an examination of the brain or prostate. Depending on the template, predetermined structures and elements are specified for the individual sections of the report, e.g., the earlier mentioned procedure, clinical information, comparison, findings, and impression.

[0012] For example, an actionable medical impression can be generated based on a structured / synoptic Findings-section using a rule-based approach. According to the rule-based approach (e.g. SmartRadiology by SmartReporting GmbH, Munich), the fully structured / synoptic sections of a findings report are used as input to generate a structured / synoptic Impression-section. For this purpose, the corresponding systems include rule-based algorithms that generate a structured / synoptic Impression-section based on the structured / synoptic parts of the findings report, and in particular of the Findings-section. The structured / synoptic portions of the findings report trigger addition of specific information elements and / or fields in the Impression-section.

[0013] The rules can either be coded into the template of the structured report or be coded into separate algorithms / software that are configured to identify specific elements of structured reports, such as fields and entries of a template, and to execute the predetermined rules upon recognition of these elements to generate or adapt the Impression-section of the structured report according to these rules. However, such rule-based approaches have some important limitations. Given that the rulebased approaches require structured reports, any deviation from the expected structure will limit the functionality of the rule-based approach.

[0014] For example, the structure usually only covers expected findings, but not incidental findings. Consequently, whenever incidental findings are made, these have to be added to the Findings- section manually. A rule-based generation of the Impression-section can only consider the expected findings, because for incidental findings there will typically be no rule such that the corresponding parts of the impression need to be added by the user manually as well, e.g., by typing or dictation. Further, given that the current standard of medical reporting still is dictation or other forms of free-text-based input, the reports often are free-text reports, i.e., all entries are provided as non-structured free-text, or hybrid reports, i.e., any mix of a free-text and a structured / synoptic report. Given, that a rule-based approach can only generate an impression based on the structured parts of the report for which a rule exists, this makes a fully automated generation of a medical impression impossible, when free-text or hybrid reports are used, or when no rules are defined in a structured report.

[0015] A different approach that provides a solution for the above issues of rule-based approaches are Artificial Intelligence (Al)-based approaches. These approaches use Al algorithms, e.g., based on Natural Language Processing (NLP) or Large Language Models (LLM), to generate an Impression-section based on the other sections of the report. The advantage of Al-based approaches is that these can handle free-text reports and, in principle, also hybrid and fully structured reports. Provided the appropriate training the Al algorithms gain a semantic understanding of the elements of the report and summarize these in the Impression-section. However, the resulting impression is not structured. Even in cases where automation (e.g., NLP or LLMs) is used to automatically create an impression from free-text, these systems yield a free-text impression. In principle, further Al could be used to retrospectively structure free text Impression-sections. However, in these cases, the resulting structure is not validated by a physician, i.e., the structure is introduced after the medicolegal report has been signed and is therefore not validated as part of the clinical routine.

[0016] A number of concepts have been proposed in the past for automating the creation of diagnostic reports and, in particular, the impression using either rule-based or Al-based approaches.

[0017] US 2013 / 0290031 Al suggests, among other things, the use of so-called macros. These macros are specific to the study as well as previously dictated entries. Based on modality, body part, and radiologist a template is selected and the template provides access to the macros. Thus, based on the given context, the macro provides text that will be entered into one or more fields of the report when selected by the reporting doctor.

[0018] US, 11,342,055 B2 proposes using a model that generates entries of a findings report based on received findings, a context of these findings and a radiologist identifier. Thus, the model is trained to generate text from entered findings that approximates a specific writing style, e.g., of a specific radiologist, group of radiologists or standard. US 2023 / 0260649 Al concerns the Findings-section of a radiology report und uses the entries of the corresponding impressions in the report. It is suggested to machine train models for generating findings and to use loss that is based on the impressions of the report for the training. Such training is supposed to render the models to predict findings more accurately.

[0019] CN 115293128 A suggests to collect radiological images and text data to train an image encoder for visual features in image data and a sentence encoder for extracting semantic features from text data. The findings and impressions sections of a radiology report are then recursively generated by fusing the visual features extracted by the image encoder and the semantic features extracted by the sentence encoder.

[0020] WO 2022 / 192893 Al suggests using a trained Al system to automatically generate impressions from reports of medical findings. In particular, for each of a plurality of medical professionals a model is trained to approximate verbiage and writing style of the corresponding medical professional. Based on an initially specified medical professional, the system selects the corresponding model to generate a medical impression based on input findings that approximates the verbiage of then specified medical professional.

[0021] However, none of these approaches automatically creates a highly structured / synoptic, actionable and validated impression of a free-text or hybrid medical report.

[0022] SUMMARY OF THE INVENTION

[0023] The present invention solves this problem by means of the method according to claim 1, the system according to claim 10, the computer program product according to claim 13, the computer readable data carrier according to claim 14 and the data carrier-signal according to claim 15. Further preferred embodiments are implemented according to the dependent claims.

[0024] According to the invention for the automatic generation of the entries in the Impression-section of a medical report, a rule-based approach is used that is supplemented by using, in addition, at least one LLM. For each piece of information / each information element that has been entered in the report and for which a rule for creating an entry in the Impression-section of the findings report is found, this rule is applied, such that a rule-based entry is created in the Impressionsection. If no rule can be found for an entry in the report, the LLM is used to create an entry in the Impression-section that reproduces or summarizes the information. Within the scope of this invention an LLM is understood to be a type of Al algorithm that uses deep learning techniques and large data sets to understand, summarize, generate and predict new content. LLMs have proven to be very efficient tools for understanding the context and meaning of written text. This happens in an agnostic way. For the implementation of the systems and methods of this invention any LLM may be used, i.e., the invention is not limited to a specific LLM.

[0025] One embodiment of the invention allows the user generate the medical report in a fully structured way, an unstructured way (free text), or a mix of both (hybrid reporting). This embodiment provides the user the flexibility to select one of these approaches. Other embodiments of the invention are limited to either a fully structured / synoptic approach, a free-text only approach or a hybrid approach.

[0026] Further according to the invention, the LLM has access to a database comprising a plurality of medical reporting templates, medical reporting modules and / or medical reporting elements.

[0027] A medical reporting template (MRT) is a set of standardized and coded medical reporting elements depicting all relevant reporting questions for a given reporting case. E.g., an MRT may be a full reporting-form that is used for a specific medical report. According to some embodiments, it covers all sections of a radiological report including the five top-level sections according to the MRRT-standard “Procedure”, “Clinical Information”, “Comparison”, “Findings” as well as “Impression”. These elements may be modified, however, or completely omitted. Each top-level element may consist of one or more medical reporting modules (MRMs) or medical reporting elements (MREs) that are inserted depending on clinical need. In case there are no top-level elements, the template may consist of a stack of MRMs or MREs. A report using MRTs entails highly qualitative data and ensures completeness, machinereadability, and human-readability as well as the validity of the medical report. Any of the set of standardized and coded medical reporting elements is uniquely identifiable and machine readable, such that, in principle, the meaning of the element can be recognized by a machine.

[0028] An MRM is a smaller snippet of a complete template, e.g., an MRT. For example, a template would cover a full reporting use case such as “prostate MRI”, while a module would cover a smaller aspect of a use case such as “suspicious lymph node” and may be limited to one report section, i.e., the Impression-section (impression MRM), only. Thus, an MRT can be built in a modular way by using several MRMs that together constitute the MRT. Thus, MRMs represent report building blocks. MRMs may, e.g., consist of decision tree elements relating to a clinical reporting task. MRMs can be of varying depth (e g , there can be any number of levels / nodes within the decision tree). An example of a data structure of an MRM is a decision tree used to describe a lymph node lesion. If a decision support engine or the user concludes that a lymph node lesion may be present, this module will be inserted into the MRT. The module can then be filled manually by the user or it can be pre-filled by computer aided diagnosis (CAD)- algorithms. In some embodiments, MRMs present in an MRT may also be rejected / deleted by the user.

[0029] A medical reporting element (MRE) is the smallest unit of a structured report and contains a singular information or information element in terms of findings (e.g., “Presence / Absence of a pulmonary nodule” or “Max. Diameter of a pulmonary nodule”) or other patient specific information. MREs always follow the question-answer design wherein the question determines the reporting question that needs to be addressed and the answer sets the framework for the reporting physician to provide the pertaining answer. MREs may have different formats (in respect to the answer format) incl. number fields, term fields, single or multi selections capturing different types of information. It is the building block for MRTs and MRMs. Thus, like an MRT an MRM can be modularly built using at least one MRE such that the one or more MRE together constitute the MRM. Thus, an MRT can be constituted by several MREs, several MRMs that comprise or group one or more MREs, or a mix of MRMs and individual MREs. An MRE uniquely corresponds to a single piece of information in the structured report and thereby to a distinct data field of a structured report. It is the most granular element of the data structure underlying the structured report.

[0030] According to the invention, the LLM tries to match any provided piece of information to an MRE for which a rule for generating an entry in the Impression-section is defined. For example, the MRTs, MRMs and / or MREs may be stored in a knowledge graph. If such a match is possible, the corresponding rule is applied to generate an entry in the Impression-section.

[0031] Further according to the invention, the Impression-section is at least partly structured. In some embodiments the Impression-section fully or in parts consists of MREs. In some of these embodiments the impression comprises at least one impression-specific MRM (impression MRM)

[0032] According to some embodiments of the invention, the rule-based approach may be implemented using a rule-based automatic impression generation engine, whereby the creation of certain entries and / or the opening of certain fields in the Impression-section is triggered by detecting specific information provided in different sections of the report such as procedure, clinical information, comparison, and findings as well. In some embodiments the creation of certain entries and / or the opening of certain fields in the Impression-section may also be triggered by detecting specific information that is not provided in the medical report but provided by other sources of patient specific information like previous reports, electronic health records (EHR), electronic medical records (EMR), or any other documented information of a Radiology Information Systems (RIS), Clinical information System (CIS), Hospital Information System (HIS), Laboratory Information System (LIS), or Picture Archiving and Communication System (PACS).

[0033] According to one embodiment, the LLM is comprised by the systems according to the invention, e g., it is an open-source LLM the code of which is included in the system and / or software. According to another embodiment, the LLM is not comprised in the system, but is interfaced, e.g., a connection to a provider for LLM-based services is established, e g., via the internet or an intranet. According to yet another embodiment a mix of both is used, e.g., a first LLM is comprised in the system and, additionally, a second LLM is interfaced, e.g., as a supplement for the first LLM. In any of these embodiments, the large language models (LLMs) may be integrated via an Application Programming Interface (API) connection.

[0034] BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Fig. 1 illustrates a process of automatically generating a highly structured Impressionsection of a medical findings reports according to some embodiments;

[0036] Fig. 2 illustrates three different approaches a)-c) to use the LLM to determine whether the database comprises a matching MRE, or an MRE or MRM associated with the Impression-section the semantic meaning of which matches with that of the piece of information; Fig. 3 illustrates an alternative process of automatically generating a highly structured Impression-section of a medical findings reports according to some embodiments;

[0037] Fig. 4 illustrates an example of a user interface and the interaction between findings entered in the Findings-section and the generation of corresponding entries in the Impression-section;

[0038] Fig. 5 illustrates a system according to some embodiments of the invention.

[0039] DETAILED DESCRIPTION

[0040] As illustrated in Fig.l, in a process 100 to generate a structured / synoptic impression as a part of a medical findings report according to the invention, patient specific information is accessed and / or received 101. This information may be one or more pieces of information that is entered into the medical report, or one or more pieces of information that are not entered into the medical report but can be accessed in another source of patient specific information like a HIS, CIS, RIS, PACS, EHR, EMR or previous medical reports.

[0041] This information or piece(s) of information may have been generated by a user, e.g., a radiologist. Alternatively, or additionally, the information or at least parts of it may have been generated fully or partly automatically using dedicated software. For example, findings from a set of medical images (e.g., MRI, PET, CT, X-Ray, Ultrasound) or other medical data (e.g., ECG, EEG, laboratory values) may be made by a user or extracted by dedicated Al-algorithms and entered into the report or be recorded elsewhere. Examples for relevant information besides the findings of the current report comprise general patient information, like name, address and age, imaging modality as well as patient information or findings from previous examinations (e g., from previous medical reports).

[0042] According to the invention, the accessed information may have any style, be it fully structured or synoptic, hybrid or free-text only. Thus, any limitation to a specific style that has to be followed by the user is overcome.

[0043] The automatic generation of the medical impression may be initiated automatically, e.g., upon completing the required parts of the report, or by a user, e.g., using a mouse click, a button on a speech microphone, or a speech command.

[0044] According to the invention, any information that can trigger a rule, e.g., because it is entered in a structured / synoptic findings report element of the medical report for which a rule has been defined 103, will also trigger the automatic creation of an entry in the Impression-section according to this rule. This means that for all structured elements of the report for which a rule for automatic generation of a part in the Impression-section has been defined, this rule applied 105, i.e., an entry of the impression is generated as defined by the rule.

[0045] For any entered piece of information for which no rule for creating an entry in the Impressionsection can be determined in the report to be generated or the associated template, e g., because no rule has been defined for it or because it has been entered in free-text only format, an LLM is used to search 107 other existing report templates, report modules and / or report elements (MRTs, MRMs and / or MREs) to determine whether a template, module or element matching the piece of information can be found from which a rule for creating an entry in the Impressionsection can be derived.

[0046] To do so, the LLM is used to first determine the semantic meaning of the piece of information. The LLM-based algorithm then searches a database comprising MRTs, MRMs and / or MREs for an MRE that matches with the piece of information 107. Such databases are also referred to as a library or corpus. If a match is found and a rule for generating an entry in the Impressionsection of a medical report has been defined for the matching MRE, this rule is applied 109.

[0047] In some embodiments, the pieces of information to be analyzed by the LLM-based algorithm, are first reformatted into a format that is suitable for the LLM, such that the LLM-based algorithm can gain an understanding of the semantic meaning of the piece of information. This may advantageously improve the performance by increasing speed and / or accuracy.

[0048] In some embodiments, the LLM has been trained using the database, i.e., the database has been used as a set of training data for the LLM, such that the LLM has access to the knowledge comprised in the database. Thus, the LLM is advantageously specifically tailored to the reporting tasks. In some of these embodiments, the database is pre-processed into a format suitable for the LLM before the training of the LLM, which may advantageously improve the performance. This is illustrated in Fig. 2 a), in which a reporting platform 201 sends as a prompt 207a a query to the LLM 203. The LLM 203 has been trained using the data of the database 205 as training data and sends a response back to the reporting platform 201.

[0049] In other embodiments, the database is provided to the LLM as a prompt or input, i.e., the LLM is not necessarily trained using the database, but is provided access to the knowledge stored within the database for each query. Thus, advantageously, the LLM does not have to be retrained when the database changes, and the LLM can be exchanged more easily. This is illustrated in Fig. 2 b), where a reporting platform 201 sends a query which together with the database 205 is provided to the LLM 203 as a prompt 207b. the LLM 203 sends back a response to the reporting platform 201. In some of these embodiments, the database is pre-processed into a format suitable for the LLM before being provided as a prompt, which may advantageously improve the performance. In some of these embodiments, a Retrieval Augmented Generation (RAG) approach is used to query the database. Thus, a query-based model is combined with the LLM in that relevant content of the database is first determined using such a query -based model and the relevant content is then together with the original query (match for piece of information) presented as a prompt to the LLM. This is illustrated in Fig. 2 c), in which the reporting platform 201 sends a query to a RAG engine 213, which initiates a semantic search using a query-based model 211 to determine one or more entries of the database 205 that are relevant for the query. These relevant entries are together with the original query provided as a prompt 207c to the LLM 203, which sends via the RAG engine 213 a response back to the reporting platform 201. Optionally, in some embodiments, a programming language for controlling LLMs, e.g., Microsoft Guidance, can be used to perform the LLM-based query and to limit the possible answers to a desired (sensible) range of answers. A RAG approach and also using a programming language for controlling LLMs advantageously may increase the quality of any output of the LLM, e.g., by reducing the chance for so called hallucinations.

[0050] Back to Fig. 1, according to the invention, if the attempt to map the piece of information to an entry of the database is not successful, the LLM is used to suggest 115 a comprehensive summary of the piece of information to be used as a free-text entry in the Impression-section. In some embodiments, the user is prompted to confirm / accept or reject this suggested entry, e.g., via a user interface displayed on a display. In some of these embodiments, the user may be provided the option to modify the entry before confirming it, e.g., via the user interface. In some embodiments, the user may be required to be a medical professional to ensure that the Impression-section remains validated even if one or more entries of this section are suggested by the LLM.

[0051] In some embodiments, as illustrated in Fig. 3, in a process 300, that is an alternative to process 100 of Fig. 1, the query of the database comprises two stages. In this Figure, steps that the embodiments of Fig. 1 and 3 have in common, have the same reference number. First only parts of the MRTs or any MRMs or MREs that are not associated with the Impression-section are taken into account. E.g., first the query 307 is performed in MRMs or MREs that are associated with the Findings-section. If a match is found and a rule is associated with the matching MRM or MRE, this rule is applied. If no match can be determined, as a second stage, the query continues 311 in all MRMs or MREs that are associated with the Impression-section. Usually, no rules for generating an entry in the Impression-section are defined for such Impression- MRMs or -MREs. Rather, these Impress! on-MRMs or MREs provide text and / or fields to be added in the Impression-section. Thus, if a match between the piece of information and an Impression-MRM or -MRE can be determined, any text or field is added / opened 313 that is associated with the respective MRM or MRE. This advantageously introduces a further step of adding structured entries to the Impression-section even when no matches comprising rules have been determined in the database.

[0052] For example, a radiologist has determined in a CT thorax, that a pulmonary nodule is present in the right upper lobe of a patient’s lung and that is has a maximum diameter of 7mm. As illustrated in Fig. 4, the radiologist may now enter this in the Findings-section of the medical report, e.g., using a predefined (Findings-)MRM “Pulmonary nodule” in the Findings-section and set the (Findings-)MRE “right upper lobe” to “affirmative” and enter the value “7 mm” in the (Findings-) MRE “max. diameter =”. In a case A, for these a rule for generating a corresponding entry in the Impression-section has been defined, which triggers one or more corresponding (Impression-) MRMs or MREs in the Impression-section. E.g., the rule may trigger opening in the Impression-section a field containing the text “Pulmonary nodule located in the right upper lobe, with a maximum diameter of 7 mm. Lung-RADS 2022: 3 (probably benign). Recommendation: low-dose CT in 6 months". In a different case B, no rules connecting the above MRMs / MREs of the Findings- and Impression-section have been defined (or the radiologist has entered the findings as free-text). Fig. 4 shows an example of a reporting platform 400. There, the entries of the Findings-section are for example entered via a sidebar 401 providing MRMs 403 and MREs 405, 407 as menus and submenus. In a separate window 409, the Impression-section comprising Impression-specific MRMs and MREs 411 is shown. In this example, according to the above embodiment, in case A, first the Findings-MRMs / MREs are queried, a match with MRMs / MREs for which a rule has been defined is determined. Consequently, the rule is applied and above field in the Impression-section is opened. In case B, the query will continue in the Impression-MRMs / MREs of the database and if a match between the semantic meaning of the piece of information and an Impression-MRM / MRE can be determined, the above field with the same text will be opened in the Impression-section.

[0053] In some embodiments, the LLM-based algorithm is used to determine 119 if the parts for which no rule could be determined are relevant for the impression generation. These parts may then be excluded 121 from the Impression-section and any further processing like the database query, if they are deemed to be not relevant, and be included 107, 307, 311 in the Impression-section and any subsequent processing if they are deemed to be relevant. To determine the relevance, the LLM may be trained accordingly and / or the prompt for the LLM may be adapted accordingly. E.g, a combination of training data, parameter weights, and information included in the prompt may enable the LLM to determine the clinical relevance of the piece of information. Advantageously, this may streamline further processing and improve the conciseness and accuracy of the report.

[0054] An important requirement for findings reports and their Impression-sections is that the relevant findings and other entries recorded there are ordered according to clinical relevance. The Impression-section should always mention the most important finding first, followed by others, in order of importance (e g., if reporting on one kidney cyst and a lung cancer vs. a kidney tumor and a benign pulmonary nodule, the order will not be the same). Purely rule based approaches do not fulfill this requirement. Thus, in some embodiments, the LLM is used to order the entries of the Impression-section according to their clinical relevance. This may be done while or after completing the Impression-section. Thus, given that the LLM has determined the semantic meaning of the entered piece of information, it may further be trained to determine the clinical relevance of a finding or entry. E.g., the LLM might have been trained using medical findings reports, in which the Impression-section is ordered according to clinical relevance of the entries. In another example, the database provided as a prompt or input to the LLM comprises this knowledge, e.g., the database is a knowledge graph comprising this information. Finally, a combination of specifically trained LLMs and specifically formulated prompts (possibly with the aid of a dedicated programming language such as Guidance) may be used to achieve this goal. Advantageously, this further improves the quality and usability of the Impression-section.

[0055] In some embodiments, all of the previously described processing steps are performed in the backend / background and the user is presented with a completed Impression-section for review Thus, the rule-based approach and any LLM-based processing and generation steps are not visible to the user and there might be no or only limited possibility for user-interaction until the report is completed. This, may advantageously streamline the process of reporting.

[0056] In some embodiments, the user is prompted to interact with the system / methods or to take further action during the process of completing the Impression-section. For example, the user is prompted to confirm the mapping of findings to MRTs, MRMs and / or MREs before the impression text is created. In addition, based on the structured framework provided by the selected MRT or MRM, the user could be given indications of missing or incomplete information or suggestions for further aspects to be investigated. Advantageously, this may increase the quality of the resulting report.

[0057] In some embodiments, all sections, sentences, findings and other entries of the report that have been used to generate the Impression-section (i.e., all parts that are deemed medically relevant for inclusion in the impression) are highlighted. This may either be only for review during the creation of the report and the highlighting disappears once the report has been approved by a medical professional, or the highlighting may be permanent, such that the basis for the provided medical impression is advantageously transparent to any reader.

[0058] In some embodiments, there may be an option to apply the previously described processing to previously generated reports. This way, as an advantage, previous, non- structured or only partly structured reports and in particular their Impression-sections can retrospectively be fully structured. Further, reports using an older structure that is based on a different or outdated standard can be updated to new or updated standard, i.e., be restructured.

[0059] In some embodiments, after an entry in the Impression-section is generated, a decision support algorithm is used to generate a recommendation for further actions or therapeutic steps to be done and the recommendation may advantageously be added to the Impression-section. The decision support algorithm may itself be rule-LLM-based, and / or RAG-based. The recommendations may be based on current guidelines, e.g., RADS.

[0060] In some embodiments, the rule-based automatic impression generation is being updated constantly as new structured information is being mapped continuously from the free text and / or structured user input by the LLM. E.g., in the course of report generation the degree of structured information increases as the user interacts with the system or the LLM maps free- text information (non-accessible for rule-based algorithms) to MREs (accessible for rule-based algorithms). Thus, accessible information are provided to the system that may lead to additional rules being executed or already executed rules being updated. This advantageously ensures maximum up-to-datedness of the impression and thus the report as a whole.

[0061] In any of the described embodiments, as an optional features, the user is requested to review and adapt the Impression-section if necessary. This advantageously provides the user the option to review the impression section and, if needed, to correct and / or modify the entries, eventually improving the quality of the report and ensuring validation.

[0062] The above-described processes can be implemented by a computer, server or any kind of processor.

[0063] A system 500 according to the invention is illustrated in Fig. 5. The system 500 comprises a processing unit 501 that is configured to execute the steps of any of the previously described methods. In particular, the processing unit 501 may comprise logic to execute the LLM-based algorithms. The processing unit 501 may comprise, e.g., a one or more single- or multi-core processors, in particular a CPU or a GPU, or an integrated circuit, e.g., an Application Specific Integrated Circuit (ASIC), a programable logic, e.g., a Field Programmable Gate Array (FPGA), a state machine, or any other suitable device or combination of these devices. The system 500 may further comprise a data storage 503, e g., one or more Hard Disk Drives (HDD) or Solid State Drives (SSD), that includes the LLM 512. Alternatively, or additionally, the system 500 may comprise an interface 505 to connect to one or more LLMs 512, e.g., via the internet. E.g., an internet connection to a provider 507 of the LLM may be established by the system.

[0064] The data storage 503 of the system may further comprise the database 511 of MRTs, MRMs and MREs.

[0065] Further, the data storage 503 may comprise computer readable instructions 513 that, when executed by the processing unit 501 cause the processing unit 501 to execute the steps of any of the previously described processes. The computer readable instructions 513 may also comprise the rules according to the rule-based approach. Further, the system may comprise a memory 509 to load the computer-readable instructions 513 before they are executed by the processing unit 501. In some embodiments, the data storage comprising the database 511, LLM 512 and computer readable instructions 513, may be several physically separated data storages. Additionally, or alternatively, the database may be stored in one or more external data storages to which a connection, e.g., via the internet or an intranet, is established to access the database. E.g., the database may be stored on a server or in a cloud. The database may comprise also the rules according to the rule-based approach.

[0066] Further, in some embodiments the data storage 503 is used to store the medical findings report. In other embodiments that medical findings report is stored in an external data storage, a server or in a cloud.

[0067] Optionally, the system may have access to external sources 514 of information like a HIS, CIS, RIS, PACS, etc. Finally, the system may comprise input devices 515 like mouse, keyboard, and microphone and output devices like a monitor or projector. The input / output devices allow the user to view the reports and / or images and other clinical data as well as to interact with the processes and results. In particular, the display may help to support the user when generating the report and to present the results to the user.

[0068] The user may have access to the system locally, or remotely, e g., the system can be part of a web-based service provided to the user. E.g., the user has access to local input and output devices, however, the other components of the system are accesses via a remote connection, e.g., the internet or an intranet.

[0069] Although omitted for conciseness, the previously described embodiments include every combination and permutation of the various system components and the various method steps. The method steps can be performed in any suitable order, sequentially or concurrently.

Claims

CLAIMS1. Computer-implemented method for automatically generating a structured Impression-section of a medical findings report, comprising a) accessing a piece of information contained in the medical findings report; b) determining the semantic meaning of the piece of information using a large language model, LLM; c) determining, by means of the LLM, whether a database comprising medical reporting elements, MREs, contains a matching MRE, wherein an MRE is a standardized and machine-readable building block of a structured medical findings report to which a unique semantic meaning is assigned, and wherein a matching MRE is present if its semantic meaning matches with the determined semantic meaning of the piece of information and if a rule for automatically generating a structured entry in the Impression-section of the medical findings report is defined for it; d) if a matching MRE can be determined, generating an entry in the Impression-section according to the mle defined for the matching MRE; and e) if no matching MRE can be determined, generating a free text entry in the Impression-section corresponding to a LLM-generated summary of the entered piece of information.

2. The method according to claim 1, wherein if the piece of information is contained in a data field of the medical findings report corresponding to an MRE for which a rule for automatically generating a structured entry in the Impression-section of the medical findings report is defined, the entry in the Impression-section is generated according to the rule and steps b)-e) are omitted.

3. The method according to any of claims 1 or 2, further comprising the following steps subsequent to step d): i) if no matching MRE associated with any other section than the Impression-section can be determined, determining, by means of the LLM, whether the database comprises an MRE associated with the Impression-Section the semantic meaning of which matches with the semantic meaning of the piece of information; and ii) if such an MRE can be determined, adding any entries corresponding to the determined MRE to the Impression-section; and iii) if no such MRE can be determined, executing step e).

4. The method according to any of the preceding claims, wherein step b) also comprises determining, by means of the LLM, whether the piece of information is relevant for the Impressionsection, and if it is determined that the piece of information is not relevant for the Impression-section, excluding it from the Impression-section; andif it is determined that the piece of information is relevant for the Impression-section, executing the steps following step b).

5. The method according to any of the preceding claims, further comprising ordering, by means of the LLM, the entries of the Impression-section according to their clinical relevance.

6. The method according to any of the preceding claims, wherein determining whether the database contains a matching MRE is carried out by means of a retrieval augmented generation, RAG, in which a query-based model is used in combination with the LLM.

7. The method according to claim 6 as far as it depends on claim 4, wherein also determining whether the database comprises an MRE associated with the Impression-Section the semantic meaning of which matches with the semantic meaning of the piece of information is carried out by means of a RAG.

8. The method according to any of the preceding claims, wherein the piece of information, is, alternatively, contained in a source of information other than the medical findings report9. The method according to claim 6, wherein the source of information other than the medical findings report is any of a previous report, an electronic health record, an electronic medical record, or data from a Radiology Information System, a Clinical Information System, a Hospital Information System or a Picture Archiving and Communication System.

10. Computer-system for automatically generating a structured Impression-section of a medical findings report, comprising a processing unit for executing computer readable instructions; one or more data storages comprising: a database including medical reporting elements, wherein a medical reporting element is a standardized and machine-readable building block of a structured medical findings report to which a unique semantic meaning is assigned; a large language model, LLM; and computer-readable instructions, which when executed by the processing unit, cause the processing unit to perform the steps of any of claims 1-7.

11. The system according to claim 10, wherein alternatively, or additionally to the one or more data storages comprising the LLM, the system is configured to interface a remote LLM.

12. The system according to claim 10 or 11, further configured to have access to a source of information other than the medical findings report, and wherein the computer-readable instructions,when executed by the processing unit, cause the processing unit to perform the steps of any of claims 8 or 9.

13. Computer-program product, comprising readable instmctions, which when executed on a processing unit cause the processing unit to execute the steps of the methods according to any of the claims 1-9.

14. Computer-readable data carrier, having stored thereon the computer-program product of claim 13.

15. Data carrier-signal carrying the computer-program product of claim 13.

Images