Method for providing a medical finding and method for providing a final medical report

Abstract

The invention relates to a computer-implemented method for providing a medical finding to a requesting entity, the method comprising a plurality of steps. A step is directed to obtaining at an evaluating entity, medical image data from a requesting entity, the requesting entity being remote from the evaluating entity. Another step is directed to determining, at the evaluating entity, at least one medical finding based on the obtained medical image data. Another step is directed to generating transmission image data. Another step is directed to embedding at least one of the determined medical findings as text in the transmission image data. Another step is directed to providing the at least one medical finding to the requesting entity by means of transmitting the transmission image data to the requesting entity.

Description

[0001] Siemens Healthineers AG

[0002] 1

[0003] METHOD FOR PROVIDING A MEDICAL FINDING AND METHOD FOR PROVIDING A FINAL MEDICAL REPORT

[0004] Independent of the grammatical term usage, individuals with male, female or other gender identities are included within the term.

[0005] The invention is related to methods for providing medical findings and to methods for providing final medical reports, in particular within a teleradiology framework.

[0006] Teleradiology is the practice of transmitting radiological image data, such as X- ray images, CT images, and MRI images, from a requesting location or entity to another location, e.g. an evaluating entity, for interpretation by radiologists and physicians. This process allows radiologists to provide their services without being physically present at the location of the patient, e.g., when specialist knowledge is required or when no radiologist is currently available at the requesting site. Teleradiology utilizes telecommunication technology, including the Internet, telephone lines, and advanced network technologies, to enable realtime access to radiology services across distances. This is particularly beneficial for providing specialized radiological expertise in remote or underserved areas.

[0007] Concepts for implementing teleradiology are generally available.

[0008] However, ensuring seamless integration of teleradiology systems with existing hospital infrastructure, such as PACS (Picture Archiving and Communication System), the reporting system, electronic medical records, etc., can be complex. Furthermore, compliance with health data security and privacy laws is essential.

[0009] To secure the protected health information (PHI) of patients, generally, only minimized data is exchanged between a requesting entity and an evaluating entity. Identifiers, which may identify a patient, are removed from the data before sending the data to the evaluating entity. However, the missing patient identifiers in the data sent to the evaluating entity negatively influence the integration of findings, which are determined at the evaluating entity, into the archiving system or into the reporting system of the requesting entity.

[0010] According to one approach, staff of the evaluating entity is provided with direct remote access to a reporting application of the requesting entity. This enables the authorized staff members to store determined findings and / or to create a medical Siemens Healthineers AG

[0011] 2 report directly within the IT-infrastructure of the requesting entity. However, this approach comes with severe drawbacks, since the granted direct access to the requesting entities IT-infrastructure requires a sophisticated contractual framework between the requesting and the evaluating entity. In certain instances, tracking of the action performed by authorized users may be required. Further, setting up and maintaining a secure remote connection between the evaluating entity and the requesting entity is cumbersome. The risk of patient health information leakage exists, since an access to the sensitive protected health information of the requesting entity is granted.

[0012] Therefore, there is a need for an improved solution that allows a better transfer of findings and / or reports from an evaluating institution to a requesting institution. In particular, the solution shall provide for a secure communication of medical findings and / or medical reports between remote entities. In particular, it shall be possible to integrate the solution into the IT-infrastructure of the respective entities with minimal efforts. In particular, the solution shall be costefficient and require less maintenance than known solutions.

[0013] This task is solved by the features of the independent claims. Further advantageous examples are included in the dependent claims.

[0014] According to an aspect of the invention, a computer-implemented method is provided. The computer-implement method is adapted for providing a medical finding to a requesting entity. The method comprises a plurality of steps:

[0015] A step is directed to obtaining, at an evaluating entity, medical image data from a requesting entity, the requesting entity being remote from the evaluating entity. Another step is directed to determining, at the evaluating entity, at least one medical finding based on the obtained medical image data. Another step is directed to generating transmission image data. Another step is directed to embedding at least one of the determined medical findings as text in the transmission image data. Another step is directed to providing the at least one medical finding to the requesting entity by means of transmitting the transmission image data to the requesting entity.

[0016] Within the framework of this disclosure the term entity shall be interpreted broadly. An entity may be a healthcare provider, wherein the information, in particular protected health information, stored within the IT-infrastructure of Siemens Healthineers AG

[0017] 3 the entity may not be shared with other entities. In an example, an entity may be or may include a healthcare organization, a hospital within a healthcare organization, a department within a hospital, a facility, an institution or a system, etc., or even a combination of the previously-mentioned examples. In particular, the evaluating entity may include a plurality of evaluating resources.

[0018] In other words, from a data-exchange perspective, the requesting entity may be separated or remote from the evaluating entity, regardless of the specific local arrangement of the requesting entity and of the evaluating entity. Each entity may be, e.g., due to legal requirements, responsible for ensuring the privacy of protected health information related to or originating from its operation. Each of the requesting entity and the evaluating entity may include a closed IT- infrastructure, such that sensitive protect health information is not accessible from the outside of the IT-infrastructure without permission.

[0019] The requesting entity and the evaluating entity may be part of a teleradiology framework. Thus, medical image data may be sent from the requesting entity to the evaluating entity for expert interpretation. The evaluated image data may be sent back to the requesting entity for further processing.

[0020] The requesting entity and / or the evaluating entity may include various components of a healthcare information system, such as components for image storage and processing, e.g. a picture archiving and communication system (PACS), for managing and reviewing medical reports, such as a reporting system, and / or for radiology reporting and / or review, e.g. radiology information system (RIS). The systems may comprise components or processes, which may be adapted for computerized provider order entry (CPOE), clinical decision support, patient monitoring, population health management (e.g., population health management system (PHMS), health information exchange (HIE), etc.), healthcare data analytics and / or cloud-based image sharing. The systems may include electronic medical record systems (e.g., electronic medical record system (EMR), electronic health record system (EHR), electronic patient record (EPR), personal health record system (PHR), etc.), and / or other health information systems (e.g., clinical information system (CIS), hospital information system (HIS), patient data management system (PDMS), laboratory information system (LIS), cardiovascular information system (CVIS), treatment planning systems (TPS), oncology information systems (OIS), etc.). Siemens Healthineers AG

[0021] 4

[0022] In a particular embodiment, the requesting entity and / or the evaluating entity comprises at least one component for storing medical image data, such as a PACS, and one component for storing medical reports, such as a reporting system.

[0023] In general, “Obtaining” may include “collecting” and / or “determining” and / or “receiving” and / or “retrieving” and / or “querying a database to obtain”, independently how the data / information / element to be obtained is specifically embodied. In particular, medical image data may be transmitted from the requesting entity to the evaluating entity for evaluation and analysis of the medical image data, e.g., for determining medical findings based on the image data. For this, the requesting entity may upload the medical image data to an image hub. The image hub may belong to the requesting entity, to the evaluating entity or to a third party. The image hub may distribute the obtained medical image data to an appropriate evaluating entity or, if the image hub belongs to the evaluating entity, to an appropriate evaluating center or evaluating institution of the evaluating entity.

[0024] The medical image data may be or may include a two-dimensional image. The medical image data may be or may include a three-dimensional image. The medical image may be or may include a four- dimensional image, where there are three spatial and one time-like dimensions. The medical image data may comprise a plurality of individual medical images.

[0025] The medical image data comprises image data, for example, in the form of a two- or three-dimensional array of pixels or voxels. Such arrays of pixels or voxels may be representative of color, intensity, absorption or other parameters as a function of two or three-dimensional position, and may, for example, be obtained by suitable processing of measurement signals obtained by a medical imaging modality or image scanning facility.

[0026] A medical image may be a radiology image depicting a body part of a patient. Accordingly, it may contain two or three-dimensional images of the patient’s body part. The medical image may be representative of an image volume or a crosssection through the image volume. The patient’s body part may be comprised in the image volume. Siemens Healthineers AG

[0027] 5

[0028] A medical imaging modality corresponds to a system used to generate or produce medical image data. For example, a medical imaging modality may be a computed tomography system (CT system), a magnetic resonance system (MR system), an angiography (or C-arm X-ray) system, a positron -emission tomography system (PET system) or the like. Specifically, computed tomography is a widely used imaging method and makes use of “hard” X-rays produced and detected by a spatially rotating instrument. The resulting attenuation data (also referred to as raw data) is processed by a computed analytic software producing detailed images of the internal structure of the patient’s body parts. The produced sets of images are called CT-scans which may constitute multiple series of sequential images to present the internal anatomical structures in cross sections perpendicular to the axis of the human body. Magnetic Resonance Imaging (MRI), to provide another example, is an advanced medical imaging technique which makes use of the effect magnetic field impacts on movements of protons. In MRI machines, the detectors are antennas and the signals are analyzed by a computer creating detailed images of the internal structures in any section of the human body.

[0029] Accordingly, the depicted body part of the patient in general will comprise a plurality of anatomies and / or organs. Taking a chest image as an example, the medical image may show lung tissue, the rib cage, lymph nodes and others.

[0030] The medical image data may comprise a plurality of images or image slices. The slices respectively show a cross-sectional view of the image volume. The slices may comprise a two-dimensional array of pixels or voxels as image data. The arrangement of slices in the medical image data may be determined by the imaging modality or by any post-processing scheme used. Further, slices may artificially be defined in the imaging volume spanned by the medical image data. Optionally, this may happen as a function of the image data comprised in the medical image data in order to optimally pre-process the medical image data for the ensuing diagnostic workflow.

[0031] The medical image data may be a two-dimensional pathology image data set, i.e., a so-called whole-slide image. A whole-slide image may show a tissue slice or slide of a patient. The tissue slice may be prepared from tissue samples taken from the patient. Further, the preparation of a tissue slice may comprise the staining of the tissue slice with a histopathological staining. The staining in this case can serve to highlight different structures in the tissue slice, such as, e.g., Siemens Healthineers AG

[0032] 6 cell walls or cell nuclei, or to test a medical indication, such as, e.g., a cell proliferation level. To create the whole-slide image, the stained tissue slices are digitized or scanned. To this end, the tissue slices are scanned with a suitable digitizing station, such as, for example, a whole-slide scanner, which preferably scans the entire tissue slice mounted on an object carrier and converts it into a pixel image.

[0033] In particular, the image data may include at least one mammography image. Mammography is an X-ray imaging method used to examine the breast, e.g., for the early detection of cancer and other breast diseases. It serves both as a diagnostic and screening tool. During a mammogram, the breast is compressed between two firm surfaces to spread out the breast tissue, and then an X-ray captures black-and-white mammography.

[0034] In addition to the actual medical image, the medical image data may further include metadata. The metadata may include at least on metadata attribute. In particular, the at least one metadata attribute comprises an attribute tag and an attribute value. The attribute tag specifies which information is comprised by the at least one metadata attribute. For example, the attribute tag can be one out of the following: ‘Body part examined’, ‘Anatomic Region Sequence’, ‘Patient Orientation’, View Position’, ‘Image Laterality’, ‘Frame Laterality’, ‘Measurement Laterality’ etc. Typically, the attribute tag cannot be changed. The attribute value provides the value corresponding to the attribute tag. In particular, the attribute value can be empty. Alternatively, the attribute value can comprise a string value and / or an integer value and / or a float value and / or a free-text value. In particular, the attribute value comprises an information about the “topic” which is provided by the corresponding attribute tag.

[0035] Further vectorial data may be included, such as structure set. The vectorial data may be a set of polygons, which may be used for segmenting structures in the image data, e.g.

[0036] The medical image data may be stored in a standard image format such as the Digital Imaging and Communications in Medicine (DICOM) format and in a memory or computer storage system such as a Picture Archiving and Communication System (PACS), a Radiology Information System (RIS), and the like. Whenever DICOM is mentioned herein, it shall be understood that this refers to the “Digital Imaging and Communications in Medicine” (DICOM) Siemens Healthineers AG

[0037] 7 standard, for example according to the DICOM PS3.1 2020c standard (or any later or earlier version of said standard).

[0038] A medical finding may relate to corresponding medical image in the medical image data. A medical finding may indicate a certain condition or pathology of the patient. The condition or pathology may be relevant for the diagnosis of the patient.

[0039] In other words, the medical finding may relate to an anatomical structure that differentiates the patient from other patients. Medical findings may be located within different organs of the patient (e.g., within the lung of a patient, or within the liver of a patient) or in between the organs of the patient. In particular, a medical finding may also relate to a foreign body.

[0040] In particular, a medical finding may relate to a neoplasm (also denoted as “tumor”), in particular, a benign neoplasm, an in-situ neoplasm, a malignant neoplasm and / or a neoplasm of uncert ain / unknown behavior. In particular, a medical finding may relate to a nodule, in particular, a lung nodule. In particular, a medical finding may relate to a lesion, in particular, a lung lesion.

[0041] Medical findings may be classified according to their type or category. This type or category is called “finding type”. A finding type may specify the general nature of medical finding. Further, the finding type may specify the anatomy or organ in which a medical finding has been found. According to some implementations, the finding type may also be conceived as a label of the medical finding. For instance, finding types may be lung nodule, liver nodule, cyst, rib fracture, undefined lesion, etc.

[0042] In some cases, the medical finding may be that there is no finding in the medical image data.

[0043] A medical finding may be determined by a human evaluator of the medical image data and / or by a computer aided detection algorithm, which may generally be configured to detect candidate medical findings in medical images. For instance, the findings detection algorithms may have two stages: the detection stage for detecting potentially relevant patterns in image data and the classification stage for classifying the potentially relevant patterns either as candidate medical findings or as false positives to be discarded. In principle, a plethora of Siemens Healthineers AG

[0044] 8 functionalities and methods is known for such computer aided detection and classification of candidate medical findings - all of which may be implemented in the findings detection algorithms. According to examples, the findings detected by an algorithm may be reviewed by an human evaluator.

[0045] In the framework of this disclosure, transmission image data is to be understood broadly and describes medical image data, which is generated by the evaluating entity and shall be sent to the requesting entity. The definitions and explanations, which are provided above with respect to medical image data may also apply to the transmission image data. Thus, the transmission image data may include images, in particular medical images, and / or metadata. The transmission image data may include at least one transmission image element. In particular, the transmission image data may include multiple transmission image elements. The transmission image data may include evidence image data, wherein the evidence image data includes at least one image with annotated findings.

[0046] At least one of the transmission image elements may be used for transmitting the determined medical findings as text, i.e. in textual form, to the requesting institution. For this, the at least one medical finding may be embedded in textual form in the transmission image element. More than one transmission image element may be used for transmitting the determined medical findings, e.g., depending on the quantity of determined medical findings.

[0047] The medical finding may be embedded in the body of the transmission image element, i.e., in the actual image data. E.g., an image, in particular a secondary capture image, or a PDF document including the at least one determined medical finding in textual form may be generated. In this case, OCR techniques may be used at the requesting entity to extract the textual medical findings from the image or from the PDF document.

[0048] In addition, or as an alternative, the medical findings may be embedded as text in the metadata of the transmission image element, as further described below. In the framework of this disclosure, the term „embedding“ shall be understood broadly and means, that the information about the medical findings is included in the transmission image data, such that it can be retrieved and understood by the requesting institution. In particular, there is not necessarily an encoding or compressing of the medical findings, although encoding or compressing of Siemens Healthineers AG

[0049] 9 medical findings is an option, in order to make the transmission of the medical findings more efficient and / or secure.

[0050] As stated before, the transmission image data is transmitted to the requesting entity, such that the medical findings are available at the requesting entity for further processing, e.g., for generating a medical report based on the medical findings and / or for review by a radiologist of the requesting entity.

[0051] Thus, the textual medical findings are transmitted from the evaluating entity to the requesting entity via the already established image reading connection. The image reading connection may have been used for transmitting images from the requesting entity to the evaluating entity. There is no separate communication or transport required for communicating the textual findings. This allows an efficient and secure transfer of medical image data between entities, wherein well-known standards for transmitting images, such as DICOM, can be used. Further, this allows applying an efficient de-identification process of the medical data, as further described below.

[0052] In particular, the solution enables the requesting entity to receive the medical findings in addition to and in context with the evidence image data without exposing the reporting system of the requesting institution to the external, remote evaluating institution. Thus, the protected health information in the reporting system of the requesting institution is not disclosed to anyone outside of the requesting institution.

[0053] In one embodiment, the method further comprises the step of generating, at the evaluating entity, a medical report based on the at least one medical finding. In addition, or as an alternative, embedding at least one of the determined medical findings as text in the transmission image data may include embedding the generated medical report in the transmission image data. Preferably, the medical report may be an HL 7 report.

[0054] Thus, not only medical findings are determined at the evaluating entity, but at least parts of a medical report are generated based on the determined medical findings. The generated medical report, or the parts of a medical report are then included in the transmission image data, e.g. as DICOM structured report. A medical report may include, e.g. textual passages, images, audio transcription and / or 4D, i.e. video data. Siemens Healthineers AG

[0055] 10

[0056] In an embodiment, the medical image data may include minimized patient information, in particular a pseudonymized patient identifier. The method further comprises a step of obtaining minimized patient information, in particular a pseudonymized patient identifier, from the medical image data. Preferably, the report is generated using the minimized patient information, in particular the pseudonymized patient identifier.

[0057] In the present disclosure, pseudonymization refers to a procedure in which all person-related data is replaced by unique identifiers, allowing to trace data back to its origins, and to preserve a patient-to-data mapping. In anonymization, all person-related data that could allow backtracking is nullified. Therefore, anonymized data may not contain any hint on the corresponding patient, which may be required due to legal privacy requirements, e.g.

[0058] Due to use of pseudonymization techniques, the requesting entity has the possibility of safely transmitting medical images to a remote evaluating entity without impairing privacy of patient data.

[0059] In particular the minimized patient information may include a hash identifier. The hash identifier may be a pseudonym, generated from the protected health information of the patient using a hash function, and may therefore allow for a mapping of the protected health information to the hash identifier.

[0060] The de -identification process ensures the patient PHI does not leave the IT- systems of the requesting entity, such that the risk of a leak of sensitive patient information is reduced.

[0061] Further, the pseudonymized identifiers may be used for generating the medical report or parts of the medical report at the evaluating entity / facility. In particular, if the generated report is a HL7 report, including personal information in the report may be mandatory. Using the pseudonymized patient information instead of the protected health information of the patient allows complying with the HL7 standard without violating privacy regulations and compromising the data security of protected health information available at the requesting entity. Siemens Healthineers AG

[0062] 11

[0063] In an embodiment, the method may further comprise the step of distributing, by means of an image hub and based on information extracted from the medical image data, the obtained medical image data to an evaluation institution, to an evaluating annotator and / or an evaluating algorithm. Then, the at least one medical finding may be determined by the evaluation institution, by the evaluating annotator and / or the evaluating algorithm.

[0064] In particular, the medical image data may be distributed based on the information included in the image data. The information may be about the imaging procedure, the used imaging modality, characteristics of the exam, the depicted body regions and / or organs, etc. The information may be compared to known characteristics of the available evaluating resources. Based on the comparison, an available evaluating resource with the best fitting characteristics may be determined, and the medical image data may be sent to this evaluating resource. E.g., chest X-ray images may be distributed to evaluating entities, evaluating centers and / or evaluating medical specialists specializing on determining medical findings for X-ray images, in particular chest X-ray images. In an example, the medical image data may be distributed based on information extracted from the metadata of the medical image data.

[0065] In an embodiment, the step of generating transmission image data further comprises the step of generating evidence image data based on the determined medical findings. The evidence image data includes at least one annotated image with an annotation of one determined finding. Optionally, the transmission image data includes an additional transmission image element, i.e., in addition to the evidence image data. The additional transmission image element may be used for transmitting the medical findings and / or the medical report to the requesting institution. Thus, the evidence images and the determined findings and / or reports may be sent to the requesting institution as a common transmission image data set. This facilitates handling and distribution of the transmission image data from the evaluating entity to the requesting entity and also facilitates the further processing of the transmission image data at the requesting facility.

[0066] In an embodiment, the determined medical findings and / or the generated medical report may be embedded in at least one of the images of the transmission image data, in particular in the image of the additional transmission image element. Siemens Healthineers AG

[0067] 12

[0068] In an embodiment, the determined medical findings and / or the generated medical report may be embedded in the metadata of the transmission image data, in particular in the metadata of the additional transmission image element. As explained above, the metadata may include attributes, each including an attribute tag and an attribute value. In particular at least one, or multiple existing attributes may be used for transmitting the medical findings and / or the generated report. Alternatively, or additionally, a new attribute may be generated and used for transmitting the medical findings and / or the report.

[0069] Preferably, the medical findings and / or the medical report, which may be in the HL7 format, are included in the metadata of the transmission image data as DICOM structured report. A DICOM Structured Report (SR) is a standard used in the field of medical imaging, normally for communications sent by modalities to, e.g., a PACS. It outlines the rules for encoding, transmitting, and storing imaging diagnostic reports. DICOM SRs are structured, linking text to other data like images, waveforms, and coordinates. DICOM SRs include well-defined, limited sets of content items, also called leafs and relationships among them. Using structured reports for embedding the determined medical findings and / or the medical reports enhances the interoperability between the systems of the requesting entity and of the evaluating entity.

[0070] Further preferably, the embedded medical findings and / or the embedded medical report is an HL7 report or a FHIR report, or complies with the HL7 format or the FHIR format, and / or includes all information, which is necessary for drawing up an HL7 or a FHIR report. In other words, the embedded medical findings and / or the embedded medical report may be sent to the requesting entity as a DICOM structured report. However, the transmitted data already contains all information and has the correct format, which is required for being included in the reporting system of the requesting entity. Back at the requesting entity, only the HL7 report information has to be extracted from the DICOM structured report framework. Thus, manual copy-paste tasks, which were commonly required at the requesting entity to transfer the data between different formats, in particular from DICOM to HL7, or vice versa, are avoided.

[0071] In some cases, when there is no finding detectable in the medical image data, the embedded finding may indicate that there is no finding. The requesting entity, after transmitting the medical data for evaluation to the evaluating entity, may await ■ in any case - a feedback of the evaluating entity. Transmitting of the Siemens Healthineers AG

[0072] 13 information that there is no finding in the medical image data, may allow the requesting institution to continue its workflow and to terminate the state, in which the requesting entity waits for the feedback of the evaluating entity.

[0073] In one embodiment, the transmission image data includes a metadata attribute indicating that the transmission image data comprises at least one textual medical finding, parts of a medical report and / or a medical report. In particular, a transmission image element may include a metadata attribute indicating that the specific transmission image data element comprises at least one textual medical finding and / or medical report.

[0074] This metadata attribute may be generated specifically for this task at the evaluating entity, in particular as a DICOM private tag. This solution provides a wide variety of design and implementation possibilities. Alternatively, an existing metadata attribute of the transmission data may be used for indicating that textual findings and / or a medical report is present in the transmission image data, and / or for transmitting the medical findings and / or the medical report. Using an existing attribute for this has the advantage, that the interoperability of the method is improved, in particular, if the attribute complies with known standards, such as the DICOM standard, and therefore can be processed by most healthcare IT-systems. At the requesting institution, this metadata attribute may be scanned in order to determine, whether a specific transmission image element includes textual findings and / or reporting information. Thus, a standardized procedure for exchange of information between the requesting entity and the evaluating entity is established.

[0075] According to an embodiment, apart from image data, in particular DICOM data, no other data is exchanged between the requesting entity and the evaluating entity. In particular, no exchange of HL 7 information between the entities is necessary.

[0076] A further aspect of the invention is related to a computer-implemented method for providing a final medical report, in particular at a requesting entity. The method comprises a plurality of steps. A step is direct to obtaining at a requesting entity, transmission image data from an evaluating entity, the requesting entity being remote from the evaluating entity, and the transmission image data including a plurality of transmission image elements. At least one transmission image element includes an embedded textual finding, parts of an Siemens Healthineers AG

[0077] 14 embedded medical report and / or an embedded medical report. Another step is directed to determining for each of the transmission image elements, whether the transmission image element includes at least one embedded textual medical finding, parts of an embedded medical report and / or an embedded medical report. If the transmission image element does not include at least one embedded textual medical finding, parts of an embedded medical report and / or an embedded medical report, another step is directed to providing the transmission image element to an image archiving system, in particular to a PACS, of the requesting entity. If the transmission image element includes at least one embedded textual medical finding, parts of an embedded medical report and / or an embedded medical report, the transmission image element is not sent to the picture archiving system, instead, another step is directed to extracting the at least one embedded textual medical finding, parts of the embedded medical report and / or the embedded medical report from the transmission image element. Another step is directed to generating a final medical report, in particular an HL7 or FHIR report, based on the at least one textual finding and / or the embedded medical report. Another step is directed to providing the final medical report to a reporting system of the requesting entity. In particular, the final medical report may be an electronic health record or a part of an electronic health record.

[0078] Some or all of the steps of the described method for providing a final medical report may be executed in combination with some or all of the steps of the method for providing a medical finding to a requesting entity. Explanations and definitions, which are provided in the context of the method for providing a medical finding to a requesting entity, may also apply to the method for providing a final medical report.

[0079] The transmission image data may be downloaded by the requesting entity from an image hub. Alternatively, the transmission image data may be directly sent from the evaluating entity to the requesting entity and received at the requesting entity. Each of the transmission image elements included in the transmission image data may be scanned and it may be checked, whether the specific transmission image element includes reporting information and / or textual medical findings, sent by the evaluating entity. All transmission image elements, which are not used for transmitting textual medical findings or reports, e.g., evidence images with annotations, are sent to an image archiving system of the requesting institution. These transmission image elements may be directly made available for review by a radiologist. The other transmission image elements, i.e. Siemens Healthineers AG

[0080] 15 the transmission image elements with textual medical findings and / or at least parts of medical reports, are filtered out. This means, that the transmission image elements are taken out of the routine image processing workflow and are further processed to generate a final medical report based on the images.

[0081] After the final report has been generated, the final report may be provided to a reporting system for review and / or approval by a radiologist of the requesting entity.

[0082] Preferably, the method further comprises a step of obtaining minimized patient information, in particular a pseudonymized patient identifier, from the transmission image data. Another step may be directed to obtaining, based on the minimized patient information, a patient identifier, from a patient database of the requesting entity. Another step may be directed to re-identifying the transmission image data based on the minimized patient information and the corresponding patient identifier, in particular by replacing the minimized patient information in the transmission image data with the patient identifier.

[0083] The patient database may be a storage device such as a cloud or local storage serving as an archive for the mapping of patient identifiers to pseudonymized patient information. In other words, the database may provide an association linking patient identifiers to corresponding pseudonymized patient information, in particular hash identifiers.

[0084] The minimized patient information included in the transmission image data may be replaced before the transmission image data is sent to the image archiving system or before it is processed to generate a final medical report.

[0085] In particular, the final medical report may be generated based on the obtained patient identifier, i.e. the patient identifier corresponding to the pseudonymized patient information included in the transmission image data. In other words, the pseudonymized patient information, which is included in the textual medical findings and / or in the medical report contained in the transmission image data, may be replaced with the actual patient identifier, wherein, at the requesting entity, a medical report may be drawn up based on the re-identified medical findings and / or medical report. Siemens Healthineers AG

[0086] 16

[0087] In one exemplary embodiment, the transmission image data may include at least one transmission image element with a DICOM structured report. The DICOM structured report may include information representing an HL7 or FHIR report. The HL7 or FHIR report may include pseudonymized patient information. In the course of generating the final report at the requesting entity, the pseudonymized patient information may be replaced with the actual patient information retrieved from the patient database, such that a HL7-conforming or FHIR- conforming report is the result.

[0088] In a preferred embodiment, the transmission image data includes a DICOM structured report, wherein an HL7 report describing the at least one medical finding is embedded in the DICOM structured report. As above-mentioned, the HL7 report is preferably generated using the pseudonymized patient identifier, which is provided with the medical image data.

[0089] The claimed approach uses the de -identification information included in the medical image data to successfully generate a report, in particular an HL7 report, which can be directly integrated into the reporting system of the requesting institution. The evidence images, generated at the evaluating entity, and the final, medical report, generated at the requesting entity based on the findings determined at the evaluating entity, are delivered to the reporting system with full patient information. Thus, the claimed concept removes the burden on the requesting entity of manually recreating a final report that fits to the reporting system of the requesting entity. Instead, the workflow at the requesting entity can be upheld and no modification is required, as the evidence images are available for review in the image archiving system and the final report is available in the reporting system. Therefore, the claimed approach simplifies and speeds up the overall remote reading workflow.

[0090] According to an aspect, a system including a requesting entity and an evaluating entity is provided, wherein the requesting entity and the evaluating entity each include at least one data processing apparatus comprising means for carrying out the steps of the method according to any of the claimed embodiments.

[0091] According to a further aspect, a data processing apparatus comprising means for carrying out the steps of the method according to any of the claimed embodiments is provided. Siemens Healthineers AG

[0092] 17

[0093] According to a further aspect, a computer program product is provided, wherein the computer program product comprises instructions or computer program elements, which, when the program is executed by a computer, cause the computer to carry out any of the above-mentioned embodiments.

[0094] The computer program product, such as a computer program means, may be embodied as a memory card, USB stick, CD-ROM, DVD or as a file which may be downloaded from a server in a network. For example, such a file may be provided by transferring the file comprising the computer program product from a wireless communication network.

[0095] According to a further aspect, a computer-readable storage medium is provided, comprising instructions or program elements, which, when readable or executed by a computer, cause the computer to carry out the steps of the method according to any of the above-mentioned embodiments.

[0096] The embodiments and features described with reference to the method of the present invention apply mutatis mutandis to the device and / or the system of the present invention.

[0097] Further possible implementations or alternative solutions of the invention also encompass combinations - that are not explicitly mentioned herein - of features described above or below with regard to the embodiments. The person skilled in the art may also add individual or isolated aspects and features to the most basic form of the invention.

[0098] Further embodiments, features and advantages of the present invention will become apparent from the subsequent description and dependent claims, taken in conjunction with the accompanying drawings, in which:

[0099] Fig.1 shows a schematical illustration of an embodiment of a system according to the invention!

[0100] Fig. 2 shows a schematical flow diagram of a method for providing a medical finding to a requesting entity! and

[0101] Fig. 3 shows a schematical flow diagram of a method for providing a final medical report. Siemens Healthineers AG

[0102] 18

[0103] In the Figures, like reference numerals designate like or functionally equivalent elements, unless otherwise indicated.

[0104] Figure 1 shows a schematical illustration of an embodiment of a system 10 for evaluating medical images. In one embodiment, the system 10 may be used for teleradiology. The system 10 includes a requesting entity 20. An evaluating entity 50 is located remote from the requesting entity 20. Apart from a network N connecting the evaluating entity 50 with the requesting entity 20, the entities do not share a common IT-infrastructure. The evaluating entity 50 and the requesting entity 20 may be at least temporarily connected to each other for data / information transfer and / or exchange by means of the network N. In an example, the network N may be realized as local area network (LAN), the internet, or a private network, such as an intranet or a wide area network (WAN). The network connection is preferably wireless, e.g., as wireless LAN (WLAN or Wi-Fi).

[0105] From a data-exchange perspective, the requesting entity 20 may be separated or remote from the evaluating entity 50. Regardless of the specific local arrangement of the requesting entity 20 and the evaluating entity 50, this means that each of the requesting entity 20 and the evaluating entity 50 represents a distinct information silo. Each information silo includes medical data, in particular medical image data, wherein the medical data in one information silo shall be secure against access from outside of the entity.

[0106] Optionally, the requesting entity 20 includes an imaging modality 22 for generating medical image data of patients. The generated medical image data may be stored in a PACS 30 of the requesting entity 20. Alternatively, the medical image data generated at the imaging modality 22 may be directly sent to an image gateway 24 of the requesting entity 20 for being forwarded to the evaluating entity 50.

[0107] The image gateway 24 of the requesting entity 20 obtains medical image data from the imaging modality 22 or from the PACS 30 of the requesting entity 20. The medical image data may include sensitive patient information, e.g. in the metadata of the medical image data, which must not be shared with other institutions than the requesting entity 20. Therefore, the medical image data is minimized in the image gateway 24. In other words, patient identifying information is removed from the medical image data. Siemens Healthineers AG

[0108] 19

[0109] The patient information may be replaced by a pseudonymized patient identifier, e.g. by a hash identifier. The mapping between the removed patient identifier and the inserted pseudonymized identifier may be stored in a patient identifier database 26 of the requesting entity 20.

[0110] The minimized medical image data may be uploaded to an image hub 54. In the depicted embodiment, the image hub 54 is in a cloud environment 52, wherein, both the cloud environment 52 and the image hub 54 are part of the evaluating entity 50. However, in other embodiments, the image hub 54 and / or the cloud environment 52 may be remote from the evaluating entity 50.

[0111] After receiving the medical image data, the image hub 54 distributes the medical image data to an appropriate evaluating institution 60.

[0112] In general, the evaluating entity 50 may comprise multiple evaluating institutions 60, wherein the different evaluating institutions 60 may be specialized on specific evaluation tasks, e.g. The evaluating institutions 60 may be part of the same evaluating entity 50. In other examples, the evaluating institutions 60 may be part of different evaluating entities 50, wherein the image hub 54 distributes data between multiple evaluating entities 50.

[0113] The distribution of the medical image data may be based on information, which is extracted from the metadata of the medical image data. In particular, the metadata can include a plurality of metadata attributes. The metadata attributes may describe for example what is depicted by the medical image, by which medical imaging system the medical image is acquired and / or which parameters are used for acquiring the medical image. In particular, a metadata attribute can comprise an information which part of the examination object is depicted in the medical image and / or in which orientation the part of the examination object is depicted and / or about the laterality of the part of the examination object and / or in which view the part of the examination object is imaged. Based on that information, the image data may be assigned to an evaluating institution 60, which is capable of evaluating the medical image data. For example, medical image data including a chest CT image may be assigned to an evaluating institution 60 specializing on evaluation of chest CT images. Siemens Healthineers AG

[0114] 20

[0115] The medical image data is received by an image gateway 62 of the evaluating institution 60. The image gateway 62 may forward the image data to a PACS 64 of the evaluating institution 60.

[0116] At least one medical finding may be determined based on the medical image data. The medical findings may be determined by an algorithm and / or or by an annotator 66, e.g., by a radiologist of the evaluating institution 60. Multiple algorithms for determining medical findings based on medical image data are available. According to exemplary embodiments, Al algorithms capable of image processing may be used for determining at least some of the determined medical findings.

[0117] Based on the determined medical findings, annotated medical images may be generated. These annotated medical images may also be called evidence images. Thus, evidence image data may include at least one, in particular a plurality of annotated medical images.

[0118] Further, a medical report may be generated based on the medical images and based on the determined findings. The report may include textual information, embedded images, audio transcription and / or video data, e.g., 4D data. In particular, the report may comprise a description of the medical findings. The report may comprise a summary of the medical findings.

[0119] Frequently, patient information is included in medical reports. However, as the medical image data, which is obtained at the evaluating entity 50 does not include patient information, the patient information is not available in the evaluating entity 50. Instead, the pseudonymized patient identifier, which is included in the medical image data may be used as a placeholder, when generating the medical report. In other words, the medical report is generated based on the pseudonymized identifier, wherein the pseudonymized identifier is included at those passages of the medical report, where normally a patient identifier would be included.

[0120] The evaluating institution 60 further includes an image bundler 68, which is capable of embedding the determined medical findings and / or the generated medical reports in the medical image data, which shall be sent back to the requesting entity 20, i.e. in the transmission image data. For this purpose, the metadata of the transmission image data may be used. In an embodiment of the Siemens Healthineers AG

[0121] 21 invention, the determined medical findings and / or the generated medical reports may be included in the metadata of the evidence images. In another embodiment, an additional transmission image element may be created, wherein the determined medical findings and / or the generated medical reports are included in the metadata of the additional transmission image element.

[0122] For including the determined medical findings and / or the generated medical reports in the metadata of an image, a metadata attribute of the image metadata may be used. The metadata attribute may be an existing attribute, whose attribute value is adjusted or, if the attribute value is empty, may be filled. Alternatively, an additional metadata attribute may be generated by the image bundler 68. The determined medical finding and / or the report may be written in the metadata attribute value of the metadata attribute.

[0123] The image metadata may include a metadata attribute, which comprises information about whether the image data includes medical findings and / or reporting information. The corresponding attribute value may be a binary value, which, depending on its value, indicates the presence of medical finings and / or report in the image metadata. The actual medical findings and / or report may be included in another metadata attribute. Alternatively or in addition, the attribute value may comprise a string value and / or an integer value and / or a float value and / or a free-text value, directly indicating the medical finding and / or report.

[0124] In an example, the textual medical findings and / or the reporting information may be included in the metadata of the image transmission data as DICOM structured report. This has the advantage that the information can be processed by the standard image processing system of the requesting entity 20.

[0125] Figure 2 shows a schematical flow diagram of a method for providing a medical finding to a requesting entity 20. The method disclosed in context with Figure 2 may be applied in a system 10 as described in context with Figure 1.

[0126] The method is preferably executed at an evaluating entity 50, in particular an evaluating entity 50 as described in context with Figure 1.

[0127] In a step Si medical image data is obtained from a requesting entity 20, wherein the requesting entity 20 is located remote from the evaluating entity 50. The Siemens Healthineers AG

[0128] 22 medical image data may be sent from the requesting entity 20 to the evaluating entity 50 for remote reading assistance. The medical image data may be uploaded over a secure communication channel, e.g., a https-channel.

[0129] In a step S2, at least one medical finding is determined based on the obtained medical image data. Annotations of the medical findings may be provided in the medical image data, such that evidence images are generated. Optionally, a report may be generated, e.g., based on the medical image data, the evidence images and / or the medical findings.

[0130] In a step S3, transmission image data is generated. In particular, the transmission image data may include an additional transmission image element. The additional transmission image element may be combined with the evidence image data, before sending the transmission image data back to the requesting entity 20.

[0131] In a step S4, at least one of the determined medical findings is embedded as text in the transmission image data. This means that the image data is used for transmitting the textual findings. Thus, apart from the image communication channel, there is no additional communication channel required.

[0132] In a step S5, the transmission image data is transmitted to the requesting entity 20. In consequence, also the at least one embedded medical finding and / or the at least one embedded medical report is transmitted to the requesting entity 20.

[0133] After embedding the findings and / or report in the transmission image data, in particular in the metadata of the transmission image data, the image data is transmitted to the requesting entity 20 via the image gateway 62 of the evaluating institution 60 and, optionally, via the image hub 54 of the evaluating entity 50.

[0134] When the transmission image data, is received at the requesting entity 20, a method for providing a final medical report may be executed, as schematically shown in Figure 3.

[0135] The transmission image data may be received, in particular downloaded, from the image hub 54. Siemens Healthineers AG

[0136] 23

[0137] If the transmission image data includes minimized patient information, the transmission image data may be re-identified. For this, a pseudonymized identifier included in the transmission image data may be extracted from the transmission image data. A local patient identifier database 26 of the requesting entity 20 may be searched in order to retrieve a matching patient identifier, in particular including protected health information (PHI). The retrieved patient identifier may be used for re-assigning the transmission image data to the patient for which the medical image data was originally captured.

[0138] For each of the elements of the transmission image data, in particular for every transmission image element, it may be determined by a data filter unit 28, whether the transmission image element includes reporting information, e.g., textual medical findings or parts of a medical report.

[0139] If it is determined that the transmission image element does not include any medical findings or reporting information, the image element may be directly sent to the PACS 30 of the requesting entity 20, where it is available for review by a radiologist 34 of the requesting entity.

[0140] If it is detected, that the transmission image element includes textual medical findings or reporting information, the transmission image element may be filtered by the data filter unit 28 of the requesting entity 20 (see also Fig. 1). In other words, the data filter unit 28 may take the transmission image element out of the “normal” image processing pipeline, and does not send the transmission image element to the PACS 30. Instead, relevant textual findings are extracted from the transmission image element.

[0141] Based on the extracted medical findings, and optionally based on the retrieved patient identifier, a medical report may be generated. For this, the medical findings may be textually described and / or the pseudonymized patient information included in the transmission image element may be replaced by the actual patient identifier.

[0142] In some embodiments, the report may be generated by combining the findings and report elements, which have been extracted from multiple transmission image elements. Siemens Healthineers AG

[0143] 24

[0144] Once the report is finished, the report may be sent to a reporting system 32 of the requesting entity 20, where it is available for review and adjustment by a reviewing radiologist 34 of the requesting entity 20.

[0145] The pre-generated report may be reviewed by the radiologist 34 and may be signed off, e.g., if the radiologist is satisfied with the report.

[0146] Thus, an efficient workflow for generating a report based on transmission image data is provided. In particular, the workflow can be executed automatically and requires only a minimal degree of human interaction.

[0147] Some of the steps disclosed in Figures 2 and 3 may be optional. Further, the order, in which the steps are executed may be changed depending on the use case.

[0148] The various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of this disclosure or the claims. Embodiments implemented in computer software may be implemented in software, firmware, middleware, microcode, hardware description languages, or any combination thereof. A code segment or machine-executable instructions may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc. Siemens Healthineers AG

[0149] 25

[0150] The actual software code or specialized control hardware used to implement these systems and methods is not limiting of the claimed features or this disclosure. Thus, the operation and behavior of the systems and methods were described without reference to the specific software code being understood that software and control hardware can be designed to implement the systems and methods based on the description herein.

[0151] When implemented in software, the functions may be stored as one or more instructions or code on a non-transitory computer-readable or processor-readable storage medium. The steps of a method or algorithm disclosed herein may be embodied in a processor-executable software module, which may reside on a computer-readable or processor-readable storage medium. A non-transitory computer-readable or processor-readable media includes both computer storage media and tangible storage media that facilitate transfer of a computer program from one place to another. A non-transitory processor-readable storage media may be any available media that may be accessed by a computer. By way of example, and not limitation, such non-transitory processor-readable media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other tangible storage medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer or processor. Disk and disc, as used herein, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media. Additionally, the operations of a method or algorithm may reside as one or any combination or set of codes and / or instructions on a non-transitory processor-readable medium and / or computer- readable medium, which may be incorporated into a computer program product.

[0152] The preceding description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the embodiments described herein and variations thereof. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the principles defined herein may be applied to other embodiments without departing from the spirit or scope of the subject matter disclosed herein. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope Siemens Healthineers AG

[0153] 26 consistent with the following claims and the principles and novel features disclosed herein.

[0154] While various aspects and embodiments have been disclosed, other aspects and embodiments are contemplated. The various aspects and embodiments disclosed are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

Claims

Siemens Healthineers AG27CLAIMS1. Computer-implemented method for providing a medical finding to a requesting entity, the method comprising the steps: obtaining, at an evaluating entity, medical image data from a requesting entity, the requesting entity being remote from the evaluating entity, determining, at the evaluating entity, at least one medical finding based on the obtained medical image data; generating transmission image data; embedding at least one of the determined medical findings as text in the transmission image data; providing the at least one medical finding to the requesting entity by means of transmitting the transmission image data to the requesting entity.

2. Method according to claim 1, wherein the method further comprises the step ofi generating, at the evaluating entity, a medical report based on the at least one medical finding; and, wherein embedding at least one of the determined medical findings as text in the transmission image data includes embedding the generated medical report in the transmission image data.

3. Method according to claim 2, wherein the medical image data includes minimized patient information, in particular a pseudonymized patient identifier, wherein the method further comprises the step: obtaining minimized patient information, in particular a pseudonymized patient identifier, from the medical image data; and wherein the report is generated using the minimized patient information, in particular the pseudonymized patient identifier.

4. Method according to claim 3, wherein the pseudonymized patient information is a hash identifier.

5. Method according to any one of the preceding claims, wherein the method further comprises the step: distributing, by means of an image hub and based on information extracted from the medical image data, in particular from metadata of the medical image data, the obtained medical image data to an evaluating institution, an evaluatingSiemens Healthineers AG28 annotator and / or an evaluating algorithm, wherein the at least one medical finding is determined by the evaluating institution, the evaluating annotator and / or the evaluating algorithm.

6. Method according to any one of the preceding claims, wherein the transmission image data includes evidence image data, wherein the evidence image data includes at least one annotated image with an annotation of a determined medical finding, wherein, optionally, the transmission image data includes an additional transmission image element, the at least one textual medical finding and / or the medical report being embedded in the additional transmission image element.

7. Method according to any one of the preceding claims, wherein the determined medical findings and / or the generated medical report are embedded in the metadata of the transmission image data.

8. Method according to any one of the preceding claims, wherein the transmission image data includes a metadata attribute indicating that the transmission image data comprises at least one textual medical finding, parts of a medical report and / or a medical report.

9. Method according to any one of the preceding claims, wherein, apart from medical image data, no other data is exchanged between the requesting entity and the evaluating entity.

10. Computer-implemented method for providing a final medical report, the method comprising the steps: obtaining at a requesting entity, transmission image data from an evaluating entity, the requesting entity being remote from the evaluating entity, and the transmission image data including a plurality of transmission image elements, wherein at least one transmission image element includes an embedded textual medical finding! determining for each of the transmission image elements, whether the transmission image element includes at least one embedded textual medical finding! if the transmission image element does not include at least one embedded textual medical finding:Siemens Healthineers AG29 providing the transmission image element to an image archiving system of the requesting entity! if the transmission image element includes at least one embedded textual medical finding: extracting the at least one embedded textual medical finding from the transmission image element! generating a final medical report based on the at least one textual finding! providing the final medical report to a reporting system of the requesting entity.

11. Method according to claim 10, wherein the method further comprises: obtaining minimized patient information, in particular a pseudonymized patient identifier, from the transmission image data! obtaining, based on the minimized patient information, a patient identifier, from a patient database of the requesting entity! re-identifying the transmission image data based on the minimized patient information and the corresponding patient identifier, in particular by replacing the minimized patient information in the transmission image data with the patient identifier.

12. Method according to claim 11, wherein the final medical report is further generated based on the patient identifier.

13. Method according to any one of the preceding claims, wherein the transmission image data includes a DICOM structured report, wherein an HL7 report or a FHIR report textually describing the at least one medical finding is embedded in the DICOM structured report.

14. A computer program product comprising program elements, directly loadable into a memory unit of a computing unit, and inducing the computing system to execute the method according to any one of the preceding claims, when the program elements are executed by the computing unit.

15. A computer-readable storage medium comprising program elements which are readable and executable by a computing unit, to execute the method of one of the claims 1 to 13, when the program elements are executed by the computing unit.

Images