Medical data processing device and method for processing medical data

Abstract

The invention relates to the processing of medical data, in particular image data from medical images and associated medical reports, for the automated generation of a draft report for a newly acquired medical image. For this purpose, image elements that correspond to documentation elements of a previous medical image are assigned to image elements in the newly acquired medical image. Based on this assignment, the draft medical report for the newly acquired medical image is automatically generated.

Description

[0001] The present invention relates to a medical data processing device, a method for processing medical data, and a computer program product for carrying out the processing of medical data.

[0002] Regardless of the grammatical use of a term, individuals with male, female, or other gender identities are included in the term.

[0003] Although the present invention is mainly described in connection with the documentation of computed tomography images, it is not limited to such images. Furthermore, the present invention is also applicable to almost any other type of medical image.

[0004] Imaging techniques have become very important in medical examinations. Such imaging techniques can include, for example, ultrasound images, radiological images, especially images from a computed tomography (CT) or magnetic resonance imaging (MRI) scanner.

[0005] After one or more images of a patient's area of ​​examination have been taken, the images are analyzed and documented by a specialist, such as a radiologist. For this purpose, the specialist prepares a medical report for each image, documenting their observations. If another image of the same patient, particularly of the same area, is taken at a later time, the specialist prepares a completely or partially new report documenting their observations for that additional image.

[0006] Creating medical reports for each medical image is relatively time-consuming. Furthermore, the specialist must ensure that all relevant observations related to an image are included in the corresponding medical report. In many cases, the specialist may also refer to other images, such as previously taken images of the same area of ​​the patient's body. All of this makes creating a medical report a very complex and time-consuming process, requiring a high degree of responsibility and prone to errors. It is essential to ensure that no information is lost.

[0007] Therefore, it is necessary to process medical data in the context of medical images that enables at least partially automated generation of medical reports for medical images. Specifically, a device and method for processing medical data are required, providing automated support in generating medical reports, up to and including the creation of a complete draft for a new medical report based on a medical image.

[0008] The present invention provides a medical data processing device, a method for processing medical data, and a computer program product with the features of the independent claims. Further advantageous embodiments are the subject of the dependent claims.

[0009] According to a first aspect, a method for processing a medical image is created. The medical data processing device comprises a first storage device, a second storage device, an input device, an allocation device, and a processing device. The first storage device is suitable for storing image data for a plurality of medical images. Each medical image can contain an associated timestamp. The second storage device is suitable for storing at least one medical report, in particular a medical report for a corresponding medical image stored in the first storage device. Each medical report can contain at least one documentation element for an image element of the corresponding medical image.The input device is capable of receiving a selection for a first medical image and / or a first medical report for the first medical image. The input device is further capable of receiving a selection for a second medical image. The mapping device is capable of calculating a mapping between the first medical image and the second medical image. The processing device is capable of determining at least one image element in the second medical image for which a documentation element is included in the first medical report. This determination is performed using the mapping between the first medical image and the second medical image. The processing device includes a documentation module.The documentation module of the processing device is capable of creating a documentation element, specifically a documentation element for a draft of a second medical report for the second medical image. A corresponding documentation element for the second medical report is automatically created using the corresponding documentation element for the corresponding image element in the first medical report. More precisely, two different instances of a documentation element can exist: a first documentation element in the first medical report and a second documentation element in the second medical report.

[0010] According to a second aspect, a device for processing a medical image is provided. The method comprises storing image data for a plurality of medical images. Each medical image may include an associated timestamp. The method further comprises storing at least one medical report for a corresponding stored medical image. The medical report includes at least one documentation element for each image element in the corresponding medical image. The method further comprises receiving a selection for a first medical image and / or a first medical report for the first medical image, and receiving a selection for a second medical image. The method further comprises calculating a mapping between the first medical image and the second medical image.Furthermore, the procedure includes identifying an image element in the second medical image for which a documentation element exists in the first medical report. The position, size, and / or shape of the image element in the second image can be determined based on the mapping between the first and second medical images. The procedure also includes creating a draft of a second medical report for the second medical image. A documentation element for the second medical report is automatically created using the corresponding documentation element for the relevant image element in the first medical report.

[0011] According to a further aspect, a computer program product is provided. The computer program product comprises a computer program that, according to the first aspect, can be loaded into a memory unit of a control device and that contains program code sections which serve to enable the control device to execute the procedure for processing the medical image data according to the second aspect when the computer program is executed in the control device.

[0012] A medical image in the context of this application can be any type of image captured for medical purposes. For example, the medical image may relate to a specific part or area of ​​a human body or to a (e.g., artificial) structure, such as an implant or stent, etc. An image may, for instance, be a scan of a specific organ or area with a lesion or disease. A medical image may be a two-dimensional (or three-dimensional) representation of a specific body part or anatomical component. The representation may be, for example, in grayscale or in color. For this purpose, the representation of the image, i.e., the individual pixels, can be determined from the corresponding image data.

[0013] Although the present invention is described in connection with medical images and the medical diagnosis of a human being, the general concept of the present invention is also applicable to similar applications, for example in the context of the medical diagnosis of animals or the like.

[0014] Image data relating to a medical image can include data specifying the individual pixels of the image, as well as other image-related data. The pixel data can be provided directly, for example, by supplying grayscale or color values ​​for each pixel. Any suitable standardized or proprietary image format can be used for this purpose. Alternatively, it may be possible to provide any type of raw or intermediate data from which the pixel values ​​for the medical image can be derived.

[0015] The medical image can be represented in a data set. The data set can include image data and / or other data. The medical image can, for example, be formatted in DICOM format and include a header section.

[0016] The image data can relate to a two-dimensional or three-dimensional representation of a depicted structure (anatomical or medical structure). If the image data relates to a three-dimensional representation, such as a computed tomography scan, it may be possible to calculate a two-dimensional image, e.g., a cross-section through the three-dimensional model.

[0017] The additional data of the image data can specify any type of supplementary information relating to the medical image, such as metadata. This additional data can be included in the header section. For example, the additional data might relate to technical aspects of the medical image or the process of capturing the medical image. The additional data can specify a timestamp, specifically the time at which the associated medical image was captured. The additional data can also specify the device used to capture the medical image and / or other information, such as the configuration or settings of the device used at the time of image capture.The additional data may also include information specifying the person / patient from whom the image was taken, a desired purpose for taking the image, etc. However, it can be assumed that the additional image data may also include other suitable information, in particular technical and / or administrative information relating to the medical image.

[0018] In addition to the image data, including the medical image data and / or other data (auxiliary data), a medical report can be generated and provided in conjunction with the associated medical image. Unlike the additional or auxiliary data, which relates to the image data, the associated medical report provides medical details related to the corresponding medical image.

[0019] The medical report typically refers to at least one medical image. Thus, a first report refers to a first medical image, and a second report refers to a second medical image. The first and second medical images can be acquired using the same or different modalities. They can also have different timestamps. Furthermore, it is possible for the first and second medical images to be acquired within the same timeframe but with different settings (e.g., different resolution, perspective, field of view, etc.).

[0020] The medical report may include one or more documentation elements. A documentation element can therefore be a part or section of the medical report. A documentation element can be provided in textual and / or verbal form. A documentation element typically relates to the corresponding medical image or a part or feature therein, particularly a visual element within the medical image. For example, a documentation element might be a text passage that specifies the tumor visible in the corresponding medical image, such as its cells, size, structure, etc.

[0021] Each documentation element can refer to a specific feature of the associated medical image. For example, a documentation element can characterize a specific property of a feature within the medical image. A documentation element can, for instance, specify the size, volume, shape, position, and / or intensity of any other feature of an element depicted in the associated medical image. In particular, if multiple corresponding medical images were acquired at different times, a documentation element for a specific feature in the medical report can also indicate variations of that feature with respect to another medical image, such as the evolution of a tumor over time in different medical images acquired at different times.Alternatively or additionally, several corresponding medical images may relate to the same time frame and / or the same anatomical structure, but originate from different modalities.

[0022] A feature (e.g., a lesion such as a tumor, calcification, or segmented organ) in a medical image can be represented by an image element within the image. Such an image element could be, for example, an image area manually selected by the user. Additionally or alternatively, it is also possible for an image element to be automatically selected for a specific feature. For instance, a suitable device can analyze the image data to identify one or more predefined properties. It is possible, for example, to automatically identify an image area related to a specific organ, lesion, tumor, etc. This can be achieved, for example, by using automatic image analysis and / or segmentation algorithms. The approach to identifying such an image area can be based on a suitable scheme.For example, an analysis can be performed using artificial intelligence, neural networks, or similar technologies. However, it may also be possible to perform such an analysis by evaluating the pixel values ​​in the medical image to identify a boundary, area, or similar feature. Furthermore, after the automatic determination of an image area has been carried out, a user may be able to manually adjust or fine-tune the result of the automatic determination process.

[0023] If a new (here also: second) medical report is to be prepared for a new / additional (here also: second) medical condition, this new medical report should preferably include documentation elements that correspond to the respective documentation elements in a previous medical report relating to a previous medical condition. It should therefore be noted that the term "documentation element" in this application is to be interpreted as encompassing two distinct cases or occurrences: a first documentation element in the first medical report and a second documentation element in the second medical report. It should also be noted that "documentation element" is to be understood as "at least one documentation element".In other words, when a documentation element is mentioned, it can generally refer to one or more documentation elements (whether it is a first or a second one).

[0024] Therefore, the person creating such a new medical report must consider the previous medical image and its associated report, identify documentation elements in the previous medical reports and their corresponding features in the previous medical image, and determine the corresponding features in the new medical image. Then, the person and / or an electronic module must create a corresponding new documentation element in the new medical report. To do this, the person and / or electronic module must analyze the relevant feature in the new medical image and compare it to the corresponding feature and / or documentation element in the previous medical report. Based on this analysis, the person and / or electronic module can create the corresponding element in the new medical report.Normally, the first medical image (and the corresponding first medical report) has a timestamp that is older than the timestamp of the second medical image, since the second medical image is typically captured later. However, other use cases are possible.

[0025] For example, it is also possible to create a report for an earlier image based on a new image with a new report. "New" in this context means an image with a timestamp that is more recent than the timestamp of an earlier image (which was taken earlier and therefore has an older timestamp).

[0026] The documentation element of a first medical report may be the same as, similar to, or different from the corresponding documentation element of a second medical report. For example, the first documentation element in the first medical report may refer to the type of lesion or structure visible in the first image, and the second documentation element in the second medical report may be the same as the first, since it still refers to the same type of lesion. However, if the first documentation element in the first medical report refers to the size of a lesion, the second documentation element may differ from the first, as the size may have changed, for example, if a tumor has grown larger.

[0027] With conventional approaches, a user, such as a specialist physician, must perform the tasks described above manually. On the one hand, manually performing all these tasks is very labor-intensive. On the other hand, the user must ensure that when creating a new medical report for a new medical image, they actually include all documentation elements from a previous medical report.

[0028] In view of these objectives, one idea of ​​the present invention is to provide technical means to assist the user in creating a new medical report and to provide an approach for at least automated creation of a report draft for a new medical image.

[0029] For this purpose, image data for multiple medical images and medical reports relating to at least some of the medical images can be provided. The image data can be stored, for example, on a primary storage device, and the medical reports can be stored, for example, on a secondary storage device. Both the primary and secondary storage devices can be suitable storage devices such as local storage devices, e.g., hard drives, solid-state drives, or network-attached storage (NAS). Alternatively, it may also be possible to store the image data and / or the medical reports in storage devices of a cloud service or similar.Although the terms "first" and "second" are used to refer to the storage devices for the image data and the medical report, it is also possible to store the image data and the medical reports on the same shared storage device. In this case, the image data and the medical report can be stored in separate files, separate folders, or similar locations. In any case, a clear and unambiguous relationship between the image data and the associated medical report can be specified. For example, the image data can include a reference to the associated medical report, and / or a medical report can include a reference to the associated image data. However, it is understood that other suitable schemes can also be used.

[0030] To perform assisted or automated creation of a new medical report, a user can specify an initial image and / or medical report. The user can also specify a second medical image for which assisted or automated report creation is to be performed. The selection of the appropriate images or reports can be made, for example, using a suitable input device. Such an input device can be any suitable device capable of receiving the required input from the user. In particular, a graphical user interface or similar can be used. The user can, for example, enter an identifier, such as a specific name, to select a medical report or medical image.In particular, the input device may provide multiple options for available medical images and reports, allowing the user to select a desired image or report. This can be done, for example, using a touchscreen, a computer mouse, or other suitable methods. Furthermore, it is also possible to select images or reports via voice commands or similar means.

[0031] After selecting the second medical image for which a medical report is to be created, and selecting the first medical image and the first medical report to serve as the basis for creating the new medical report, a mapping is performed between the first medical image and the second medical image.

[0032] The mapping establishes a relationship between the first and second medical images. For example, the mapping can specify a transformation between positions in the first and second medical images. One possibility of such a mapping is to establish a relationship between pixels in a coordinate system of the first medical image and pixels in a coordinate system of the second medical image. The mapping can be applied to the medical images themselves, thus mapping a first medical image to a second. Alternatively or additionally, the mapping can be applied to image elements within the respective images, thus mapping a first image element to a second image element. Based on such mappings, it is possible to represent or illustrate the development of the body structure under investigation over time, such as...Developmental stages of a tumor or a decrease in an inflammatory response.

[0033] Furthermore, it may be possible to map image elements in the first medical image to corresponding image elements in the second medical image. In particular, such a mapping can be performed for image elements that relate to documentation elements in the first medical report. To perform such a mapping, any suitable existing or future approach for mapping the first and second medical images can be applied. For example, such a mapping can be performed using a mapping device. A mapping can also be based on a segmentation algorithm.

[0034] In this way, it is possible to identify image elements in the second medical image, particularly those for which a corresponding documentation element exists in the first medical report. For example, based on the mapping, for each image element in the first medical image that relates to a corresponding documentation element in the first medical report, a corresponding image element in the second medical image can be determined. This determination of the associated image elements in the second medical image can be carried out, for example, by a suitable processing device.

[0035] Once image elements in the second medical image that correspond to the relevant image elements in the first medical image have been identified, a draft for a new medical report for the second medical image can be created. For this purpose, a corresponding documentation element can be created in the new medical report for each documentation element in the first medical report. For each such "new" or "second" documentation element in the "new" or "second" medical report ("second" here refers to the medical report to be created), a reference to the corresponding image in the second medical image can be created in the new medical report.Furthermore, it is conceivable to reference the documentation element in the first medical report and / or the corresponding image element in the first medical image within the documentation element of the new medical report. It may also be possible to automatically perform an analysis of the corresponding image element in the second medical image to determine one or more predefined properties of this image element. For example, it may be possible to automatically determine size, volume, length, shape, and / or another property. Such automatically determined properties of the image element in the second medical image can also be added to the expected documentation element in the new medical report. If required, further operations can also be performed to automate the drafting of the new medical report.All these operations can be performed, for example, using the processing device.

[0036] The allocation device and / or the processing device can be implemented, for example, by any suitable computer device. For instance, the allocation device and / or the processing device can include a processor and associated memory in which program instructions are stored to perform the desired operations. The first and second memory devices, the input device, the allocation device, and the processing device can communicate with each other to perform the necessary data exchange.

[0037] In one possible embodiment, the device further comprises an output device. The output device may be capable of outputting the first image, the corresponding first report for the first image, and / or the second image and the draft for the second report. The input device may be capable of receiving user input for a selection of an image element in the output images and / or a documentation element in the output reports. The processing device may comprise a processing module capable of identifying an image element corresponding to the selection in each of the first and second images and of identifying a documentation element corresponding to one of the first and second reports. The output device may, for example, be a display such as an OLED or TFT screen.However, any other suitable device can also be considered for displaying the required information, in particular the medical images and the medical report. Specifically, the output device can be part of a system for providing a graphical user interface (GUI). For example, the output device can include a touchscreen for displaying information and receiving user input. However, it is also conceivable to receive user input via a keyboard, a computer mouse, voice commands, etc.

[0038] In one possible embodiment, the input device is capable of receiving a setting for mapping an image element in the first image to an image element in the second image. In such a configuration, the processing device can be capable of adjusting the mapping between the image elements in the first and second images based on the received setting. Any suitable approach can be used to adjust the mapping between the image elements in the first and second medical images. For example, a GUI can receive a user command to select an image element and adjust / set the image element in one of the images. For this purpose, the user can adjust the position, size, shape, and / or other characteristic of the respective image element.In this way, the user can, for example, fine-tune the automated mapping between the image elements in the first and second medical images.

[0039] According to one possible embodiment, the device further comprises a third storage device. This third storage device can be configured to store the mapping between the first and second images and / or a mapping between a pixel in the first image and a corresponding pixel in the second image. The stored mapping, particularly the mapping relationship after further user adjustments, can be used to improve the mapping approach between medical images. For example, the information from the adjusted mapping can be used as training data for a neural network, especially for a neural network that applies the mapping. However, it can be assumed that other approaches are also possible.

[0040] According to one embodiment, the first, second and / or third memory can be implemented in a common memory or platform, or provided separately.

[0041] According to one possible embodiment, the device includes an analysis module. The analysis module can be configured to determine at least one property of the image element in the second image. In this configuration, the documentation module can be configured to generate the draft of the second report using the determined at least one property of the image element. Such a property of an image element can be, for example, a length, size, volume, shape, density, and / or another suitable feature for characterizing the associated image element. For example, the analysis can be performed based on a suitable image processing algorithm. The algorithm can, for example, identify a boundary of the feature in the respective image element and determine one or more characteristic features of the image element within this boundary, in particular by means of a segmentation algorithm.However, another approach may also be possible, in particular an approach that uses artificial intelligence, neural networks or the like.

[0042] In one possible embodiment, the analysis module can be configured to determine at least one property to be determined using a semantic analysis of the corresponding documentation element in the first report. For example, the semantic analysis can identify a keyword in the associated documentation element and / or in the image metadata. Based on this identified keyword, the analysis can automatically determine the feature associated with that keyword. For instance, if the semantic analysis identifies the term "length" in the associated documentation element of the first medical report, the analysis performed by the module can automatically determine the length of the corresponding image element in the second medical image.The specified length can be automatically inserted into the respective documentation element of the newly created second medical report. However, it goes without saying that this approach can also be applied to any other keyword and any other type of semantic analysis.

[0043] According to one possible embodiment, the device includes an evaluation device. The evaluation device can be capable of identifying a corresponding image element in a plurality of medical images and a corresponding property of each identified image element. Furthermore, the evaluation device can be capable of determining a progression of the image element's property as a function over time, i.e., over the associated timestamps. In this way, a progression of the feature relating to an image element can be automatically determined in multiple medical images over the desired period and, if necessary, visualized. Moreover, it is also possible to use this progression, or a description relating to such a progression, in an associated documentation element in the newly generated medical report.

[0044] In particular, it is possible to automatically perform the analysis of this progression in relation to other medical images, especially other medical images from the past, without the user having to manually open the relevant medical images.

[0045] According to one possible embodiment, the documentation module is capable of creating a corresponding documentation element in the second report for each image element in the first image and / or for each documentation element in the first report. The automatic creation of the respective documentation elements in the (newly created) second medical report ensures that the new report includes a documentation element for each image element / documentation element of the first report. In this way, the completeness of the new medical report can be guaranteed.

[0046] According to one possible embodiment, the device includes a first data interface. This first data interface is suitable for connection to a medical diagnostic imaging system. Furthermore, the first data interface can be designed to receive medical image data. It can also be designed to store the received data on the first storage device. In this way, the device can be easily integrated into a medical system, particularly one dealing with medical images, for processing medical data and, in particular, for the automated generation of medical reports.

[0047] According to one possible embodiment, the medical images may include ultrasound images or radiological images, in particular images from a computed tomography (CT) scan, a positron emission tomography (PET) scan, and / or a magnetic resonance imaging (MRI) scan. However, it is understood that any other type of medical image, in particular medical images for diagnostic purposes, is also conceivable.

[0048] According to one possible embodiment, the device includes a second data interface. This second data interface can be configured to provide data from a medical image and a corresponding medical report to an external processing device. Additionally or alternatively, the second data interface can be configured to receive information for an image element of a medical image or a documentation element of a medical report. Furthermore, the second data interface can be configured to automatically add the received information to the corresponding image and / or report. In this way, further data exchange with additional devices, particularly additional devices of a medical system that handle images, can be implemented.The automated creation of the medical report can be further improved by receiving additional information and automatically adding this information to a documentation element of the medical report.

[0049] According to one possible embodiment, the device further comprises a matching device. The matching device is designed to identify at least one documentation element in a first medical report. The identification of at least one documentation element can be achieved by analyzing the first medical report. In particular, a semantic analysis can be performed. The matching device can also be designed to determine a corresponding image element in the first medical image for each of the identified documentation elements. In this way, it is possible to use medical images and corresponding medical reports even if there are no links or relationships between the first medical image and the first medical report in the original information.Furthermore, the matching device can perform an automated process for referencing the initial medical image and the initial medical report. Therefore, it is possible to identify relevant image elements in the initial medical image to which the initial report refers. This allows the use of additional data sources, particularly further medical images and medical reports not originally generated by the medical data processing device, thus eliminating any correlation between medical images and reports in the original data sources.

[0050] According to one possible embodiment, the image elements are positioned at a corresponding (embedded) location within the first or second medical image. Therefore, the image elements can be limited to a predefined area within the medical image. Additionally or alternatively, the image elements can include an indication of a position, organ, lesion, and / or size.

[0051] According to one possible embodiment, the mapping between the first and second medical images involves mapping two-dimensional or three-dimensional coordinates. In this way, the mapping operation can provide a transformation scheme based on which pixels, image areas, or the like in the first medical image can be easily mapped to corresponding pixels, image areas, or the like in the second medical image. Thus, a fast and simple scheme for defining the relationship between medical images or image elements can be implemented. It may be possible to save a specific mapping between medical images and reuse this mapping at a later time. This saves computing resources.

[0052] The following detailed description of the figures explains non-limiting embodiments with their features and further advantages, based on the drawings. The figures include: Fig. Figure 1 shows a schematic diagram of a graphical interface provided by a device for processing medical data, according to one embodiment; Fig. Figure 2 shows a schematic block diagram representing a device for processing a medical image, according to one embodiment; Fig. Figure 3 shows a block diagram illustrating a method for processing a medical image according to one embodiment.

[0053] Fig. Figure 1 shows a schematic diagram of a graphical interface 100, in which some main elements related to the processing of medical data for automated or assisted creation of a medical report are shown.

[0054] For example, a new (second) medical report 220 can be created for a related and / or corresponding (second) medical image 210. For this purpose, the creation of a draft for the new (second) report 220 can be carried out automatically based on another (first and / or, for example, previous) medical report 120 and a first medical image 110 in relation to this first medical report 120. Hereinafter, the other medical image is referred to as the first medical image 110, and the related further or other medical report is referred to as the first medical report 120. Accordingly, the new medical report to be created is referred to as the second medical report 220, and the related image for this second medical report is referred to as the second medical image 210.

[0055] As in Fig. As shown in Figure 1, the first medical report 120 can include one or more documentation elements 121, 122. Each documentation element can refer to a corresponding image element 111, 112 in the first medical image 110. An image element 111, 112 can, for example, specify a particular position, area, and / or volume in the medical image 110. For example, the image element 111, 112 can be specified by a user when creating the first medical report 120. An image element 111, 112 can specify any type of feature or characteristic in the associated medical image 110. For example, an image element can refer to an organ, a lesion, a tumor, or another property in the medical image 110.The image element 111, 112 can be represented by a point, a line, an arrow, a circle, a polygon or another suitable graphic or non-graphic element.

[0056] The first medical report 120, which refers to the first medical image 110, may include one or more documentation elements 121, 122. Each documentation element 121, 122 may, for example, refer to a corresponding image element 111, 112 in the associated medical image 110. For example, a documentation element may specify that "image element xx shows a tumor measuring 10 mm." Accordingly, such a documentation element 121, 122 may include at least a reference to an image element 111, 112 and a description relating to that image element 111, 112.

[0057] On the right side of Fig. Figure 1 shows a second medical image 210 and a second medical report 220. For example, the second medical image 210 may be a newly acquired medical image for which a new (or second) medical report 220 is to be created. For this purpose, an automated or at least assisted approach to creating such a new second medical report 220 may be used.

[0058] To perform such a (semi-)automated generation of a draft for a second medical report 220, a mapping is carried out between the first medical image 100 and the second medical image 210. This mapping can, for example, include a transformation scheme between pixels or positions in the first medical image 110 and the corresponding pixels or positions in the second medical image 210. Based on this mapping, it is possible to automatically identify the image elements 211 and 212 in the second medical image 210 that correspond to the image elements 111 and 112 in the first medical image.

[0059] In addition to identifying image elements 211 and 212 in the second medical image 210 based on the mapping scheme, further operations can be performed to identify characteristic regions or areas in the second medical image 210 with features or characteristics that correspond to the features or characteristics of the associated image element 111 or 112 in the first medical image 110. This will further improve the identification of the corresponding image elements 211 and 212 in the second medical image 210.

[0060] Once image elements 211 and 212 have been identified in the second medical image 210, an automatic draft of the second medical report 220 can be generated. For this purpose, the corresponding documentation elements 121 and 122 in the first medical report 120 can be identified, and corresponding documentation elements 221 and 222 can be created in the draft of the second medical report 220.

[0061] In this context, documentation elements 221 and 222 in the second medical report 220 can be adapted based on the properties of the corresponding image elements 211 and 212 in the second medical image. For example, if documentation element 121 or 122 in the first medical report 120 describes the size of an image element 111 or 112 in the first medical image 110, the size of the corresponding image element 211 or 212 in the second medical image can be automatically determined, and the result of this determination can be inserted into the corresponding documentation element 221 or 222 in the second medical report 220. In this way, the automated creation of the new, second medical report 220 for the second medical image 210 can be achieved.

[0062] The user can read and evaluate the draft of the second medical report 220 and / or adjust the automatically generated second medical report 220 if necessary. For example, the user can directly edit the draft of the second medical report 220. This can be done using any suitable means, such as a keyboard, a computer mouse, or speech recognition.

[0063] Furthermore, the user can edit or adjust the automatic selection of image elements 211 and 212 in the second medical image 210. After adjusting an image element 211 or 212 in the second medical image 210, the associated properties of the image element 211 or 212 can be determined automatically, and the corresponding documentation element 221 or 222 in the second medical report 220 can also be adjusted accordingly. For example, if a user adjusts the shape or dimensions of an image element 211 or 212 in the second medical image 211, a new size for this image element 211 or 212 can be determined, and the newly determined size can be automatically inserted into the associated documentation element 221 or 222.

[0064] A user can also add further documentation elements 221, 222 to the second medical report 220. For example, the user can identify an existing or new image element 211, 212 in the second medical image 210. In response, a new documentation element 221, 222 can be created in the second medical report 220. Furthermore, the corresponding position or area in the first medical image 110 can also be determined and displayed. This determination of an image element in the first medical image that corresponds to the newly selected image element 211, 212 in the second medical image 210 can also be performed, for example, based on the specific mapping between the first medical image 110 and the second medical image 210.

[0065] The following describes a schematic concept of a device 1 for processing medical data, in particular for processing medical data to create medical reports.

[0066] Fig. Figure 2 shows a schematic block diagram of a device 1 for processing medical data according to one embodiment. The device 1 can comprise a first storage device 11 and a second storage device 12. Optionally, the device 1 can include further storage devices, for example, a third storage device 13. Although the storage devices are shown as separate storage devices 11, 12, 13, it is also possible that the individual storage devices 11, 12, 13 are at least partially implemented by individual storage areas of a shared storage device 10.

[0067] The first storage device 11 can store image data, in particular image data from multiple medical images. A medical image in this context can be any type of image that provides a representation for medical / diagnostic purposes. The medical image can be, for example, an ultrasound image or a radiological image. In particular, the radiological image can be an image obtained from a computed tomography (CT) scanner and / or a magnetic resonance imaging (MRI) scanner. However, other suitable images for medical purposes, such as positron emission tomography (PET) or ultrasound, are also conceivable.

[0068] The image data for the medical images can be obtained, for example, from a medical diagnostic imaging system. Such a medical diagnostic imaging system can include, for example, the aforementioned computed tomography scanner, positron emission tomography, magnetic resonance imaging, or ultrasound. Other suitable medical imaging systems are also possible. For data exchange between the medical imaging system and the device 1 for processing the medical data, the device 1 can be connected to the medical imaging system via a first data interface 71. In particular, the first data interface 71 can receive image data from the medical imaging system and store the received image data in the first storage device 11.

[0069] For this purpose, a suitable communication scheme, such as Ethernet or the like, is conceivable.

[0070] The second storage device 12 can store data from medical reports; medical reports for corresponding medical images are stored in particular in the first storage device 11. The medical reports can be, for example, medical reports such as those already mentioned above in connection with Fig. 1 described. Accordingly, each medical report comprises one or more documentation elements that relate to corresponding image elements of an associated medical image stored in the first storage device 11.

[0071] A second data interface 72 can further be provided for data exchange between the device 1 for processing the medical data and any type of additional external processing device. Such an additional external device could be, for example, a remote storage device, a cloud service, or the like. Furthermore, an additional external device could be another device of a medical infrastructure, e.g., a device that provides further medical information, such as blood values, a pathological diagnosis, etc.

[0072] Furthermore, the additional device of the medical infrastructure can be a device that provides medical information, for example, a medical report, or a device that requests medical information, for example, a medical report stored in the second storage device 12, and / or a medical image stored in the first storage device 11. Communication between the additional external processing device and the device 1 for processing the medical data via the second data interface 72 can also be carried out based on a suitable communication scheme, for example, network communication using Ethernet.

[0073] The device 1 for processing medical data can further comprise an output device 40. The output device 40 can, for example, be a computer screen, such as an OLED or TFT display. However, another suitable device for outputting graphical content is also conceivable.

[0074] Furthermore, the device 1 for processing medical data includes an input device 50. The input device 50 can be any type of input device for receiving user input. For example, the input device 50 can include a keyboard, a computer mouse, or a microphone for receiving voice commands. The input devices 50 and the output device 40 can be implemented, for example, by means of a touchscreen or the like. In particular, a graphical user interface that uses the output device 40 and the input device 50 can be used for interaction with a user.

[0075] The device 1 for processing medical data further comprises a processing device 20. The processing device 20 can read image data stored in the first storage device 11, which processes image data, and cause the output device 40 to display a medical image corresponding to the read image data. Furthermore, the processing device 20 can read data from a medical report stored in the second storage device 12 and cause the output device 40 to display the contents of the respective medical report. In particular, a medical image and the corresponding medical report can be provided by the output device 40 in any suitable relationship. For example, the associated medical report can be displayed below a displayed medical image.However, another suitable system for providing a medical image and a corresponding medical report is also conceivable.

[0076] To create a new medical report or to adapt a medical report for a different (i.e., new or second) medical image, a user can select the medical image. This medical image will subsequently be referred to as the second medical image. For this purpose, the user can, for example, use the graphical user interface of output device 40 and input device 50.

[0077] Furthermore, the user can select another medical image, in particular a medical image that was recorded at an earlier time for a corresponding or even the same purpose. Hereinafter, this additional medical image is referred to as the first medical image, so that the user can reference this first image when creating the new (second) medical report. Additionally or alternatively, the user can select a medical report created at an earlier time, in particular a medical report that refers to an image captured at an earlier time. Hereinafter, this medical report is referred to as the first medical report. If the user selects the first image from an earlier time, the processing device 20 can automatically identify a related first medical report for this first medical image.If a user selects a first medical report that was generated at an earlier time, the processing device 20 can automatically determine the associated first medical image for that first medical report. The processing device 20 can then output the first medical image and the associated first medical report, along with the second medical image for which a second medical report is to be generated.

[0078] As described above, a medical report can include one or more documentation elements, in particular documentation elements that relate to the corresponding image elements in the relevant medical image. Therefore, the processing device 20 can determine the individual documentation elements in the first medical report and the corresponding image elements in the associated first medical image. The image elements and / or documentation elements can be marked, e.g., highlighted, if necessary.

[0079] The processing device 20 can include a processing module 22 for identifying an image element that corresponds to a user selection. The processing module 22 can identify an image element that corresponds to a user input / selection. For example, the user can click on a specific image element or a documentation element linked to an image element. If the user selects a specific documentation element, the corresponding image element in the associated medical image can be highlighted. If the user selects a specific image element in a medical image, the corresponding documentation element in the medical report can also be highlighted.

[0080] The following describes the automated or assisted creation of a new medical report based on the selected previous medical report and the corresponding previous medical image.

[0081] After selecting the second medical image for which the new, second medical report is to be generated, and further selecting / identifying a first medical image and a first medical report, a mapping between the first and second medical images is performed. For this purpose, a mapping device 30 for processing medical data can be provided in the device 1. Any suitable existing or future method can be used in the mapping to determine a relationship between the first and second medical images. For example, the mapping can determine a transformation scheme by specifying a relationship between a pixel position in the first medical image and the position of a corresponding pixel in the second medical image.In particular, the mapping can specify a relationship between positions in the first medical image and positions in the second medical image such that corresponding pixels refer to the same features, e.g., an organ, a lesion, a tumor, a bone, a cavity, a cyst, etc. The mapping can be performed, for example, using artificial intelligence, neural networks, etc. However, it is understood that other suitable approaches are also possible.

[0082] Once the mapping between the first and second medical images has been established, the processing device 20 can identify one or more image elements in the second medical image that correspond to the respective image elements in the first medical image. At the user's request, the identified image elements in the second medical image can be marked, e.g., highlighted, in the second medical image displayed on the output device 40.

[0083] Additionally or alternatively, the processing device 20 can identify the individual documentation elements of the first medical report that relate to the first medical image. Based on the identified documentation elements of the first medical image, the processing device 20 can determine the corresponding image elements in the first medical image. Furthermore, the processing device 20 can determine the corresponding image elements in the second medical image based on the identified mapping between the first and second medical images.

[0084] To create a draft for the new, i.e., the second, medical report, the processing device 20 can include a documentation module 21 that can automatically create a new documentation element in the second medical report for each documentation element in the first medical report and / or for each identified image element in the second medical report.

[0085] In particular, the documentation module 21 can create (second) documentation elements for the second medical report that refer to the associated image elements in the second medical image. Optionally, a (second) documentation element in the second medical report can also include a reference to the associated (first) documentation element in the first medical report and / or to an associated (first) image element in the first medical image. Furthermore, the documentation element in the design for the second medical report can be automatically adapted according to the characteristics / properties of an associated image element in the second medical image. For this purpose, the processing device 20 can include an analysis module 23.Analysis Module 23 can analyze an image element that relates to a documentation element to determine a property of the image element that the user needs to document or specify. The desired properties can be determined, for example, based on a semantic analysis of the respective documentation element. If, for instance, a documentation element contains an expression such as "the size is," Analysis Module 23 can automatically determine that the desired characteristic is a size, and therefore an analysis of the associated image element is performed to determine the size of the image element. It should be noted that the "size" of the characteristic is just one example, and any other type of characteristic, such as volume, density, shape, etc., can also be determined.

[0086] In the preceding description, an image element in the second medical image is automatically determined based on a mapping between the first and second medical images. While such automated mapping can already yield very good results, it may be desirable to adjust and automatically recognize detected image elements. For this purpose, the user can manually adjust the automatically detected image element. For example, the user can use the input device 50 to adjust the image element. The user can, for instance, use a computer mouse, a touchscreen, or another suitable component of the input device 50 to adjust an image element to a displayed medical image, particularly the second medical image.For example, the user can move the position of the image element, adjust the frame of the image element, add further features such as an arrow or a line, change the shape of the image element, or make other suitable changes.

[0087] After the user adjusts an image element, the processing device 20, in particular the analysis module 23, can perform a further analysis to determine the desired feature based on the adapted image element. For example, if the user changes the shape or size of an image element, the analysis module 23 can automatically determine the new size of the modified image element. The newly determined feature can then be automatically updated in the corresponding documentation element of the medical report.

[0088] If the user adjusts or changes an automatically determined image element in the second medical image, the adjusted relationship between an image element in the first medical image and the second medical image can also be stored. For example, the mapping, in particular the adjusted mapping between the first medical image and the second medical image, can be stored in a further storage device, such as a third storage device 13. In this way, the mapping is available at a later time. Therefore, a new mapping procedure is not required if the second medical report is to be modified later. Furthermore, the mapping, in particular the adjusted mapping stored in the third storage device 30, can also be used, for example, to train a neural network.In this way, the neural network can be further improved for performing the assignment operations based on the additional data provided by the third storage device 13.

[0089] After the draft of the second medical report has been automatically generated by the processing device 20, specifically by the documentation module 21, the user can manually modify this draft report if necessary. For example, the user can add further text, such as explanations, diagnoses, etc., to one or more documentation elements. To do this, the user can enter the desired changes or additions using the input device 50. Specifically, the user can enter text via a keyboard or use voice commands to modify the draft of the second medical report.In this context, a user can select a specific documentation element or image element, and the processing device 20 can automatically highlight all relevant documentation elements in the first and second medical reports, as well as the corresponding image element in the first and second medical images. Accordingly, the user can easily identify all relevant information provided by the output device 40.

[0090] The foregoing assumes that a medical image exists with one or more image elements, and that the corresponding medical report refers to these image elements with one or more documentation elements. However, in some cases, it may be desirable to import medical images and medical reports directly, for example, from external sources. In this case, there may be no relationship between a medical image and a corresponding medical report. For the automated generation of medical reports described above, however, it is desirable to establish such a relationship between medical images and reports. For this purpose, the processing device 20 can include a matching device 25.The matching device 25 can perform an analysis of a medical report and an associated medical image to identify one or more documentation elements within the medical report. This analysis can be based, for example, on a semantic analysis of the respective medical report. Furthermore, the matching device 25 can perform an analysis of the associated medical image to determine a corresponding image element for each documentation element. This relationship between an identified documentation element and the corresponding image element can then be added to the respective medical report and / or the medical image.

[0091] The described concept of automatically determining the characteristics of corresponding image elements in multiple medical images can also be extended to compare more than two related medical images. For example, it may be possible to determine a corresponding image element in several medical images at different times. In this case, it may also be possible to automatically determine one or more properties of such corresponding image elements for the multiple medical images and to provide information about the development of such a characteristic. For example, it is possible to determine the size, volume, etc., of a specific element in a series of images. In such a case, it may also be possible to enrich a documentation element in the second medical report by adding information about this development.For example, information about the individual values ​​of several previous medical images can be retrieved. Furthermore, it may also be possible to determine a control or reduction rate and include this specific rate in a corresponding documentation element in the second medical report.

[0092] Such an analysis of a development with respect to a specific image element can also be performed in a separate task, for example, before a corresponding documentation element is manually adapted to that image element by a user. For example, a device 1 for processing medical images can further comprise an evaluation device 60 for performing such an analysis of the features of an image element in a series of several medical images. For this purpose, a user can select a specific image element or, alternatively, a documentation element for the image element. The user can also select multiple medical images. Alternatively, the time period of several medical images can be selected automatically. For example, a user can specify a time period, and the evaluation device 60 can automatically select suitable medical images for that period.However, there may also be another suitable approach to selecting multiple images for such an analysis.

[0093] The evaluation device 60 can then identify a corresponding image element in each of the selected medical images. For this purpose, the mapping between the medical images described above can be applied to identify the desired image areas. Furthermore, an automated analysis of the identified image area across the multiple medical images is performed to determine one or more values ​​for one or more specific features of the image elements in all medical images. After determining the desired values ​​for the image elements in all medical images, a temporal progression of the respective values ​​can be determined. The determined progression can be communicated to the user, for example, by displaying the results on the output device 40.Furthermore, it may also be possible to automatically add the specific progression or one or more pieces of information derived from this progression to a documentation element in the second medical report.

[0094] Although the imaging device 30 and the evaluation device 60 are described as devices separate from the processing device 20, it may also be possible to include the imaging device 30 and / or the evaluation device 60 in the processing device 20.

[0095] For example, the processing device 20 can be a computer device with a processor and a memory for storing instructions to be executed by the processor. Therefore, the memory can provide instructions to cause the processor to perform the operations described above. In this context, the processing device 20 can be communicatively coupled to the first, second, and third memories 11, 12, 13, the output device 40, and the input device 50 via suitable interfaces.

[0096] Fig.Figure 3 shows a block diagram illustrating a method for processing a medical image according to one embodiment. The method may include all suitable steps for performing operations as already described in connection with the medical data processing device 1. Furthermore, the medical data processing device 1 described above may also include suitable devices or components for carrying out the method described below.

[0097] The procedure for processing medical data includes a step S1 of storing image data for a plurality of medical images. Each medical image can include an associated timestamp, in particular a timestamp representing the time at which the respective medical image was recorded.

[0098] In step S2, at least one medical report is saved for a corresponding stored medical image. The medical report can include at least one documentation element for each image element in the corresponding medical image.

[0099] The procedure further includes step S3 of receiving a selection for a first medical image and / or a first medical report for the first medical image. The procedure further includes step S4 of receiving a selection for a second medical image.

[0100] In step S5, a mapping between the first and second medical images is calculated. This mapping can be based on the image elements in the first and second medical images.

[0101] Furthermore, the procedure includes step S6 of determining an image element in the second medical image for which a documentation element exists in the first medical report. For this purpose, the position, size, and / or shape of the image element in the second image can be determined based on the correlation between the first and second medical images.

[0102] Finally, the procedure includes step S7 of creating a draft of a second medical report for the second medical image, whereby a documentation element for the second medical report is automatically created using the corresponding documentation element for the corresponding image element in the first medical report.

[0103] In summary, the present invention relates to the processing of medical data, in particular image data from medical images and associated medical reports, for the automated generation of a draft report for a newly recorded medical image. For this purpose, the image elements that relate to corresponding documentation elements for a previous medical image are mapped to or assigned to image elements in the newly recorded medical image. Based on this assignment, the draft medical report for the newly recorded medical image is automatically generated.

[0104] Unless expressly stated otherwise, individual embodiments or their individual aspects and features described with reference to the drawings may be combined or interchanged without limiting or extending the scope of the described invention, wherever such combination or interchangeability is meaningful and in line with the purpose of this invention. Advantages described with respect to a particular embodiment of the present invention or with respect to a particular figure may also be advantages of other embodiments of the present invention.

Claims

[1] Medical data processing device (1), comprising: - a first storage device (11) adapted to store image data for a plurality of medical images, each medical image being provided with a corresponding time stamp; - a second storage device (12) adapted to store at least one medical report for a corresponding medical image stored in the first storage device (11), wherein the medical report comprises at least one documentation element for an image element in the corresponding medical image; - an input device (50) adapted to receive a selection for a first medical image and / or a first medical report for the first medical image, and adapted to receive a selection for a second medical image; - an association device (30) adapted to calculate an association between the first medical image and the second medical image; - a processing device (20) adapted to determine at least one image element in the second medical image for which a documentation element is contained in the first medical report, wherein the determination is carried out using the association between the first medical image and the second medical image; wherein the processing device (20) comprises a documentation module (21) adapted to create a documentation element for a draft of a second medical report for the second medical image, wherein a corresponding documentation element for the second medical report is automatically created using the corresponding documentation element for the corresponding image element in the first medical report. [2] Device (1) according to claim 1, comprising an output device (40) adapted to output the first image, the corresponding first report for the first image, the second image, and the draft for the second report, and / or wherein the input device (50) is adapted to receive user inputs for a selection of an image element in the output images and / or a documentation element in the output reports, and / or wherein the processing device (20) comprises a processing module (22) adapted to identify an image element corresponding to the selection in each first and second image and to identify a documentation element corresponding to one of the first report and the second report; wherein the output device (40) is further adapted to highlight the identified image elements in the first and second images and the identified documentation elements in the first and second reports. [3] Device (1) according to one of the preceding claims, wherein the input device (50) is adapted to receive a setting for an assignment of a picture element in the first image and a picture element in the second image, and wherein the processing device (20) is adapted to adjust the assignment between the picture elements in the first image and the second image based on the received setting. [4] Device (1) according to one of the preceding claims, comprising a third storage device (13) adapted to store the association between the first image and the second image and / or to store an association between a pixel in the first image and a corresponding pixel in the second image. [5] Device (1) according to one of the preceding claims, wherein the processing device (20) comprises an analysis module (23) adapted to determine at least one property of the image element in the second image, and wherein the documentation module (21) is adapted to create the draft of a second report using the determined at least one property of the image element. [6] Device (1) according to the immediately preceding claim, wherein the analysis module (23) is suitable for determining the at least one property to be determined by means of a semantic analysis of the corresponding documentation element in the first report. [7] Apparatus (1) according to any one of the preceding claims, comprising an evaluation device (60) adapted to identify an associated pixel in a plurality of medical images and a corresponding property of each identified pixel and to determine a progression of the property of the pixel as a function of the associated time stamps. [8] Device (1) according to one of the preceding claims, wherein the documentation module (21) is adapted to generate a corresponding documentation element in the second report for each image element in the first image and / or each documentation element in the first report. [9] Device (1) according to one of the preceding claims, comprising a first data interface (71) adapted to be coupled to a medical diagnostic imaging system, to receive medical image data, and to store the received data in the first storage device (11). [10] Device (1) according to one of the preceding claims, comprising a second data interface (72) adapted to provide data of a medical image and a corresponding medical report to an external processing device and / or to receive information for an image element of a medical image or a documentation element of a medical report and to automatically add the received information to the corresponding image and / or report. [11] Device (1) according to one of the preceding claims, comprising a comparison device (25) which is designed to identify at least one documentation element in a first medical report using an analysis of the first medical report, in particular a semantic analysis, and to determine a corresponding image element in the first medical image for each of the identified documentation elements. [12] Device (1) according to one of the preceding claims, wherein the image elements are positioned within the first or second medical image at an associated embedded position and / or wherein the image elements comprise an indication of a position, an organ, a lesion and / or a size indication. [13] Device (1) according to one of the preceding claims, wherein the association between the first medical image and the second medical image comprises two-dimensional or three-dimensional coordinates. [14] Method for processing medical data, comprising the steps of: Storing (S1) image data for a plurality of medical images, each medical image having an associated timestamp; Storing (S2) at least one medical report for a corresponding stored medical image, wherein the medical report comprises at least one documentation element for each image element in the corresponding medical image; Receiving (S3) a selection for a first medical image and / or a first medical report for the first medical image; Receiving (S4) a selection for a second medical image; Calculating (S5) an association between the first medical image and the second medical image; Determining (S6) an image element in the second medical image for which a documentation element is present in the first medical report, wherein a position, size and / or shape of the image element in the second image is determined based on the association between the first medical image and the second medical image; and Creating (S7) a draft of a second medical report for the second medical image, wherein a documentation element for the second medical report is automatically created using the corresponding documentation element for the corresponding image element in the first medical report. [15] Computer program product comprising a computer program which can be loaded into a memory unit of a control device according to one of the preceding device claims and which contains program code sections which serve to ensure that the control device carries out the method for processing the medical image data according to the preceding method claims when the computer program is executed in the control device.

Images