CINEMATIC RENDERING SHORT FILM TO ILLUSTRATION A CLINICAL FINDING
Patent Information
- Application Number
- DE502017016557
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2017-09-29
- Publication Date
- 2024-12-05
- Estimated Expiration
- 2037-09-29
AI Technical Summary
The current radiological diagnosis process is hindered by the time-consuming and inefficient communication of complex medical image information, as large image data sets require significant network resources and specialized knowledge, often resulting in loss of illustrative information when transmitted between diagnostic units and referrers.
A method for generating a cinematic rendering short film based on medical image data sets and radiological reports, using a computing unit to select clinical findings, determine parameter sets for a cinematic rendering algorithm, and create a short film that adapts to specific anatomical structures and medical questions, allowing for clear visualization of clinical findings.
This approach enables the preparation of complex visual information in a simple and clear manner, enhancing communication of medical findings by converting medical image data into a cinematic rendering short film that can be easily understood by non-radiological professionals, thereby improving diagnostic reporting without overloading hospital networks.
Description
[0001] To record and assess the health status of an examination subject, increasingly extensive image data sets of a body region of interest are being acquired, e.g., using computed tomography, magnetic resonance imaging, C-arm X-ray equipment, or similar techniques. These data sets are interpreted and assessed with regard to various medical questions. Depending on the medical question and / or the body region being examined, an acquired image data set can comprise an image series with up to several thousand individual images. Thus, a diagnosis is no longer limited to the assessment of a depicted anatomical condition based on a medical image, as was the case with a diagnosis based on a conventional X-ray image, for example.In particular, diagnostic reporting is increasingly no longer performed simultaneously with image generation, but must be viewed, at least outside of emergency medicine, as a time-consuming process separate from image data acquisition. In this respect, radiological reporting is experiencing a significant development boost, along with the technological advancements in medical image data acquisition technology. Trends in this development include the establishment of large radiological reporting units or reporting centers with multiple parallel reporting workstations, which are located either locally in a health center or hospital or centrally. In the latter case, the radiological reporting unit can act as a service provider for several hospitals or a hospital chain.Trends in this development also include international standardization of radiological reports, so that a report can be read in a uniform and understandable manner regardless of country, region and / or radiologist making the report, and is suitable for integration into an electronic patient record that is thus essentially available everywhere.
[0002] A current standard process, for example, looks like this: the radiology department of a hospital receives a medical request regarding patient XY from a non-radiology department, also known as a referrer in technical terms, e.g., cardiology, oncology, or pathology. After patient XY has been scanned or examined using an MRI, CT, or similar method, a radiologist interprets the acquired image data (usually DICOM images) for patient XY and, while viewing the images, dictates clinical findings into a radiological report. The report contains all of the medical findings extracted from the image data. The report, also called a report, is then stored, together with the underlying image data, as a so-called DICOM study in a local archiving system, e.g., a PACS (Picture Archiving and Communication System), for retrieval.In addition, the report is sent to the referrer, without the image data. The additional transmission of image data is omitted for technical reasons, as the sheer volume of data to be transmitted would overload a typical hospital network infrastructure. Furthermore, the requesting medical department often lacks the necessary knowledge and / or time to properly interpret the image data. Accordingly, a large amount of additional illustrative visual information is frequently lost during the communication of the diagnosis results between the reporting unit and the referrer.
[0003] WO 2003 / 071779 A1 discloses a system and method for generating a sequential display of medical image data. For this purpose, a plurality of medical image data are acquired, sorted into groups, and displayed sequentially in a continuous film loop.
[0004] In contrast, the object of the present invention is to provide means for enriching the information content of a report of findings that require neither special knowledge of findings on the part of a referrer nor increased capacities of a hospital network infrastructure.
[0005] This object is achieved by a method for generating a cinematic rendering short film based on a medical image dataset and a radiological report, by means of a corresponding computing unit, a corresponding computer program, a corresponding computer-readable data storage medium, and a corresponding diagnosis workstation according to the independent claims. Preferred and / or alternative, advantageous embodiments are the subject of the dependent claims.
[0006] The inventive solution to the problem is described below with reference to the claimed method and with reference to claimed devices. Features, advantages, or alternative embodiments mentioned here are also applicable to the other claimed subject matter, and vice versa. In other words, claims (which, for example, are directed to a method) can also be developed with features described or claimed in connection with one of the devices. The corresponding functional features of the method are implemented by corresponding modules or units.
[0007] The present invention relates in a first aspect to a method for generating a cinematic rendering short film. The generation of the cinematic rendering short film is based on a medical image data set comprising a series of images, wherein the image data set represents a body region of interest of an examination subject, and a radiological findings report, wherein the radiological findings report comprises at least one clinical finding relating to the body region of interest represented by means of the image data set.
[0008] The method comprises a number of steps. In a first step, a clinical finding is selected from the report. In a second step, a partial image series from the image dataset is selected based on the selected clinical finding. In a third step, a parameter set for a cinematic rendering algorithm is determined, wherein the parameter set takes into account information about specific protocol parameters and / or anatomical structures within the imaged body region of interest. In a fourth step, a cinematic rendering short film is generated by applying the cinematic rendering algorithm to the selected partial image series using the parameter set.The parameter set is preferably determined taking into account the selected clinical finding, the underlying medical question transmitted by a referrer to a radiology department and / or the anatomical structures within the body region of interest, such as specific organs or tissue, shown in the selected partial image series. The question provides information, for example, about certain protocol parameters for image data acquisition and thus about the information content of the image series. The anatomical structure shown, e.g. bone or soft tissue, requires different settings for cinematic rendering for comparable image or video quality. In this way, the configuration of the cinematic rendering algorithm can be adapted to the individual circumstances of the patient being examined, and a particularly advantageous visualization or representation of the anatomical structure, e.g.a diagnosed lesion, in the short film.
[0009] According to the invention, it is therefore provided that, within the framework of a diagnosis in a diagnosis unit, complex, clinical, visual additional information can be prepared in a simple and clear manner for a viewer, in particular a physician outside the radiological field, and made available alongside a report. For this purpose, medical image information contained in a medical image dataset underlying a diagnosis is transferred or converted into a cinematic rendering short film, taking into account a finding also derived from it. By selecting or selectingTo determine a parameter set, preferably depending on the selected clinical finding, the underlying medical question, and / or the anatomical structures depicted in the selected partial image series, a cinematic rendering algorithm is preconfigured according to the invention such that the cinematic rendering short film clearly depicts the selected finding or an anatomical situation corresponding to the finding, or a state of health or disease, even for an untrained viewer. The short film according to the invention can advantageously be made available to a referrer, e.g., a neurologist or a cardiologist, together with or separately from the findings report.
[0010] In the following, without loss of generality, we assume a patient as the object of investigation, usually a human being. In principle, the patient can be any living being or tissue. Therefore, the terms "object of investigation" and "patient" are used synonymously. The object of investigation can alternatively be a plant or a non-living object, e.g., a historical artifact or the like.
[0011] A body region of interest within the meaning of the invention corresponds to a (partial) area of the examination subject imaged using any medical imaging system, in particular a specific body region such as the head, thorax, hip, lower leg, etc. The body region of interest is, in particular, a symptom-relevant body region. In this case, image data is acquired only with respect to this (partial) area; however, the body region of interest can also encompass the entire examination subject. In this case, the medical imaging system acquires image data relating to the entire body of the examination subject.
[0012] A medical image dataset in the sense of the invention is to be understood as a set of image data that was acquired as a result of a medical question using a medical imaging system relating to the body region of interest. According to the invention, the image dataset is present in image space or in spatial space. The image dataset can be embodied as a three-dimensional or four-dimensional image dataset. A three-dimensional image dataset comprises three spatial dimensions. A four-dimensional image dataset comprises a time dimension in addition to the three spatial dimensions. In other words, the image dataset can comprise an image series with at least two image elements, wherein individual image elements or images of the image series are acquired essentially simultaneously and have a spatial offset from one another and / or individual image elements of the image series have a temporal offset from one another, e.g.when contrast agent flow or contrast agent enhancement in a specific tissue within the body region of interest, or cardiac function, is to be visualized. Individual image elements can be generated from existing image elements using known interpolation techniques, for example. A typical medical image dataset, for example, comprises several hundred to several thousand individual image elements.
[0013] According to the invention, a partial image series of the image data set comprises a subset of connected image elements of the image series of the image data set, which in individual cases can comprise the entire image series. The partial image series comprises at least enough individual image elements to be suitable for the inventive generation of a cinematic rendering short film.
[0014] In this context, a cinematic rendering short film is to be understood as a temporal sequence of individual video elements, i.e. a short film or video clip with a short running time, e.g. a few seconds, e.g. 5 - 30 seconds. Other, particularly longer running times are also possible. A cinematic rendering short film is further characterized by the fact that it was created using a cinematic rendering process or algorithm. Individual video elements each correspond to a rendered frame or time frame of the short film; an individual video element can in particular correspond to an individual image element of the partial image series, e.g. if the medical image dataset includes a temporal component. The cinematic rendering short film can be in various formats, e.g. as a DICOM Secondary Capture Element, as a DICOM Video Stream (MPEG-4 AVC), as an Animated GIF Element or MPEG Stream.In particular, it is intended that a cinematic rendering short film is compatible and integrable into a workflow of a hospital information system (HIS) or a radiology information system (RIS), typically according to the FHIR or HL7 standard.
[0015] A cinematic rendering algorithm within the meaning of the invention is to be understood as a novel post-processing method for three-dimensional volume image datasets, such as those obtained using CT or MRI. Cinematic rendering can be used to generate a particularly natural or photorealistic representation of a body region of interest represented in the image data. Cinematic rendering is particularly suitable for visualizing anatomical structures with high density or high contrast, such as bone tissue or contrast-enhanced blood vessels. When calculating a pixel entry of a rendered image, cinematic rendering techniques take into account that light can be scattered from any direction along a light beam.Monte Carlo simulations are typically used to randomly determine a bundle of light rays with a favorable spatial distribution, which is then taken into account in ray tracing. A final, rendered image is typically obtained by averaging a large number of Monte Carlo samples, each with a different beam distribution and scattered ray directions. When generating a cinematic rendering short film, the cinematic rendering algorithm is applied to each individual image element or the series of partial images. In particular, cinematic rendering in the short film can create the impression that the viewer is moving around or through the body region of interest by changing the viewer's perspective or the position of a virtual camera relative to the displayed body region of interest.The cinematic rendering algorithm can advantageously include caching of light beam reflections to generate a fast, particularly noise-free and low-interruption preview or view of the cinematic rendering short film. This allows the user to quickly and in advance decide whether the cinematic rendering short film, or more precisely the selected sub-image series and / or the specific parameter set, is optimally tailored to the selected findings, and to make adjustments if necessary before the final cinematic rendering short film is created.
[0016] The parameter set of the cinematic rendering algorithm comprises at least one, but usually several parameters for configuring the cinematic rendering. In a preferred embodiment of the invention, the parameter set comprises at least one parameter from the group of the following cinematic rendering parameters: lighting model, zooming, panning, clipping, transfer function, camera position, camera trajectory. The lighting model comprises various configurations regarding the artificial light source used in the calculation, such as its position in or on an object within or outside the body region of interest, or its trajectory if the position of the lighting source is to change during the cinematic rendering short film. The artificial light source can be a virtual lamp or a virtual spotlight.
[0017] A radiological report within the meaning of the invention is a report in the form of an electronic document that arises as a result of a radiological diagnosis. The form or medium for archiving and / or distribution of the report is not specified. However, the DICOM standard is now a widely used option, in which the report is archived and communicated as a DICOM Structured Report. Alternatively or additionally, a report can also be in the form of an HL7 report, which can be saved in a hospital information system (HIS / HIS). It can be provided that a DICOM Structured Report is converted into an HL7 report using HL7 CDA conversion methods, and vice versa. Specifications regarding its structure and content components are standardized. A report includes at least information on the identification of the patient and the examination performed.Furthermore, the report of findings must also include at least one piece of medical content, which must contain at least clinical information, a research question, and an assessment, summarizing a clinical finding. Ideally, a report of findings should be complete, i.e. it refers only to the associated image data set archived, for example, together with the report of findings as a DICOM study. The assessment of the clinical information addresses the formulated research question, and other findings identifiable in the image data of the image data set must also be assessed. The structure of the report of findings is ideally also standardized, e.g. to enable integration into an electronic patient record. For example, a clinical finding can be texturally structured. In addition, supplementary multimedia presentation formats can be created, which allow, for example, dynamic visualizations of components of the findings.
[0018] Advantageously, according to the invention, the generated cinematic rendering short film can be archived or saved as a DICOM study together with a radiological report and the corresponding image data set.
[0019] In a preferred embodiment of the invention, the method also comprises the following step: Inserting a direct access element into the cinematic rendering short movie, which allows direct access from the cinematic rendering short movie to the clinical finding in the report.
[0020] This embodiment creates a link or connection between the cinematic rendering short film and the clinical findings. The direct access element can, for example, be designed as image content in one or a series of connected frames or individual video elements of the short film, e.g. as the display of a button or as interactive text. Other embodiments are also conceivable. Alternatively, a direct access element can be designed as an interactive marker, e.g. an arrow or a line within a timeline displayed in addition to the cinematic rendering short film over the course of the short film. Other embodiments of the direct access element are also conceivable. What all embodiments have in common is that when the direct access element is actuated or activated by a user, in this case a viewer, e.g. a physician outside of radiology, the radiological findings report is displayed to the user.In particular, the selected finding from the report to which the cinematic rendering short film refers is displayed. In addition to the clinical findings, and if necessary simultaneously, the viewer is provided with illustrative image information on the health condition or disease described in the report.
[0021] It is further provided that the procedure shall also include the following step: Inserting a direct access element into the diagnosis report, which allows direct access from a clinical finding in the diagnosis report to the cinematic rendering short film.
[0022] This creates a connection or link between the cinematic rendering short film and the clinical finding, but this time starting from the selected clinical finding and pointing towards the cinematic rendering short film. The direct access element in the report can, for example, be designed as an interactive reference or link to the storage location of the cinematic rendering short film. The direct access element can, for example, be designed as a partial reference path if the cinematic rendering short film is archived locally, e.g., in a hospital's PACS, or as a complete reference path to a storage location if the cinematic rendering short film is stored centrally, e.g., on a cloud server. Alternatively, the cinematic rendering short film can be embedded directly as a direct access element in the report relating to the selected clinical finding, e.g., as a DICOM Secondary Capture object.This exemplary embodiment also provides that when a user activates or operates the direct access element in the diagnosis report, the cinematic rendering short film is automatically displayed or played, for example, using suitable playback software that is also called up when the direct access element is activated. In this respect, a user, again a viewer who reads or analyzes the selected clinical finding in the diagnosis report, is instantly presented with an additional visual illustration of the anatomical structure or situation underlying the finding, if desired.
[0023] In summary, the present invention thus establishes a uni-directional and particularly preferably a bi-directional interface between a clinical finding and a cinematic rendering short film, whereby the content of information regarding a finding or short film is mutually supplemented in a mutually advantageous and clear manner.
[0024] In another embodiment of the invention, a position of the direct access element in the cinematic rendering short film is selected such that a representation of the direct access element correlates with a representation of the body region of interest on which the selected clinical finding is based. This embodiment therefore provides in particular that the direct access element is faded in or displayed in the cinematic rendering short film when a displayed image content with regard to the body region of interest best illustrates the clinical finding. In other words, the direct access element is placed or positioned or arranged in the cinematic rendering short film such that the individual frame or frames or individual video elements of the short film that comprise the direct access element best illustrate a diagnosed illness or health condition, e.g.the size and / or location of a lesion relative to another, particularly sensitive organ or a functional weakness, e.g. of a heart valve.
[0025] In a further preferred embodiment, the position, i.e. the temporal position, of the direct access element in the cinematic rendering short film is determined by means of a machine learning method, a statistical method or an artificial neural network.
[0026] Machine learning methods can be described as the artificial generation of knowledge from experience. An artificial system learns from examples in a training phase and can generalize after the training phase is completed. This allows an algorithm or the underlying selection heuristic for determining the optimal position for the direct access element to be continuously adapted. The use of machine learning methods can include the recognition of patterns and regularities in the training data.
[0027] A statistical method can include, for example, fuzzy logic, a self-organizing map, resampling, pattern recognition, or support vector machines. Based on experience, an algorithm for recognizing patterns or regularities can advantageously be used to determine a suitable position for the direct access element.
[0028] A trained algorithm for determining a position of a direct access element in the cinematic rendering short film can comprise a neural network, a system, a learning transfer method or an algorithm based on a database or on mathematical functions or equations. The trained determination algorithm can be created, improved or adapted on the basis of known data from historical image datasets and corresponding image content relating to the body region of interest as well as the associated clinical findings. Combinations of the following information can serve as input data for training the determination algorithm: measurement protocols from old image data recordings, information about the imaging system used, the body region to be examined, image preprocessing steps used such as segmentation and / or artifact correction steps, as well as for the assessment of a disease orAnatomical structures classified as relevant to the health status and marked accordingly in the image data, such as lesions or a specific tissue type, information about the parameter set of the cinematic rendering algorithm, or the like. This data can be obtained, for example, from log files of all imaging systems in the field and / or from historical DICOM studies, each comprising at least one cinematic rendering short film with at least one direct access element, archived locally or centrally. In the event that the determination algorithm is a neural network, supervised learning can be used as a training method, since the determined positions of the direct access elements can also be supplied as input data for the old DICOM studies.A position determined for a direct access element can correspond to a unique relation or a relation determined by probability. Patterns or regularities from the historical training data can advantageously be applied to a current position determination for a direct access element. In individual cases, it may be possible for the machine-determined position for a direct access element of a short film to be adjusted, changed, or corrected by a user, for example, if a different position appears more advantageous to the user.
[0029] In a further preferred embodiment, the clinical finding and / or the partial image series is selected by a user. In this embodiment of the invention, it is provided that the user, in this case a diagnosing radiologist, makes at least one selection of the clinical finding or the partial image series from the entire image data set for which they wish to create a cinematic rendering short film. Accordingly, the invention can provide that, for example, the selection of a suitable partial image series takes place automatically based on a user input regarding the clinical finding or is displayed to the user for confirmation or adjustment. Alternatively, it can be provided that, for each clinical finding in a findings report, a partial image series of the image data set is automatically displayed to the user for selection or adjustment. An automatic selection of a partial image series is based, for example, onbased on an analysis of the content and anatomical structures represented in the individual images of the image series of the image dataset, in particular after at least one post-processing step such as segmentation, noise correction, and / or motion artifact correction. In alternative embodiments, at least one of the two selection steps occurs automatically without user input.
[0030] In a further embodiment of the invention, the parameter set is determined taking into account the selected clinical finding and / or the selected partial image series using a machine learning method, a statistical method, or an artificial neural network. Regarding the implementation of the various alternative variants, the above applies.
[0031] A trained determination algorithm can be created, improved, or adapted based on known data from historical image datasets and corresponding cinematic rendering films, especially short films, regarding the body region of interest and the associated clinical findings. Input data for training the determination algorithm can be, for example, combinations of the following information: measurement protocols from previous image data acquisitions, information about the imaging system used, the body region to be examined, and image preprocessing steps used, such as segmentation steps, noise reduction, and / or motion artifact correction, as well as for the assessment of a disease or condition.Anatomical structures classified as relevant to the health status and marked accordingly in the image data, information about the parameter set of old cinematic rendering algorithms and information about the cinematic rendering algorithm used. This data can be obtained, for example, from log files of all imaging systems in the field and / or from historical DICOM studies, each comprising at least one cinematic rendering film, or archived locally or centrally. If the determination algorithm is a neural network, supervised learning can be used as a training method, since the parameter sets used for the old DICOM studies can also be supplied as input data. A parameter set for a cinematic rendering algorithm can correspond to a unique relationship or a relationship based on a probability.Advantageously, patterns or regularities from the historical training data can be applied to a current parameter set determination. In individual cases, it may be possible for a machine-determined parameter set to be adjusted, modified, or corrected by a user, for example, if the user considers a different configuration of the cinematic rendering algorithm more advantageous.
[0032] Alternatively, the parameter set can be specified by default, for example, depending on a selected clinical finding, the body region of interest depicted in the image data set, the measurement protocol used and / or the like.
[0033] The invention relates in another aspect to a computing unit for generating a cinematic rendering short film based on a medical image data set comprising a series of images, wherein the image data set represents a body region of interest of an examination subject, and a radiological findings report, wherein the radiological findings report comprises at least one clinical finding relating to the body region of interest represented by means of the image data set.
[0034] According to the invention, the computing unit has means for carrying out a method according to the invention.
[0035] In a preferred embodiment of the computing unit, its means for carrying out the method are arranged at least partially on the server side. In other words, at least one of its means is arranged on the server side, for example, in the cloud. This has the advantage that the centrally arranged means can be used by several computing units located locally at different locations. This is resource-saving and efficient.
[0036] Particularly preferably, the step of generating the cinematic rendering short film takes place centrally in the cloud using a cloud-based service. In this context, the invention can provide that, before transmitting the selected partial image series and a selected parameter set for the cinematic rendering, the data is anonymized with regard to patient identification data. For this purpose, for example, a text recognition and / or an OCR (optical character recognition) method can be applied to the data to be transmitted, and corresponding patient identification data can be removed, blacked out, blended, and / or anonymized.
[0037] In another aspect, the invention relates to a computer program with program code for carrying out a method according to the invention when the computer program is executed on a computer.
[0038] In a further aspect, the invention relates to a computer-readable data carrier with program code of a computer program in order to carry out a method according to the invention when the computer program is executed on a computer.
[0039] In a final aspect, the invention relates to a diagnostic workstation for generating a cinematic rendering short film based on a medical image data set comprising a series of images, wherein the image data set represents a body region of interest of an examination subject, and a radiological findings report, wherein the radiological findings report comprises at least one clinical finding relating to the body region of interest represented by means of the image data set.
[0040] This diagnostic workstation has a computing unit according to the invention. A diagnostic workstation preferably comprises an input and output unit. The output unit serves to display image information, diagnostic reports, and / or cinematic rendering short films. For this purpose, the output unit comprises at least one display screen. The input unit enables interaction with a user, in this case a radiologist, who is performing a diagnostic analysis of an image dataset at the diagnostic workstation. The user can make inputs via the input unit, for example, regarding the selection of a partial image series or a desired parameter set. The selection or selection options can be displayed to the user via an output unit. The diagnostic workstation is, for example, located in a radiology unit, alongside a multitude of other diagnostic workstations.The diagnostic workstation can be located locally within a hospital or centrally in an independent unit, such as a radiology center that performs diagnostic services for multiple hospitals in parallel. In particular, a diagnostic workstation can also be mobile; for example, an input and output unit can be configured as a portable tablet, and the computing unit in this variant can be configured as a central computing unit, with the input and output unit connecting to it for data communication.
[0041] The above-described properties, features, and advantages of this invention, as well as the manner in which they are achieved, will become clearer and more readily understood in connection with the following description of the exemplary embodiments, which are explained in more detail in conjunction with the drawings. This description does not limit the invention to these exemplary embodiments. In different figures, identical components are provided with identical reference numerals. The figures are generally not to scale. They show: FIG 1 shows a view of a system comprising a diagnosis workstation according to the invention and a computing unit according to the invention according to an embodiment of the present invention, FIG 2 shows a schematic representation of a method according to the invention according to an embodiment of the present invention, FIG 3 shows a representation of a cinematic rendering short film according to the invention including a direct access element in an embodiment of the present invention, and FIG 4 shows a representation of a diagnosis report according to the invention including a direct access element in an embodiment of the present invention.
[0042] Figure 1shows a system for storing, processing, transmitting and / or displaying medical image data, e.g., a CT image data set or an MRI image data set. A comparable system can be located, for example, in a hospital or any medical care center. Individual units, modules or components of the system can be located locally, in the medical care center, centrally at another location, or virtually in the cloud. The system comprises an output unit 48 and an input unit 50. The system further comprises a computing unit 46 according to the invention. The computing unit 46 and the output and input units 48, 50 are in data communication via a connection 14. These components together form a diagnostic workstation BA according to the invention. In particular, the diagnostic workstation BA according to the invention is located centrally or remotely, e.g.This is the case when a medical care center utilizes teleradiology as a service. With teleradiology, radiological images are transmitted to a remote location for evaluation by a qualified physician. The location of the diagnosis and the location of the examination are thus separated, and communication takes place via secure, electronic data transmission (e.g., the internet).
[0043] The output and input units 48, 50 are used for information exchange or interaction with a user, here a radiologist or a diagnosing physician. The output unit 48 is used, for example, to graphically display a medical image dataset and / or a cinematic rendering short film KF according to the invention for a user. The output unit is also used, for example, to display selection options regarding a parameter set P for a cinematic rendering algorithm or regarding a partial image series that is to serve as the basis for generating a cinematic rendering short film KF. The input unit 50 is used, for example, for confirmation or input of selection information by the user or information regarding the display of a specific image dataset BD, radiological report BB, or the like. A user can, in particular, also create a clinical finding or a corresponding finding report BB via the input unit 48.Via the input unit 50, the user N has in particular the option of intervening in the sequence of the method according to the invention or of controlling it. The output unit 48 can, for example, be at least one LCD, plasma or OLED screen. Preferably, a plurality of screens, in particular two screens, are included in order to be able to display various information simultaneously, for example an image data set BD and an associated radiological finding as well as an associated cinematic rendering short film KF. The output unit 48 can also be a touch-sensitive screen, which is also designed as an input unit 50. In particular, the output unit 48 is designed as a portable output unit, for example a cell phone or tablet, and is therefore also decoupled from the location of the computing unit 46, so that a finding orThe method according to the invention can be carried out regardless of the location of the expert radiologist. The input unit 50 is, for example, a keyboard, a mouse, a so-called "touchscreen," a microphone for voice input, and / or a foot switch. The microphone and foot switch have the particular advantage of leaving the user's hands free. The input unit 50 can also be configured to recognize a user's movements and translate them into corresponding commands.
[0044] The computing unit 46 is designed to carry out all steps of the method according to the invention. For this purpose, the computing unit 46 can comprise various modules or units. In particular, the computing unit 46 communicates with an image storage unit 20 and a findings storage unit 30 via connections 14. The image storage unit 20 can be designed as a Picture Archiving and Communication System (PACS) and serves for the digital archiving and communication of digital image data in the form of medical image datasets BD in a network. Image data from all modalities and from different patients (Pat X, Pat Y, Pat Z) can be archived in the image storage unit 20. For one patient, several image datasets BDX1 - BDZ1 can be stored, corresponding to various examinations of a patient. The PACS 20 enables fast, uncomplicated access to the data, rapid availability, and reproducibility.If a findings report BB is saved in DICOM Structured Report Format, it can also be archived as part of a DICOM study in the image storage unit 20. The findings storage unit 30 can, for example, be designed as a radiology information system (RIS) of a hospital's radiology department, but also as a hospital information system (HIS / HIS). The findings storage unit 30 primarily manages patient master data. For this purpose, it communicates with other system components. Communication usually takes place using the HL7 standard. For example, the patient master data is used to equip image data records with corresponding metadata. In addition, the findings storage unit 30 also serves to digitally archive radiological findings reports BB comprising at least one clinical finding Bx1, Bx2 that is available for a specific patient, e.g., patient Pat X.In particular, the archiving of BB reports takes place in the report storage unit 30 in HL7 format. Using the patient metadata already mentioned, the archived BB reports can be clearly assigned to the respective image data sets BD in the PACS.
[0045] The computing unit 46 can advantageously comprise a receiving unit 44, which receives any selection or input commands, e.g. regarding a partial image series TB, regarding a position of a direct access element DZE in a cinematic rendering short film KF or a radiological report BB and / or regarding a clinical finding Bx1, Bx2 of a report BB, from a user and processes them for further consideration. In addition, the receiving unit 44 is also configured to receive image data records BD and / or report findings BB from the image storage unit 20 or report storage unit 30 via connection 14. The computing unit 46 further comprises a determination unit 42 for determining a parameter set P for a cinematic rendering algorithm. For this purpose, the determination unit 42 canof the selected clinical finding Bx1, Bx2, the medical question underlying an examination and thus information on certain protocol parameters and / or anatomical structures within the imaged body region of interest. In particular, the determination unit 42 can be configured to determine the parameter set P using a machine learning method. The computing unit 46 also includes a second determination unit 40 for determining a position of a direct access element in a cinematic rendering short film. This is configured, in particular, to evaluate and analyze a selected partial image series or its image content in such a way that it identifies individual image elements whose image content best matches or correlates with a selected finding Bx1.In addition, the computing unit 46 also includes a rendering unit 38, which is configured to create a cinematic rendering short film from all information provided by the other units using a cinematic rendering algorithm. In particular, the rendering unit 38 can be outsourced as a standalone unit and implemented, for example, in the form of a web-based service via a cloud server.
[0046] Furthermore, all of the aforementioned units of the computing unit 46 may be physically and / or functionally connected or separated from one another.
[0047] The computing unit 46 can interact with a computer-readable data carrier, in particular to carry out a method according to the invention using a computer program with program code. Furthermore, the computer program can be stored retrievably on the machine-readable carrier. In particular, the machine-readable carrier can be a CD, DVD, Blu-ray Disc, a memory stick, or a hard disk. The computing unit 46 can be implemented in the form of hardware or software. For example, the computing unit 46 is implemented as a so-called FPGA (acronym for "Field Programmable Gate Array") or comprises an arithmetic logic unit.
[0048] At least one computer program is stored in a memory 36 of the computing unit 46, which performs all method steps of the method according to the invention when the computer program is executed on the computer. The computer program for executing the method steps of the method according to the invention comprises program code. Furthermore, the computer program can be designed as an executable file and / or on another computer system. For example, the system shown can be designed such that the computing unit 46 loads the computer program for executing the method according to the invention into its internal memory via an intranet or the internet.
[0049] Figure 2shows a flowchart of a method according to the invention in one embodiment of the invention. The method is carried out, for example, at a diagnosis workstation BA according to the invention. At such diagnosis workstations BA, grayscale images and / or false-color images in 2D, 3D, or 4D (time-dependent representations, e.g., changes in activity in a body region of interest or representations of body fluid movements) acquired using a recording technology such as CT, MRI, PET, or similar are evaluated, i.e., assessed, and diagnosed. This means that the image data is searched for pathological patterns. In addition, characteristics of the image data must be analyzed, measured, and interpreted. Findings are dictated and either forwarded for further processing or fed directly into the computer using appropriate software.
[0050] Preferably, the method according to the invention is carried out directly during or after the preparation of a report BB for a clinical finding Bx1, Bx2 included therein. The method according to the invention comprises a plurality of steps, which are described in more detail below:
[0051] In a first step S1, an image data set BDx1 is displayed to the user by means of output unit 48. In parallel, in a step S3, a report BB based on the image data set is displayed to the user. As shown in Figure 1shown. In a step S4, the user selects, e.g., by marking with the mouse, a finding Bx1 contained in the finding report BB, which they would like to additionally illustrate visually for a viewer in the recipient chain of the finding report BB using a cinematic rendering short film KF. For or during the generation of the finding Bx1, the user N can optionally use image processing tools of a reporting application, e.g., to segment organs and / or blood vessels, detect landmarks, synchronize image data and heartbeats, or perform other preprocessing steps. The results of these preprocessing steps can preferably be comprehensibly recorded in the image data set BD1 by appropriate markings, coloring, or additional overlays.In a step S2, the user can now select a partial image series TB from the image dataset BDx1 based on the selected finding Bx1, which will be used to create the cinematic rendering short film KF. For example, this selection can be made by marking individual image elements of the image dataset BDx1 using the mouse. Alternatively, the user can define a partial image series TB by entering relative coordinates to the imaged body region of interest. Particularly with four-dimensional image data BD, a selection can also determine the start and end of the partial image series by specifying the exposure times associated with individual image elements. The partial image series can comprise the entire image dataset or just an arbitrary subset of individual image elements of the image dataset BD. In a step S5, a parameter set P for a cinematic rendering algorithm is determined.Preferably, the determination is made automatically by the computing unit 46. In particular, it can be provided that the parameter set P results from default settings which, depending on, for example, the cinematic rendering algorithm used, keywords contained in the finding Bx1, values of protocol parameters used during image data acquisition and / or image information included in the partial image series TB, e.g., a specific tissue type and / or a specific body region such as the head or thorax, according to a look-up table in the memory 36 of the computing unit 46. This is advantageous if the user has insufficient knowledge of cinematic rendering techniques and would be overwhelmed by an individual selection or adjustment of parameters. Alternatively and / or additionally, it can be provided that in this step the user adjusts at least one default parameter of the parameter set P using the input unit 50.Alternatively, the user can set all parameters of the parameter set directly themselves. In particular, this variant can allow the user to set parameters relating to a virtual light source or camera. For example, they can choose between a static light source within the rendered body region of interest, a static light source positioned at a fixed reference point outside the rendered body region of interest, and / or a light source whose position is tied to the position of the virtual camera. Furthermore, the user can set the following parameters themselves: transfer function, zooming, panning, clipping, etc.
[0052] Alternatively, a machine learning method or the like can be used to determine the parameter set for the cinematic rendering. In a further method step S6, a position for a direct access element DZEv for the cinematic rendering short film KF is determined. This step can also include specifying the design form of the direct access element DZEv. The direct access element DZEv can, for example, be designed as an overlay and / or marker in a timeline displayed parallel to the playback of the short film KF. It establishes direct access from the short film KF to the corresponding finding Bx1 of the findings report BB. More precisely, the direct access element DZEv links a specific location or specific individual video elements of the short film KF, comprising a particularly clear illustration of the anatomical situation underlying the finding Bx1 and / or the corresponding state of health, with the finding Bx1.This step can also be implemented either manually by the user, who, for example, marks the corresponding individual image elements in the selected partial image series that are to be assigned the direct access element DZEv during rendering. Alternatively, this step can be performed again using machine learning methods (see above) or similar, whereby various information from the image data and / or the findings and / or the measurement protocol underlying the image data and / or old cinematic rendering short films with direct access elements DZEv or their positions in the short film can be taken into account.
[0053] An embodiment of a cinematic rendering short film KF according to the invention including direct access element DZEv is shown in Figure 3Here, the direct access element DZEv is designed as an interactive marker bar above all individual video elements of the short film KF, which are displayed on a time scale T in parallel with the playback of the short film KF.
[0054] In a further step S7, the cinematic rendering takes place based on the selected partial image series TB and the determined parameter set. In other words, in step S7, the series of individual image elements of the partial image series TB is converted into a series of individual video elements of the cinematic rendering short film KF. The cinematic rendering short film KF can, for example, be archived as part of a DICOM study together with the image data set BDx1 in the image storage unit 20 of the system. Alternatively, the short film KF can be stored in the cloud, preferably anonymized. Alternative storage locations are also possible. In a further, optional step S8, the position and design of a direct access element DZEb is determined, which is to be included in the findings report BB, preferably near or within theis integrated into the selected clinical finding Bx1 and enables direct access from the findings report BB to the cinematic rendering short film KF, so that when the direct access element DZEb is activated, both pieces of information, i.e. the illustrative short film KF and the underlying findings Bx1, are available to a viewer in parallel. The direct access element DZEb can be designed as a partial path or a complete path depending on the storage location of the short film KF, for example, if the short film KF is stored locally within the system; however, the direct access element DZEb can also be designed as a web link if the short film KF has been saved in the cloud. In principle, any reference path can be integrated into the findings report BB. The direct access element DZEv is preferably stored in the document tree of a structured report, e.g. as a known DICOM attribute.This prevents the direct access element DZEv from being lost during archiving in the PACS or during conversion to an HL7 CDA format for inclusion in an electronic patient record in the findings storage unit 30.
[0055] Figure 4 shows a representation of a findings report BB according to the invention for a patient Pat X, including a direct access element DZEb, in one exemplary embodiment of the invention. The findings report comprises two findings Bx1 and Bx2, each provided with a direct access element DZEb for a respective cinematic rendering short film KF.
[0056] For training purposes and / or communication with physicians outside of radiology, it may be planned to integrate a known web link into a report BB that directly refers to the cinematic rendering short film KF associated with a report, for example in the following form: http: / / intranet.someClinic.org / referenceRenderings / Stream:UID _proceeding_02-12-17 / 0:06:40 / 0:42:32
[0057] It can be provided that the user N, in a further method step (not shown), enters via the input unit 50 where and / or in what form or format the generated cinematic rendering short film KF is to be saved.
[0058] The method according to the invention can be run through several times in an optional repeat loop IT, wherein for different findings Bx1, Bx2 of a finding report BB, the steps S2 to S8 can be run through based on the same image data set BD1 in order to generate several cinematic rendering short films KF.
[0059] These can be combined into a single video, with each direct access element DZEb from one of the findings Bx1, Bx2 then linking to the corresponding short film KF within the single video. Starting from the single video, a multitude of direct access elements DZEv can exist, each linking to different findings in one or more reports.
[0060] Challenging and complex clinical questions are commonplace in everyday clinical practice. These also place high demands on clear and unambiguous communication between radiology and other clinical units. Image data provided along with findings and corresponding markings or labels in the image data are often difficult or inadequate to understand or interpret for less experienced and / or trained personnel. A good and easily understandable presentation of supporting image information using the latest cinematic rendering techniques according to the present invention can remedy this situation.
[0061] A possible clinical application of the present invention is, for example, a radiology inquiry regarding a patient's ventricular dysfunction or impairment. Based on all available clinical information regarding the patient's condition, a suitable and, in particular, gentle treatment option must be determined from a multitude of possible therapeutic options. Among other options, invasive treatment procedures and medication may be considered. To determine the most suitable form of therapy, stress test results are also considered, particularly comprehensive statements regarding biomarkers such as natriuretic factor type B.A catchy and easily understandable representation of the patient's anatomical conditions through an inventive 3D cinematic rendering short film based on a 4D image data set can improve or simplify both the communication of the findings and the subsequent choice of therapy.
[0062] Another example use case is a radiology inquiry regarding a patient with an aneurysm, e.g., in the head or abdomen. A patient exhibits symptoms of blood congestion; here, too, there are a variety of different treatment options, such as endovascular closure of the affected blood vessel or surgical treatment. Using a 3D cinematic rendering short film according to the invention, changes or improvements in a health condition can be easily illustrated from image data sets acquired at different times and made available to a viewer outside of the radiology department.
[0063] In addition to creating a report, possibly including individual image elements with labels or markings to illustrate a finding, a radiologist can be given the option, via the software application used for the diagnosis, to convert the image information contained in the image data set into a 3D cinematic rendering short film and to forward this to the referrer along with the report.
[0064] The rendering function according to the invention can be designed as one of a plurality of tools of image display, image processing and / or diagnostic software in the form of an application with which a radiologist interacts during a diagnostic assessment. In particular, it can be provided that the radiologist can manually select or set rendering parameters such as the lighting model, zooming, panning, transfer function, camera position, camera trajectory, viewing angle or the like. Alternatively, the cinematic rendering function can be provided as an automatically running diagnostic process step that the radiologist can start or skip after selecting a partial image series. In this variant, the application generates at least one cinematic rendering short film for at least one finding using preset or automatically determined parameters.Particularly preferably, the rendering tool determines specific rendering settings or parameters based on a data analysis of existing, old cinematic rendering short films. In particular, if the reporting application uses reporting guidelines and / or templates for report findings, it can be provided that the reporting application instructs the radiologist or recommends to him for which finding a cinematic rendering short film should be created, depending on the type of finding. Particularly preferably, the execution of step S7 of the method according to the invention is outsourced to a central server in the form of a cloud-based service, so that the cinematic rendering functionality is not arranged locally on a client, e.g., a reporting workstation BA according to the invention, or in the computing unit 46.
[0065] Where not explicitly stated, but appropriate and in accordance with the spirit of the invention, individual embodiments, individual aspects, or features thereof may be combined or interchanged without departing from the scope of the present invention. Advantages of the invention described with reference to one embodiment also apply to other embodiments without explicit mention, where transferable.
Claims
1. Method for generating a cinematic rendering short film (KF), based on - a medical image data set (BDx1) comprising a series of images, wherein the image data set represents a body region of interest of an examination object (Pat X), and - a radiological findings report (BB), wherein the radiological findings report comprises at least one clinical finding (BDx1) relating to the body region of interest represented by means of the image data set, with the following steps: - selecting (S4) a clinical finding (Bx1) of the findings report, - selecting (S2) a partial image series (TB) of the image data set based on the selected clinical finding, - determining (S5) a parameter set (P) for a cinematic rendering algorithm, wherein the parameter set takes into consideration information regarding particular protocol parameters and / or anatomical structures within the mapped body region of interest, - generating (S7) a cinematic rendering short film by applying the cinematic rendering algorithm to the selected partial image series, using the parameter set, and - inserting (S8) a direct access element (DZEb), which allows direct access of a clinical finding of the findings report to the cinematic rendering short film, into the findings report.
2. Method according to claim 1, which also comprises the following step: - inserting (S6) a direct access element (DZEv), which allows direct access from the cinematic rendering short film to the clinical finding in the findings report, into the cinematic rendering short film.
3. Method according to one of claims 1 or 2, wherein a position of the direct access element in the cinematic rendering short film is chosen in such a manner that a representation of the direct access element correlates with a representation of the body region of interest underlying the selected clinical finding.
4. Method according to claim 3, wherein the position of the direct access element in the cinematic rendering short film is determined by means of a machine learning method, a statistical method or an artificial neural network.
5. Method according to one of the preceding claims, wherein the selecting of the clinical finding and / or the selecting of the partial image series takes place by way of a user (N).
6. Method according to one of the preceding claims, wherein the parameter set comprises at least one parameter from the group of the following cinematic rendering parameters: lighting model, zooming, panning, clipping, transfer function, camera position, camera trajectory.
7. Method according to one of the preceding claims, wherein the determining of the parameter set takes place while taking into consideration the selected clinical finding and / or the selected partial image series by means of a machine learning method, a statistical method or an artificial neural network.
8. Computing unit (46) for generating a cinematic rendering short film (KF), based on - a medical image data set (BDx1) comprising a series of images, wherein the image data set represents a body region of interest of an examination object (Pat X), and - a radiological findings report (BB), wherein the radiological findings report comprises at least one clinical finding (Bx1) relating to the body region of interest represented by means of the image data set, having means (36, 38, 40, 42, 44) for performing a method according to one of claims 1 to 7.
9. Computing unit according to claim 8, of which the means for performing the method are arranged at least partially on the server side.
10. Computer program with program code, in order to perform the method according to one of claims 1 to 7, when the computer program is executed on a computer.
11. Computer-readable data carrier with program code of a computer program, in order to perform the method according to one of claims 1 to 7, when the computer program is executed on a computer.
12. Diagnostic workstation (BA) for generating a cinematic rendering short film (KF), based on - a medical image data set (BDx1) comprising a series of images, wherein the image data set represents a body region of interest of an examination object (Pat X), and - a radiological findings report (BB), wherein the radiological findings report comprises at least one clinical finding (Bx1) relating to the body region of interest represented by means of the image data set, with a computing unit (46) according to claim 9.
