Methods and systems for providing an observable that indicates a medical diagnosis

Abstract

Computer-implemented methods and systems are provided for generating observables that indicate a medical diagnosis. These methods / systems implement several steps. One step involves obtaining a patient's image data series, where the image data series comprises multiple medical image records acquired at different times over a period of time. Another step involves extracting a time series from the medical image data series. A further step involves determining the observables based on the time series. Finally, a further step involves making the observables available.

Description

[0001] Embodiments of the invention relate to methods and systems configured to derive observables based on a time series analysis that can indicate a medical diagnosis. Furthermore, embodiments of the invention relate to methods and systems for deriving, selecting, and comparing or correlating such time series from raw data, such as medical image data or other patient data, to provide observables. Furthermore, embodiments of the invention relate to the use of the observables in the further clinical process, such as in the creation of a medical report, a (partially) automated medical diagnosis, or the identification of a treatment option, each based on the observables.

[0002] In medical diagnostics, temporal progression plays a crucial role. Often, not only the current value of a parameter is decisive, but also how this value has developed. For example, a shrinking lesion will have a completely different diagnostic indication than one that is growing or is being detected for the first time.

[0003] Such an analysis is complicated, however, by the fact that individual values ​​are often viewed in the context of other values. For example, a patient's weight loss may be considered non-critical depending on other values, or it may indicate a deterioration in the patient's condition. More generally, an episode of an elevated or decreased first value that coincides with an episode of an elevated or decreased second value may indicate a completely different disease pattern than one with a constant second value.

[0004] One problem in the practical implementation of this finding is that such information is often inaccessible, especially when evaluating medical image data. Therefore, the physician performing the evaluation is often limited to viewing image data sets acquired at different times side by side. Quantitative measurements are frequently not recorded. The inclusion of additional time series outside of radiology is often thwarted by the lack of access to such data and the limited time available for evaluation. In clinical routine, there is often simply not enough time to load individual documents from the patient file to check whether they contain information relevant to the evaluation.

[0005] A further problem arises from the fact that data from various medical disciplines, such as radiology, laboratory, or pathology, are largely collected and analyzed separately. While findings from the various disciplines are compiled for complex illnesses, this often takes the form of separate presentations on the various aspects. This means that the radiologist reports on the radiology findings, the pathologist on the pathology findings, and the internists on the patient's symptoms. This makes a systematic, i.e., quantitative, and possibly comparative, analysis of the temporal progression of individual values ​​difficult.

[0006] Against this background, it is a task to provide methods and devices that support a user in diagnosing a patient and, in particular, facilitate the consideration of temporal courses in clinical routine.

[0007] These and other objects are achieved by a method, a system, a computer program product, or a computer-readable storage medium according to the main claim and the subordinate claims. Advantageous further developments are specified in the dependent claims.

[0008] Below, the inventive solution to the problem is described both with respect to the claimed systems and with respect to the claimed methods. Features, advantages, or alternative embodiments / aspects mentioned here are also applicable to the other claimed subject matter, and vice versa. In other words, the subject claims (which, for example, are directed to a system) can also be developed with the features described or claimed in connection with a method. The corresponding functional features of the method can be implemented by corresponding subject modules.

[0009] According to one aspect, a computer-implemented method for providing an observable indicating a medical diagnosis is provided. The method comprises several steps. One step is directed to obtaining an image data series of a patient, wherein the image data series comprises multiple medical image data sets that were acquired over a period of time, each at different times. Another step is directed to extracting a time series from the medical image data series. Another step is directed to determining the observable based on the time series. Another step is directed to providing the observables.

[0010] Medical image datasets generally refer to datasets acquired using a medical imaging modality. A medical image dataset includes image data depicting a patient's body, body region, or body part. The medical image datasets in the image data series may each depict essentially the same body, body region, or body part, but may have been acquired at different times.

[0011] Image data can refer to medical image data with two or three spatial dimensions. The medical image dataset comprises image data, for example, in the form of a two- or three-dimensional array of pixels or voxels. Such arrays of pixels or voxels can be representative of color, intensity, absorption, or other parameters as a function of two- or three-dimensional position and can be obtained, for example, by appropriate processing of measurement signals obtained from a medical imaging modality.

[0012] Imaging modalities can include, for example, computed tomography (CT) devices, magnetic resonance imaging (MRI) devices, X-ray devices, ultrasound devices, and the like. Image data acquired with these or similar modalities is also referred to as radiology image data.

[0013] The medical image datasets may be formatted in a standard image format such as the Digital Imaging and Communications in Medicine (DICOM) format, for example, according to the DICOM PS3.1 2020c standard (or a later or earlier version of this standard). The medical image datasets may be stored in an electronic archive or storage facility, such as a picture archiving and communication system (PACS).

[0014] The time period may be predefined and may be, for example, weeks, months, or days. According to some examples, the time period may be predefined based on context information described herein. An image data series comprises at least two, preferably more than two, image data sets acquired within the time period.

[0015] According to some examples, obtaining the image data series can comprise retrieving the image data sets from an electronic storage device. In particular, this can comprise retrieving further image data sets based on an image data set to be diagnosed. The image data set to be diagnosed can be the most recent image data set in the image data series in terms of time. The further image data sets can, for example, be automatically selected from several image data sets of the patient in the storage device. According to some examples, this can be done with the proviso that the further image data sets "match" the image data set to be diagnosed in terms of content and / or time, e.g. by showing essentially the same body, the same body part or the same body part as the image data set to be diagnosed or by being taken within the time period.

[0016] The time series can comprise a series of measured values ​​that were recorded at different points in time. In particular, the measured values ​​in the time series can be of the same type. “Similar” can mean, in particular, that the measured values ​​are measured values ​​of the same measurand. In particular, the time series can be a time series of a measurand associated with the patient (or the patient’s image data series). In particular, the measurand can be a medical or physiological measurand, such as the size of an abnormality or the body composition of the patient. The measured values ​​are based on the image data sets or are taken from them. Each measured value in the time series can correspond to or be taken from one, in particular exactly one, image data set.

[0017] The measured values ​​of the first time series can be derived, in particular, from the image data of the image data sets. Extracting or retrieving the measured values ​​from the image data sets can be carried out, in particular, using image processing or image data analysis tools.

[0018] According to some examples, the step of extracting comprises applying an image data analysis function to the medical image data sets of the medical image data series, wherein the image data analysis function is configured to extract an image data-based measurement value from a medical image data set, and the time series comprises the image data-based measurement values.

[0019] According to some examples, the observable includes a temporal behavior of the first time series or is based on a temporal behavior of the first time series. In particular, the observable can be a measure of a temporal behavior of the first time series. In particular, the observable can include or be a measure of an increase or a decrease of the first time series. Optionally, the observable can include, be based on, or be a measure of a temporal behavior of the first time series relative to one or more other time series.

[0020] According to some examples, providing the observable includes outputting the observable to a user. The user can, for example, be a diagnosing physician, in particular a radiologist. Output can, for example, be performed via a user interface, in particular a graphical one.

[0021] Furthermore, providing the observable may include providing the observable for further processing. Further processing may include generating a result based on the observable that is usable by the user in the diagnostic process. For example, this may include a medical report or parts of one, or a recommended course of action.

[0022] By automatically extracting a time series from a series of medical image datasets, the user is initially relieved of the task of comparing individual image datasets and manually extracting comparative measurements. By subsequently determining the observables, an automated analysis of the time series is then performed to determine whether a medical diagnosis is indicated within the time series. This not only supports the user in diagnosing a patient but also facilitates the consideration of temporal processes in clinical routine.

[0023] According to one aspect, the step of determining the observables comprises obtaining a second time series, different from the time series, of a measured variable associated with the patient, which second time series covers a second time period that at least partially overlaps with the time period, wherein the determination of the observables is additionally carried out based on the second time series.

[0024] The second time series can, for example, be automatically extracted from an electronic patient record or from the patient's patient data. The patient data can comprise image data, in particular the image data series, and non-image data. Alternatively, the second time series can also be selected by a user from several available second selection time series and thus obtained. According to some examples, the available second time series can, in particular, be automatically extracted from the patient's patient data. Like the time series, the second time series can comprise a series of measured values ​​that were recorded at different points in time. In particular, the measured values ​​of the second time series can be of the same type. “Similar” can, in particular, mean that the measured values ​​are measured values ​​of the same measurand.

[0025] The measurement of the second time series may be different from the measurement of the time series.

[0026] By considering the second time series, different temporal developments can be examined in relation to one another. This allows for the identification of dependencies that are not accessible when considering individual measured variables in isolation.

[0027] According to some examples, the step of obtaining the second time series comprises extracting the second time series from the medical image data series.

[0028] The second time series then corresponds in its basic design to the time series, but can in particular be based on a different measured value. In particular, the explanations and definitions given herein for the time series then also apply to the second time series. In particular, extracting the second time series can comprise applying a further image data analysis function to the medical image data sets of the medical image data series, wherein the further image data analysis function (different from the image data analysis function) is designed to extract a further image data-based measured value (different from the image data-based measured value of the time series) from a medical image data set, and the time series comprises the further image data-based measured values.

[0029] This allows complementary information to be extracted and compared from the image data, which can improve the diagnostic significance of the observables.

[0030] According to some examples, the step of obtaining the second time series comprises extracting the second time series from a second medical image data series different from the medical image data series.

[0031] The second image data series may comprise multiple medical image data sets acquired at different times over the second time period.

[0032] According to some examples, the image data sets of the image data series may have been acquired by a first imaging modality, and the image data sets of the second image data series may have been acquired by a second imaging modality that differs in its imaging type from the first imaging modality. For example, the first imaging modality may be a magnetic resonance modality, and the second imaging modality may be a computed tomography modality.

[0033] By using different image data series, complementary information can be extracted and compared, which can improve the diagnostic significance of the observables.

[0034] According to one aspect, the step of determining the observables comprises: determining correlation information based on the first time series and the second time series, wherein the correlation information indicates a measure of a temporal correlation between the first time series and the second time series, and determining the observables based on the correlation information.

[0035] The correlation information can indicate the temporal relationship between the time series and the second time series. To evaluate the correlation information, known correlation functions can be applied to the time series and the second time series, for example.

[0036] According to some examples, the observable includes a correlation of the time series with the second time series, and in particular a correlation in the temporal behavior of the time series with the second time series, or is based on such a correlation. In particular, the observable can include a correlation or anticorrelation of the time series with the second time series, or be a measure thereof.

[0037] By evaluating correlation information, measured variables can be systematically analyzed in relation to one another. This allows for the quantification of dependencies that are not accessible when considering individual measured variables in isolation. This automatically provides the user with information to quickly arrive at an accurate diagnosis for the patient.

[0038] According to one aspect, the measure of the second time series is not based on medical image data.

[0039] In other words, the second time series was not extracted from image datasets. In particular, the second time series may have been extracted from patient datasets that are not image data.

[0040] Non-image data refers to patient data that is not image data. It can include, for example, one or more medical reports, e.g., from radiology, pathology, the laboratory, or other disciplines. Non-image data can include, particularly longitudinal data, one or more of the patient's medical values ​​and / or elements from the patient's medical history. This can include laboratory data, vital signs, and / or other patient-related measurements or examinations.

[0041] The non-image data may, for example, be contained in an electronic medical record (EMR) or electronic health record (EHR). The electronic medical record may be retrieved from one or more storage devices, which storage devices may be integrated into a medical information network. For example, storage devices may be queried for the non-image data or the patient's electronic health record. An electronic identifier, such as a patient ID or an access number, may be used for this purpose. Accordingly, the non-image data may be received from one or more of the available storage devices, each of which stores at least portions of the electronic health record.The storage devices can, for example, be part of medical information systems, such as hospital information systems and / or PACS systems and / or laboratory information systems, etc.

[0042] The electronic patient record can store all available information about the patient, especially non-image data, but also image data.

[0043] By considering a non-imaging-based measurement, additional information beyond the imaging diagnostics can be taken into account. The corresponding observable can thus correlate various temporal developments. This provides the user with a better basis for making a diagnosis without requiring significant additional effort.

[0044] In one aspect, obtaining the second time series comprises retrieving an electronic medical record of the patient and extracting the second time series from the electronic medical record.

[0045] According to one aspect, the method comprises extracting a plurality of time series from the image data series and / or obtaining a plurality of second time series, wherein in the step of determining, for a plurality of different combinations (pairs) of time series selected from the time series and / or the second time series, correlation information and an intermediate observable based thereon are determined, individual combinations (pairs) of time series are selected based on the intermediate observables, and the selected pairs are provided.

[0046] The selected combinations (pairs) can be displayed to the user, for example, by providing them (e.g., in a corresponding user interface). In particular, the selected combinations (pairs) can be displayed together with the associated intermediate observable to provide an explanation for the selection. A selection can, for example, include checking whether an intermediate observable exceeds a threshold. A selection can further include a comparison with an observable pattern that indicates a medically relevant fact.

[0047] By calculating correlations between various combinations of time series (which can be done in the background), a systematic search for anomalies relevant for diagnostics can be made. Such anomalies can then be automatically brought to the user's attention. This eliminates the need for the user to manually create and compare different time series; relevant correlations are automatically identified and displayed.

[0048] According to one aspect, the measured variable of the second time series comprises a laboratory value, in particular a blood value, of the patient.

[0049] By considering laboratory values, complementary data can be systematically incorporated into the imaging. This shows that (minor) abnormalities in imaging are often accompanied by changes in laboratory or blood values. While the isolated examination of individual values ​​often does not indicate disease, a comparison of these values ​​can certainly point to a significant condition.

[0050] According to some examples, the measurements include an oxygen saturation, an ECOG value, an inflammation value, an alkaline phosphatase (AP) level, and / or an erythrocyte level in the patient's blood.

[0051] Oxygen saturation provides information about breathing and the efficiency of oxygen transport, and thus about the patient's constitution. Poor oxygen saturation can indicate other problems the patient may have.

[0052] The ECOG score, or performance status score, describes the physical condition of cancer patients and is used to quantify their overall well-being and limitations in activities of daily living. Based on this score, disease progression can be assessed, appropriate treatment determined, and a prognosis can be established.

[0053] Inflammatory markers, such as C-reactive protein (CRP), provide information about inflammation in the body. Inflammatory markers are elevated in cases of infection, inflammation, and tissue damage. Alkaline phosphatase (ALP) testing serves as an indicator for diseases, such as liver and bile duct disease, tumors, or changes in bone metabolism.

[0054] Too many or too few red blood cells in the blood can also indicate certain diseases. For example, a low red blood cell count can indicate bone marrow disease, such as leukemia, or tumors.

[0055] The values ​​mentioned can therefore complement the imaging well and help to better assess the measured values ​​from the imaging.

[0056] According to some examples, the measure of the second time series includes a patient's medication.

[0057] Medication can include a type and amount of medication administered to the patient.

[0058] By taking medication into account, it is possible to assess, for example, whether a patient's physical condition, as seen in the imaging, is caused by medication or not.

[0059] According to some examples, the step of determining the observables comprises: obtaining a third time series of a further measurement variable associated with the patient, which third time series is different from the time series and the second time series, which third time series covers a third time period that at least partially overlaps with the first and second time periods, determining correlation information based on the first time series, the second time series and the third time series, wherein the correlation information indicates a measure of a temporal correlation between the first time series, the second time series and the third time series, and determining the observables based on the correlation information.

[0060] By including a third time series, additional information can be taken into account. In particular, the measured variable in the third time series can be a patient's medication. This allows, for example, a measured variable from imaging (e.g., a physical characteristic of the patient, such as fluid retention) to be correlated with a measured variable from the laboratory (such as an AP value) and a medication administration.

[0061] According to one aspect, the step of determining the observables comprises a step of obtaining an individual event associated with the patient which lies within the time period, a step of determining correlation information based on the time series and the individual event, wherein the correlation information indicates a measure of a temporal correlation between the time series and the individual event, and a step of determining the observables based on the correlation information.

[0062] According to one aspect, the step of determining the observables comprises a step of obtaining an individual event associated with the patient which lies within the second time period, a step of determining correlation information based on the second time series and the individual event, wherein the correlation information indicates a measure of a temporal correlation between the second time series and the individual event, and a step of determining the observables based on the correlation information.

[0063] An individual event may, for example, relate to a patient's therapy or treatment. In particular, the individual event may relate to radiotherapy, a surgical procedure, chemotherapy, and / or immunotherapy.

[0064] Furthermore, an individual event may relate to a patient's health condition. In particular, the individual event may include an accident, a seizure (such as a stroke), the onset of an inflammatory disease, and the like.

[0065] According to some examples, the step of obtaining the individual event may include extracting the individual event based on the time series in an electronic medical record or from the patient's patient data. In particular, this may include searching for individual events in an electronic medical record or from the patient's patient data based on the time period.

[0066] The correlation information can indicate the temporal relationship between the time series, and in particular a temporal behavior of the time series, and the individual event. To evaluate the correlation information, for example, a temporal behavior of the time series can be determined and related to extracted individual events.

[0067] The correlation information allows the time series to be related to possible individual events. This allows for automatic determination of whether a temporal development in the time series coincides with an individual event or is temporally related. This not only supports the user in diagnosing a patient but also facilitates the consideration of temporal developments in clinical routine.

[0068] According to one aspect, the method further comprises a step of obtaining context information relating to a clinical picture of the patient, wherein the step of extracting comprises selecting an image data-based measurement variable from a plurality of different image data-based measurement variables based on the context information, and the first time series comprises image data-based measurement values ​​of the selected measurement variable.

[0069] The context information can, in particular, specify framework conditions relevant to a user's diagnostic activity. Examples of framework conditions include: "Diagnosis of a chest CT scan after a trauma," "follow-up examination as part of cancer treatment of organ X," "analysis of an MRI scan of the lung," "confirmation of a suspected diagnosis Y," etc. The context information can be obtained, for example, based on a corresponding user input and / or the electronic patient record and / or an electronic worklist.

[0070] Based on the context information, a measurement can be automatically selected that is particularly relevant to the current case. For example, if a cancer patient is to be diagnosed based on the context information, the proportion of muscle tissue can be used as a measurement to provide information about the deterioration and progression of the disease. Likewise, the size of a lesion can be used as a measurement to indicate whether a treatment is working.

[0071] According to some examples, observables for initially unselected measurement variables can be determined in the background.

[0072] According to some examples, the step of extracting further comprises selecting an image data analysis function from a plurality of provided image data analysis functions, wherein the selected image data analysis function is configured to extract an image data-based measured value of the selected measurand from medical image data sets, and applying the selected image data analysis function to the medical image data sets of the medical image data series.

[0073] According to some examples, the step of obtaining the second time series comprises selecting a non-image data-based metric from a plurality of different non-image data-based metrics based on the context information, and extracting non-image data-based metrics of the selected metric from non-image data of the patient.

[0074] By selecting a metric based on context information, a metric that is particularly relevant to the current case can be automatically selected. For example, if a cancer patient is being diagnosed, the ALP value or the erythrocyte count can provide information about disease progression.

[0075] According to some examples, the step of obtaining the image data series of a patient comprises selecting a plurality of image data sets for the image data series from a plurality of available image data sets of the patient based on the context information.

[0076] According to some examples, obtaining the second time series includes retrieving an electronic medical record of the patient, selecting records from the patient record based on the context information, and extracting the second time series from the selected records.

[0077] This makes it possible, for example, to specifically select image data sets or data sets from the patient's file for screening a patient's lungs that depict the area of ​​the lung or contain relevant information.

[0078] According to some examples, the selection can be rule-based, e.g., always selecting a certain type of image dataset(s) for a context. The datasets can be, for example, individual documents, such as reports or tables. According to some examples, the selection can be based on metadata stored in the image datasets, such as DICOM tags, or on document titles or keywords in the documents.

[0079] According to one aspect, the medical image data sets each comprise an image of an entire body part or a whole-body image of the patient.

[0080] A body part can be an area of ​​the patient's body that includes multiple body parts or organs. For example, the area can be the patient's thorax, chest, or upper body.

[0081] By using image data sets that show an entire body part or even the entire body as a whole-body image, the time series can provide information about the patient's overall condition as determined from the imaging.

[0082] In particular, the measured variable in the time series can indicate the condition of a body part or the patient's entire body. This allows values ​​regarding the body condition to be quantified in the measured variable that are difficult for users to determine simply by visually examining the image data.

[0083] According to some examples, the time series measure is selected from: - an analysis value of a body composition, BCA value, in particular comprising a muscle percentage and / or a body fat percentage and / or a water percentage, - the size of a lesion, - a tumor burden in a part of the patient's body or in the patient's entire body, and / or - FDG distribution in a part of the patient's body or in the patient's entire body.

[0084] BCA stands for "Body Composition Analysis" and describes a process for extracting values ​​from medical image data that generally characterize a body's condition. These values ​​are therefore not focused on individual lesions, but on overall body properties. Examples include the composition of muscle and fatty tissue or the extent of water retention. Changes in these values ​​can indicate, for example, the progression of cancer, but are very difficult for the user to interpret from the image data. This is because human perception is trained to detect individual changes, such as lesions, but has more difficulty making proportional assessments. This is all the more true since the human body is a three-dimensional object.Consequently, the image datasets are also three-dimensional, making them difficult for a user to view in a two-dimensional screen. The inventors further recognized that correlation with blood values, such as the ALP or erythrocyte count, in particular, allows for a well-founded statement about the progression of a disease and represents a valuable observable.

[0085] To provide such BCA values, appropriately designed image analysis functions can be applied to the image data sets. In particular, such image analysis functions can be configured to provide a BCA value based on three-dimensional image data sets for the entire body or a body part.

[0086] Determining the size of a lesion across multiple image datasets is also challenging and time-consuming for the user. The lesion must be identified and measured by the user from image dataset to image dataset. An image analysis function capable of tracking and measuring lesions from image dataset to image dataset can relieve the user of this task.

[0087] A variety of well-known detection algorithms are available for this purpose.

[0088] Tumor burden indicates the extent to which a patient's body, or a specific part of the body, is affected by tumors. Again, this is a value that is difficult for the user to estimate or even quantify in its entirety.

[0089] For example, tumor burden can be determined by administering appropriate contrast agents in a PET-CT scan, followed by image analysis by a suitably trained image analysis function. PET-CT scanning is a combination of positron emission tomography (PET) and computed tomography (CT). PET can visualize metabolic processes in the body. For this purpose, tiny amounts of radioactively labeled substances are administered to the patient. The substances are distributed throughout the body and accumulate in certain tissues, such as tumors. FDG (F-18 deoxyglucose) can be used as a marker. FDG is a glucose molecule labeled with radioactive fluorine. Since cancer cells consume more glucose than healthy cells, FDG accumulates more rapidly in diseased cells.

[0090] The different distributions in the body's cells are visualized using a PET camera. Even cancerous lesions measuring just a few millimeters can be detected.

[0091] The PET-CT combination scanner allows for the almost simultaneous performance of a CT scan. Combining both techniques allows for the detection of cell regions with high glucose metabolism activity, thus providing information about tumor burden.

[0092] The inventors have recognized that, in particular, a correlation of tumor load or FDG values ​​with blood values ​​such as AP or erythrocyte values ​​enables a well-founded statement about the progression of a disease and represents a valuable observable.

[0093] According to some examples, the step of providing includes comparing the observables to a predetermined observable pattern, and displaying the time series and / or the observables based on the comparison.

[0094] If a correlation is performed with a second time series according to some aspects and examples described herein, the second time series can also be displayed together with the first.

[0095] The observable pattern can, for example, include one or more conditions for one or more observables, such as a threshold for an anticorrelation between two measured variables or for a decrease or increase in a measured variable. An observable pattern can include a combination of multiple conditions for different observables. The observable pattern can, for example, be considered fulfilled if all conditions are true.

[0096] The above selection can also be made when multiple time series or multiple second time series are present. In this case, only those time series or second time series are displayed for which the corresponding (intermediate) observable falls within an observable pattern (e.g., by being above a threshold).

[0097] Selective display allows users to be specifically alerted to relevant observables. The display can therefore serve as a warning to the user about a worsening of the clinical picture. At the same time, the user is only informed when there are actually abnormalities, which relieves the user's stress.

[0098] According to some examples, the method further comprises the steps of determining a medical diagnosis based on the observable, and providing the medical diagnosis.

[0099] A medical diagnosis can include a prediction of a medical finding, such as confirming or rejecting a suspected diagnosis. Furthermore, a medical diagnosis can indicate the probability of the presence of a medical finding.

[0100] According to some examples, determining the medical diagnosis may include selecting the medical diagnosis from a plurality of suspected diagnoses.

[0101] The multiple suspected diagnoses can, for example, be provided based on context information. They represent a preselection from which the diagnosis that best fits the case based on the observables can be selected. Each suspected diagnose can be associated with one or more indicators that indicate the suspected diagnosis. The diagnosis can thus be determined based on a comparison between the observables and the indicators.

[0102] If several observables are available (e.g. in the case of several (second) time series), all of them can be used to determine the medical diagnosis, or to select from several suspected diagnoses, or to compare them with the indicators.

[0103] By providing the diagnosis, the user is presented with a suggestion for a possible diagnosis based on the observable. In other words, a diagnosis is predicted. This effectively supports the user in diagnosing a patient without the need to manually create and evaluate time series.

[0104] According to some examples, providing the medical diagnosis includes generating a medical report based on (and / or using) the observable and / or the diagnosis.

[0105] For example, the diagnosis can be converted into a template for a medical report, optionally together with the observable and / or the time series(es) or the second time series(es). In particular, it can also be provided to select the template based on the observable (and / or diagnosis) from several available templates.

[0106] A medical report can be the result of a diagnostic process aimed at determining a patient's condition with regard to one or more clinical aspects based on the relevant medical data. A medical report can comprise a document. In particular, a medical report can comprise a structured document or structured document parts. Furthermore, a medical report can comprise an unstructured document or unstructured document parts. Furthermore, the medical report can be constructed from one or more templates that input patient-specific information during the diagnosis process.

[0107] By providing a medical report, the user can be provided with a processing result that can be used directly.

[0108] According to one aspect, the step of determining the medical diagnosis comprises selecting a disease pattern from a plurality of selected disease patterns based on the observable, querying an electronic health record of the patient from a database, comparing the selected disease pattern with entries in the electronic health record, and determining the medical diagnosis based on the matching.

[0109] A disease pattern can have one or more characteristics or indicators that indicate a disease. The characteristics or indicators can, for example, be value ranges for a measurand and / or an observable based on one or more measurands. The measurands can be an image-based measurand or other non-image-based measurands.

[0110] Selection disease patterns can be predefined. Furthermore, selection disease patterns can be preselected from a variety of available disease patterns based on context information.

[0111] Furthermore, selected disease patterns can be extracted from electronic compendia or textbooks. An example of such an electronic textbook is Thieme's eRef. A disease pattern can be extracted, for example, using a so-called Large Language Model (LLM), as described herein. Furthermore, selected disease patterns can be obtained based on disease patterns of comparison patients.

[0112] A comparison can involve searching for information in the electronic patient record and comparing it with the disease pattern. In particular, the electronic patient record can be searched for characteristics or indicators from the disease pattern. Furthermore, the electronic patient record can be searched for measured variables and / or observables, and the associated values ​​can be compared with value ranges in the disease pattern.

[0113] Using disease patterns allows for a more accurate diagnosis prediction, as it eliminates the need to rely solely on observables. This provides the user with reliable and easily usable output.

[0114] According to some examples, matching includes determining a lack of an observable and / or measure in the electronic health record compared to the disease pattern and, optionally, applying an analysis function to the electronic health record to provide the missing observable and / or measure and / or creating an examination protocol for an examination of the patient to provide the missing observable and / or measure.

[0115] The examination protocol may include general recommendations or instructions for the user or other personnel performing the examination. Furthermore, the examination protocol may include control commands for controlling a modality to perform the examination. The examination may, for example, be an imaging examination using an imaging modality.

[0116] By identifying missing measurements, a statement can be made about the reliability of the provided diagnosis. It also provides the user with information about how to further confirm the diagnosis. The optional automated application of an analysis function or the optional automated creation of an examination report can further reduce the user's workload, as the next steps to confirm the diagnosis are initiated automatically.

[0117] According to some examples, matching the selected disease pattern with the electronic medical record may be performed using a transformer network and, in particular, a large language model (LLM).

[0118] A transformer network is a neural network architecture that generally consists of an encoder, a decoder, or both an encoder and a decoder. In some cases, the encoders and / or decoders consist of several corresponding encoding and decoding layers. Within each encoding and decoding layer is a so-called attention mechanism.

[0119] The attention mechanism, sometimes referred to as self-attention, relates data items (e.g., words) within a set of data items to other data items within the set. For example, the self-attention mechanism allows the model to examine a group of words within a sentence and determine the relative importance that other groups of words within that sentence have to the examined group of words.

[0120] In particular, the encoder can be configured to convert the input (an element of the disease pattern or a document in the electronic medical record) into a numerical representation. The numerical representation can comprise one vector per input token (e.g., per word). The encoder can be configured to implement an attention mechanism so that each vector of a token is influenced by the other tokens in the input. This allows the tokens to be correlated and, for example, identified as relevant to the disease pattern even if there is no literal equivalent.

[0121] Furthermore, the transformer network may comprise a classification module or unit configured to map the output of the encoder or decoder to a set of learned outputs, such as an element of the disease pattern.

[0122] Training a transformer model can be performed in two stages based on a few examples: pre-training and fine-tuning. In the pre-training phase, a transformer network can be trained with a large data corpus to learn the underlying semantics of the problem. Such pre-trained transformer networks are available for various languages. For certain applications described herein, fine-tuning can be performed based on medical texts with verified (annotated) meanings and / or medical ontologies such as RADLEX and / or SNOMED. This allows the transformer network to learn typical relations and synonyms of medical terms.

[0123] For further information on transformer networks, please refer to the paper “Attention Is All You Need,” in arXiv: 1706.03762, June 12, 2017, the contents of which are incorporated herein by reference in their entirety.

[0124] An LLM can be based on a transformer network. An LLM is a language model characterized by its size. The artificial neural networks they are built with can contain tens of millions to billions of weights and are pre-trained using self-supervised and semi-supervised learning. Transformer networks contribute significantly to faster learning. Examples of LLMs that can be used for comparing disease patterns with the electronic health record include the GPT series (GPT-3.5 and GPT-4; used in ChatGPT), the LLaMa series, PaLM (used in Google Bard), BLOOM, Ernie 3.0 Titan, or Claude.

[0125] One advantage of transformer networks is that, thanks to their attention mechanism, they can efficiently account for long-range dependencies in the input data. Furthermore, encoders are capable of processing data in parallel, which saves computational resources. Furthermore, thanks to autoregression, decoders are able to iteratively generate a sequence of output values ​​with high confidence.

[0126] According to one aspect, the method comprises the steps of: comparing the observables with corresponding observables of a plurality of reference patients different from the patient, selecting a comparison patient from the plurality of reference patients based on the step of comparing, providing information about the comparison patient as comparison information.

[0127] According to some examples, the information about the comparison patient may include an electronic medical record of the comparison patient or an extract from the electronic medical record, such as a medical report.

[0128] Electronic patient records of the reference patients can, for example, be stored in a storage device of a medical information system. The electronic patient records can each contain the corresponding observable of the respective reference patient. According to some examples, corresponding observables can be obtained using a transformer network, in particular an LLM.

[0129] According to one aspect, the method further comprises a step of selecting a comparison patient from a plurality of reference patients based on the observable.

[0130] According to some examples, selecting the comparison patient may be based on the selected disease pattern. According to some examples, selecting may include comparing the selected disease pattern with the electronic medical records of each of the reference patients.

[0131] According to one aspect, the step of comparing comprises, for each corresponding observable, determining a similarity measure with the observable, wherein the step of selecting comprises selecting based on the similarity measures.

[0132] According to one aspect, providing the comparison information includes issuing a warning if a critical condition is indicated for the comparison patient.

[0133] Through the automated selection of comparison patients, similar cases / patients can be brought to the user's attention. The user can then use these similar cases as a guide when diagnosing and selecting treatment options. This can effectively support the user in diagnosing and treating the patient. According to some examples, the method further comprises the steps of determining a treatment option based on the observable and providing the treatment option.

[0134] According to some examples, the step of determining a treatment option comprises selecting a treatment option from a plurality of choice treatment options based on the observable.

[0135] According to some examples, the step of selecting the treatment option is additionally performed based on the context information and / or the selected disease pattern and / or the particular medical diagnosis.

[0136] According to some examples, the method further comprises pre-selecting the selection treatment option from a plurality of treatment options based on the context information and / or the selected disease pattern and / or the determined medical diagnosis.

[0137] According to some examples, the step of determining a treatment option further comprises obtaining an indication pattern for the selected treatment option, querying an electronic health record of the patient from a database, matching the indication pattern with entries in the electronic health record, and verifying the treatment option based on the matching.

[0138] An indication pattern may include signs and / or contraindications for a treatment option. The indication pattern may be associated with the selected treatment option, for example, based on one or more medical guidelines.

[0139] According to some examples, matching the indication pattern includes determining a lack of an observable and / or measure in the electronic health record compared to the indication pattern and, optionally, applying an analysis function to the electronic health record to provide the missing observable and / or measure and / or creating an examination protocol for an examination of the patient to provide the missing observable and / or measure.

[0140] According to one aspect, a system for providing an observable indicating a medical diagnosis is provided. The system comprises a computing device and an interface. The interface is configured to receive an image data series of a patient, wherein the image data series comprises a plurality of medical image data sets that were acquired over a period of time, each at different times. The computing device is configured to extract a time series from the medical image data series. Furthermore, the computing device is configured to determine the observables based on the time series. Furthermore, the computing device is configured to provide the observables via the interface.

[0141] The computing device can be configured as a back-end computing device. In particular, the computing device can be configured as a server system. The computing device can comprise a cluster or a group of computing devices and data storage devices. The back-end computing device can be connected to a front-end computing device via a medical information network, which front-end computing device hosts a user interface for the user. The computing device can be connected to several different (but especially similar) front-end computing devices via the medical network. The front-end computing device(s) can belong to a medical organization, such as a practice, a hospital, or a hospital network. The back-end computing device can also belong to the medical organization or be configured outside the medical organization.The back-end computing device may be in data communication with several different front-end computing devices, each belonging to different medical organizations, via the medical information network.

[0142] Alternatively, the computing device may be distributed across the back-end and front-end computing devices or may only include the front-end computing device.

[0143] According to embodiments, the medical information network can be based on the HL7 standard. Health Level 7 (HL7) is a group of international standards for the exchange of data between healthcare organizations and their computer systems. In particular, communication and / or data connections can be based on the FHIR standard. Fast Healthcare Interoperability Resources (FHIR) is a standard developed by HL7. It supports the exchange of data between healthcare software systems. By using the HL7 or FHIR standard, data can be transferred in a structured manner, eliminating the need for reformatting.

[0144] The interface can generally be designed for data exchange between the computing device and other components. The interface can be implemented in the form of one or more individual data interfaces, which can have a hardware and / or software interface, e.g. a PCI bus, a USB interface, a FireWire interface, a ZigBee or a Bluetooth interface. The interface can furthermore have an interface of a communications network, wherein the communications network can have a local area network (LAN), for example an intranet or a wide area network (WAN) or an internet. Accordingly, the one or more data interfaces can have a LAN interface or a wireless LAN interface (WLAN or Wi-Fi).

[0145] Furthermore, the system can have a data storage unit connected to the interface, in which the image data sets and / or electronic patient records and / or analysis functions and / or image data analysis functions and / or a transformer network / LLM are stored and can be retrieved and thus made available via the interface. The data storage unit can be configured as a centralized or decentralized storage unit or cloud storage.

[0146] Furthermore, the system may include one or more imaging modalities, such as a computed tomography system, a magnetic resonance system, an angiography system, an x-ray system, a positron emission tomography system, a mammography system, and / or the like.

[0147] The advantages of the proposed systems essentially correspond to the advantages of the proposed methods. Features, advantages, or alternative embodiments / aspects of the methods can also be transferred to the other claimed subject matter, and vice versa.

[0148] In a further aspect, the invention relates to a computer program product which comprises a program and is loadable directly into a memory of a programmable controller and has program means, e.g. libraries and auxiliary functions, in order to execute a method for providing an observable indicating a medical diagnosis, in particular according to the aforementioned embodiments / aspects, when the computer program product is executed.

[0149] Furthermore, in a further aspect, the invention relates to a computer-readable storage medium on which readable and executable program sections are stored in order to carry out all steps of a method for providing an observable indicating a medical diagnosis according to the aforementioned embodiments / aspects when the program sections are executed by the controller.

[0150] The computer program products can comprise software with source code that still needs to be compiled and linked, or that only needs to be interpreted, or executable software code that only needs to be loaded into the processing unit for execution. The computer program products enable the methods to be executed quickly, identically repeatably, and robustly. The computer program products are configured such that they can execute the method steps according to the invention using the computing unit. The computing unit must have the prerequisites, such as appropriate RAM, a suitable processor, a suitable graphics card, or a suitable logic unit, so that the respective method steps can be executed efficiently.

[0151] The computer program products are stored, for example, on a computer-readable storage medium or on a network or server, from where they can be loaded into the processor of the respective computing unit, which can be directly connected to the computing unit or formed as part of the computing unit. Furthermore, control information of the computer program products can be stored on a computer-readable storage medium. The control information of the computer-readable storage medium can be designed such that it carries out a method according to the invention when the data carrier is used in a computing unit. Examples of computer-readable storage media are a DVD, a magnetic tape, or a USB stick on which electronically readable control information, in particular software, is stored.If this control information is read from the data carrier and stored in a computing unit, all inventive embodiments / aspects of the methods described above can be implemented. Thus, the invention can also be based on said computer-readable medium and / or said computer-readable storage medium. The advantages of the proposed computer program products or the associated computer-readable media essentially correspond to the advantages of the proposed methods.

[0152] Further features and advantages of the invention will become apparent from the following explanations of exemplary embodiments based on schematic drawings. Modifications mentioned in this context can be combined with one another to form new embodiments. In different figures, the same reference numerals are used for the same features.

[0153] They show: Fig. 1 is a schematic representation of an embodiment of a system for providing an observable indicating a medical diagnosis, Fig. 2 is a flowchart of a method for providing an observable indicating a medical diagnosis according to an embodiment, Fig. 3 is a diagram of optional method steps in a method for providing an observable indicating a medical diagnosis according to one embodiment, Fig. 4 is a data flow diagram of a method for providing an observable indicating a medical diagnosis according to one embodiment, Fig. 5 shows a schematic representation of a graphical user interface in a method for providing an observable indicating a medical diagnosis, Fig. 6 is a diagram of optional method steps in a method for providing an observable indicating a medical diagnosis according to one embodiment, Fig. 7 is a diagram of optional method steps in a method for providing an observable indicating a medical diagnosis according to one embodiment, Fig. 8 is a diagram of optional method steps in a method for providing an observable indicating a medical diagnosis according to one embodiment, and Fig. 9 is a diagram of optional method steps in a method for providing an observable indicating a medical diagnosis according to one embodiment.

[0154] In Fig. 1 illustrates a system 1 for providing an observable OBS indicating a medical diagnosis according to one embodiment. The system 1 comprises at least one front-end computing device 10 with a user interface (or simply "user interface 10"), a medical information system 40, and a back-end computing device 20 (or simply "computing device 20"), which are communicatively connected to one another via a medical network or a data interface 26.

[0155] The medical image data sets SET and further information, such as electronic patient records EPA, can be provided to the front-end computing device 10 via suitable interfaces 26 from the medical information system 40. Typically, a Fig. The system shown in Figure 1 comprises a plurality of front-end computing devices 10, all of which access the same medical information system 40 or exchange data with the back-end computing device 20. In the embodiment shown, the back-end computing device 20, the medical information system 40, and the front-end computing device(s) 10 are part of the same medical organization. A medical organization can be, for example, a practice, a practice network, a hospital, or a hospital network. Accordingly, the network connecting these components via the interface 26 can be designed as an internal network of the organization and, for example, comprise an intranet (such as a local area network and / or a wireless local area network).

[0156] The front-end computing device 10 can be configured, for example, as a diagnosis station or diagnosis workstation, at which a user N can view and analyze patient data from an electronic patient record (EPR) or a medical image dataset (SET), as well as create, review, modify, assess, and determine medical diagnoses (DIG) and treatment options (BHO). The front-end computing device 10 can therefore also be referred to as a user interface 10. The front-end computing device 10 can have a physical user interface, for example, comprising a display and / or an input device. The front-end computing device 10 can have a processor.The processor may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an image processing processor, an integrated (digital or analog) circuit, or combinations of the aforementioned components, and other devices for providing an observable OBS, a diagnostic DIG, and / or a treatment option BHO according to embodiments. The front-end computing device 10 may include, for example, a desktop PC, laptop, or tablet.

[0157] The medical information system 40 can generally be configured to capture and / or store and / or forward medical image data sets SET and electronic patient records EPA. For example, the medical information system 40 can have one or more databases (not shown). In particular, the databases can be implemented in the form of one or more cloud storage modules. Alternatively, the databases can be implemented as local or distributed storage, e.g., as a PACS (Picture Archiving and Communication System), a hospital information system (HIS), a laboratory information system (LIS), an electronic medical record (EMR) information system, and / or other medical information systems. According to some examples, the medical information system 40 can also include one or more medical imaging modalities (not shown), such as, for example,a computed tomography system, a magnetic resonance system, an angiography system, a C-arm X-ray system, a positron emission tomography system, an X-ray system, or the like.

[0158] The medical image dataset SET may comprise medical image data. Image data in this context may refer to medical image data with two or three spatial dimensions. The image data may, for example, have been generated using a medical imaging modality, such as an X-ray, computed tomography, magnetic resonance imaging, positron emission tomography, or angiography device, or other devices. Such image data may also be referred to as radiology image data.

[0159] The image data contained in the medical image dataset SET can, for example, be formatted according to the DICOM format. DICOM (=Digital Imaging and Communications in Medicine) is an open standard for the communication and administration of medical image data and related data.

[0160] A patient’s image data sets (SET) can be contained in the patient’s electronic health record (EPA).

[0161] In addition to medical image datasets (SET), the electronic patient record (EPA) can also contain non-image data. Non-image data can be, for example, examination results that are not based on medical imaging. This can include laboratory data, vital signs, spirometry data, or the protocols of neurological examinations. In addition, non-image data can include text datasets, such as structured and unstructured medical reports. Non-image data can also be patient-related data. This can include, for example, demographic information about the patient, such as their age, gender, or body weight. The non-image data can be integrated into the image data, for example, as metadata. Alternatively or additionally, the non-image data can also be stored in separate documents.

[0162] The back-end computing device 20 may include a processor. The processor may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an image processing processor, an integrated (digital or analog) circuit, or combinations of the aforementioned components, and other devices for providing an observable OBS according to embodiments. The back-end computing device 20 may be implemented as a single component or include a group of computers, such as a cluster. Such a system may be called a server system. Depending on the embodiment, the back-end computing device 20 may be configured as a local server. Furthermore, the back-end computing device 20 may include a working memory, such as a RAM, for temporarily storing, for example, image data sets SET, electronic patient records EPA, or time series ZR1, ZR2. The back-end computing device 20 is, for example,by computer-readable instructions, by design and / or hardware such that it can perform one or more method steps according to embodiments of the present invention.

[0163] The back-end computing device 20 can be connected to the front-end computing device 10 and / or the storage device RD and / or the medical information system 40 via the interface 26. Via this interface 26, the back-end computing device 20 can receive medical image data sets SET, electronic patient records EPA, and / or user inputs INP, on the basis of which time series ZR1, ZR2 can be automatically selected and observables OBS can be created and provided with computer support.

[0164] To provide an Observable OBS, the back-end computing device 20 may have various modules.

[0165] Module 21 is configured as a data retrieval module. It can be configured to access the medical information system 40 and search for medical image data sets (SET) or electronic patient records (EPA). In particular, module 21 can be configured to formulate search queries and parse them to the medical information system 40.

[0166] Module 22 can be configured as a user interaction module or unit. Module 22 can be configured to provide the user with N time series ZR1, ZR2 and observables OBS, as well as diagnoses DIG and / or treatment options BHO based thereon. Furthermore, module 22 can be configured to capture one or more user inputs INP and provide them for processing in back-end computing device 20. Such user inputs INP can include, for example, speech, gestures, eye movements, handling of input devices such as a computer mouse, etc. The user inputs INP can be directed toward an interaction with the medical image data set SET.

[0167] Module 23 can be understood as an analysis module. Module 23 is designed to extract the time series ZR1, ZR2 and to analyze them. For example, module 23 can be designed to extract time series ZR1 from image data sets SET. Such time series ZR1 are time series of an image data-based measured variable BD-MG. Furthermore, module 23 can be designed to extract time series ZR2 from electronic patient records EPA that relate to a non-image data-based measured variable NBD-MG. Module 23 can be designed to execute corresponding analysis functions AF. For example, one or more such analysis functions AF can be designed to extract measured values ​​for an image data-based measured variable BD-MG from image data sets SET by image data processing (image data analysis function).Furthermore, one or more analysis functions AF can be configured to extract electronic patient record EPA measurement values ​​to a non-image data-based measurement variable NBP-MG.

[0168] To analyze the extracted time series ZR1, ZR2 and to provide the observable OBS, the module 23 can further be configured to execute a time series analysis function ZR-AF. The time series analysis function ZR-AF is then configured to determine a temporal development of one or more time series ZR1, ZR2. In particular, the time series analysis function ZR-AF can comprise a correlation algorithm configured to determine correlation information KOR between two time series ZR1, ZR2, on the basis of which the observable OBS can be provided.

[0169] Module 24 can be understood as a follow-up module or LLM module. It can be configured to convert the observable OBS provided by module 23 into usable results for user N. For example, module 24 can be configured to provide the observable OBS as text in natural language. Furthermore, module 24 can be configured to provide a diagnosis (DIG) and / or a treatment option (BHO) based on the observable OBS—optionally with further comparison with the electronic patient record (EPA). Furthermore, module 24 can be configured to search the medical information system 40 for comparison patients who have a similar case constellation to the patient based on the observable OBS.

[0170] Module 24 can be configured to host and execute a transformer network or large language model (LLM). The LLM can be used to translate the observable OBS into natural language and / or perform a comparison with electronic health records (EHRs).

[0171] The division of the back-end computing device 20 into modules 21-24 serves merely to simplify the explanation of the functionality of the back-end computing device 20 and is not to be understood as limiting. The modules 21-24 or their functions can also be combined into a single element. The modules 21-24 can in particular also be understood as computer program products or computer program segments, which, when executed in the back-end computing device 20, implement one or more of the method steps described herein. In Fig. Figure 2 shows a schematic flow diagram of a method for providing an observable OBS. The order of the method steps is not restricted by the illustrated sequence or the chosen numbering. Thus, the order of the steps can be reversed if necessary, and individual steps can be omitted. Furthermore, one or more steps, in particular a sequence of steps, and optionally the entire method, can be executed repeatedly. Fig. 4 shows a corresponding diagram, which is comparable to the Fig. 2. Unless otherwise stated, optional steps and data streams in the Fig. 2 and Fig. 4 shown in dashed lines.

[0172] In an optional step S0, context information AI can first be obtained. The context information AI comprises a medical context for the patient to be diagnosed. This can, for example, be a diagnosis task that the user N has to complete based on the available patient data. Additionally or alternatively, the context information AI can include an indication of the patient's clinical stage. Additionally or alternatively, the context information AI can include an indication of a possible disease or diagnosis of the patient.

[0173] The context information AI can, for example, be derived from the patient's electronic health record (EPA). Additionally or alternatively, the context information AI can take into account user inputs via the user interface 10.

[0174] In step S10, an image data series SE of the patient is obtained. The image data series SE comprises several individual image data sets SET of the patient, which were acquired at different times. The image data sets SET can be loaded in step S10, for example, from the medical information system 40. The context information AI can optionally be taken into account, so that only those image data sets SET that correspond to the context information AI are loaded. The loading of the image data series SE can be triggered, for example, by a manual selection of the respective case / patient by the user N via the user interface 10.

[0175] In step S20, one or more time series ZR1 are extracted from the image data sets SET. These time series ZR1 are also called first time series ZR1 or image data-based time series ZR1. The time series ZR1 comprise a longitudinal series of measured values ​​for a measured variable BD-MG. The measured variables BD-BG can be extracted from the image data sets in step S20, for example, by applying a corresponding image analysis function AF to the image data sets SET.

[0176] The BP-MG measurement(s) can be predefined. Alternatively, BP-MG measurement(s) can be selected specifically for the respective patient based on several possible BP-MG measurements. This can be done, for example, based on the context information (AI).

[0177] The first time series ZR1 generally contain measured values ​​that can be extracted from medical image data sets (SET). Examples include BCA values ​​such as body fat percentage, muscle percentage, etc., as they result from images of the patient's body.

[0178] In step S30, an observable OBS is determined from the first time series(es) ZR1 that is relevant for a diagnosis or treatment of the patient or indicates a diagnosis or treatment. In particular, the observable OBS can comprise a temporal development of one or more time series ZR1, ZR2. For example, the observable OBS can comprise a temporal derivative of a time series ZR1, ZR2.

[0179] In step S40, the observable OBS is provided. This may include displaying the observable OBS in the user interface 10. Optionally, underlying time series ZR1, ZR2 may be displayed.

[0180] Furthermore, the provision in step S40 may include forwarding the observables for optional further processing. Some possible further processing steps are listed below with reference to steps S50-S100. The steps are optional and may follow step S40 individually or in any combination.

[0181] In the optional step S50, a medical diagnosis DIG can be predicted based on the observable OBS. For example, a temporal development of a time series ZR1, ZR2 (either alone or in relation to other time series ZR1, ZR2) can indicate a medical diagnosis DIG. Optionally, in step S50, further information such as context information AI or information from the patient's electronic patient record (EPR) can be considered when predicting the medical diagnosis DIG.

[0182] The prediction of the medical diagnosis DIG can be rule-based. Alternatively, the prediction can be determined in step S50 by applying a trained algorithm to the observable OBS, optionally the context information AI, and / or the electronic patient record (EPR).

[0183] In step S60, the diagnosis DIG is provided. This may include displaying the diagnosis DIG in the user interface 10. Optionally, underlying time series ZR1, ZR2, and other decision-relevant information may be displayed.

[0184] In step S70, one or more comparison patients who exhibit a medical similarity to the patient can be determined based on the observable OBS. For this purpose, several reference patients can be provided in step S70, from which comparison patients can be selected based on the observable OBS who exhibit increased similarity to the patient (based on the observable OBS).

[0185] In addition, further information such as context information (AI) or information from the patient's / reference patient's electronic health records (EPA) can also be taken into account when selecting comparison patients.

[0186] In step S80, information VI relating to the comparison patients can then be provided (also referred to as comparison information VI). For example, comparison information VI can be extracted from the electronic patient records (EPA) of the comparison patients. The comparison information VI can include, for example, a diagnosis, a treatment performed, a course of illness, and the like. Alternatively, the comparison information VI can include the respective electronic patient records (EPA) of the comparison patients or parts thereof.

[0187] The comparison information VI can be provided to the user N by displaying it in the user interface 10. For example, the user N can be provided with a link to the electronic patient records (EPA) of the comparison patients.

[0188] In step S90, a treatment option BHO for the patient can be determined based on the observable OBS. For example, a temporal development of a time series ZR1, ZR2 can indicate a treatment option BHO. Optionally, in step S90, further information such as the medical diagnosis DIG, the context information AI, or information from the patient's electronic patient record EPA can be considered when determining the treatment option BHO. Similar to the prediction of the medical diagnosis DIG, the treatment option BHO can be determined rule-based or using a trained algorithm.

[0189] In step S100, the treatment option BHO is provided. This may include displaying the treatment option BHO in the user interface 10. Optionally, underlying time series ZR1, ZR2, and other decision-relevant information may be displayed.

[0190] In Fig. Figure 3 shows a schematic flow diagram of optional substeps for determining an observable OBS. The order of the process steps is not limited by the sequence shown or by the chosen numbering. Thus, the order of the steps can be reversed if necessary, and individual steps can be omitted. Furthermore, one or more steps, in particular a sequence of steps, can be executed repeatedly. Fig. 3 can be executed within step S30. In Fig. 4 shows a corresponding diagram, which is comparable to the Fig. 3 shows an example of the data streams associated with the method shown.

[0191] For some diseases, it may be important to consider time series ZR1, ZR2 not only individually, but also in relation to each other. Fig. The embodiment shown in Figure 3 deals with this aspect.

[0192] First, in step S31, one or more second time series ZR2 are obtained that are different from the (first) time series ZR1, but at least partially overlap in time with the first time series ZR1. The second time series ZR2 can also be based on image data. Alternatively, the second time series ZR2 can not be based on image data. For example, the measured variables NBP-MG reproduced in the second time series ZR2 can be based on laboratory and, in particular, blood values ​​of the patient. Such second time series ZR2 can, for example, be taken from the patient's electronic patient record (EPA). Which second time series ZR2 are to be provided in step S31 can be determined, for example, based on the context information AI.

[0193] According to some examples, the second time series ZR2 may also comprise only a single event. The single event may, for example, relate to a therapy and / or a patient's health condition.

[0194] In step S32, a first time series ZR1 is mathematically correlated with a second time series ZR2 to determine correlation information KOR. Selected first and second time series ZR1, ZR2 can be correlated with each other, whereby a selection from several first and second time series ZR1, ZR2 can be made, for example, based on the context information AI.

[0195] Alternatively, all possible combinations of the first and second time series ZR1, ZR2 can first be correlated to obtain corresponding correlation information KOR or Observable OBS. Based on the correlation information KOR, relevant combinations of time series ZR1, ZR2 can then be identified and provided.

[0196] One or more observables (OBS) can then be determined from the correlation information (KOR). This can, for example, involve selecting medically significant combinations of time series (ZR1, ZR2) based on the correlation information (KOR). These can be time series combinations that exhibit striking temporal correlations.

[0197] Furthermore, step S33 can comprise transferring correlation information KOR into a format that user N can easily interpret, which is then provided to user N as an observable OBS. For example, the correlation information KOR itself can be transferred back into a time series, which can be displayed together with the underlying time series ZR1, ZR2. Furthermore, an observable OBS determined based on the correlation information KOR can comprise a text that describes the temporal dependencies encoded in the correlation information KOR for user N (e.g., "decrease in muscle mass with simultaneous increase in AP values ​​in the past two months"). Such a text can be provided, for example, with an LLM.

[0198] In Fig. Figure 5 illustrates a graphical user interface (GUI) according to one embodiment. The graphical user interface (GUI) may be implemented, for example, in user interface 10. The graphical user interface (GUI) may provide user N with one or more results of the processing described herein.

[0199] Time series ZR1 and ZR2 can be displayed in a section of the graphical user interface (GUI). Time series ZR1 and ZR2 can be automatically selected for display, for example, based on the observable OBS. Alternatively, time series ZR1 and ZR2 can be selected by user N.

[0200] In Fig. The example shown in Figure 5 shows a time series ZR1 based on an image data-based BP-MG measurement. A BCA value is shown as an example, which can include individual BP-MG measurements relating to the patient's body composition, such as body fat percentage or muscle percentage, or a combination of such BP-MG measurements. Each measurement can correspond to an imaging examination (e.g., a whole-body CT scan).

[0201] Furthermore, Fig. Figure 5 shows three (second) time series ZR2, which are based on a non-image data-based measurement variable, NBP-MG. In this case, these are the patient's AP, erythrocyte, and MCV values, as obtained, for example, from the patient's blood tests. Individual measured values ​​can refer to the patient's blood tests. The MCV value stands for "mean corpuscular volume." This blood value indicates the average volume of a single red blood cell (erythrocyte). This value is important, for example, for determining the cause of anemia. It is determined as part of the blood test, but is not very meaningful on its own.

[0202] Furthermore, in the section of time series ZR1 and ZR2, a correlation between the time series ZR1 and ZR2 is shown as an observable OBS. Initially, the time series exhibit a similarly constant pattern. From a certain point in time, a decrease in the BCA value and an increase in the AP and MCV values ​​can be observed. This is evident in a decrease in the observable OBS, which indicates this anticorrelation.

[0203] Additionally or alternatively, the observable OBS can be provided as text output. This is displayed in a lower left section of the graphical user interface (GUI). The text output may be more easily accessible to user N than a correlation plot. Furthermore, this text output can be directly integrated into a report. In addition, possible diagnoses (DIG) and / or treatment options (BHO) can be displayed in other sections of the graphical user interface.

[0204] In Fig. Figure 6 shows a schematic flow diagram of optional sub-steps for providing / displaying an Observable OBS. The order of the process steps is not limited by the sequence shown or the chosen numbering. Thus, the order of the steps can be reversed if necessary, and individual steps can be omitted. Furthermore, one or more steps, in particular a sequence of steps, can be executed repeatedly. Fig. The substeps shown in Figure 6 can be executed within step S40.

[0205] Depending on the available data, a variety of observable OBSs are possible. In particular, multiple time series ZR1, ZR2 open up a variety of possible combinations for determining correlation information KOR. To assist user N in locating relevant observable OBSs or time series ZR1, ZR2, step S41 can provide for a selection based on the observable OBSs.

[0206] For this purpose, the observable OBS can be compared, for example, with a threshold value or a specific pattern of observables. This allows us to check whether the observable OBS indicate, for example, a medically relevant condition. Based on the comparison, the observable OBS and thus the associated time series ZR1, ZR2 can be selected.

[0207] Based on the selection in step S41, the selected observables OBS and / or time series ZR1, ZR2 can be displayed in step S42 (cf. Fig. 5).

[0208] In Fig. Figure 7 shows a schematic flow diagram of optional sub-steps for determining a medical diagnosis (DIG). The sequence of the process steps is not limited by the sequence shown or the selected numbering. Thus, the order of the steps can be reversed if necessary, and individual steps can be omitted. Furthermore, one or more steps, in particular a sequence of steps, can be performed repeatedly. Fig. The substeps shown in Figure 7 can be executed within step S50.

[0209] To determine a medical diagnosis DIG, a disease pattern can first be identified in step S51. For this purpose, the disease pattern that corresponds to the observable OBS(s) can be selected from several disease patterns. In particular, the disease pattern that is most likely based on the observable OBS can be selected.

[0210] The multiple disease patterns can be preselected from a plurality of predefined disease patterns based on the context information AI. A disease pattern can generally include signs and countersigns for a specific disease and thus a diagnosis. At least one of these signs can be an observable OBS described herein.

[0211] In step S52, an electronic patient record (EPR) of the patient is retrieved from the medical information system 40. This can be done, for example, using an electronic patient identifier such as a patient ID with which the patient is registered in the medical information system 40.

[0212] In step S53, the disease pattern can then be compared with the electronic patient record. This can be checked to determine the extent to which the signs and countersigns of the disease pattern are reflected in the electronic patient record. According to some examples, this can be done using a transformer network or LLM. This can also identify signs and countersigns in the electronic patient record that have no direct (literal) equivalent in the electronic patient record (EHR). In step S54, a medical diagnosis (DIG) can be determined based on the comparison. If, for example, it can be verified in step S53 that the electronic patient record (EHR) reflects the signs and countersigns of the disease pattern, it can be concluded that the disease underlying the disease pattern is present. This can be provided as a DIG diagnosis.Conversely, if the signs or contraindications are absent, the disease can be ruled out. This can also be broadly interpreted as a medical diagnosis of DIG.

[0213] Optionally, step S54 can be followed by steps S51 and S53 to compare further disease patterns with the electronic patient record (EPA). In step S51, the next most probable disease pattern can be selected based on the observable OBS.

[0214] In Fig. Figure 8 shows a schematic flow diagram of optional sub-steps for identifying comparison patients. The sequence of the process steps is not limited by the sequence shown or by the chosen numbering. Thus, the order of the steps can be reversed if necessary, and individual steps can be omitted. Furthermore, one or more steps, in particular a sequence of steps, can be performed repeatedly. Fig. The substeps shown in Figure 8 can be executed within step S70.

[0215] Step S70 deals with determining comparison patients from a plurality of reference patients. The reference patients can each be registered in the medical information system 40. An electronic patient record (EPR) can be created for each reference patient in the medical information system 40. The electronic patient records (EPR) can each already contain observables that correspond to the observables (OBS) determined for the patient. Alternatively, such corresponding observables can be generated for the reference patients using processing similar to that used for the patient.

[0216] In step S71, a comparison of the observable OBSs with corresponding observables of the reference patients can be performed. One or more observable OBSs can be used for the comparison. Optionally, the reference patients can be preselected from a larger group of patients registered in the medical information system 40 based on the context information AI.

[0217] In step S72, based on step S71, those reference patients can then be identified as comparison patients who exhibit a certain similarity to the patient based on the observable OBS. These can, in particular, be reference patients who exhibit similar corresponding observables as the observable OBS.

[0218] The comparison patients can be brought to the attention of user N. Additionally, step S73 can provide for determining information about the selected comparison patients (comparison information VI). The comparison information VI can be extracted from the respective electronic patient records (EPA) of the comparison patients. The comparison information VI can, for example, include a diagnosis of DIG or a treatment option of BHO. In addition, the comparison information VI can include one or more findings reports of the comparison patients.

[0219] In Fig. Figure 9 shows a schematic flow diagram of optional sub-steps for providing / displaying a BHO treatment option. The sequence of the process steps is not limited by the sequence shown or the selected numbering. Thus, the order of the steps can be reversed if necessary, and individual steps can be omitted. Furthermore, one or more steps, in particular a sequence of steps, can be performed repeatedly. The Fig. The substeps shown in Figure 9 can be executed within step S90.

[0220] First, in step S91, a disease pattern can be selected based on the observable OBS(s). In principle, the procedure can be the same as in step S51. In this embodiment, the disease patterns are each associated with at least one BHO treatment option. The BHO treatment options can be specified, for example, based on a medical guideline for the disease underlying the disease pattern. The BHO treatment options can each be associated with further indicators that indicate whether the BHO treatment options are suitable for the respective patient. Such indicators can include, for example, allergies to certain medications or an age restriction, and the like.

[0221] In step S92, an electronic patient record (EPA) of the patient can be retrieved from the medical information system 40. In principle, the procedure can be the same as in step S52.

[0222] In step S93, the disease pattern is compared with the electronic patient record (EPA). This can generally be done in the same way as in step S53.

[0223] Based on the comparison in step S93, a BHO treatment option can be determined in step S94. If, for example, it can be verified in step S93 that the disease pattern is present, the corresponding BHO treatment option can be provided. To ensure the BHO treatment option, additional information about the patient can be automatically requested (e.g., from user N) for signs or countersigns of the disease pattern or indicators of the BHO treatment option for which no information is found in the electronic patient record (EPA), and further examinations of the patient can be suggested or initiated in the medical information system 40.

[0224] While embodiments have been described in detail, particularly with reference to the figures, it should be noted that a multitude of modifications are possible. Furthermore, it should be noted that the exemplary embodiments are merely examples which are not intended to limit the scope of protection, application or structure in any way. Rather, the preceding description provides the person skilled in the art with a guide for implementing at least one embodiment, wherein various modifications, in particular alternative or additional features and / or modifications of the function and / or arrangement of the described components, can be made as desired by the person skilled in the art without deviating from the subject matter defined in the appended claims and its legal equivalents and / or leaving their scope of protection.Regardless of the grammatical gender of a particular term, persons with male, female or other gender identities are included. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited non-patent literature

[0000] Attention Is All You Need", in arXiv: 1706.03762, June 12, 2017

[0123]

Claims

[1] Computer-implemented method for providing an observable indicating a medical diagnosis, comprising the following steps: - Obtaining (S10) a medical image data series (SE) of a patient, wherein the medical image data series (SE) comprises several medical image data sets (SET) that were recorded over a period of time at different times, - Extracting (S20) a time series (ZR1) from the medical image data series (SE), - Determining (S30) the observables (OBS) based on the time series (ZR1), and - Providing (S40) the observables (OBS). [2] The method of claim 1, wherein the step (S30) of determining the observables (OBS) comprises: - Obtaining (S31) a second time series (ZR2) of a measured variable (NBD-MG, BP-MG) associated with the patient, which second time series covers a second time period which at least partially overlaps with the time period, and - Determine (S33) the observables (OBS) additionally based on the second time series (ZR2). [3] The method according to claim 2, wherein the step (S30) of determining the observables (OBS) comprises: - determining (S32) a correlation information (KOR) based on the time series (ZR1) and the second time series (ZR2), wherein the correlation information (KOR) indicates a measure of a temporal correlation between the time series (ZR1) and the second time series (ZR2), and - Determination (S33) of the observables (OBS) based on the correlation information (KOR). [4] Method according to one of claims 2 or 3, wherein the measured variable (NBD-MG) of the second time series (ZR2) is not based on medical imaging. [5] Method according to claim 4, wherein the measured variable (NBD-MG) of the second time series (ZR2) comprises one or more laboratory values, in particular blood values, of the patient. [6] Method according to one of the preceding claims, wherein the step (S30) of determining the observables (OBS) comprises: - Obtaining (S31) a single event associated with the patient which lies within the time period, - determining (S32) a correlation information (KOR) based on the time series (ZR1) and the individual event, wherein the correlation information (KOR) indicates a measure of a temporal correlation between the time series (ZR1) and the individual event, and - Determination (S33) of the observables (OBS) based on the correlation information (KOR). [7] Method according to one of the preceding claims, further comprising the step: - Obtaining (S0) context information (KI) concerning a clinical picture of the patient, where - the extraction step (S20) comprises selecting an image data-based measurement variable (BD-MG) from several different image data-based measurement variables (BD-MG) based on the context information (KI), and - the time series (ZR1) is a time series of the selected image data-based measurement quantity (BD-MG). [8] Method according to one of the preceding claims, in which medical image data sets (SET) each comprise an image of an entire body part or a whole-body image of the patient. [9] Method according to one of the preceding claims, in which a measured variable (BD-MG) of the time series (ZR1) is selected from: - an analysis value of a body composition, BCA value, in particular comprising a muscle percentage and / or a body fat percentage, - a size of a lesion, and / or - a tumor burden in a part or region of the patient's body or in the patient's entire body. [10] Method according to one of the preceding claims, wherein the step of providing (S40) comprises: - comparing (S41) the observables (OBS) with a given observable pattern, and - Display (S42) the time series (ZR1) and / or the observables (OBS) based on the comparison. [11] Method according to one of the preceding claims, further comprising the steps: - Determining (S50) a medical diagnosis (DIG) based on the observable (OBS), and - Providing (S60) the medical diagnosis (DIG). [12] The method of claim 11, wherein the step of determining (S50) the medical diagnosis (DIG) comprises: - Selection (S51) of a disease pattern from several selection disease patterns based on the observable (OBS), - querying (S52) a patient’s electronic patient record (EPA) from a database, - Matching (S53) the selected disease pattern with entries in the electronic patient record (EPA), and - Determining (S54) the medical diagnosis (DIG) based on the matching (S53). [13] Method according to one of the preceding claims, further comprising the step of - comparing (S71) the observables (OBS) with corresponding observables of several reference patients different from the patient, - selecting (S72) a comparison patient from the plurality of reference patients based on the comparing step, - Providing (S73) information (VI) on the comparison patient. [14] Method according to one of the preceding claims, further comprising the steps: - Determining (S80) a treatment option (BHO) based on the observable (OBS), and - Provision (S90) of the treatment option (BHO). [15] System for providing an observable indicating a medical diagnosis (OBS) comprising a computing device (20) and an interface (26), wherein the interface (20) is designed to - to obtain an image data series (SE) of a patient, wherein the image data series (SE) comprises a plurality of medical image data sets (SET) which were recorded over a period of time at different times, and the computing device (20) is designed to: - extract a time series (ZR1) from the medical image data series (SE) - to determine the observable (OBS) based on the time series (ZR1), and - to provide the Observable (OBS) via the interface (26). [16] Computer program product comprising a program and being directly loadable into a memory of a programmable computing unit, with program means for carrying out a method according to claims 1 to 14 when the program is executed. [17] A computer-readable storage medium on which readable and executable program sections are stored to carry out all steps of the methods according to any one of claims 1 to 14 when the program sections are executed.

Images