Computer-implemented method, medical technology system and computer program product for automatic medical image evaluation and / or image diagnosis and their use

Abstract

A multi-stage computer-implemented method, medical-technical system, and computer program product for automatic medical image evaluation and / or image diagnosis is described. The multi-stage method, system, and computer program product implements the following stages: a first stage (103, 104) for the automatic evaluation of medical data (ID, SD), in particular medical image data (ID, SD), wherein a probability measure and / or a probability distribution (hij) of freedom from findings is determined for the medical data (ID, SD); and a second stage (106) for the automatic determination of deviations of the probability measure and / or the probability distribution (hij) of freedom from findings from a probability measure threshold value (hSij) and / or from a probability measure criterion (hCij) of freedom from findings; and optionally a third stage (107) for the automatic creation of medical findings.

Description

[0001] The invention relates to a computer-implemented method, a medical-technical system and a computer program product for automatic medical image evaluation and / or image diagnosis and their use, as well as a computer-implemented training method.

[0002] The invention is particularly concerned with assisting physicians in the creation and / or evaluation and / or presentation of medical image data and / or medical findings on the basis of acquired and / or stored imaging data, in particular with regard to the automatic or automated evaluation and diagnosis of radiological and / or medical-technical and / or general image content in the context of the creation and / or evaluation and / or presentation of medical findings.

[0003] Physicians in many disciplines are responsible for conducting diagnostic examinations throughout a patient's treatment period. Examples include imaging examinations and monitoring various patient information, such as symptoms, blood values, tumor markers, and measurements related to the patient's physical condition (e.g., characterization of organs, tissue condition, and potentially pathological tissue changes, etc.).

[0004] Previous automatic or automated creation and / or evaluation and / or presentation and / or diagnosis of radiological and / or medical-technical image content and / or general image content relating to medical findings - for example in radiology and other medical disciplines - can be carried out in particular using computer-implemented trainable functions such as neural networks, in particular using so-called "convolutional neural networks" (CNNs), which are generally known from the general technical knowledge of computer-implemented methods, see for example: https: / / en.wikipedia.org / wiki / Convolutional_neural_network.

[0005] The use of CNNs in the context of automatic or automated creation and / or evaluation and / or presentation and / or diagnosis of radiological and / or medical image data and / or medical image content and / or general data and / or image content relating to medical findings is generally known from the following earlier patent applications or patents of the applicant of this patent application: Document US 10,595,727 B2 describes a machine learning-based segmentation approach for cardiac medical imaging. For cardiac segmentation in magnetic resonance imaging or other medical imaging techniques, deep learning trains a neural network. The neural network, such as U-net, contains at least one long-short-term memory (LSTM), such as a convolutional LSTM. The LSTM integrates temporal and spatial features to improve the accuracy of the machine-learned network's segmentation.

[0006] Document US 10,7796,917 B2 describes a method and system for compensating motion artifacts using machine learning. One method is used to train a convolutional neural network of a compensation unit.The method comprises: providing a machine learning device, wherein the machine learning device is configured to train the convolutional neural network; providing a starting compensation unit, including an untrained convolutional neural network, on or at the machine learning device; providing a training image dataset comprising a plurality of medical training input images and at least one training output image, wherein a reference object is shown substantially without motion artifacts in the at least one training output image and the reference object in question is included in the plurality of medical training input images with different motion artifacts; and training the convolutional neural network of the compensation unit in accordance with a machine learning principle using the training image dataset.A compensation unit, a machine learning device, a control device for controlling a medical imaging system are also disclosed.

[0007] Document US 10,452,899 B2 describes deep representation learning for fine-grained body part recognition. This document discloses a method and apparatus for deep learning-based fine-grained body part recognition in medical imaging data. A paired convolutional neural network (P-CNN) for slice ordering is trained on unannotated (unlabeled) medical image volumes. A convolutional neural network (CNN) for fine-grained body part recognition is trained by fine-tuning the learned weights of the trained P-CNN for slice ordering. The CNN for fine-grained body part recognition is trained to calculate a normalized height value for a transverse input slice of a medical imaging volume, which indicates a normalized height of the transverse input slice in the human body.

[0008] Regarding the general technical background of the use of CNNs in the context of automatic or automated creation and / or evaluation and / or presentation and / or diagnosis of radiological and / or medical image data and / or medical image content and / or general data and / or image content related to medical findings, it should be noted that computer-implemented neural networks – especially CNNs – as a core technology of computer vision over the last 10 years of computer-implemented data processing or information processing, are modeled on the visual cortex of the human brain. Visual content is translated into higher-dimensional feature maps, in which, for example, spatial relationships of characteristic image content are encoded. Objects can be identified and classified automatically or automatically based on this trained content.

[0009] A fundamental problem with this approach in radiology, medical imaging examinations, and medicine in general is the large number of potentially automated or automatically detected imaging and / or medical indications. For example, a chest X-ray image, a relatively simple case in medical imaging, can contain imaging and / or medical indications for around 150 different known diseases. For each possible disease, a trainable function such as a neural network, e.g., a CNN, must be globally trained. This requires corresponding training data—especially training image data—for each disease. Statistically necessary data sets of hundreds to thousands of images are not uncommon.In multiplication, this translates into hundreds of thousands to millions of high-quality annotated image and health data as necessary training data for the aforementioned trainable functions, especially neural networks such as CNNs. The data and training effort required for this is currently a major reason and technical obstacle why differential diagnosis based on imaging data, particularly in radiology—or in other medical disciplines—cannot currently be performed using CNN-based products, and will not be possible in the foreseeable future.

[0010] In addition, the inherent bias of the data, which is always present, is a significant problem even in individual applications. The training and validation data would have to contain an equal distribution of gender, ethnicity and age, among other things, in order to achieve adequate representation of the disease to be classified in its real-life form in the population. This is virtually impossible in practice. Any compromise leads to a significantly deterioration in model performance in reality. It should also be noted that the trained classifications are often linked to a specific imaging device and its specific technical parameters and calibrations and can, under certain circumstances, produce different results on a different imaging device. For example, they may function significantly less well or at least the results may not be sufficiently comparable or reproducible.

[0011] Current neural networks such as CNNs and their derivatives are therefore generally only capable of automatically or automatically identifying specific medical indications in special cases. Radiographs and CT scans of the lungs have so far been the primary method of choice for this. Comprehensive automated or automatic differential diagnosis is generally not performed.

[0012] The technical teaching and the technical features as well as combinations or sub-combinations of technical features of the aforementioned earlier patent application or patents US 10,595,727 B2, US 10,7796,917 B2, US 10,452,899 B2 of the patent applicant as well as the aforementioned technical features of the general technical background and / or technical expertise are hereby incorporated into the disclosure of the present invention as possible developments and / or possible further or supplementary embodiments of the present invention. In particular, those individual technical aspects of the aforementioned earlier patent application of the patent applicant are incorporated into the disclosure of the present invention which a person skilled in the art would use as helpful or supportive for the further development or for the implementation of the present invention.

[0013] It is an object of the present invention to provide an improved computer-implemented method, an improved medical technology system and an improved computer program product for medical image evaluation and / or image diagnosis and their use as well as an improved training method.

[0014] This object is achieved according to the invention by the subject matter of the independent patent claims. Advantageous embodiments and expedient further developments are the subject matter of the dependent claims.

[0015] Aspects of the present invention are described below. The inventive aspects and / or features of the described aspects can, within the scope of each aspect and / or within the scope of each combination or sub-combination of features of the present invention, be designed or further developed in particular as - Method, in particular as a computer-implemented method, comprising one or more of the steps described in this document in any combination or sub-combination of features and / or aspects of the invention as described in this document, and / or - as a medical-technical data processing unit or medical-technical system, in particular designed to carry out a method according to one of the aspects described in this document or according to a combination or sub-combination of features and / or a method according to one of the aspects described in this document, and / or - as a computer program product or computer program system consisting of several computer program product modules, each with individual program steps, each individually or jointly designed to carry out corresponding steps of a method according to one of the aspects described in this document and / or designed to interact with a medical data processing unit or medical system according to one of the aspects described in this document.

[0016] Within the scope of the present invention, a multi-stage method or a multi-stage computer program product and a multi-stage medical data processing unit or medical system for automatic medical image evaluation and / or image diagnosis are provided: In a first step, a (particularly location-dependent or spatial) probability measure and / or a (particularly location-dependent or spatial) probability distribution is determined for recorded and / or stored medical data - in particular medical image data - using a suitable method for processing, evaluating or assessing this data. Stored medical data may contain historical image data from previous imaging examinations (i.e., from preliminary imaging examinations) and may additionally contain personal data of the patient in question and / or historical medical data from non-imaging examinations (i.e., from further preliminary examinations) and / or historical findings data (i.e.,The acquired and / or stored medical image data may contain medical findings (e.g., findings data based on previous examinations) of the patient in question and / or reference patients and / or data on the complete medical history of the patient in question and / or reference patients. The aforementioned location-dependent probability measure and / or the aforementioned spatial probability distribution indicates how likely a pixel or image region corresponds to an expectation of freedom from findings (in other words, of “normal health”) for the patient in question or the examination object. The acquired and / or stored medical image data contain at least one one-, two-, or three-dimensional image content. In addition, other medical and / or personal and / or historical data such as patient age and gender, previous findings, etc. may also be used if necessary.

[0017] In a second stage, the determined (particularly location-dependent or spatial) probability distribution is examined for significant deviations from the expectation of freedom from findings (in other words, from "normal health"). In individual (particularly segmented) data regions – especially image regions – a local procedure based on the information from the first stage is applied to identify, classify, or quantify a local potential finding. This means that potentially finding local data regions – especially image regions – are identified and evaluated. These local procedures can vary and be optimized, for example, for individual organ systems, previous findings, etc.

[0018] In a third, optional stage, in addition to the results of the first two stages, i.e. in addition to the determined data of the (in particular location-dependent or spatial) probability distribution of an expectation of freedom from findings and the determined data of the determination or localization and evaluation of potentially finding data regions - in particular image regions - further stored or determined medical and / or personal data and information such as size, malignancy, organ affiliation etc. of a determined medical finding, patient age and sex, previous findings etc. can also be used, on the basis of which, in conjunction with the aforementioned results of the first two stages, a medical finding report can be created automatically and computer-implemented and made available to a user such as a treating physician, a radiologist or a nuclear medicine specialist.

[0019] According to one aspect of the present invention, a computer-implemented method for automatic medical image evaluation and / or image diagnosis is therefore provided, which can be further developed with features or a subcombination of features of any further aspect of the present invention, wherein the computer-implemented method is designed as a multi-stage method and comprises the following stages: - a first stage for the automatic computer-implemented evaluation of recorded and / or stored medical data, in particular medical image data, whereby a probability measure and / or a probability distribution of the absence of findings is determined for the recorded medical data and / or for the stored medical data; and - a second stage for the automatic computer-implemented determination of deviations of the probability measure and / or the probability distribution of freedom from at least one probability measure threshold and / or at least one probability measure criterion of freedom from the recorded and / or stored medical data; and optionally - a third stage for the automatic computer-implemented creation of medical findings on the basis of the recorded and / or stored medical data and the determined deviations of the determined probability measure and / or the determined probability distribution of freedom from at least one probability measure threshold value and / or at least one probability measure criterion of freedom from findings, and for the automatic computer-implemented provision of the created medical findings to a user by means of a display unit.

[0020] According to one aspect of the present invention, a computer-implemented method for automatic medical image evaluation and / or image diagnosis is provided, in particular according to the aforementioned aspect or according to features of the aforementioned aspect of the invention, wherein the computer-implemented method comprises the following steps: - automatic reading of captured medical image data and / or stored medical data representing at least one one-dimensional, two-dimensional or three-dimensional medical image content; - automatically applying a first function trained using predominantly finding-free and / or exclusively finding-free training data and computer-implemented in a neural network computing unit to the acquired medical image data and / or to the stored medical data; - automatically determining a location-dependent probability measure and / or a spatial probability distribution of freedom from findings for the acquired medical image data and / or for the stored medical data by means of the trained computer-implemented function; and - automatically displaying a spatial representation of the location-dependent probability measure and / or the spatial probability distribution by means of a display unit; and / or - automatically applying a second function, trained using training data containing findings and computer-implemented in a neural network processing unit, to such acquired medical image data and / or to such stored medical data whose determined probability measure falls below or exceeds a predefined and / or learned probability measure threshold and / or fulfills a probability measure criterion correlated with the medical image content, for the automatic determination of findings data; and - automatic determination and display of the determined findings data for a user by means of a display unit.

[0021] According to one aspect of the present invention, a computer-implemented method for automatic medical image evaluation and / or image diagnosis is provided, in particular according to one of the aforementioned aspects of the invention or according to features of an aforementioned aspect of the invention, wherein the computer-implemented method comprises the following steps: - automatic reading of captured medical image data and / or stored medical data representing at least one one-dimensional, two-dimensional or three-dimensional medical image content; - automatically segmenting and / or encoding the image data into a plurality of image segments; - automatically applying a first function to the image segments, which is trained using predominantly free and / or exclusively free training data and computer-implemented in a neural network computing unit; - automatically determining a probability measure of freedom from findings for each of the image segments using the trained computer-implemented function; and - automatically displaying a spatial representation of the probability measure by means of a display unit; and / or - automatically applying a second function, trained using training data containing findings and computer-implemented in a neural network processing unit, to image segments with a probability measure that falls below or exceeds a predefined and / or learned probability measure threshold and / or meets a probability measure criterion correlated with the medical image content, for the automatic determination of findings data; and - automatic determination and display of the determined findings data for a user by means of a display unit.

[0022] A method according to the invention according to each of the aforementioned aspects of the invention is designed as a computer-implemented method, and it can be realized in conjunction with each individual feature from each further aspect of the invention, in conjunction with each further aspect of the invention, or in conjunction with any combination or sub-combination of further features and / or aspects of the invention as described in this document.

[0023] Within the scope of each of the aforementioned aspects of the present invention as well as within the scope of each further aspect of the present invention, a computer-implemented method, in particular a computer-implemented method for automatic medical image evaluation and / or image diagnosis or a computer-implemented training method for automatically training computer-implemented trainable functions, may comprise or additionally comprise the following steps: - automatic training of a first computer-implemented function using predominantly free and / or exclusively free image training data to learn parameters for automatically characterising a probability measure of freedom from findings in image data and / or at least one probability measure threshold of freedom from findings in image data and / or at least one probability measure criterion of freedom from findings in image data correlated with medical image content, and - Automatic training of a second computer-implemented function using training data containing findings to learn parameters for the automatic characterization of findings data.

[0024] Within the scope of each of the aforementioned aspects of the present invention as well as within the scope of each further aspect of the present invention, a computer-implemented method, in particular a computer-implemented method for automatic medical image evaluation and / or image diagnosis or a computer-implemented training method for automatically training computer-implemented trainable functions, may comprise or additionally comprise individual, several or all of the following features and / or steps: - automatically applying a second function, trained using training data containing findings and computer-implemented in a neural network processing unit, to image segments with a probability measure that falls below or exceeds a predefined and / or learned probability measure threshold and / or meets a probability measure criterion correlated with the medical image content, taking into account and / or correlating with stored medical data, for the automatic determination of findings data; and - automatic display of captured medical image data and / or stored medical data as well as automatically determined findings data, in particular location-dependent findings data, for a user by means of a display unit.

[0025] Within the scope of each of the aforementioned aspects of the present invention as well as within the scope of each further aspect of the present invention, a computer-implemented method, in particular a computer-implemented method for automatic medical image evaluation and / or image diagnosis or a computer-implemented training method for automatically training computer-implemented trainable functions, may comprise or additionally comprise individual, several or all of the following features and / or steps: - automatic training of a second computer-implemented function using training data with findings and stored medical data to learn parameters for the automatic characterization of findings data and to learn a language model for the linguistic description of automatically determined and / or stored findings data; - automatically applying the second function, trained using training data containing findings and computer-implemented in a neural network processing unit, for automatically determining findings and automatically creating a linguistic description based on the learned language model to such acquired medical image data and / or to such stored medical data and / or to such image segments with a probability measure that falls below or exceeds a predefined and / or learned probability measure threshold and / or fulfills a probability measure criterion correlated with the medical image content, taking into account and / or correlating with stored medical data; and - automatic display of the determined findings data as well as the linguistic description of the determined findings data automatically created on the basis of the learned language model for a user by means of a display unit.

[0026] Within the scope of each of the aforementioned aspects of the present invention as well as within the scope of each further aspect of the present invention, a computer-implemented method, in particular a computer-implemented method for automatic medical image evaluation and / or image diagnosis or a computer-implemented training method for automatically training computer-implemented trainable functions, may comprise or additionally comprise individual, several or all of the following features and / or steps: - automatically segmenting and / or encoding the image data into a plurality of image segments by means of a neural transformer network architecture (e.g. a vision transformer architecture) and / or by means of a generative neural network architecture (e.g. a generative adversarial network architecture); - automatically applying a first neural network function to the image segments, trained using predominantly and / or exclusively free training data and computer-implemented in a neural network processing unit; and - Automatic application of a second neural network function, trained using training data containing findings and computer-implemented in a neural network processing unit, to image segments with a probability measure that falls below or exceeds a predefined and / or learned probability measure threshold and / or fulfills a probability measure criterion correlated with the medical image content, for the automatic determination of findings data.

[0027] Image segmentation using a Vision Transformer (ViT) architecture is generally known from the state of the art and essentially follows the same principles as other image segmentation methods, but uses a transformer-based approach. The basic process of computer-implemented image segmentation using a Vision Transformer architecture is divided into an input preprocessing step, a transformation step, a segmentation step, a training step, and an inference step. In the input preprocessing step, an input image is divided into so-called patches. Each patch can be generated as a vector and then used as input for the actual transformer. Position information of the patches relative to each other or about their position in the input image is usually added using position embeddings.In the transformation step, the patches are encoded using a transformer. The transformer architecture consists of so-called attention layers that enable the model to capture both local and global context information. In the segmentation step, a decoder processes the output of the transformer encoder to generate, for example, pixel-wise, region-wise, or segment-wise predictions. It can consist of upsampling layers to restore spatial resolution and generate segmentation masks. In a training step, the model can be trained, in particular, with annotated (labeled) data, in which each pixel, region, or segment in an image is assigned a corresponding class or segmentation label. In the inference step, the trained model is applied to new, previously unprocessed images to generate segmentation masks.These masks represent the predicted class labels for each pixel or region or segment in the image.

[0028] Convolutional neural networks (CNNs) are generally well-known in the art and represent a special type of computer-implemented, trainable neural network that is particularly well-suited for image processing. CNNs use special layers, called convolutional layers, to extract features from the input data. The basic sequential elements by which a computer-implemented convolutional neural network typically functions are: one or more convolutional layers, a pooling layer, one or more fully connected layers, one or more flatten layers, and an output layer. A convolutional layer of a CNN is essentially a filter that is passed over input data, calculating the dot product between the filter values ​​and the input data. This process is repeated for different portions of the input data, creating so-called feature maps.One or more convolutional layers are usually followed by a pooling layer. This pooling serves to reduce the dimensions of the feature maps while still preserving the relevant information. The convolutional and pooling layers can be followed by fully connected layers, which convert the previously extracted information or features into a form suitable for classification or regression. Such fully connected layers learn weights for the various features. One or more flatten layers can be used before the fully connected layers to convert multidimensional feature maps into lower-dimensional representations such as vectors, which then serve as input for the fully connected layers. Every CNN ends with an output layer, which has a corresponding number of neurons depending on the task (classification, regression, etc.).

[0029] According to a further aspect of the present invention, a computer-implemented training method for automatically training computer-implemented trainable functions using medical image data is provided, the computer-implemented training method comprising the following steps: - automatic training of a first computer-implemented function using predominantly free and / or exclusively free image training data to learn parameters for automatically characterising a probability measure of freedom from findings in image data and / or at least one probability measure threshold of freedom from findings in image data and / or at least one probability measure criterion of freedom from findings in image data correlated with medical image content, and - Automatic training of a second computer-implemented function using training data containing findings to learn organ-specific and / or tissue structure-specific parameters for the automatic characterization of disease-specific findings data.

[0030] Within the scope of each of the aforementioned aspects of the present invention as well as within the scope of each further aspect of the present invention, a computer-implemented method, in particular a computer-implemented method for automatic medical image evaluation and / or image diagnosis or a computer-implemented training method for automatically training computer-implemented trainable functions, may comprise or additionally comprise individual, several or all of the following features and / or the following step or steps: - Automatic training of a second computer-implemented function using training data with findings and stored medical data to learn organ-specific and / or tissue structure-specific parameters for the automatic characterization of disease-specific findings data as well as for learning a language model for the linguistic description of automatically determined and / or stored findings data.

[0031] According to a further aspect of the present invention, a medical data processing unit or medical system is provided, in particular designed to carry out a computer-implemented method according to one of the aforementioned aspects of the present invention and within the scope of any further aspect of the present invention, and such a medical data processing unit or medical system can be implemented in conjunction with each individual feature from any further aspect of the invention, in conjunction with any further aspect of the invention, or in conjunction with any combination or sub-combination of further features and / or aspects of the invention as described in this document, wherein the medical data processing unit or the medical system comprises at least the following units that are in data communication with one another: - at least one image acquisition unit designed to automatically acquire medical image data; - at least one storage unit designed to store medical data; - at least one image data processing unit designed for the automatic processing of medical image data, in particular designed for the automatic segmentation and / or encoding of medical image data into a plurality of image segments; - at least one neural network computing unit designed to automatically apply trainable and / or trained computer-implemented functions to medical image data and / or to stored medical data; - at least one display unit designed for the automatic display of medical image data and / or stored medical data and / or medical findings data; wherein the neural network computing unit is designed for - automatically applying a first computer-implemented neural network function trained using predominantly free and / or exclusively free training data to the image segments; and - automatically applying a second computer-implemented neural network function trained using training data containing findings to image segments with a probability measure that falls below or exceeds a predefined and / or learned probability measure threshold and / or meets a probability measure criterion correlated with the medical image content, for the automatic determination of findings data.

[0032] According to a further aspect of the present invention, a computer program product or computer program system comprising a plurality of computer program product modules is provided, in particular designed to carry out a computer-implemented method according to one of the aforementioned aspects of the present invention and within the scope of any further aspect of the present invention, and in particular designed to interact with a medical-technical data processing unit or medical-technical system according to one of the aforementioned aspects of the present invention and within the scope of any further aspect of the present invention, and such a computer program product or computer program system comprising a plurality of computer program product modules can, in conjunction with each individual feature from each further aspect of the invention,in conjunction with any further aspect of the invention or in conjunction with any combination or subcombination of further features and / or aspects of the invention as described in this document. The computer program product or computer program system consisting of several computer program product modules comprises instructions which, when the computer program product or the computer program product modules are executed by a data processing unit, cause this data processing unit to carry out program steps of a computer-implemented method according to one of the aforementioned aspects of the present invention or according to any further aspect of the present invention,in particular in cooperation with a medical-technical data processing unit or medical-technical system according to one of the aforementioned aspects of the present invention or according to any further aspect of the present invention. The aforementioned computer program product or computer program system comprises, in particular, instructions which, when the computer program product or the computer program product modules are executed by a data processing unit, cause this data processing unit to execute the following computer-implemented program steps: - Program steps of a first program stage for the automatic computer-implemented evaluation of recorded and / or stored medical data, in particular medical image data, whereby a probability measure and / or a probability distribution of the absence of findings is determined for the recorded medical data and / or for the stored medical data; and - Program steps of a second program level for the automatic computer-implemented determination of deviations of the probability measure and / or the probability distribution of freedom from at least one probability measure threshold and / or at least one probability measure criterion of freedom from recorded and / or stored medical data; and optionally - Program steps of a third program level for the automatic computer-implemented creation of medical findings on the basis of the recorded and / or stored medical data and the determined deviations of the determined probability measure and / or the determined probability distribution of freedom from at least one probability measure threshold value and / or at least one probability measure criterion of freedom from findings, and for the automatic computer-implemented provision of the created medical findings to a user by means of a display unit.

[0033] Within the scope of each of the aforementioned aspects of the present invention as well as within the scope of each further aspect of the present invention, a computer program product or computer program system comprising several computer program product modules may comprise or additionally comprise individual, several or all of the following features and / or the following program step or steps: - Program step for the automatic computer-implemented reading of acquired medical image data and / or stored medical data representing at least one one-dimensional, two-dimensional or three-dimensional medical image content; - Program step for the automatic computer-implemented application of a first function, trained using predominantly finding-free and / or exclusively finding-free training data and computer-implemented in a neural network computing unit, to the acquired medical image data and / or to the stored medical data; - Program step for the automatic computer-implemented determination of a probability measure of freedom from findings for the acquired medical image data and / or for the stored medical data by means of the trained computer-implemented function; and - Program step for the automatic computer-implemented display (S105) of a spatial representation of the probability measure by means of a display unit; and / or - Program step for the automatic computer-implemented application of a second function, trained using training data containing findings and computer-implemented in a neural network processing unit, to such acquired medical image data and / or to such stored medical data, the determined probability measure of which falls below or exceeds a predefined and / or learned probability measure threshold and / or fulfills a probability measure criterion correlated with the medical image content, for the automatic determination of findings data; and - Program step for the automatic computer-implemented presentation of the determined findings data for a user using a display unit.

[0034] Within the scope of each of the aforementioned aspects of the present invention as well as within the scope of each further aspect of the present invention, a computer program product or computer program system comprising several computer program product modules may comprise or additionally comprise individual, several or all of the following features and / or the following program step or steps: - Program step for the automatic computer-implemented reading of acquired medical image data and / or stored medical data representing at least one one-dimensional, two-dimensional or three-dimensional medical image content; - Program step for automatically computer-implemented segmenting and / or encoding the image data into a plurality of image segments; - Program step for the automatic computer-implemented application of a first function, trained using predominantly finding-free and / or exclusively finding-free training data and computer-implemented in a neural network computing unit, to the image segments; - Program step for the automatic computer-implemented determination of a probability measure of freedom from findings for each of the image segments using the trained computer-implemented function; and - Program step for the automatic computer-implemented display of a spatial representation of the probability measure by means of a display unit; and / or - Program step for the automatic computer-implemented application of a second function, trained using training data containing findings and computer-implemented in a neural network processing unit, to image segments with a probability measure that falls below or exceeds a predefined and / or learned probability measure threshold and / or fulfills a probability measure criterion correlated with the medical image content, for the automatic determination of findings data; and - Program step for the automatic computer-implemented presentation of the determined findings data for a user using a display unit.

[0035] Within the scope of each of the aforementioned aspects of the present invention as well as within the scope of each further aspect of the present invention, a computer program product or computer program system comprising several computer program product modules may comprise or additionally comprise individual, several or all of the following features and / or the following program step or steps: - Program step for the automatic computer-implemented training of a first computer-implemented function using predominantly free and / or exclusively free image training data for learning parameters for the automatic characterization of a probability measure of freedom from findings in image data and / or of at least one probability measure threshold of freedom from findings in image data and / or of at least one probability measure criterion of freedom from findings in image data correlated with medical image content, and - Program step for the automatic computer-implemented training of a second computer-implemented function using training data with findings to learn organ-specific and / or tissue structure-specific parameters for the automatic characterization of disease-specific findings data.

[0036] Each of the aforementioned aspects of the present invention as well as each further aspect of the present invention or any combination or sub-combination of further features and / or aspects of the invention as described in this document, in particular a computer-implemented method for automatic medical image evaluation and / or image diagnosis and / or a medical-technical data processing unit or a medical-technical system and / or a computer program product or computer program system according to one aspect of the present invention or according to any combination or sub-combination of further features and / or aspects of the invention as described in this document, can be used for automatic medical image evaluation and / or image diagnosis of radiological image data and / or nuclear medicine image data and / or ultrasound image data and / or optical image data.A further aspect of the present invention is therefore such aforesaid use of a computer-implemented method for automatic medical image evaluation and / or image diagnosis and / or a medical-technical data processing unit or a medical-technical system and / or a computer program product or computer program system.

[0037] Within the scope of the inventive methods, in the aspects of the invention described above and / or in further aspects of the invention described in this document, a trained, computer-implemented function is particularly automatically applied. Each trained function described within the scope of the present invention can, in particular, be embodied as a trained neural network function, but can also be embodied as another data processing function suitable and adapted for the respective data processing step.

[0038] A trained function, in particular a trained neural network function, generally maps input data to output data. The output data can depend, in particular, on one or more parameters of the trained function. The one or more parameters of the trained function can be determined and / or adjusted through training. The determination and / or adjustment of the one or more parameters of the trained function can, in particular, be based on a pair of training input data and associated training output data, wherein the trained function is applied to the training input data to generate training mapping data.

[0039] In particular, it can be provided that, within the scope of one or more aspects of the present invention, automatic, in particular computer-implemented, feedback loops are provided for changing the training basis. This means that the training basis can be changed via feedback loops - i.e., it can improve over time - because, through feedback on the training data - such as image training data from patients that are free of findings and / or exclusively free of findings and / or training data with findings, such as image training data from patients that are particularly free of findings - the underlying model can be adapted in further development, and a better set of training data can be used for further automatic medical image evaluations and / or image assessments of new patients.

[0040] In particular, the determination and / or adaptation can be based on a comparison of the training image data and the training output data. For the purposes of this invention, a trainable function, i.e., a function with parameters that have not yet been adapted, is also understood as a trained function. In particular, the trained function can be contained in a single filter component of the data filter. In addition, the data filter can have further filter components that do not include trained functions because they operate, for example, in a rule-based manner. Furthermore, the data filter can also have multiple trained functions.

[0041] A trained function within the meaning of this invention can be embodied as a trained mapping rule, a mapping rule with trained parameters, a function with trained parameters, an algorithm based on artificial intelligence, or a machine learning algorithm. An example of a trained function is an artificial neural network, such as a convolutional neural network. Parameters within the meaning of the invention can, in particular, be weight parameters of neurons of an input layer and / or an output layer and / or of one or more hidden layers of such an artificial neural network.

[0042] Values ​​for a probability measure and / or a probability criterion and / or for probability measure threshold values ​​of functions described within the scope of the invention can be designed as error values ​​or cost values ​​of one or more error functions or cost functions of an aforementioned neural network, or as values ​​derived from such error values ​​or cost values.

[0043] A method according to one of the aforementioned aspects of the invention, as well as according to a further aspect as described in this document, can in particular enable the integration of multidisciplinary previous (i.e., historical in the sense of the invention) analytical, medical and / or diagnostic information and the automatic evaluation of the relevance of such information. A method according to one of the aforementioned aspects of the invention, as well as according to a further aspect as described in this document, can alternatively or additionally support, in particular, the diagnosis of current evidence, such as the diagnosis of current image examinations based on medical imaging methods, in particular through context-specific integration of relevant previous (i.e., historical in the sense of the invention) evidence of the same patient and / or of reference patients, such asprevious imaging information and clinical patient information to increase diagnostic confidence and assist in the creation of reports and subsequent quantification of findings.

[0044] The properties, features, and advantages of aspects of the invention, as well as the manner in which they can be achieved, are explained below by way of example with reference to specific embodiments described in conjunction with the drawings. These specific embodiments do not limit the invention to these embodiments. In different figures, identical components are provided with identical reference numerals. The figures are generally not to scale. FIGURE DESCRIPTION

[0045] They show: Fig. 1 a schematic representation of a specific embodiment of a medical data processing unit or a medical system according to the invention, designed to carry out a computer-implemented method according to the invention for automatic medical image evaluation and / or image diagnosis; Fig. 2 a schematic representation of steps of a specific embodiment of a computer-implemented method according to the invention for automatic medical image evaluation and / or image diagnosis, in particular using a medical-technical data processing unit according to the invention or a medical-technical system according to Fig. 1; Fig. 3 a schematic representation of steps of an alternative specific embodiment of a computer-implemented method according to the invention for automatic medical image evaluation and / or image diagnosis, in particular using a medical-technical data processing unit according to the invention or a medical-technical system according to Fig. 1; Fig. 4 a schematic representation of steps of a specific embodiment of a computer-implemented convolutional neural network according to the invention, in particular in the form of a computer program product; Fig. 5 a schematic representation of steps of a specific embodiment of a computer-implemented training method for automatically training computer-implemented trainable functions and of steps of a specific embodiment of a computer-implemented method according to the invention for automatic medical image evaluation and / or image diagnosis, in particular using a medical-technical data processing unit according to the invention or a medical-technical system according to Fig. 1; and Fig. 6 a schematic representation of program steps of a specific embodiment of a computer program product or computer program system consisting of several computer program product modules, with program steps for carrying out a computer-implemented training method for automatically training computer-implemented trainable functions and of program steps of a specific embodiment of a computer-implemented method according to the invention for automatic medical image evaluation and / or image diagnosis, in particular according to the embodiments according to Fig. 1 to Fig. 4 and / or using a medical data processing unit according to the invention or a medical system according to Fig. 1.

[0046] The embodiment according to Fig. 1 shows a schematic representation of a special embodiment of a medical-technical data processing unit according to the invention or of a medical-technical system SYS, designed to carry out a computer-implemented method according to the invention for automatic medical image evaluation and / or image diagnosis, as described in particular in the following Fig. 2 to 6 including the corresponding description.

[0047] The medical-technical data processing unit or the medical-technical system SYS comprises the following units, which are connected to each other via data: - an image acquisition unit IDCU, designed to automatically acquire medical image data ID (for example, radiological image data and / or nuclear medicine image data and / or ultrasound image data and / or optical medical image data such as endoscopy image data or colonoscopy image data or eye examination image data or skin cancer scan image data) representing a one-dimensional or two-dimensional or three-dimensional medical image content B(r) (for example, a body region, an organ, a tissue region or a tissue sample), - a storage unit SDMU designed to store medical data SD (e.g. current or historical patient data, medical findings data, medical image data, medical findings texts, each as current and / or historical personal data and / or medical data of the same patient and / or of other patients and / or collected data of reference patients and / or of medical reference image data and / or of medical reference texts), - at least one image data processing unit IDPU designed for the automatic processing of medical image data ID, SD, in particular designed for the automatic segmentation and / or encoding of medical image data ID, SD into a plurality of image segments IS ij (for example, using a Vision Transformer architecture, as described with reference to Fig. 2 and Fig. 3 will be explained further below), - at least one neural network computing unit NNCU designed for the automatic application of trainable and / or trained computer-implemented functions TF1, TF2 (for example application of one or more trained neural network functions, in particular convolutional neural network functions, as described with reference to Fig. 4 is further explained below) to the aforementioned recorded medical image data ID and / or to the aforementioned stored medical data SD (in particular to stored medical image data and / or to stored medical text data SD); - at least one display unit DU designed to automatically display medical image data ID and / or stored medical data SD and / or automatically and computer-implemented process results of the automatic application of the aforementioned trainable and / or trained computer-implemented functions TF1, TF2 to the aforementioned acquired medical image data ID and / or to the aforementioned stored medical data SD for a user of the medical-technical data processing unit or the medical-technical system SYS (for example for a treating physician, a radiologist, a nuclear medicine specialist, a medical diagnostician, an employee of a medical-technical institute or a medical-technical research facility),in particular for the automatic display of automatically and computer-implemented generated medical findings data MD(r) and / or in particular for the automatic display of automatically and computer-implemented spatial representations of a probability measure h, ij (r) the absence of medical findings and / or the presence of medical findings for the captured medical image data ID and / or for the stored medical data SD, as further explained below.

[0048] The medical-technical data processing unit or the medical-technical system SYS is designed to execute a computer-implemented method for automatic medical image evaluation and / or image diagnosis according to Fig. 5 or a computer program product according to Fig. 6, wherein the computer-implemented method or the computer program product is designed in several stages and comprises the following stages, as described below with reference to the Fig. 5 and Fig. 6 is further explained: - a first stage 103, 104, S103, S104 for the automatic computer-implemented evaluation of recorded and / or stored medical data ID, SD, in particular medical image data ID, SD, wherein a probability measure and / or a probability distribution h ij the absence of findings for the recorded medical data ID) and / or for the stored medical data SD is determined; and - a second stage 106, S106 for the automatic computer-implemented determination of deviations of the probability measure and / or the probability distribution h ij the absence of findings of at least one probability measure threshold hS ijand / or of at least one probability measure criterion hC ij the medical data collected and / or stored in the absence of findings ID, SD; and optionally - a third stage 107, S107 for the automatic computer-implemented creation of medical findings based on the recorded and / or stored medical data ID, SD and the determined deviations of the determined probability measure and / or the determined probability distribution h ij the absence of findings of at least one probability measure threshold hS ij and / or of at least one probability measure criterion hC ij the absence of findings, and for the automatic computer-implemented provision of the created medical findings to a user by means of a display unit DU.

[0049] The aforementioned, in Fig. 1 schematically illustrated neural network computing unit NNCU is in accordance with the embodiment according to Fig. 1 trained in the automatic computer-implemented determination of diagnostic data, in particular as described in Fig. 1 schematically shown, for automatically applying a first computer-implemented neural network function TF1, trained using predominantly free and / or exclusively free training data TD1, to the image segments IS ij , as well as for automatically applying a second computer-implemented neural network function TF2, trained using training data TD2 with findings, to such image segments IS ij , for which a probability measure h is automatically and computer-implemented ij which has a predefined and / or learned probability measure threshold hS ijfalls below or exceeds and / or a probability measure criterion hC correlated with the medical image content B(r) ij fulfilled (which depends, for example, on the typical and / or expected nature of a body region, an organ, a tissue region or a tissue sample, whereby these data can in particular also be correlated with stored data SD, such as with current or historical patient data, medical findings data, medical image data, medical findings texts, in each case as current and / or historical personal data and / or medical data of the same patient and / or of other patients and / or collected data from reference patients and / or from medical reference image data and / or from medical reference texts).

[0050] As in Fig. 1, a computer-implemented training method for automatically training the computer-implemented trainable functions TF1, TF2 can be carried out using medical image data, in particular using training data TD1 without findings and training data TD2 with findings, which are stored in the medical-technical data processing unit or in the medical-technical system SYS in a training data storage unit TDMU, wherein this training data storage unit TDMU can be partially or completely identical to the data storage unit SDMU or the training data TD1, TD2 can be partially or completely identical to the stored data SD.The computer-implemented training method for automatically training the computer-implemented trainable functions TF1, TF2 using medical image data, in particular using training data TD1 without findings and training data TD2 with findings, can also be carried out separately from the medical-technical data processing unit or in the medical-technical system SYS in a separate, in . Fig. 1 data processing unit not shown.

[0051] According to a specific embodiment of a computer-implemented training method for the automatic training of computer-implemented trainable functions, the first step involves the practical training and learning of data representing freedom from findings, or in other words, training and learning of "normal health." The training of computer-implemented trainable functions and thus their learning from data representing freedom from findings can be carried out through "unsupervised learning" based on as many images and, if necessary, secondary data of healthy, i.e., finding-free, references (reference image data) as possible. Alternatively, the training of computer-implemented trainable functions and thus their learning from data representing freedom from findings can be carried out through "supervised learning."

[0052] The diagnosis-free (i.e. “normally healthy”) references can be obtained by different approaches, in particular by means of the following procedural steps, which can be implemented and carried out automatically and computer-implemented: a) A model or a computer-implemented trainable function is trained on areas in medical images and clinical parameters that are clearly not affected by a medical finding or diagnosis, and that are also not mentioned in stored data such as stored medical report data. Simple example: When a collarbone fracture is diagnosed, medical image data that exhibit this medical finding are excluded from the training process, and a model or a computer-implemented trainable function is trained based on the remaining available or stored data, which are characterized, classified, or assumed to be free of a medical finding or diagnosis, as free of findings or "normal health." b) A model or a computer-implemented trainable function learns based on data from completed diagnostic processes or based on data that represent and / or document current and / or historical disease progressions. The advantage here is that the additional trained parameters, while training-intensive, also have a stabilizing effect on the model or the computer-implemented trainable function and its parameters (particularly weight parameters of neurons in an input layer and / or an output layer and / or one or more hidden layers of a neural network), as the variance from invisible parameters decreases. c) A model or a computer-implemented trainable function learns based on medical image data with non-actionable disease (NAD) classification. This represents a compromise and a special form of supervised learning, in which a finite list of disease patterns has been excluded by domain experts for the medical image data, and the medical image data can be described as NAD-normally healthy. d) In addition, the interpretation of the difference between a finding (i.e. “normal health”) and a finding (i.e. medical “abnormality”) is often tied to further parameters beyond pure medical image data and in particular correlates with personal parameters and data of a specific patient or a specific patient group (such as age, gender, ethnicity, weight, laboratory parameters, etc.). A simple example: the size of the liver correlates with the size of the patient, but also with gender and age. The assessment of an abnormality can also be time-dependent, so that follow-up examinations can also play a role in training and analysis. Such non-image-based data or parameters can be assigned to a model ora computer-implemented, trainable function in a training procedure and a trained computer-implemented function within the framework of an automatic medical image analysis and / or image reporting. This can be done based on image data or text data, for example, using a large language model (e.g., GPT) based on the stored text data and / or image data (SD) of a patient record. e) A general model or a general computer-implemented trainable function can be trained from image, text and other quantitative information, which can assess the absence of findings (normality) or the presence of findings (abnormality) as a whole. f) Computer-implemented or computer-readable information from medical guidelines and medical textbooks stored electronically or available in electronic databases can also be incorporated into the training. This allows training not only on relevant medical example images, but also on correlations between medical image data and other medical and / or personal data, such as laboratory values, typical for specific disease cases.

[0053] Fig. 2 shows a schematic representation of steps of a specific embodiment of a computer-implemented method according to the invention for automatic medical image evaluation and / or image diagnosis, in particular using a medical-technical data processing unit according to the invention or a medical-technical system according to Fig. 1. The Fig. 2 schematically illustrated method comprises the following automatic computer-implemented steps: automatic acquisition and reading of acquired medical image data ID which represents a one-dimensional or two-dimensional or three-dimensional medical image content B(r), automatic segmentation IS (and if necessary encoding) of the image data ID into a plurality of image segments IS ij , where these image segments IS ij contain location information r1, r2, automatically applying a first computer-implemented function TF1 trained using predominantly finding-free and / or exclusively finding-free training data to the image segments ISij, automatically determining a (location-dependent) probability measure h ij (r1, r2) of the freedom from findings for each of the image segments IS ijusing the trained computer-implemented function TF1; and automatically displaying a spatial or location-dependent representation of the probability measure h ij (r), in particular by means of a representation unit DU as in Fig. 1 shown.

[0054] The aforementioned image segmentation can generally be performed using a suitable image segmentation method known from the state of the art. An input image is divided into so-called patches. Each patch can be defined as a group of individual image segments IS ij generated, as shown schematically in Fig. 2, and then used as input for an actual transformer. As already explained at the beginning, position information of the patches relative to each other or about the position in the input image is added with position embeddings in order to obtain location information IS ij (r1, r2) of the individual image segments IS ijA decoder can then process the output of the transformer encoder to obtain IS ij To generate predictions regarding certain parameters, in the example according to Fig. 2 with respect to a (location-dependent) probability measure h ij (r1, r2) of the absence of findings. A sufficiently trained model of such image segmentation, transformation, encoding, and decoding can be applied to new, previously unprocessed images to generate corresponding segmentation masks. These masks represent the predicted class labels for each individual image segment IS ij .

[0055] Fig. 3 shows a schematic representation of steps of an embodiment according to Fig. 2 alternative specific embodiment of a computer-implemented method according to the invention for automatic medical image evaluation and / or image diagnosis, in particular using a medical-technical data processing unit according to the invention or a medical-technical system according to Fig. 1.

[0056] The aforementioned image segmentation according to Fig. 3 can be carried out in particular with a Vision Transformer (ViT) architecture already mentioned above, whose basic process, as already explained above, is divided into an input preprocessing step, a transformation step, a segmentation step IS, and an inference step. In the input preprocessing step, an input image or input image data ID is divided into so-called patches. Each patch can be generated as a vector that contains individual image segments IS. ij contains, as shown schematically in Fig. 3, and then used as input for the actual transformer. As already explained, position information IS(r) of the patches relative to each other or about the position in the input image can be added using position embeddings. In the segmentation step, a decoder processes the output of the transformer encoder IE to generate IS for each individual image segment. ij To generate predictions regarding certain parameters, in the example according to Fig. 3 with respect to a (location-dependent) probability measure h i (r) of the absence of findings. A sufficiently trained model of such image segmentation, transformation, encoding, and decoding can be applied to new, previously unprocessed images to generate corresponding segmentation masks. These masks represent the predicted class labels for each individual image segment IS ij .

[0057] Fig. Fig. 4 shows a schematic representation of steps of a specific embodiment of a computer-implemented Convolutional Neural Network TF1, TF2 according to the invention as in Fig. 1 and already described at the beginning, in particular in the form of a computer program product or a computer-implemented neural network.

[0058] As already mentioned at the beginning, Convolutional Neural Networks TF1, TF2 are basically known from the state of the art and represent a special type of computer-implemented trainable neural networks that are particularly well-suited for image processing. CNNs use convolutional layers, which are generated by convolution steps C1, C2, to extract features from input data I. The basic sequential elements according to which such a computer-implemented Convolutional Neural Network usually functions are described in Fig. 4: one or more convolutional layers CL1, CL2, one or more pooling layers PL1, PL2, one or more fully connected layers FCL, and an output layer OL. A convolutional layer of a convolutional neural network TF1, TF2 is essentially a filter that is passed over input data. This process is repeated for different areas of the input data, creating so-called feature maps FM1, FM2, FM3, FM4. After one or more, in Fig. 4 after each Convolutional Layers CL1, CL2 follows in Fig. 4 each have a subsampling S1, S2 to form a pooling layer PL1, PL2. This pooling serves to reduce the dimensions of the feature maps FM1, FM3 generated by the convolutions C1, 2 while still preserving the relevant information. After the convolutional layers CL1, CL2 and the pooling layers PL1, PL2, a fully connected layer FCL follows, which converts the previously extracted information or features into a form suitable for classification or regression and converts the generated multidimensional feature maps FM1, FM2, FM3, FM4, in particular the final feature map FM4, into a lower-dimensional representation and finally flows into an output layer OL, which has a corresponding number of neurons depending on the task (classification, regression, etc.).

[0059] Fig. 5 shows a schematic representation of steps of a specific embodiment of a computer-implemented training method for automatically training computer-implemented trainable functions and of steps of a specific embodiment of a computer-implemented method according to the invention for automatic medical image evaluation and / or image diagnosis, in particular using a medical-technical data processing unit according to the invention or a medical-technical system according to Fig. 1. The computer-implemented training method for the automatic training of computer-implemented trainable functions and the (subsequent) computer-implemented method for automatic medical image evaluation and / or image diagnosis comprise the following process steps: - automatic training 100a of a first computer-implemented function TF1 using predominantly free of findings and / or exclusively free of findings image training data TD1 for learning parameters for automatically characterizing a probability measure h of the freedom from findings of image data ID and / or of at least one probability measure threshold value hS ij the absence of findings from image data ID and / or from at least one probability measure criterion hC correlated with medical image content B(r) ij the absence of findings from image data ID, as well as - automatic training 100b of a second computer-implemented function TF2 using training data TD2 containing findings to learn parameters for the automatic characterization of findings data, and further the steps: - automatic reading 101 of captured medical image data ID and / or of stored medical data SD, which represent at least one one-dimensional or two-dimensional or three-dimensional medical image content B(r), - automatic segmentation 102 and / or encoding of the image data ID, SD into a plurality of image segments IS ij , - automatically applying 103 a first function TF1, trained using predominantly free and / or exclusively free training data TD1 and computer-implemented in a neural network computing unit NNCU, to the image segments IS ij , - automatic determination 104 of a probability measure h ij the absence of findings for each of the image segments IS ij using the trained computer-implemented function TF1, and either as a further step: - automatic display 105 of a spatial representation of the probability measure h ij (r) by means of a representation unit DU, or alternatively as further steps: - automatic application 106 of a second function TF2, trained using training data TD2 with findings and computer-implemented in a neural network computing unit NNCU, to image segments IS ij with a probability measure h ij which has a predefined and / or learned probability measure threshold hS ij falls below or exceeds and / or a probability measure criterion hC correlated with the medical image content B(r) ij fulfilled, for the automatic determination of findings data, and - automatic display 107 of the determined findings data for a user by means of a display unit DU.

[0060] For the last two steps 106 and 107 mentioned above (as well as in the program steps S106 and S107 described below), the following can be provided: Within the scope of the automatic application 106 of a second computer-implemented function TF2, in particular, organ- and structure-specific detectors for specific clinical pictures can be selected and executed. Here, too, the aforementioned training of the local models of a second computer-implemented function TF2 can be carried out purely image-based or with the aid of optional secondary information. The training can either be supervised, and existing solutions with CNNs can be used. Alternatively, it can be provided that, for example, Vision Transformer-like models generate or learn locally specialized detectors for specific indications and organ systems solely from annotated training data from image annotation and, if applicable, secondary annotation.

[0061] Fig. 6 shows a schematic representation of program steps of a specific embodiment of a computer program product or computer program system consisting of several computer program product modules, with program steps for carrying out a computer-implemented training method for automatically training computer-implemented trainable functions and of program steps of a specific embodiment of a computer-implemented method according to the invention for automatic medical image evaluation and / or image diagnosis, in particular according to the embodiments according to Fig. 1 to Fig. 4 and / or using a medical data processing unit according to the invention or a medical system according to Fig. 1. The specific embodiment of a computer program product or computer program system according to Fig. 6 includes instructions which, when the computer program product or the computer program product modules are executed by a data processing unit, cause this data processing unit to execute the following program steps: - Program step for the automatic computer-implemented training S100a of a first computer-implemented function TF1 using predominantly free of findings and / or exclusively free of findings image training data TD1 for learning parameters for the automatic characterization of a probability measure h of the freedom from findings of image data ID and / or of at least one probability measure threshold value hSij of the freedom from findings of image data ID and / or of at least one probability measure criterion hCij of the freedom from findings of image data ID correlated with medical image content B(r), and - Program step for the automatic computer-implemented training S100b of a second computer-implemented function TF2 using training data TD2 containing findings for learning organ-specific and / or tissue structure-specific parameters for the automatic characterization of disease-specific findings data, and furthermore the program steps: - Program step for the automatic computer-implemented reading S101 of acquired medical image data ID and / or of stored medical data SD, which represent at least one one-dimensional or two-dimensional or three-dimensional medical image content B(r), - Program step for automatic computer-implemented segmentation S102 and / or encoding of the image data ID, SD into a plurality of image segments IS ij , - Program step for the automatic computer-implemented application S103 of a first function TF1, trained by means of predominantly free and / or exclusively free training data TD1 and computer-implemented in a neural network computing unit NNCU, to the image segments IS ij ; - Program step for the automatic computer-implemented determination S104 of a probability measure h ij the absence of findings for each of the image segments IS ij using the trained computer-implemented function TF1, and either as a further program step: - Program step for the automatic computer-implemented display S105 of a spatial representation of the probability measure h ij (r) by means of a representation unit DU, or alternatively as further program steps: - Program step for the automatic computer-implemented application S106 of a second function TF2, trained using training data (TD2) with findings and computer-implemented in a neural network computing unit NNCU, to image segments ISij with a probability measure h ij which has a predefined and / or learned probability measure threshold (hS ij ) and / or a probability criterion hC correlated with the medical image content B(r) ij fulfilled, for the automatic determination of findings data, and - Program step for the automatic computer-implemented display S107 of the determined findings data for a user by means of a display unit DU.

[0062] A computer-implemented method for automatic medical image evaluation and / or image diagnosis and / or a medical data processing unit or a medical system and / or a computer program product or computer program system as described in the Fig. 1 to 6 and as described above can be used for automatic medical image evaluation and / or image reporting of any type of medical images, such as for automatic medical image evaluation and / or image reporting of radiological image data and / or nuclear medicine image data and / or ultrasound image data and / or optical image data, but they can also be used for evaluation and / or analysis for general vision tasks.

[0063] In summary, the aspects of the present invention provide methods and data processing devices or systems as well as computer program products which exclude healthy biological systems and regions from further diagnostics by the principle of exclusion and subsequently analyze and evaluate conspicuous regions for specific localized findings.

[0064] The described aspects, features and embodiments of the invention solve, in comparison to the prior art, essential problems of automatic medical image evaluation and / or image diagnosis through the described multi-stage procedure.

[0065] As described, in a first stage, we train nonspecifically for "no findings" or "normal health"—and thus, conversely, for nonspecific abnormalities. This is also possible unsupervised with a large amount of radiological data from healthy references. Similar to how large language models learn the structure of language and content with large amounts of data, in a first stage, "no findings" or "normal health" can be learned nonspecifically and evaluated globally.

[0066] In a second stage, trained functions such as neural networks or CNNs and derivatives can be used, or new approaches such as vision transformers can be applied. Unlike the state of the art, these can be trained in a more localized manner and, if necessary, also advantageously receive relevant prior information from the first stage. This reduces the need for specially annotated image sets. Vision transformers benefit from the ability of transformers to learn patterns from relatively few individual cases using attention mechanisms.

[0067] The aforementioned multi-stage approach or combination also has the advantage of reducing the number of false positive reports in the overall system, since all general abnormalities are double-checked by a local model.

[0068] A further advantage is that in the second stage, “unclear findings” can also be localized and presented to another decision-making body (e.g., a human radiologist or a computer-implemented second model, etc.) for review, analysis and / or evaluation.

[0069] It is also possible to have several alternative models test the same finding in the second stage. The result may be no finding, a finding, or a need for clarification.

[0070] The described multi-stage procedure with an optional third stage for automatic medical diagnosis can achieve extensive automation of a medical procedure and / or a medical device and relieve users of such procedures and devices, such as attending physicians, radiologists or nuclear medicine specialists, of routine work.

[0071] Regardless of the grammatical gender of a particular term, persons with male, female or other gender identities are included. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] US 10,595,727 B2 [0005, 0012] US 10,7796,917 B2 [0006, 0012] US 10,452,899 B2 [0007, 0012] Cited non-patent literature

[0000] https: / / en.wikipedia.org / wiki / Convolutional_neural_network

[0004]

Claims

[1] Computer-implemented method for automatic medical image evaluation and / or image diagnosis, wherein the computer-implemented method is designed as a multi-stage method and comprises the following stages: - a first stage (103, 104) for the automatic computer-implemented evaluation of recorded and / or stored medical data (ID, SD), in particular medical image data (ID, SD), wherein a probability measure and / or a probability distribution (h ij ) the absence of findings is determined for the recorded medical data (ID) and / or for the stored medical data (SD); and - a second stage (106) for the automatic computer-implemented determination of deviations of the probability measure and / or the probability distribution (h ij ) the absence of findings from at least one probability measure threshold (hS ij) and / or at least one probability measure criterion (hC ij ) the absence of findings in the recorded and / or stored medical data (ID, SD); and optionally - a third stage (107) for the automatic computer-implemented creation of medical findings on the basis of the recorded and / or stored medical data (ID, SD) and the determined deviations of the determined probability measure and / or the determined probability distribution (h ij ) the absence of findings from at least one probability measure threshold (hS ij ) and / or at least one probability measure criterion (hC ij ) of freedom from findings, and for the automatic computer-implemented provision of the created medical findings to a user by means of a presentation unit (DU). [2] Computer-implemented method for automatic medical image evaluation and / or image diagnosis according to claim 1, wherein the computer-implemented method comprises the following steps: - automatic reading (101) of acquired medical image data (ID) and / or of stored medical data (SD) which represent at least one one-dimensional or two-dimensional or three-dimensional medical image content (B(r)); - automatically applying (103) a first function (TF1) trained using predominantly finding-free and / or exclusively finding-free training data (TD1) and computer-implemented in a neural network computing unit (NNCU) to the acquired medical image data (ID) and / or to the stored medical data (SD); - automatically determining (104) a location-dependent probability measure and / or a spatial probability distribution (h ij(r)) the absence of findings for the acquired medical image data (ID) and / or for the stored medical data (SD) using the trained computer-implemented function (TF1); and - automatically displaying (105) a spatial representation of the location-dependent probability measure and / or the spatial probability distribution (h ij (r)) by means of a display unit (DU); and / or - automatically applying (106) a second function (TF2) trained by means of training data (TD2) containing findings and computer-implemented in a neural network computing unit (NNCU) to such acquired medical image data (ID) and / or to such stored medical data (SD), the determined probability measure (h ij ) a predefined and / or learned probability measure threshold (hS ij) and / or a probability measure criterion (hC ij ) for the automatic determination of diagnostic data; and - automatic determination and presentation of findings data (107) for a user by means of a presentation unit (DU). [3] Computer-implemented method for automatic medical image evaluation and / or image diagnosis according to claim 1 or 2, wherein the computer-implemented method comprises the following steps: - automatic reading (101) of acquired medical image data (ID) and / or of stored medical data (SD) which represent at least one one-dimensional or two-dimensional or three-dimensional medical image content (B(r)); - automatic segmentation (102) and / or encoding of the image data (ID, SD) into a plurality of image segments (IS ij ); - automatically applying (103) a first function (TF1) trained using predominantly finding-free and / or exclusively finding-free training data (TD1) and computer-implemented in a neural network computing unit (NNCU) to the image segments (IS ij ); - automatically determining (104) a probability measure (h ij ) of the absence of findings for each of the image segments (IS ij ) using the trained computer-implemented function (TF1); and - automatically displaying (105) a spatial representation of the probability measure (h ij (r)) by means of a display unit (DU); and / or - automatically applying (106) a second function (TF2) trained by means of training data (TD2) with findings and computer-implemented in a neural network computing unit (NNCU) to image segments (IS ij ) with a probability measure (h ij) which has a predefined and / or learned probability measure threshold (hS ij ) and / or a probability measure criterion (hC ij ) for the automatic determination of diagnostic data; and - automatic determination and presentation (107) of findings data for a user by means of a presentation unit (DU). [4] Computer-implemented method for automatic medical image evaluation and / or image diagnosis according to one of claims 1 to 3, wherein the computer-implemented method additionally comprises the following steps: - automatic training (100a) of a first computer-implemented function (TF1) using predominantly free of findings and / or exclusively free of findings image training data (TD1) for learning parameters for automatically characterizing a probability measure (h) of the freedom from findings of image data (ID) and / or of at least one probability measure threshold value (hS ij ) the absence of findings from image data (ID) and / or from at least one probability measure criterion (hC) correlated with medical image content (B(r)) ij ) the absence of findings in image data (ID), as well as - automatically training (100b) a second computer-implemented function (TF2) using training data (TD2) containing findings in order to learn parameters for the automatic characterization of findings data. [5] Computer-implemented method for automatic medical image evaluation and / or image diagnosis according to one of claims 1 to 4, wherein the computer-implemented method comprises the following steps: - automatically applying (106) a second function (TF2) trained by means of training data (TD2) with findings and computer-implemented in a neural network computing unit (NNCU) to image segments (IS ij ) with a probability measure (hij) which exceeds a predefined and / or learned probability measure threshold (hS ij ) and / or a probability measure criterion (hC ij ), including and / or correlating with stored medical data (SD), for the automatic determination of diagnostic data; and - automatic display (107) of recorded medical image data (ID) and / or stored medical data (SD) as well as of automatically determined findings data (MD(r)), in particular of location-dependent findings data (MD(r)), for a user by means of a display unit (DU). [6] Computer-implemented method for automatic medical image evaluation and / or image diagnosis according to one of claims 1 to 5, wherein the computer-implemented method comprises the following steps: - automatic training (100b) of a second computer-implemented function (TF2) using training data (TD2) containing findings and using stored medical data (SD) to learn parameters for the automatic characterization of findings data and to learn a language model for the linguistic description of automatically determined and / or stored findings data; - automatically applying (106) the second function (TF2), which is trained by means of training data (TD2) containing findings and is computer-implemented in a neural network computing unit (NNCU), for automatically determining findings data and for automatically creating a linguistic description on the basis of the learned language model to such acquired medical image data (ID) and / or to such stored medical data (SD) and / or to such image segments (IS ij ) with a probability measure (h ij ) which has a predefined and / or learned probability measure threshold (hS ij ) and / or a probability measure criterion (hC ij ) is fulfilled by including and / or correlating with stored medical data (SD); and - automatic display (107) of the determined findings data as well as the linguistic description of the determined findings data automatically created on the basis of the learned language model for a user by means of a display unit (DU). [7] Computer-implemented method for automatic medical image evaluation and / or image diagnosis according to one of claims 1 to 6, wherein the computer-implemented method comprises the following steps: - automatic segmentation (102) and / or encoding of the image data (ID, SD) by means of a neural transformer network architecture and / or by means of a generative neural network architecture into a plurality of image segments (IS ij ); - automatically applying (103) a first neural network function (TF1) trained using predominantly finding-free and / or exclusively finding-free training data (TD1) and computer-implemented in a neural network computing unit (NNCU) to the image segments (IS ij ); and - automatically applying (106) a second neural network function (TF2) trained using training data (TD2) and computer-implemented in a neural network computing unit (NNCU) to image segments (IS ij ) with a probability measure (h ij ) which has a predefined and / or learned probability measure threshold (hS ij ) and / or a probability measure criterion (hC ij ) for the automatic determination of diagnostic data. [8] A computer-implemented training method for automatically training computer-implemented trainable functions using medical image data, the computer-implemented training method comprising the following steps: - automatic training (100a) of a first computer-implemented function (TF1) using predominantly free of findings and / or exclusively free of findings image training data (TD1) for learning parameters for automatically characterizing a probability measure (h) of the freedom from findings of image data (ID) and / or of at least one probability measure threshold value (hS ij ) the absence of findings from image data (ID) and / or from at least one probability measure criterion (hC) correlated with medical image content (B(r)) ij ) the absence of findings in image data (ID), as well as - automatic training (100b) of a second computer-implemented function (TF2) using training data (TD2) containing findings in order to learn organ-specific and / or tissue structure-specific parameters for the automatic characterization of disease-specific findings data. [9] A computer-implemented training method according to claim 8, wherein the computer-implemented training method comprises the following step: - automatic training (100b) of a second computer-implemented function (TF2) using training data (TD2) containing findings and using stored medical data (SD) to learn organ-specific and / or tissue structure-specific parameters for the automatic characterization of disease-specific findings data and to learn a language model for the linguistic description of automatically determined and / or stored findings data. [10] Medical-technical data processing unit or medical-technical system (SYS), in particular designed to carry out a computer-implemented method according to one of claims 1 to 7, wherein the medical-technical data processing unit or the medical-technical system (SYS) has at least the following units which are in data communication with one another: - at least one image acquisition unit (IDCU) designed to automatically capture medical image data (ID); - at least one storage unit (SDMU) designed to store medical data (SD); - at least one image data processing unit (IDPU) designed for the automatic processing of medical image data (ID, SD), in particular designed for the automatic segmentation (102) and / or encoding of medical image data (ID, SD) into a plurality of image segments (IS ij ); - at least one neural network computing unit (NNCU) designed to automatically apply trainable and / or trained computer-implemented functions (TF1, TF2) to medical image data (ID) and / or to stored medical data (SD); - at least one display unit (DU) designed for the automatic display of medical image data (ID) and / or stored medical data (SD) and / or medical findings data (MD(r)); wherein the neural network computing unit (NNCU) is designed to - automatically applying (103) a first computer-implemented neural network function (TF1) trained using predominantly free and / or exclusively free training data (TD1) to the image segments (IS ij ); and to - automatically applying (106) a second computer-implemented neural network function (TF2) trained using training data (TD2) to image segments (IS ij ) with a probability measure (h ij ) which has a predefined and / or learned probability measure threshold (hS ij ) and / or a probability measure criterion (hC ij ) for the automatic determination of diagnostic data. [11] A computer program product or computer program system comprising a plurality of computer program product modules, comprising instructions which, when the computer program product or the computer program product modules are executed by a data processing unit, cause this data processing unit to execute program steps for carrying out a computer-implemented method according to one of claims 1 to 7; in particular in cooperation with a medical-technical data processing unit or medical-technical system (SYS) according to claim 10, comprising instructions which, when the computer program product or the computer program product modules are executed by a data processing unit, cause this data processing unit to execute the following computer-implemented program steps: - program steps (S103, S104) of a first program stage for the automatic computer-implemented evaluation of recorded and / or stored medical data (ID, SD), in particular medical image data (ID, SD), wherein a probability measure and / or a probability distribution (h ij ) the absence of findings is determined for the recorded medical data (ID) and / or for the stored medical data (SD); and - Program steps (S106) of a second program stage for the automatic computer-implemented determination of deviations of the probability measure and / or the probability distribution (h ij ) the absence of findings from at least one probability measure threshold (hS ij ) and / or at least one probability measure criterion (hC ij ) the medical data recorded and / or stored (ID, SD) of the absence of findings; and optionally - Program steps (S107) of a third program stage for the automatic computer-implemented creation of medical findings based on the recorded and / or stored medical data (ID, SD) and the determined deviations of the determined probability measure and / or the determined probability distribution (h ij ) the absence of findings from at least one probability measure threshold (hS ij ) and / or at least one probability measure criterion (hC ij ) of freedom from findings, and for the automatic computer-implemented provision of the created medical findings to a user by means of a presentation unit (DU). [12] A computer program product or computer program system comprising a plurality of computer program product modules according to claim 11, comprising instructions which, when the computer program product or the computer program product modules are executed by a data processing unit, cause this data processing unit to execute the following computer-implemented program steps: - Program step for the automatic computer-implemented reading (S101) of acquired medical image data (ID) and / or of stored medical data (SD) which represent at least one one-dimensional or two-dimensional or three-dimensional medical image content (B(r)); - Program step for the automatic computer-implemented application (S103) of a first function (TF1) trained using predominantly finding-free and / or exclusively finding-free training data (TD1) and computer-implemented in a neural network computing unit (NNCU) to the acquired medical image data (ID) and / or to the stored medical data (SD); - Program step for the automatic computer-implemented determination (S104) of a probability measure (h ij ) the absence of findings for the acquired medical image data (ID) and / or for the stored medical data (SD) using the trained computer-implemented function (TF1); and - Program step for the automatic computer-implemented display (S105) of a spatial representation of the probability measure (h ij (r)) by means of a display unit (DU); and / or - Program step for the automatic computer-implemented application (S106) of a second function (TF2) trained by means of training data (TD2) containing findings and computer-implemented in a neural network computing unit (NNCU) to such acquired medical image data (ID) and / or to such stored medical data (SD), the determined probability measure (h ij ) a predefined and / or learned probability measure threshold (hS ij ) and / or a probability measure criterion (hC ij ) for the automatic determination of diagnostic data; and - Program step for the automatic computer-implemented display (S107) of the determined findings data for a user by means of a display unit (DU). [13] A computer program product or computer program system comprising a plurality of computer program product modules according to claim 11, comprising instructions which, when the computer program product or the computer program product modules are executed by a data processing unit, cause this data processing unit to carry out the following steps: - Program step for the automatic computer-implemented reading (S101) of acquired medical image data (ID) and / or of stored medical data (SD) which represent at least one one-dimensional or two-dimensional or three-dimensional medical image content (B(r)); - Program step for automatic computer-implemented segmentation (S102) and / or encoding of the image data (ID, SD) into a plurality of image segments (IS ij ); - Program step for the automatic computer-implemented application (S103) of a first function (TF1) trained using predominantly finding-free and / or exclusively finding-free training data (TD1) and computer-implemented in a neural network computing unit (NNCU) to the image segments (IS ij ); - Program step for the automatic computer-implemented determination (S104) of a probability measure (h ij ) of the absence of findings for each of the image segments (IS ij ) using the trained computer-implemented function (TF1); and - Program step for the automatic computer-implemented display (S105) of a spatial representation of the probability measure (h ij (r)) by means of a display unit (DU); and / or - Program step for the automatic computer-implemented application (S106) of a second function (TF2) trained using training data (TD2) and computer-implemented in a neural network computing unit (NNCU) to image segments (IS ij ) with a probability measure (h ij ) which has a predefined and / or learned probability measure threshold (hS ij ) and / or a probability measure criterion (hC ij ) for the automatic determination of diagnostic data; and - Program step for the automatic computer-implemented display (S107) of the determined findings data for a user by means of a display unit (DU). [14] A computer program product or computer program system comprising a plurality of computer program product modules according to claim 11, comprising instructions which, when the computer program product or the computer program product modules are executed by a data processing unit, cause this data processing unit to carry out the following steps: - Program step for the automatic computer-implemented training (S100a) of a first computer-implemented function (TF1) using predominantly free of findings and / or exclusively free of findings image training data (TD1) for learning parameters for the automatic characterization of a probability measure (h) of the freedom from findings of image data (ID) and / or of at least one probability measure threshold value (hS ij ) the absence of findings from image data (ID) and / or from at least one probability measure criterion (hC) correlated with medical image content (B(r)) ij) the absence of findings in image data (ID), as well as - Program step for the automatic computer-implemented training (S100b) of a second computer-implemented function (TF2) using training data (TD2) containing findings for learning organ-specific and / or tissue structure-specific parameters for the automatic characterization of disease-specific findings data. [15] Use of a computer-implemented method for automatic medical image evaluation and / or image diagnosis according to one of claims 1 to 7 and / or use of a medical data processing unit or a medical system (SYS) according to claim 10 and / or use of a computer program product or computer program system according to one of claims 11 to 14 for automatic medical image evaluation and / or image diagnosis of radiological image data and / or nuclear medicine image data and / or ultrasound image data and / or optical image data.

Images