Method and system for creating a medical image database using a convolutional neural network

Description

[0001] The invention relates to a method and a system for creating a medical image database according to the preamble of patent claim 1 and patent claim 13, respectively.

[0002] The invention relates to the technical field of image processing, particularly medical image processing, for example, of medical image data resulting from imaging diagnostic methods. These include, for example, two- or three-dimensional CT or MRI images, ultrasound images, or microscopic images. Preferred aspects of the invention also relate to methods for processing medical texts, such as those from specialist journals, written diagnostic texts, and patient reports.

[0003] The main challenges in this area are the identification of the features relevant for the diagnosis, which can provide reliable support to the physician concerned.

[0004] Especially in the medical field, the search for image data, such as X-rays or computed tomography images, which show similar anatomical structures, changes or symptoms of disease and can therefore be assigned to similar cases of disease, is very difficult and time-consuming, especially if only visual information is available as a basis for the search, since a disease often only causes small-scale changes in the human body.

[0005] However, particularly in the clinical field, a precise search for similar image data that can be carried out quickly, even in extensive medical databases, is desirable in order to be able to identify similar cases of illness for a current case of illness as quickly as possible and, based on this, to make a diagnosis for the current case of illness.

[0006] US 6,760,714 B1 describes a method in which image features are generated by performing wavelet transforms on sample points of images stored in electronic form. Multiple wavelet transforms at a point are combined to form an image feature vector. A prototypical set of feature vectors, or atoms, is derived from the set of feature vectors to form an "atom vocabulary." The prototypical feature vectors are derived using a vector quantization method, which also generates a vector quantization network. The atomic vocabulary is used to define new images. Meaning is established between atoms in the atomic vocabulary. Each atom is associated with high-dimensional context vectors. The context vectors are then trained as a function of the proximity and co-occurrence of each atom to other atoms in the image.After training, the context vectors associated with the atoms that comprise an image are combined to form a summary vector for the image. Images are retrieved using a number of query methods (e.g., images, image parts, vocabulary atoms, index terms).

[0007] The object of the invention is therefore to provide a method for the efficient creation of medical image databases so that they can be searched precisely and quickly. In particular, the object of the invention is to perform an image search based on content criteria. In this context, it is advantageous if modern deep learning methods, such as convolutional neural networks (CNN) or recurrent neural networks (RNN), can also be used.

[0008] The invention solves this problem by a method for creating a medical image database, wherein a) data sets are specified which comprise partial images of two-dimensional or multi-dimensional output images of parts of the human body, and wherein for each partial image of an initial image, the respective position in an anatomical reference atlas is known, and the individual initial images or partial images are optionally provided with additional textual, numerical or semantic information, and a partial image can also correspond to an entire initial image, with the characterising features of patent claim 1.

[0009] According to the invention, it is provided that b) a projection is created to obtain feature vectors from the partial images, which projection maps similar partial images, in particular visually or semantically, onto feature vectors with a small distance, and wherein, in preparation for the execution of the projection, a neural network, in particular a convolutional neural network, is created on the basis of predetermined learning partial images, wherein the data sets or a part of the data sets are used by the neural network as part of a metric learning process to learn the projection and the creation of the feature vectors from learning partial images or groups of learning partial images as well as a predetermined similarity to be achieved between the learning partial images, wherein the metric learning process in question [schroff2015] is based on one or more of the following specifications: Specification of n-tuples of learning partial images or groups of learning partial images as similar, which are slightly shifted or rotated relative to one another,are distorted or stretched and were created from the same source image and / or specifying n-tuples of learning sub-images or groups of learning sub-images as similar, which are created from the same sub-area of ​​the source image, wherein at least one of the learning sub-images is modified compared to the sub-area of ​​the source image such that the learning sub-images have different noise and / or different image intensity and / or different contrast and / or specifying n-tuples of sub-areas originating from the same source image or groups of source images as learning sub-images, wherein the similarity to be achieved between the respective learning sub-images and images of the n-tuple depends on the spatial distance of the relevant sub-areas in the source image, wherein in particular learning sub-images are considered to be more similar the closer the relevant sub-areas are to each other in the source image,and / or creating a compressed representation of the information contained in a partial image, learning partial images or groups of learning partial images from the same source image 4a, 4b, 4c or from different source images 4a, 4b, 4c, which are to be regarded as similar due to external features stored with the respective source images 4a, 4b, 4c, such as textual, numerical or semantic information. c) wherein the projection is applied to the partial images of the data sets and / or to a number of further partial images of further data sets and accordingly at least one feature vector is obtained for each of these partial images, and d) wherein the feature vectors created in this way, in particular linked to the source images and / or data sets or further data sets, are stored in an index data structure, wherein the respective position of the partial images of an initial image is determined relative to the human body, and in particular, wherein the information on the position of the partial images is used by a neural network to learn a projection for estimating the positions of partial images, wherein the projection is learned with the objective function that by mapping pairs or groups of partial images the spatial constellation of the pairs / groups before and after the projection is similar and wherein the projection is learned based on a known mapping of partial images to positions wherein in addition to the feature vectors the learned or known position information is stored in the database wherein when a search request is present the searched position in the body is determined and the database is searched for feature vectors of partial images for which the same position is stored in their data sets or whose position does not exceed a distance threshold value specified by the user from the searched position.

[0010] The invention is particularly advantageous because, for example, a radiologist can use the image database created in this way to locate similar partial images stored in the created database for a given medical image or partial image, which may contain additional information. Likewise, similar cases of partial images can be extracted from the database, which contain improved information, for example, for the radiologist. The invention enables a particularly rapid and effective search in the database for image data of medical cases relating to specific body parts.

[0011] For example, in order to efficiently store information from a patient’s medical record in the database, it can be provided that additional information is stored in the data records, whereby the additional information is specified as text information and / or semantic information and / or numerical information and that optionally, if text information or numerical information is available, the text information or numerical information is stored in the database as tags and / or semantic representations

[0012] In order to be able to obtain feature vectors from the partial images which can be searched particularly precisely and quickly, it can be provided that the additional information is used by the neural network to create the projection, whereby the projection is created in such a way that learning partial images are specified as similar, which originate from source images or correspond to partial images to which the same additional information is assigned.

[0013] In order to carry out a precise, computing-efficient search in the database, for example for image data on specific cases of illness, it can be provided that the user provides text information, numerical information and / or semantic information to create a search query, that optionally the text information or numerical information is converted into tags and / or semantic representations that, when a search query is present, the database is searched for feature vectors that have similar tags and / or semantic representations and that, to create a search query, at least one query image, in particular of a region of interest, is selected from at least one two- or multi-dimensional examination image or in an examination image sequence, a feature vector is determined for the query image according to step c of claim 1 according to the learned projection, the database is searched for data records with feature vectors that are close to the feature vector of the query image based on a predetermined metric, and

[0014] As a result of the search query, data records are output whose subimages have a similar appearance to the selection area or which are semantically relevant.

[0015] In order to be able to carry out a search in a database comprising two-dimensional image data, for example starting from three-dimensional image data, it can be provided that in order to search for similar images for an examination image or an examination image sequence, a dimensionally reduced and, if necessary, reduced in its dimensions query sub-image is created by creating a section and this query sub-image is used for the search query.

[0016] In order to keep the number of search results as low as possible so that only the search results most relevant to the user are determined and output, it can be provided that when searching for similar feature vectors in the database, search results are excluded based on the tags and / or semantic representations stored in the associated data records in the database, in particular that search results are excluded based on criteria specified by the user for the tags and / or semantic representations and / or that as a result of the search query, a ranking of the data records determined in each case is created and output according to their similarity or correspondence to the tags and / or semantic representations of the search query and / or according to their similarity to at least one query image of the search query, and / or as determined by the distance of the relevant feature vectors.

[0017] In order to provide a user with not only image data but also additional information or statistical data as a result of the search query, it can be provided that the text information, numerical information and / or the semantic information stored as tags in the respective data records are output as a result of the search query and / or that the tags of the partial images similar to the selection area are statistically evaluated and the statistical result is output.

[0018] To ensure the protection of patient data, for example when a user submits a search query to the database via the Internet, it can be provided that the examination images and / or examination image sequences and / or text information and / or numerical information and / or semantic information underlying the search query are anonymized before the user submits the search query to the database.

[0019] In order to prepare the results obtained for a search query in a structured manner for the user, it can be provided that, without a search query, groups of partial images and / or text information and / or numerical information and / or semantic information with similar feature vectors are created and output directly on the basis of the data available in the database and, if necessary, additional information regarding these groups is output, and / or that in response to a user's search query, neighboring groups of partial images and / or text information and / or numerical information and / or semantic information with similar feature vectors are determined and output in the database and, if necessary, additional information regarding these groups is output.

[0020] In order to be able to carry out a cascade-like search in succession in several databases, with each further search query being based on the search results already determined, it can be provided that the search results determined as a result of a first search query to the database, in particular partial images, group information, text information, numerical information and / or semantic information, are used to create at least one further search query, and that the further search query is transmitted to at least one further database, and / or that further information output to the database as a result of the first search query is used to create the at least one further search query to the at least one further database, in particular that output statistical results and / or output images and / or similar partial images are used, wherein the dimension of the output original images and / or the output partial images for the further search query in the at least one further database is reduced if necessary.

[0021] In order to create a particularly precise search query using search results from a database to which the user does not have direct access, it can be provided that individual data records from the database are used exclusively for the creation of a projection function, but are not made available to the user for viewing as query results.

[0022] In order to be able to use source images or partial images whose pixel or voxel dimensions are unknown to create the database or to create a search query for the database, it can be provided that size information regarding the pixel dimensions or voxel dimensions of the source images or partial images is stored in the data sets, or that the pixel dimensions or voxel dimensions are specified for the source images or partial images by searching for similar reference source images or partial images with known pixel or voxel dimensions, in particular those originating from the same body part. subsequently, by image comparison, a scaling is sought with which the source image or partial image can be optimally aligned with the reference source image or reference partial image and, based on this scaling and the known pixel dimensions or voxel dimensions of the reference source image or reference partial image, the pixel dimensions or voxel dimensions of the source image or partial image are determined and stored in the database.

[0023] In a preferred embodiment of the invention, images can be found that are similar to a given search image.

[0024] In a further advantageous embodiment of the invention, images relevant to a query can be learned based on a target similarity learned through training. This target similarity can be learned through training based on known images. In particular, a distance function that specifies a distance between two images can also be learned through training.

[0025] A further advantageous embodiment of the invention enables an improved determination of a distance function between two images based on an improved database comprising information about the patient in question, curves, diagnoses, data on the course of the disease, prognoses, and image data. After training, an approximate target similarity can be determined based on a portion of the data, for example, solely on the biomedical image data. This target similarity can be used to later make diagnoses or prognoses regarding the course of the disease. This embodiment of the invention is capable of creating a distance function that determines the essential characteristic of the target similarity even based on only partially available data, such as, in particular, solely on the image data.

[0026] In a preferred embodiment of the invention, it is also possible to perform training based on similarity or distance functions, the creation or maintenance of an index exclusively based on CNNs for image processing and RNNs for text processing.

[0027] A further advantage of preferred embodiments of the invention is the retrieval of relevant cases, patents, or patent data based on available image information of a source case or query case, whereby the user only has one image available as a search basis for the query. A direct association between similar cases is usually very difficult to model due to semantic differences in the data.

[0028] In a preferred embodiment of the invention, a distance function for ranking search results can be determined based on a search region within the image specified by the user, optionally in combination with further patient information such as age or gender.

[0029] In a further embodiment of the invention, the most relevant or highest-ranked results for the user can be visually displayed. These results can be confirmed by the user based on their experience.

[0030] It is particularly advantageous that individual embodiments of the invention can learn a multitude of relationships, create an index, determine a distance function and learn statistical evaluations and models on the basis of data that arise in the daily operation of a hospital, without the need for manual annotation.

[0031] It is also particularly advantageous that, in individual embodiments of the invention, additional data sources such as specialist articles, recordings, images and representations, teaching materials and similar additional information can be included in the search, thus making diagnosis and evaluation significantly easier. This is made possible in particular by the simple use of a data model that was obtained from a large database that has accrued in the daily operation of a hospital. It is also possible to conduct a search or evaluation based on a query image on the basis of a broad database, but the individual data records underlying the database do not need to be displayed, since in particular only other data records can be accessed.

[0032] Another preferred embodiment of the invention uses 2D, 3D or higher dimensional images, for example radiological images, CT images or MR images, to provide the user with visually similar medically relevant cases from a hospital-internal database.

[0033] Another embodiment of the invention uses the available semantic information associated with individual diagnostic reports for training. An automated full-text search in medical literature can also be used.

[0034] A further preferred embodiment of the invention relates to a system according to claim 13 for, in particular, the simultaneous search of medical image data, medical text information, and semantic information based on a user's search query consisting of an image and a region of interest. The region of interest can also comprise the entire image. The system determines a ranking of relevant data records as a result of the search query. More relevant or more similar data records are ranked higher than less relevant or less similar data records. Typically, data records consist of: medical image data such as, but not limited to, computed tomography or magnetic resonance imaging image data, medical text information such as, but not limited to, radiological reports, medical websites, publications, literature, didactic information sources or other material, semantic information such as, but not limited to, marked sub-areas of examination images and semantic information from clinical texts and

[0035] To carry out the individual operations, the system has an indexing unit, a learning unit and a search unit.

[0036] Preferably, an indexing unit and a learning unit are trained with data from different areas, such as (1) medical image data such as, but not exclusively, computed tomography or magnetic resonance imaging image data, and simultaneously (2) image data from documents such as, but not exclusively, websites, publications, literature, didactic information sources or other material and simultaneously (3) semantic information.

[0037] Preferably, the indexing unit and the learning unit will be additionally trained with patient-specific data such as radiological reports and semantic information from radiological reports.

[0038] It is particularly advantageous if the learning unit and the indexing unit use semantic labels or text information from the domain as additional cost terms in the training and indexing of data from the other domain.

[0039] Furthermore, it is advantageous if the learning unit implicitly focuses on sub-areas of image data that are semantically meaningful, such as partial images based on data from other domains such as computed tomography or magnetic resonance imaging volumes.

[0040] It is also advantageous if the learning unit can additionally determine the scale of image data in one domain based on known scales in the other domain, even if the physical size or resolution in the first domain is unknown. The indexing unit also indexes the determined scales together with the image data.

[0041] Finally, it is advantageous if the learning unit can train a CNN that maps blocks into a common spatial reference frame, whereby the resulting model can be used to map individual blocks into the reference frame, but also to map entire volumes into this reference frame and thus register the image data.

[0042] Furthermore, it can advantageously be provided that the learning unit can estimate a position, in particular of certain image sections to be determined, by means of CNNs, wherein the indexing unit additionally indexes the estimated position together with the image data, or wherein the search unit automatically determines the position of a query image and the positions in the query image in an anatomical reference reference system, thereby enabling the display and further use, for example for enriching the search query, of reference coordinates, or names of anatomical structures, or illustrations of anatomical structures as a result of the search query.

[0043] Furthermore, it is advantageous if the learning unit, given one or more words, can make predictions regarding the probability that one of a series of semantic terms or identifiers is present that are given in a terminology or ontology such as RadLex, MESH, Snomed or others.

[0044] Furthermore, it is advantageous if the learning unit trains an RNN, given one or more words, whose probability of representing one of a series of semantic identifiers of medical terms can be determined. Additionally, a score can be trained and predicted that indicates, for example, "missing," "more likely present," or "present" with respect to these identifiers. The training can advantageously combine a prediction cost function with additional cost functions to model word or character sequences.

[0045] The partial images do not necessarily have to be sections or blocks and can also have a non-cubic or non-rectangular shape, e.g. circular or spherical.

[0046] The indexing unit preferably stores a compact representation of the partial image, section or block of each image in addition to the metadata or additional information such as patient age, block position, semantic information, in a data structure optimized for fast searching.

[0047] Advantageously, the search engine uses the index to find the most similar results, with metadata being able to be used as search constraints if desired.

[0048] A further advantageous development of the invention provides that the data structure is used in the indexing unit to search the indexed data for structures, e.g., finding clusters in the data or metadata.

[0049] Furthermore, it can advantageously be provided that the indexing unit stores indices of the terms or text information in the form of paragraph vectors or word vectors.

[0050] The indexing unit advantageously learns and processes low-dimensional separable filters to increase search speed.

[0051] It can be particularly advantageous for the search unit to combine indices of image data, text data and semantic data.

[0052] It can be particularly advantageous for the search unit to determine a result of ranked relevant data records or cases, so that more relevant or similar data records are ranked higher than less relevant or less similar data records, where the ranking is determined by a projection (representation function) that takes relevant semantic similarity into account, even if it only receives the image and an ROI as input information, where the projection (representation function) is based on CNNs and RNNs trained by a relative position-oriented cost function and / or a semi-supervised cost function or a weakly supervised cost function.

[0053] Furthermore, it is also possible to advantageously cascade the method according to the invention in order to gain additional information within the framework of the cascade.

[0054] It can advantageously be provided that the search unit enriches a user's search query with the results of search queries in various indices from different domains, for example, with images or text information, or semantically, in order to perform a subsequent search query. This search can provide relevant or more relevant information, especially if it is used to create further queries.

[0055] Finally, it may be provided that the search data provided by the user or another system is anonymized in the browser before the data is transmitted.

[0056] Further advantages and embodiments of the invention will become apparent from the description and the accompanying drawings.

[0057] Particularly advantageous, but not limiting, embodiments of the invention are shown schematically below with reference to the accompanying drawings and described by way of example with reference to the drawings: Fig. 1 shows schematically the creation of a data record in a database. Fig. 2 shows schematically a search query to the database Fig. 1 . Fig. 3 shows schematically the process of a multi-database search procedure Creation of a database with medical image data

[0058] Fig. 1 shows a schematic representation of a database 2 with medical image data, which is recorded, for example, in a hospital during the examination of patients. The database 2 is accessible on a server in the hospital or on the Internet and contains Fig. 1several data sets 1a, 1b, 1c, each data set 1a, 1b, 1c comprising partial images 3a, 3b, 3c of two- or multi-dimensional output images 4a, 4b, 4c of parts of the human body (see Fig. 2 ).

[0059] To create the database 2, medical image data, such as image data obtained using radiological procedures or nuclear magnetic resonance procedures, are first specified as source images 4a, 4b, 4c. These include, for example, two-dimensional X-ray images, sonographic image data or data from imaging microscopy procedures, but also three-dimensional X-ray or magnetic resonance images or four-dimensional contrast agent image sequences. In the source images 4a, 4b, 4c, large numbers of sub-regions are systematically selected, for example by shifting a pixel grid with one or different grid sizes along the respective source image 4a, 4b, 4c, whereby the selected sub-regions are stored as sub-images 3a, 3b, 3c in the data sets 1a, 1b, 1c of the database 2.By selecting partial images 3a, 3b, 3c, for example, by systematically shifting a pixel grid along the respective source image 4a, 4b, 4c, their position in the source image 4a, 4b, 4c is also known, and from each source image 4a, 4b, 4c, for example, 100,000 partial images 3a, 3b, 3c are selected. However, one partial image 3a, 3b, 3c can also correspond to an entire source image 4a, 4b, 4c.

[0060] The Fig. 1 The output image 4a of data set 1a shown is a three-dimensional computed tomography image of a patient's lung. In the output image 4a of data set 1a, it can be seen that a portion of the patient's right lung exhibits changes. The output images 4b, 4c and the selected partial images 3b, 3c of data sets 1b, 1c, which are also shown in the Fig. 1 schematically represented database 2 are stored in Fig. 2 shown.

[0061] When creating the database 2, a projection is created to obtain feature vectors 6a, 6b, 6c from the partial images 3a, 3b, 3c in order to save memory space and to quickly search the database 2 for content-relevant partial images. The projection maps, in particular visually or semantically, similar partial images 3a, 3b, 3c onto similar feature vectors 6a, 6b, 6c and, when applied to a partial image 3a, 3b, 3c, delivers a feature vector 6a, 6b, 6c, wherein in particular the number of entries of a respective feature vector 6a, 6b, 6c is less than the number of pixels of the partial images 3a, 3b, 3c.

[0062] This reduction in the number of entries in a feature vector 6a, 6b, 6c compared to the number of pixels in a partial image 3a, 3b, 3c advantageously leads to faster searchability of the database 2. Instead of a time-consuming and computationally intensive search of the database 2 for similar source images 4a, 4b, 4c or partial images 3a, 3b, 3c of the source images 4a, 4b, 4c, a search for similar feature vectors 6a, 6b, 6c that represent the partial images 3a, 3b, 3c or the source images 4a, 4b, 4c is sufficient. Furthermore, this projection allows partial images 3a, 3b, 3c with different appearances but which contain similar semantic information, such as the same disease, to be mapped to similar feature vectors, which will be discussed in more detail below.Thus, partial images 3a, 3b, 3c are also mapped to similar feature vectors 6a, 6b, 6c, on which, for example, the imaged tissue shows changes with different visual appearance, which, however, can be assigned to the same disease.

[0063] To prepare for the projection execution, a neural network, specifically a convolutional neural network, is created based on predefined learning subimages from a training unit. The data sets 1a, 1b, 1c, or a subset of the data sets 1a, 1b, 1c, are used by the neural network as part of a metric learning process to learn the projection and the creation of the feature vectors 6a, 6b, 6c from the learning subimages, as well as a predefined similarity to be achieved between the learning subimages.

[0064] To effectively learn the projection or the creation of the feature vectors 6a, 6b, 6c, the metric learning method in question [e.g. yang2006, schroff2015] is given n-tuples of learning sub-images or groups of learning sub-images of one or more of the following types as similar: Training partial images that are slightly shifted, rotated, distorted or stretched relative to one another and are created starting from the same source image 4a, 4b, 4c and / or training partial images that are created starting from the same partial area of ​​the source image 4a, 4b, 4c, wherein at least one of the training partial images is modified compared to the partial area of ​​the source image 4a, 4b, 4c such that the training partial images have different noise and / or different image intensity and / or different contrast and / or training partial images of partial areas originating from the same source image 4a, 4b, 4c, wherein the similarity to be achieved between the respective training partial images of the n-tuple depends on the spatial distance of the respective partial areas in the source image 4a, 4b, 4c, wherein in particular training partial images are considered to be more similar the closer the respective partial areas are in the source image 4a, 4b,4c and / or creating a compressed representation of the information contained in a subimage 3a, 3b, 3c, as exemplified in Bengio, Yoshua, Aaron Courville, and Pascal Vincent. "Representation learning: A review and new perspectives." IEEE transactions on pattern analysis and machine intelligence 35.8 (2013): 1798-1828. , Goodfellow, Lan, et al. "Generative adversarial nets." Advances in neural information processing systems. 2014 ; Learning partial images from the same source image 4a, 4b, 4c or from different source images 4a, 4b, 4c, which are to be regarded as similar due to external features stored with the respective source images 4a, 4b, 4c, such as textual, numerical or semantic information.

[0065] Following this learning phase, the projection is applied by an indexing unit to the partial images 3a, 3b, 3c of the data sets 1a, 1b, 1c and / or to a number of partial images of further data sets, and at least one feature vector is created for each of these partial images. The feature vectors 6a, 6b, 6c created in this way are stored in the data sets, in particular linked to the source images.

[0066] A for the data set 1a of the database 2 in Fig. 1 The exemplary feature vector 6a contains 18 entries and comprises a significantly smaller number of entries than the three-dimensional image section of each partial image 3a shows.

[0067] The individual source images 4a, 4b, 4c or partial images 3a, 3b, 3c specified for creating the database 2 can optionally be provided with additional information 5, wherein the additional information 5 is specified as text information and / or semantic information and / or numerical information. The additional information 5 can, for example, be medical information. For example, it can be stated that a section of the imaged body part shows changes, what type of changes these are, or what disease caused these changes. The additional information 5 can also be personal information such as the age and gender of the patient. This additional information 5 can optionally also be stored in the data records 1a, 1b, 1c in the database 2.In order to store such additional information 5 compactly in the database 2 and to map semantically relevant relationships, text information or numerical information is stored as tags and / or semantic representations 7a, 7b, 7c and / or semantic information as semantic representations 7a, 7b, 7c in the index data structure.

[0068] In the Fig. 1In the example shown, the entries in the patient's medical record are available as additional information 5 for data set 1a in database 2. Instead of text content from the medical record, semantic representations 7a of the entries in the patient's medical record for age, gender, and clinical picture are stored in data set 1a in database 2. The example concerns a 93-year-old male patient who has been diagnosed with lung cancer. In this case, number combinations are stored as semantic representations 7a: "82" represents the patient's gender, "93" the age, and "16" the diagnosed disease. More complex information, such as more extensive descriptions in clinical findings, can be represented by specially trained neural networks such as recurrent neural networks (RNNs), as exemplified in Sundermeyer, Martin, Ralf Schlüter, and Hermann Ney. "LSTM neural networks for language modeling."Thirteenth Annual Conference of the International Speech Communication Association. 2012. ., or Convolutional Neural Networks (CNNs) with attention mechanisms, exemplified in Vaswani, Ashish, et al., "Attention Is All You Need." arXiv preprint arXiv:1706.03762 (2017) ), converted into semantic codes, optionally along with a weighting (such as "absent" / "slightly pronounced" / "possibly present" / "definitely present") and saved. Alternatively, entire paragraphs and reports can be embedding using neural networks such as Paragraph Vectors, as exemplified in Dai, Andrew M., Christopher Olah, and Quoc V. Le. "Document embedding with paragraph vectors." arXiv preprint arXiv:1507.07998 (2015) ), be mapped and stored.

[0069] Optionally, such additional information 5 can be used by the neural network to create the projection, which maps partial images 3a, 3b, 3c to feature vectors 6a, 6b, 6c. The projection is created in such a way that training partial images are specified as similar, which originate from source images 4a, 4b, 4c or correspond to partial images 3a, 3b, 3c to which the same additional information 5 is assigned.

[0070] In the example shown in Fig. 1For example, to learn the projection or create the feature vectors 6a, 6b, 6c, learning partial images are provided in which the imaged lung tissue shows similar changes and to which the same disease name is assigned, for example, as additional information 5. As a result, partial images 3a, 3b, 3c, in which lung tissue with similar changes is depicted and the respective patient suffers from a potentially similar disease, are mapped to similar feature vectors 6a, 6b, 6c.

[0071] Furthermore, the respective position of the partial images 3a, 3b, 3c of an initial image 4a, 4b, 4c relative to the human body is stored in the data sets 1a, 1b, 1c in the database 2. The information on the position of the partial images 3a, 3b, 3c can be used by a neural network, in particular, to learn a projection for finding body parts using feature vectors 6a, 6b, 6c of partial images 3a, 3b, 3c. Search query to the database

[0072] To generate a search query to the database 2, the user first selects at least one query image 3' from at least one two- or multi-dimensional examination image 4' or from an examination image sequence. Additional information 5 that may be available can also be used by a search unit to generate the search query, provided it is available.

[0073] Alternatively or additionally, to create a search query, the user can also provide text information such as a diagnostic text, numerical information such as a patient's age or age group, and / or semantic information such as a disease. The search unit can convert the text information or numerical information into tags and / or semantic representations 7a, 7b, 7c. Subsequently, it searches for feature vectors 6a, 6b, 6c that map similar tags and / or semantic representations 7a, 7b, 7c.

[0074] For example, it is possible to create a search query for records associated with a specific patient name or disease. For example, if a user creates a search query with "wrist fracture" as text information, the given text information is converted into a semantic representation, in this example the number combination 53, and transmitted to database 2. The result of the search is, in the example, Fig. 2 In this case, the data set 1b is output, in which a corresponding semantic representation 7b is stored.

[0075] To create a search query based on image information to the database 2, one or more feature vectors 6' of the query image 3' are first determined for the query image 3' as described above according to the learned projection. Subsequently, the database 2 is searched for data sets 1a, 1b, 1c with feature vectors 6a, 6b, 6c that are close to the feature vector 6' of the query image 3' based on a predetermined metric. The search results output optionally sorted partial images 3a, 3b, 3c that have a similar appearance to the selection area, optionally together with or replaced by the data sets 1a, 1b, 1c.

[0076] The sorting of the resulting partial images is determined by the similarity of the corresponding feature vectors to the feature vector(s) of the query image. The distances of the data sets of 1a, 1b, 1c to the query image, and thus an optional sorting, can be determined, for example, by accumulating the distances of the result vectors for each data set by the number of result vectors per data set within a selected similarity threshold, by analyzing the spatial configuration of the partial images corresponding to the result vectors within the result data sets, and by analyzing the additional information stored in the database for the result vectors.

[0077] Thus, partial images 3a, 3b, 3c, which depict, for example, similar clinical pictures, can be found in the database 2 using the feature vectors 6a, 6b, 6c assigned to them, without having to perform a time-consuming and computationally intensive search directly for partial images 3a, 3b, 3c. A comparatively rapid search for feature vectors 6a, 6b, 6c that are similar to the feature vector 6' of the query image 3' is sufficient to find similar partial images 3a, 3b, 3c and the associated source images 4a, 4b, 4c. Furthermore, this procedure allows to find feature vectors 6a, 6b, 6c and corresponding partial images 3a, 3b, 3c, which are visually different but are semantically relevant according to the projection created by the training unit, since they are assigned to the same clinical picture, for example.

[0078] Furthermore, criteria can optionally be specified by the user, for example to reduce the number of potential hits when searching in a database 2. For example, when searching for similar feature vectors 6a, 6b, 6c in the database 2, search results can also be excluded based on the tags and / or semantic representations 7a, 7b, 7c stored in the associated data sets 1a, 1b, 1c in the database 2, whereby in particular search results can be excluded based on criteria specified by the user for the tags and / or semantic representations 7a, 7b, 7c. For example, the search can only be for results from patients of the same gender and the same age group.

[0079] Optionally, a ranking of the respectively determined data sets 1a, 1b, 1c can be output as a result of the search query. The determined data sets 1a, 1b, 1c are ranked by the search unit according to their similarity or correspondence to the tags and / or semantic representations 7a, 7b, 7c of the search query and / or according to their similarity to at least one query image 3' of the search query, determined in particular by the distance between the relevant feature vectors 6a, 6b, 6c, and the ranking thus created is displayed to the user.

[0080] Optionally, the text information, numerical information, and / or the semantic information stored as tags in the respective data sets 1a, 1b, 1c can be output as a result of the search query, and / or the tags of the partial images 3a, 3b, 3c that are similar to the selection area can be statistically evaluated. The statistical result thus obtained can then be output. This statistic can, for example, be used in differential diagnosis to group different clinical pictures associated with visually similar appearances or changes and present them to the user. Furthermore, the user can easily create statistics based on the search results, for example, on how often male or female patients suffer from a certain disease or how frequently a certain age group is affected by a certain disease.

[0081] Fig. 2 shows a database 2 of a hospital with three data sets 1a, 1b, 1c stored therein, which contain information on patients or cases of illness. Each of the data sets 1a, 1b, 1c comprises a feature vector 6a, 6b, 6c, which were each created from partial images 3a, 3b, 3c systematically selected in the source images 4a, 4b, 4c, for example, by shifting a pixel grid. The source images 4a, 4b, 4c are in Fig. 2 Three-dimensional computed tomography images, with the original images 4a and 4c showing images of the lungs, and the original image 4b ​​showing an image of a hand. The lungs shown in the original images 4a and 4c each exhibit changes in the lung tissue. The hand shown in the original image 4b ​​exhibits a wrist fracture.

[0082] Each record 1a, 1b, 1c in the database in Fig. 2further includes semantic representations 7a, 7b, and 7c, respectively. In the example, number combinations represent the gender and the diagnosed disease, which are recorded in the medical record of the patient with whom the respective source images 4a, 4b, and 4c are associated. In the example, 82 represents "male," 89 "female," 16 the diagnosis "lung cancer," and 53 "wrist fracture."

[0083] In the Fig. 2In the example shown, a user submits an initial search query to a database 2. A three-dimensional computed tomography image of a lung is specified as the examination image 4'. In the examination image 4', the user selects a section of the left lung as the query image 3' because this area of ​​the lung shows changes. In order to verify his preliminary diagnosis of "lung cancer," the user wishes to receive, as a result of his search query to the database 2, data sets containing partial images of lung tissue with similar changes and to compare the associated diagnoses with the diagnosis he has provisionally made.

[0084] The query image 3' initially serves solely to create the first search query, whereby data records in the database 2 are to be determined in which partial images similar to the query image 3' are stored in the database 2. To create the search query, as described above, a feature vector 6' of the query image 3' is created based on the learned projection and transferred to the database 2 as a search query, which is Fig. 2 is schematically indicated as a solid arrow pointing towards database 2. Subsequently, those data records are identified in database 2 whose feature vectors are similar to the feature vector 6' of the query image 3'.

[0085] When in Fig. 2In the example shown, data sets 1a and 1c are output as the result of the first search query; this fact is indicated by solid arrows from database 2. The feature vectors 6a, 6c of data sets 1a, 1c were each created based on partial images 3a, 3c of an initial image 4a, 4c, each of which shows a three-dimensional computed tomography image of a lung with similar changes to those in the query image 3'. The feature vector 6b stored in data set 1b does not have sufficient similarity to the feature vector 6' of the query image 3', since the feature vector 6b was created based on partial images 3b of an initial image 4b, which shows a section of a three-dimensional computed tomography image of a patient's hand, and is therefore not output.

[0086] Additionally, the semantic representations 7a and 7c stored in data sets 1a and 1c are output, each containing the number combination 16 for the diagnosis "lung cancer." Thus, as a result of their search query, the user now has access to data sets 1a and 1c containing images of lung tissue exhibiting similar changes to those of their current patient and the respective diagnosis, in the example "lung cancer," to verify their preliminary diagnosis.

[0087] In the Fig. 2 In the example shown, semantic representations 7' of the entries in the medical record of the patient on whose examination image 4' the query is based are also available for generating the search queries. The semantic representations 7' include the number combinations 82 for "male" and 16 for the preliminary diagnosis "lung cancer."

[0088] The user now wishes to specifically search for records of male patients diagnosed with lung cancer whose lung tissue shows similar changes to that of the current patient. The user therefore submits a second search query to database 2, in which, in addition to the query image 3', the user specifies "male" and "lung cancer" as search criteria. To create the second search query, the query image 3' and the semantic representations 7', comprising 82 for "male" and 16 for "lung cancer," are transmitted to database 2, resulting in Fig. 2 is schematically shown as a dashed arrow in the direction of database 2.

[0089] The second search query now searches database 2 for records with feature vectors that, based on a given metric, are close to the feature vector 6' of the query image 3', while excluding records whose semantic representations 7' do not include the number combinations 82 for "male" and 16 for "lung cancer".

[0090] In the Fig. 2 In the example shown, the result of the second search query is data set 1a, which is indicated as a dashed arrow from database 2. Data set 1a includes a feature vector 6a that is similar to the feature vector 6' of the query image 3', and additionally, the semantic representations 7a include the number combinations 82 for "male" and 16 for "lung cancer."

[0091] Dataset 1c is not returned as a result of the second query, even though the feature vector 6c is similar to the feature vector 6' of the query image 3', because the semantic representations 7c include the number combination 89 for "female" and thus do not match all criteria of the second search query. The user is thus provided with dataset 1a, which contains images of the lungs of a male patient with lung cancer, to verify their preliminary diagnosis.

[0092] Alternatively, the result of an initial search query, e.g., "lung cancer," can be used to search for relevant content in reference databases, websites, databases of scientific articles, or hospital information systems. Furthermore, the additional information in the data sets regarding the position within the human body can be used to output information related to the corresponding anatomical position ("lower left lung") or the corresponding organ ("left lower lobe of the lung").

[0093] Alternatively, to create a search query for an examination image 4' or a sequence of similar images, a dimensionally reduced and, if necessary, reduced-dimensional query sub-image can be created by creating a section, and this query sub-image can be used for the search query. Thus, for example, higher-dimensional examination images 4' can be used to create a search query for a lower-dimensional database 2. A two-dimensional section, for example, from a three-dimensional computed tomography image, can thus be used as a query sub-image for a search query in a database 2 comprising data sets with images from scientific articles.

[0094] Alternatively, to create a search query to a database 2, a user can specify, for example, a two-dimensional query image 3' from a scientific article or from a website, or an excerpt thereof. The database 2 to be queried can contain data sets 1a, 1b, 1c with two-dimensional or three-dimensional source and / or partial images.

[0095] Such a search query to a higher-dimensional database 2 starting from a lower-dimensional query image 3' is possible if the training unit, when creating projections for obtaining feature vectors 6a, 6b, 6c from partial images 3a, 3b, 3c, was trained to the fact that, in particular visually or semantically, similar partial images 3a, 3b, 3c map to similar feature vectors 6a, 6b, 6c regardless of their partial image format such as partial image dimension or size.

[0096] Thus, partial images 3a, 3b, 3c, which are mapped to similar feature vectors 6a, 6b, 6c by a first projection learned from training partial images with a first dimension, are also mapped to similar feature vectors 6a, 6b, 6v by a second projection learned from training partial images with a second dimension. Therefore, structures that are similar in a three-dimensional space, for example, are also recognized as similar in a cross-section.

[0097] Furthermore, a user can optionally specify a query image 3' with an unknown position in the human body and transmit it to database 2 as a search query. In this case, the desired position in the human body is first determined, and database 2 is searched for feature vectors 6a, 6b, 6c of partial images 3a, 3b, 3c for which the same position relative to the human body is stored in their data sets 1a, 1b, 1c or whose position does not exceed a distance threshold value from the desired position specified by the user. As a result of the search query, partial images 3a, 3b, 3c are determined that show a spatially similar section of a human body as the query image 3'.

[0098] Optionally, the examination images 4' and / or examination image sequences and / or text information and / or numerical information and / or semantic information underlying a search query can be anonymized before the user transmits the search query to database 2. In this way, the user can ensure that a patient's personal data is not transmitted to database 2 when submitting a search query. This can be done, for example, according to the DICOM PS3.15 2013 anonymization guidelines.

[0099] Furthermore, optionally in response to a user's search request in the database 2, groups of partial images 3a, 3b, 3c and / or text information and / or numerical information and / or semantic information with similar feature vectors 6a, 6b, 6c can be created and output and, if necessary, additional information 5 with regard to these groups, and / or neighboring groups of partial images 3a, 3b, 3c and / or text information and / or numerical information and / or semantic information with similar feature vectors 6a, 6b, 6c can be determined and output and, if necessary, additional information 5 with regard to these groups can be output. Multi-database method (cascade search)

[0100] One embodiment of the invention offers the possibility of sequentially performing a multi-stage search process in several databases. First, as described above, a first search query is transmitted to a database 2. The search results determined as a result of this first search query to the database 2, in particular partial images 3a, 3b, 3c, group information, text information, numerical information, and / or semantic information, are subsequently used to generate at least one further search query, and this further search query is transmitted to at least one further database 2a.

[0101] To create the at least one further search query to the at least one further database 2a, alternatively or additionally, further information output to the database 2 as a result of the first search query, in particular output statistical results and / or output images 4a, 4b, 4c and / or similar partial images 3a, 3b, 3c can be used, wherein the dimension of the output original images 4a, 4b, 4c and / or the output partial images 3a, 3b, 3c for the search query in the at least one further database 2a, for example a literature database, is reduced if necessary.

[0102] For example, it is possible to create a further search query based on the determined three-dimensional partial images 3a, 3b, 3c and to transmit it to a literature database which only contains two-dimensional images, whereby the result is, for example, data sets with two-dimensional images from scientific publications.

[0103] Fig. 3 shows schematically the sequence of such a search procedure for searching in the Fig. 1 and Fig. 2 shown database 2, which includes data sets 1a, 1b, 1c with information obtained during examinations of patients, and in a further database 2a, which includes further data sets 1a', 1b' with information from the specialist literature.

[0104] First, a user, a doctor employed at a hospital, specifies an examination image 4' and selects a query image 3' from it to generate an initial search query to database 2. In the example shown, the user also specifies additional information 5 as search criteria to generate the initial search query to database 2, although this is by no means mandatory. Starting from query image 3', a feature vector 6' of query image 3' is created after the projection created as described above. The additional information 5 is converted into semantic representations 7', and both are transmitted to database 2 as the initial search query, indicated by a solid arrow.

[0105] As a result of the first search query, the database 2 outputs the data sets 1a, 1b, which include feature vectors 6a, 6b that are close to the feature vector 6' of the query image 3' and which include semantic representations 7a, 7b that are similar to the semantic representations 7' specified by the user.

[0106] In the example in Fig. 3The data sets 1a, 1b obtained in the first search query to database 2 are then used to generate another search query. For this purpose, sections through the three-dimensional partial images 3a, 3b are created and used to generate the next search query, which is then transmitted to the next database 2a. As a result of the second search query, the next database 2a outputs the next data sets 1a', 1b', which contain two-dimensional images from scientific publications whose feature vectors are close to the feature vectors 6a, 6b of the data sets 1a, 1b obtained as a result of the first search query.

[0107] Thus, in the example, the user has Fig. 3A total of four data sets are now available which are similar to his selected query image 3', whereby in the example the data sets 1a, 1b of the database 2 contain three-dimensional partial images 3a, 3b or source images 4a, 4b of patients in the hospital and the further data sets 1a', 1b' of the further database 2a contain two-dimensional images from a scientific journal.

[0108] Additionally, semantic representations 7a, 7b stored in the data sets 1a, 1b obtained in the first search query can also be used to generate the second search query. In this case, data sets are determined in the further database 2a that contain semantic representations similar to the semantic representations 7a, 7b.

[0109] Optionally, in such a multi-stage search procedure, individual data records 1a, 1b, 1c of the database 2 can also be used exclusively for the formation of a projection function, but are not made available to the user for viewing as query results.

[0110] This case is in Fig. 3 indicated by the dotted border of database 2. A user, for example, a doctor with his own practice, does not have direct access to the data sets 1a, 1b, 1c in database 2, but can use database 2 to create a multi-stage search query. In this case, the user selects a query image 3' from the examination image 4' to create an initial search query. For the query image 3', a feature vector 6' of the query image 3' is determined and transmitted to database 2.

[0111] As a result, as previously described, the data sets 1a, 1b are determined whose partial images 3a, 3b have a similar appearance to the selected query image 3'. However, the data sets 1a, 1b are not displayed to the user, but are used exclusively to create another search query, which is transmitted to the additional database 2a. Finally, as a result of their search query, the user receives the additional data sets 1a', 1b' of the additional database 2a, whose partial images are similar to the selected query image 3' and the partial images 3a, 3b of the data sets 1a, 1b.

[0112] The case where the user directly submits a search query to the further database 2a is in Fig. 3represented by the dashed arrows. In this case, the user submits a search query directly to the additional database 2a, and as a result of the search query, five records are retrieved in the additional database 2a and displayed to the user. The displayed search results in this case also contain fewer relevant records, since, compared to the multi-stage search, the criterion of similarity to records 1a', 1b' of database 2 is omitted. This shows that a multi-stage search procedure delivers more precise search results.

[0113] Optionally, when creating the database 2, size information regarding the pixel dimensions or voxel dimensions of the source images 4a, 4b, 4c or partial images 3a, 3b, 3c can also be stored in the data sets 1a, 1b, 1c and / or the pixel dimensions or voxel dimensions for the source images 4a, 4b, 4c or partial images 3a, 3b, 3c can be specified.

[0114] To specify the pixel dimensions or voxel dimensions, a search is carried out for similar reference output images or reference partial images with known pixel or voxel dimensions, in particular those originating from the same body part, and then a scaling is sought by image comparison with which the output image or partial image can be optimally aligned with the reference output image or reference partial image. Based on this scaling and the known pixel dimensions or voxel dimensions of the reference output image or reference partial image, the pixel dimensions or voxel dimensions of the output image 4a, 4b, 4c or partial image 3a, 3b, 3c are determined and stored in the database 2.

[0115] Similarly, if required, for a query image 3' for which no correspondence between pixel / voxel size and physical units of measurement such as mm is known, such a correspondence can be estimated by creating a search query to the database 2 and using correspondences between pixel / voxel size and physical unit of measurement of the reference output or reference sub-images determined as a result of the search query to estimate the correspondence in the query image 3'.

Claims

1. Method for creating a medical image database, a) wherein data records (1a, 1b, 1c) are created that comprise component images (3a, 3b, 3c) of two- or multidimensional initial images (4a, 4b, 4c) of parts of the human body, and wherein - for each component image (3a, 3b, 3c) of an initial image (4a, 4b, 4c), the respective position in an anatomical reference atlas is known, and - the individual initial images (4a, 4b, 4c) or component images (3a, 3b, 3c) are optionally provided with additional textual, numerical or semantic information (5a, 5b, 5c), and - a component image (3a, 3b, 3c) can also correspond to an entire initial image (4a, 4b, 4c), wherein b) a projection to obtain feature vectors (6a, 6b, 6c) is created from the component images (3a, 3b, 3c), - which, in particular visually or semantically, maps similar component images (3a, 3b, 3c) onto feature vectors (6a, 6b, 6c) at a small distance, and - wherein, to prepare for the execution of the projection a neural network, in particular a convolutional neural network, is created on the basis of prescribed learning component images, wherein the data records (1a, 1b, 1c) or some of the data records (1a, 1b, 1c) are used by the neural network as part of a metric learning method to learn the projection and the creation of the feature vectors (6a, 6b, 6c) of a compressed representation of the information contained in a learning component image from learning component images or groups of learning component images as well as a prescribed similarity between the learning component images to be achieved, wherein the relevant metric learning method is based on one or more of the following prescriptions: - prescribing as similar n-tuples of learning component images or groups of learning component images that are slightly shifted, rotated, distorted or elongated in relation to one another and have been created from the same initial image (4a, 4b, 4c), and / or - prescribing as similar n-tuples of learning component images or groups of learning component images that are created starting from the same component region (3a, 3b, 3c) of the initial image (4a, 4b, 4c), wherein at least one of the learning component images is modified relative to the component region of the initial image (4a, 4b, 4c) in such a way that the learning component images have different noises and / or different image intensities and / or different contrasts, and / or - prescribing as learning component images n-tuples of component regions originating from the same initial image or groups of initial images (4a, 4b, 4c), wherein the similarity between the respective learning component images of the n-tuple to be achieved depends on the spatial distance of the respective component regions in the initial image (4a, 4b, 4c), wherein in particular learning component images are viewed as being all the more similar, the closer the respective component regions in the initial image (4a, 4b, 4c) are to each other, and / or - prescribing learning component images or groups of learning component images from the same initial image (4a, 4b, 4c) or from different initial images (4a, 4b, 4c), which are to be viewed as similar owing to external features of the additional textual, numerical or semantic information stored with the respective initial images (4a, 4b, 4c), c) wherein the projection is applied to the component images (3a, 3b, 3c) of the data records (1a, 1b, 1c) and / or to a number of further component images of further data records, and accordingly at least one feature vector is obtained for each of these component images, and d) wherein the feature vectors created in this way, linked to the initial images and / or data records (1a, 1b, 1c) or further data records, are stored in an index data structure, wherein the respective position of the component images (3a, 3b, 3c) of an initial image (4a, 4b, 4c) relative to the human body is determined in the anatomical reference atlas, and wherein the information on the position of the component images (3a, 3b, 3c) is used by a neural network to learn a projection for estimating the positions of component images (3a, 3b, 3c), - wherein the projection is learned with the target function that, by mapping pairs or groups of component images (3a, 3b, 3c), the spatial configuration of the pairs / groups before and after the projection is similar and - wherein the projection is learned based on a known mapping of component images onto positions, - wherein, in addition to the feature vectors (6a, 6b, 6c), the learned or known position information is stored in the database, - wherein, if a search request is made, the position in the body being searched is determined and the database (2) is searched for feature vectors (6a, 6b, 6c) of component images (3a, 3b, 3c) for which the same position is stored in their data records (1a, 1b, 1c) or whose position does not exceed a distance threshold prescribed by the user from the position searched for.

2. Method according to claim 1, characterized in that additional information (5) is stored in the data records (1a, 1b, 1c), wherein the additional information (5) is provided as text information and / or semantic information and / or numerical information, and in that if text information or numerical information is available, the text information or numerical information is optionally stored in the database (2) as tags and / or semantic representations (7a, 7b, 7c), and in that the additional information (5) is optionally used by the neural network to create the projection, wherein the projection is created in such a way that learning component images are prescribed as similar if they originate from initial images (4a, 4b, 4c) or correspond to component images (3a, 3b, 3c) to which the same additional information (5) is assigned.

3. Method according to one of claims 1 to 2, characterized in that, - text information, numerical information and / or semantic information is provided by the user to create a search request, - the text information or numerical information is optionally converted into tags and / or semantic representations (7a, 7b, 7c) - if a search request is made, the database (2) is searched for feature vectors (6a, 6b, 6c) that have similar tags and / or semantic representations (7a, 7b, 7c).

4. Method according to one of claims 1 to 2, characterized in that, - at least one query image (3'), in particular of a region of interest, is selected from at least one two- or multi-dimensional examination image (4') or in an examination image sequence to create a search request, - for the query image (3') according to step c of claim 1, a feature vector is determined according to the learned projection, - the database (2) is searched for data records (1a, 1b, 1c) with feature vectors (6a, 6b, 6c) which are based on a prescribed metric close to the feature vector (6') of the query image (3'), and as a result of the search request, data records (1a, 1b, 1c) are output whose component images (3a, 3b, 3c) have a similar appearance to the selection region or which are semantically relevant, and optionally, wherein, in order to search for similar images for an examination image (4') or an examination image sequence, a dimensionally reduced query component image which possibly has its dimensions reduced is created by creating a section, and this query component image is used for the search request.

5. Method according to one of claims 3 or 4, characterized in that, when searching the database (2) for similar feature vectors (6a, 6b, 6c), search results are excluded on the basis of the tags and / or semantic representations (7a, 7b, 7c) stored in the associated data records (1a, 1b, 1c) in the database (2), in particular, in that search results are excluded under criteria prescribed by the user for the tags and / or semantic representations (7a, 7b, 7c).

6. Method according to one of claims 3 to 5, characterized in that, as a result of the search request, a ranking of the data records (1a, 1b, 1c) determined in each case is created and output according to their similarity or correspondence to the tags and / or semantic representations (7a, 7b, 7c) of the search request and / or according to their similarity, determined by the difference from the respective feature vectors (6a, 6b, 6c), to at least one query image (3') of the search request.

7. Method according to one of claims 3 to 6, characterized in that, - as a result of the search request, the text information and numerical information stored as tags in the respective data records (1a, 1b, 1c) and / or the semantic information stored as semantic representations (7a, 7b, 7c) are output and / or - the tags of the component images (3a, 3b, 3c) similar to the selection region are statistically evaluated and the statistical result is output.

8. Method according to one of claims 3 to 7, characterized in that the examination images (4') and / or examination image sequences and / or text information and / or numerical information and / or semantic information underlying the search request are anonymized by the user before the search request is sent to the database (2).

9. Method according to one of claims 3 to 8, characterized in that, - without a search request, groups of component images (3a, 3b, 3c) and / or text information and / or numerical information and / or semantic information with similar feature vectors (6a, 6b, 6c) are created and output directly in the database (2) on the basis of the data provided in the database and, if necessary, additional information (5) is output with regard to these groups, and / or - adjacent groups of component images (3a, 3b, 3c) and / or text information and / or numerical information and / or semantic information with similar feature vectors (6a, 6b, 6c) are determined and output following the search request by a user in the database (2) and, if necessary, additional information (5) with regard to these groups is output.

10. Method according to one of the methods according to one of claims 3 to 9, characterized in that - the search results obtained as a result of an initial search request to the database (2), in particular component images (3a, 3b, 3c), group information, text information, numerical information and / or semantic information, are used to create at least one further search request, and - the further search request is sent to at least one further database (2a), and / or - further information that is output as a result of the initial search request to the database (2) is used to create the at least one further search request to the at least one further database (2a), - in particular, statistical results and / or initial images (4a, 4b, 4c) and / or similar component images (3a, 3b, 3c) that are output are used, wherein the dimension of the output initial images (4a, 4b, 4c) and / or the output component images (3a, 3b, 3c) is reduced if necessary for the further search request in the at least one further database (2a).

11. Method according to one of the preceding claims, characterized in that individual data records (1a, 1b, 1c) of the database (2) are used exclusively for the formation of a projection function, but are not made available to the user for viewing as results of queries.

12. Method according to one of the preceding claims, characterized in that size specifications concerning the pixel dimensions or voxel dimensions of the initial images (4a, 4b, 4c) or component images (3a, 3b, 3c) are stored in the data records (1a, 1b, 1c), or in that, for the initial images (4a, 4b, 4c) or component images (3a, 3b, 3c), the pixel dimensions or voxel dimensions are prescribed by searching for similar reference initial or reference component images, in particular from the same body part, with known pixel or voxel dimensions - then, by comparing images, a scaling is sought that can optimally match the initial or component image with the reference initial or reference component image, and - based on this scaling and the known pixel dimensions or voxel dimensions of the reference initial or reference component image, the pixel dimensions or voxel dimensions of the initial image (4a, 4b, 4c) or component image (3a, 3b, 3c) are determined and stored in the database (2).

13. System for creating a medical image database, comprising a training unit and an indexing unit downstream of the training unit, wherein the training unit is designed, prescribing data records (1a, 1b, 1c) that comprise component images (3a, 3b, 3c) of two- or multi-dimensional initial images (4a, 4b, 4c) of parts of the human body, wherein the respective position of the component images (3a, 3b, 3c) of an initial image (4a, 4b, 4c) is determined with respect to the human body, wherein the information on the position of the component images (3a, 3b, 3c) is used by a neural network to learn a projection for estimating the positions of component images (3a, 3b, 3c), wherein - for each component image (3a, 3b, 3c) of an initial image (4a, 4b, 4c), the respective position in an anatomical reference atlas is known, and - the individual initial images (4a, 4b, 4c) or component images (3a, 3b, 3c) are optionally provided with additional textual, numerical or semantic information (5a, 5b, 5c), and - a component image (3a, 3b, 3c) can also correspond to an entire initial image (4a, 4b, 4c), characterized in that - a projection to obtain feature vectors (6a, 6b, 6c) is created from the component images (3a, 3b, 3c), - which, in particular visually or semantically, maps similar component images (3a, 3b, 3c) onto similar feature vectors (6a, 6b, 6c) and / or feature vectors (6a, 6b, 6c) at a small distance, and - to prepare for the execution of the projection, a neural network, in particular a convolutional neural network, is created on the basis of prescribed learning component images, wherein the data records (1a, 1b, 1c) or some of the data records (1a, 1b, 1c) are used by the neural network as part of a metric learning method to learn the projection and the creation of the feature vectors (6a, 6b, 6c) of a compressed representation of the information contained in a learning component image from learning component images as well as a prescribed similarity between the learning component images to be achieved, wherein the relevant metric learning method is based on one or more of the following prescriptions: - prescribing as similar n-tuples of learning component images that are slightly shifted, rotated, distorted or elongated in relation to one another and have been created from the same initial image (4a, 4b, 4c), and / or - prescribing as similar n-tuples of learning component images that are created starting from the same component region (3a, 3b, 3c) of the initial image (4a, 4b, 4c), wherein at least one of the learning component images is modified relative to the component region of the initial image (4a, 4b, 4c) in such a way that the learning component images have different noises and / or different image intensities and / or different contrasts, and / or - prescribing as learning component images n-tuples of component regions originating from the same initial image (4a, 4b, 4c), wherein the similarity between the respective learning component images of the n-tuple to be achieved depends on the spatial distance of the respective component regions in the initial image (4a, 4b, 4c), wherein in particular learning component images are viewed as being all the more similar, the closer the respective component regions in the initial image (4a, 4b, 4c) are to each other, and / or - prescribing learning component images or groups of learning component images from the same initial image (4a, 4b, 4c) or from different initial images (4a, 4b, 4c), which are to be viewed as similar owing to external features of the additional textual, numerical or semantic information stored with the respective initial images (4a, 4b, 4c), - wherein the projection is learned with the target function that, by mapping pairs or groups of component images (3a, 3b, 3c), the spatial configuration of the pairs / groups before and after the projection is similar and / or - wherein the projection is learned based on a known mapping of component images onto positions, - wherein, in addition to the feature vectors (6a, 6b, 6c), the learned or known position information is stored in the database, wherein the indexing unit is designed - to apply the projection created by the training unit to the component images (3a, 3b, 3c) of the data records (1a, 1b, 1c) and / or to a number of further component images of further data records, and accordingly at least one feature vector is obtained for each of these component images, and - the feature vectors created in this way, linked to the data records (1a, 1b, 1c) or further data records, are stored in an index data structure, wherein the respective position of the component images (3a, 3b, 3c) of an initial image (4a, 4b, 4c) relative to the human body is determined in the anatomical reference atlas, - wherein, if a search request is made, the position in the body being searched is determined and the database (2) is searched for feature vectors (6a, 6b, 6c) of component images (3a, 3b, 3c) for which the same position is stored in their data records (1a, 1b, 1c) or whose position does not exceed a distance threshold prescribed by the user from the position searched for.

14. System according to claim 13 comprising one of the training unit and search unit downstream of the indexing unit, wherein the search unit is designed - to create a search request prescribing at least one query image (3'), wherein the query image (3') is selected from at least one two- or multi-dimensional examination image (4') or in an examination image sequence, - to determine a feature vector according to the projection created by the training unit for the query image (3'), - to search the database (2) for data records (1a, 1b, 1c) with feature vectors (6a, 6b, 6c) which are based on a prescribed metric close to the feature vector (6') of the query image (3'), and as a result of the search request, to output data records (1a, 1b, 1c) whose component images (3a, 3b, 3c) have a similar appearance to the selection region or which are semantically relevant.