Method for determining target value, processing device, computer program and data carrier

Abstract

The invention relates to a computer-implemented method for determining a respective target value (33-35) of at least one control parameter (36-38) for controlling the detection and / or processing and / or representation of medical image data (39), comprising the following steps: during the viewing of at least one representation (42) by a reference person (41), determining the target value (33-35) of the at least one control parameter (36-38); at least one measurement data set (40) is obtained, which is based on the sensing detection of the respective brain activity of the respective reference person (41), and wherein the representation (42) is based on the respective medical image data set (43), and-target values (33-35) are determined from the measurement data set (40).

Description

Technical Field

[0001] The invention relates to a computer-implemented method for determining a corresponding setpoint value for at least one control parameter for controlling the acquisition and / or processing and / or display of medical image data. The invention also relates to a method, a processing device, a computer program and a data carrier for displaying medical image data. Background Art

[0002] In the field of medical imaging, image data are usually stored for visualization by an observer, who uses the visualization, for example, for diagnostic purposes or for monitoring medical operations. Therefore, the observer should be provided with images that are as readable as possible. However, this goal often conflicts with other requirements of imaging, such as limiting the incident X-ray intensity and / or achieving a high imaging rate or a short measuring time.

[0003] It is therefore known that parameters of the acquisition and / or processing and / or display of medical image data should be adjusted during the production and design of imaging medical examination devices and sometimes also dynamically during measurement operation in order to optimize the image quality.

[0004] On the one hand, the adjustment can be based on automatically determinable quality criteria, such as minimizing the noise content and / or maximizing the contrast. It is known that such optimization sometimes does not result in an image impression that allows the observer to actually recognize the image content ideally easily and robustly.

[0005] On the other hand, learning how different image impressions affect observers can be implemented, for example by showing different illustrations of medical image data based on different parameterizations to a selected group of observers, such as doctors and / or other medical professionals. Subjective image impressions can then be obtained and statistically evaluated by questionnaires or the like.

[0006] However, in corresponding learning, the viewed representation is evaluated according to visual preferences, so that, for example, there is the risk that real features are suppressed in the representation due to synthetic parameterization, even though this would apparently result in a clearer image impression or the like. Moreover, corresponding learning can only take into account very limited considerations that differently parameterized representations may be advantageous for different observers. Summary of the invention

[0007] Therefore, the technical problem to be solved by the present invention is to further improve the detection, processing or representation of medical image data.

[0008] According to the invention, the technical problem is solved by a method of the aforementioned type, which comprises the following steps:

[0009] - during viewing of at least one representation by a reference person, at least one measurement data set is obtained, which is based on sensory detection of a corresponding brain activity of the corresponding reference person, wherein the respective representation is based on a corresponding medical image data set, and

[0010] - determining a setpoint value based on the measurement data set.

[0011] By taking into account the brain activity of at least one reference person in the parameterization of the method according to the invention, an optimization of the visualization of the medical image data can be achieved which takes into account the real human perception of the at least one reference person and / or the medical image data can be enriched with semantic content which has been previously recognized by the reference person, for example. Different methods will be explained in more detail later on.

[0012] However, an overview of the evaluation of measurement data related to brain activity should first be given. Since the methods discussed below are known per se, they will be briefly introduced. Further implementation details can be found in the technical articles discussed below, etc.

[0013] In the course of research on computer-brain interfaces, it has been recognized that the observed image can be at least approximately reconstructed based on sensory detected brain activity. Suitable measurement data sets can be determined, for example, by functional magnetic resonance imaging of the brain and / or by measuring brain currents (electroencephalogram) and / or by a chip or interface implanted in the brain.

[0014] Corresponding methods are discussed, for example, in Yu Takagi et al., High-resolution image reconstruction with latent diffusion models from human brain activity, bioRxiv2022.11.18.517004; doi: https: / / doi.org / 10.1101 / 2022.11.18.517004, and Zijiao Chen et al., Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding, arXiv preprint arXiv: 2211.06956 (2022), https: / / arxiv.org / abs / 2211.06956. Here, algorithms trained by machine learning are used to reconstruct the original observed images from the measurement data of functional magnetic resonance imaging.

[0015] As described in detail in the document Yu Takagi et al., Improving visual image reconstruction from human brain activity using latent diffusion models via multiple decoded inputs, arXiv preprint arXiv:2306.11536[q-bio.NC](2023), https: / / doi.org / 10.48550 / arXiv.2306.11536, although images generated directly based on brain activity are usually not clear, they can still robustly determine the semantic content of the image, that is, for example, keywords related to the image content or the title of the image content.

[0016] The semantic content can be determined directly from the measured brain activity or measurement data set, whereby in particular the semantic recognition of displayed features by a reference person can be taken into account. In addition or alternatively, however, images generated, for example based on measurement data on the visual cortex, can be taken into account when recognizing the semantic content.

[0017] It is known within the scope of the present invention that the determination of the perceived image content based on measured brain activity, which is known per se from the aforementioned prior art, can also be used to parameterize the acquisition, processing and / or representation of medical image data.

[0018] For example, in the method according to the invention, an evaluation of the measurement data set can be used to check whether the respective reference person recognized one or more relevant existing features in the corresponding representation, that is to say, for example, the presence of a microcatheter shown in the fluorescence image. In other words, the information about whether a specific feature was semantically correctly recognized can be used as a quality criterion for the automatic determination of the representation or its parameterization in order to determine or optimize one or more control parameters. As an alternative or in addition, this information can be evaluated in order to determine whether a specific content is present in the representation, if the reference person can recognize said content.

[0019] If a conventional classification algorithm, in particular one trained by machine learning, is used to determine the semantic image content, a measure of robustness or the unambiguity of the recognition can also be determined here, so that, additionally or alternatively, how well or how easily the corresponding reference person recognizes or perceives the corresponding feature is also taken into account as a quality measure.

[0020] Thus, for example, at least one control parameter can be varied until the relevant feature is identified, or in order to maximize a measure for robustness or the unambiguity of the identification.

[0021] Alternatively or additionally, it can also be checked whether certain features are recognized, in particular the evaluation for the robustness measure or the recognition univocity is also used to check whether the reference person focused his attention on the representation during the measurement data acquisition. In other words, the measure can be used as an attention measure, or the attention measure can be determined based on the quality measure, and the nominal parameter or at least one nominal parameter can be determined based on the attention measure.

[0022] For example, it is expedient to evaluate the measurement data sets within the scope of continuous or repeated acquisition of medical image data, for example within the scope of fluorescence imaging, in order to reduce the X-ray dose, for example, if the reference person is not paying attention or only paying little attention to the current representation.

[0023] As an alternative or in addition, for example, a fixed preset reference value set for the control parameters can be used, from which a rated parameter set can be selected or determined based on it, which maximizes the robustness or the recognition unambiguity for a specific reference person or maximizes the average value of a quality assessment criterion, for example for a group of reference persons.

[0024] This can be used, for example, to optimize the so-called “flavor” of the representation in order to optimally recognize features or objects shown in the image, for example to define how much noise should be suppressed and / or how much edges should be sharpened.

[0025] The method can also be used, for example, to identify limits of control parameters. For example, during fluorescence imaging, it can be monitored whether a specific feature, such as a microcatheter, is recognized by a reference person or whether its recognition is robust, and the X-ray dose can be increased if the recognition is not completed or is not sufficiently robust.

[0026] Thus, it is generally possible to determine a quality criterion for the corresponding representation by the method according to the invention based on the measured data set and thus determine the control parameter on which the representation is based. As with other quality criteria (e.g. contrast or noise component), the target value of the control parameter can also be optimized based on the quality criterion. Since the quality criterion can be determined automatically, this optimization can also be carried out during operation, for example during fluorescence imaging. However, since the actual perception of the reference person or persons is taken into account, the aforementioned problems that may arise when only commonly used quality criteria are used can be avoided.

[0027] Furthermore, due to the use of concrete measurement data, distortions caused by subjective judgments of the image impression, as may occur in the panel studies based on questionnaires or similar methods mentioned at the beginning, can be avoided.

[0028] As a supplement or alternative to optimizing image quality in terms of identifying specific image features, the method according to the present invention can also be used to enrich medical image datasets or processing results generated by processing medical image datasets with semantic information, and / or train algorithms that can implement said enrichment.

[0029] For example, semantic information determined based on the measurement data set relates to the content of the representation seen by the reference person and is used for segmenting and / or annotating or labeling image parts and / or for training an algorithm that performs segmentation or labeling.

[0030] Since segmentation or labeling also involves the processing of medical image data, the determined semantic information or, in general, the information determined based on the semantic information or the measurement data set (for segmentation and / or labeling), or the determined parameterization of the algorithm suitable for this purpose, constitutes a control parameter in the sense of the present invention.

[0031] Once determined, the control parameters can be used directly to control the detection and / or processing and / or illustration of medical image data, and / or provide the control parameters via an output interface, i.e., for example, transmit them to an external device or store them in a database, which output interface can in particular be a hardware interface or a software interface.

[0032] The measurement data set can be determined within the scope of the method or, for example, already exist before the method starts and be obtained via an input interface, which can be a hardware interface or a software interface. For example, the measurement data set can be stored in a database before the method starts and retrieved from the database for carrying out the method.

[0033] Regardless of whether the measurement data set is obtained as part of the method or before the method begins, the detection of the measurement data set may comprise, in particular, the following steps:

[0034] - outputting a representation of at least one medical image data set and / or a representation of a processing result of each medical image data set to at least one reference person, and

[0035] - sensory detection of measurement data related to brain activity of the respective reference person while the respective reference person is viewing the respective illustration.

[0036] The one or more image data sets may include medical image data to be processed or to be illustrated, but this is not essential. For example, the determination of at least one control parameter may be done on the basis of medical reference data which were previously detected or which may also be determined, for example, by simulation. However, it is particularly expedient if the detection of the medical image data is parameterized.

[0037] Multiple representations based on the same medical data set can also be considered, wherein, however, preferably different control parameters are used for processing and / or representation of the image data set. This can be used, for example, to select a preferred set of control parameters from multiple parameter sets and / or for iterative optimization of at least one control parameter.

[0038] The individual representations may depend on reference values ​​for the individual control parameters which are assigned to the individual images, wherein corresponding setpoint values ​​for the individual control parameters may also be determined as a function of the reference values ​​of the individual control parameters which are assigned to the representation or at least one of the representations.

[0039] The individual diagrams can thus be viewed as examples of the effects of the corresponding reference values ​​of the individual control parameters. The measured data determined based on the diagrams can thus be evaluated in order to determine the suitability of the reference value associated with at least one of the individual diagrams as a setpoint value. This can be used, for example, to select a suitable set of setpoint values ​​for the control parameters from a plurality of reference value sets or, for example, to adjust the control parameters in an iterative method until the desired quality of the diagram is achieved.

[0040] The individual representations, i.e. the detection of the medical image data set and / or the processing of the medical image data set for providing the processing result to be represented and / or the representation of the medical image data set and / or the processing result, can depend on reference values ​​for the individual control parameters assigned to the individual representations. In particular, the detection and / or the processing and / or the representation of the individual medical image data set or the processing result can be performed by the same method as for the medical image data, wherein the individual control parameters are here predefined by corresponding reference values. The processing of the medical image data and the generation of the representation can differ only in the values ​​of the control parameters and / or in the objects displayed by the medical image data or the medical image data set.

[0041] Based on individual measurement data sets or on groups of measurement data sets, respective quality measures for the individual diagrams can be determined, wherein the rated value or at least one rated value is determined according to the quality measure, wherein the measurement data sets relate to the sensory detection of brain activity of different reference persons when viewing the same diagram. Here, "viewing the same diagram" refers in particular to viewing diagrams generated by the same method with the same control parameters based on the same measurement data set, even if the processing and / or the diagram is performed by different devices. The quality measure indicates in particular whether or how clearly and / or robustly the measured brain activity indicates the recognition of specific relevant features of the individual diagrams. Examples for this have been explained before.

[0042] Thus, the setpoint value of at least one control parameter can be set in particular depending on which quality criterion is achieved when using corresponding reference values ​​of the control parameters, and the reference value can be used as the setpoint value depending on the quality criterion or the setpoint value can be determined by adjusting the reference value.

[0043] For some control parameters, in particular for control parameters related to the detection of medical image data, such as the incident X-ray intensity and / or the used measuring time, it is known in which direction the control parameter must be changed in order to improve the quality criterion. In this case, based on the determined quality criterion, for example, an offset of a reference value can be determined, which can be added to the offset in order to determine a setpoint value. In repeated or continuous imaging, the quality criterion can be adjusted according to the setpoint value or within a setpoint interval.

[0044] In addition or as an alternative, in particular in repeated or continuous imaging, the quality measure can also be used to determine whether the reference person's attention is focused on the diagram. For example, if the quality measure falls below a certain limit value and / or the quality measure falls by a certain value or factor within a preset number of imaging cycles, it can be indicated that the reference person has deviated or moved his attention away from the diagram. In particular in X-ray imaging, this can be used as a trigger for adjusting the measurement scheme, i.e., in particular for reducing the incident X-ray intensity or even stopping imaging, thereby, for example, being able to avoid unnecessary stress on the examination object and / or unnecessary energy consumption.

[0045] For some control parameters, for example for edge sharpening and / or the degree of noise suppression and / or the choice of filter kernel, the relationship between the quality measure and the control parameter value may be non-monotonic, or different control parameters may interact in a complex way. In particular in these cases, the quality measure can be optimized by changing one or more control parameters. This can be achieved by conventional optimization methods or by selecting the reference value set of the diagram with the highest quality measure as the set of rated values. For this purpose, the measurement data set can be evaluated for diagrams based on different reference values ​​of the control parameters.

[0046] In the case of repeated or continuous imaging, this optimization can be performed during operation. As an alternative, this optimization can also be performed offline by outputting corresponding differently parameterized diagrams to specific reference persons or groups of reference persons in order to determine and evaluate a large number of measurement data sets. The setpoint parameters determined as a quality measure for the optimization can then be made available to these reference persons or, in particular, when evaluating the measurement data of a group of reference persons, generally also to all users or to a specific group of users for the detection, processing and / or visualization of medical image data in subsequent imaging.

[0047] Control parameters or at least one control parameter can be used to control the processing and / or illustration of medical image data, wherein a plurality of mutually different illustrations are generated based on respective medical image data sets, in that the respective medical image data sets are processed and illustrated based on the same method as for the medical image data, wherein respective reference values ​​assigned to the illustrations are used to preset respective control parameters for providing the respective illustrations, wherein the mutually different illustrations differ from each other at least with respect to the reference value of at least one control parameter, wherein a subgroup of illustrations including one of the illustrations or not including all of the illustrations is selected based on a quality measure determined for the respective illustrations, whereby the rated value of the respective control parameter is set based on the selected illustration or on the respective reference value assigned to the respective selected illustration.

[0048] For example, the diagram with the highest quality criterion can be selected, and the reference value associated with this control parameter in the selected diagram can be used as the setpoint value for the individual control parameters. However, in particular when a plurality of diagrams generate similar quality values, it can be advantageous to select a subset of the diagrams with the best quality values ​​and to determine the setpoint value for the individual control parameters, for example by averaging or medianing the reference values ​​for this control parameter in this subset, or to interpolate between the reference values ​​according to the quality criterion.

[0049] Thus, the method according to the invention can be used, for example, after imaging has been completed in order, for example, to optimize the processing and / or illustration of medical image data so that, given the image data, information relating to a specific reference person can be particularly easily detected in the generated illustration, or, based on learning from a plurality of reference persons, information relating to all observers or to a specific group of observers can be predictively extracted particularly easily from the illustration.

[0050] In particular, different reference value sets can be predefined, by using which different image impressions are generated, for example, sharper or less sharp images, different contrasts and / or image brightnesses, etc. Since the selection of control parameters (which primarily influence the image impression) has hitherto been largely dependent on the user's taste and reflects subjective preferences to a certain extent, the different possible image impressions or parameter sets are also referred to as "styles" of the illustration. As control parameters that influence the "style" of the illustration, for example, parameters for edge sharpening, noise suppression and / or time communication can be used.

[0051] It is known that a subjectively preferred image impression does not always correspond to a parameterization of the illustration in which the user can particularly easily and robustly recognize specific anatomical features or related objects. It may therefore be expedient for the “style” of the illustration to automatically indicate or prompt the user at least one “style” that is particularly suitable for the user or a specific target group.

[0052] It may be expedient to determine an optimized image impression for exactly one reference person in each case, in particular for specifying a default setting for the reference person when viewing the medical image data later. However, in order to determine an image impression which, at least for the vast majority of users, enables good identification of relevant features or objects in the image, in particular in a specific field of application, and which can thus be used, for example, as a universal default setting for processing and / or displaying medical image data, the brain activity of different reference persons when viewing the same display can be evaluated, as described above, in order to determine a quality criterion, thus taking into account the perception of the display by different reference persons.

[0053] The medical image data set on which the illustration is based can be a medical image data set determined within the scope of an imaging sequence, wherein the imaging sequence includes repeated or continuous detection of medical graphic data of the same examination object, wherein the rated value can be used to control the detection and / or processing and / or illustration of at least part of the medical image data, which is detected within the scope of the same imaging sequence after detecting the medical image data set.

[0054] The viewing of the representation by the reference person and the detection of the measurement data set thus performed can thus be carried out within the scope of repeated or continuous imaging (e.g. within the scope of fluorescence imaging), and the control parameters can thus be adjusted dynamically within the imaging. In this case, for example, detection-related control parameters, in particular the X-ray intensity or the frame rate, can be adjusted dynamically. Additionally or alternatively, for example, parameters that influence the subjective image impression of the representation, in particular the aforementioned "style", can be adjusted dynamically.

[0055] The method according to the invention can include detecting the medical image data set and / or the medical image data that still needs to be detected or overall repeated or continuous imaging. As an alternative, however, the detection of the medical image data set and / or the medical image data can also be performed outside the method. The method is implemented, for example, in a time interval after the detection of the medical image data set has ended and before the detection of the medical image data that still needs to be detected. The method can be used, for example, to optimize any of the above-mentioned quality measures.

[0056] In addition or alternatively, at least one object and / or at least one anatomical feature can be checked separately to determine whether a perception condition is met, the satisfaction of which depends on the respective measurement data sets and indicates the perception of the respective object or anatomical feature by the respective reference persons in the respective illustrations, wherein the rated value or at least one rated value is determined as a function of the satisfaction of the respective perception condition.

[0057] For example, the perception of a particular object or a particular feature by each reference person can indicate that this object or anatomical feature is currently particularly critical within the context of an imaging sequence to be performed during surgery. Thus, without prior knowledge or at least without specific prior knowledge of the surgical procedure or surgical plan, it can be confirmed in near real time whether the fact that a reference person can easily and reliably identify a particular object or anatomical feature is currently critical, for example during microcatheter positioning.

[0058] If this is the case, in particular with respect to fulfilling the perception conditions, for example a higher frame rate and / or a higher x-ray dose can be used in order to improve the image quality and thus further simplify the recognition of relevant objects or features.

[0059] Otherwise, i.e. in particular when the reference person deviates his or her attention from the object or feature and the perception condition is thus no longer met, the control parameters can be adjusted, for example, so that the radiation exposure or other exposures to the patient can be reduced and, in addition or as an alternative, the costs and / or energy consumption of the imaging can also be reduced, wherein for this purpose it can be taken into account that the reference person makes the identification of the object or feature more difficult.

[0060] If the recognition of the object or feature or of another object or feature is relevant again at a later time, the reference person takes note of this recognition and thereby recognizes the perception of the corresponding object or feature again and can adjust the imaging or at least one control parameter accordingly.

[0061] The control parameter or the individual control parameters may specify an X-ray dose incident on the examination object for the detection of medical image data and / or an imaging rate for the detection of the medical imaging object. This may be expedient in particular in fluorescence imaging or in other imaging modes with near real-time imaging, for example, in imaging during surgery.

[0062] Additionally or alternatively, setpoint values ​​may also be determined for other control parameters used in the detection of medical image data. Additionally or alternatively, setpoint values ​​may also be determined for control parameters related to the representation of medical image data or the representation of the processing results of image data determination. This may be, for example, control parameters for a display device, for example for adjusting image brightness or contrast. Alternatively or additionally, setpoint values ​​for control parameters in the processing of image data, for example parameters in the area of ​​image reconstruction, edge sharpening, time communication and / or noise suppression, may be preset.

[0063] The control parameter or at least one control parameter (with a preset rated value for the control parameter) can control the enrichment of medical image data using semantic information, and / or control the segmentation of medical image data or processing results determined based on the medical image data, wherein the enrichment and / or segmentation is part of the medical image data processing.

[0064] For this purpose, the above-mentioned perception conditions can be evaluated, for example, in order to determine whether a specific anatomical feature or object is recognized by the reference person in the representation. It can be particularly useful if the measurement data also relate to at least brain regions that are involved in the semantic recognition of the image content, since in this case the brain of the reference person can be used as a recognition algorithm or as part of the recognition algorithm.

[0065] As in the prior art cited above, for everyday objects such as airplanes, teddy bears, watches, and animals, measurement data related to the visual cortex of the brain can be combined with such semantic information to determine the location and / or size of the object, or the anatomical features described by the semantic information in the representation. For example, this can be used to add classifications or labels or similar information to anatomical features or objects as part of adding semantic information, and / or to support segmentation of medical image data or processing results. For example, control parameters can be used to set an initial segmentation, which is then refined using known segmentation methods or similar methods.

[0066] In particular, the medical image data set itself may be part of the medical image data set or form part of the medical image data set. Thus, the method according to the invention may also be used to segment the image data set using the brain of a reference person or to supplement it with semantic information.

[0067] In addition or alternatively, a training function can also be trained for processing medical image data or for use in processing medical image data, wherein the control parameters are parameters of the training function whose rated values ​​are preset by machine learning in the case of supervised learning, wherein multiple training data sets are used for supervised learning, each training data set includes at least one measurement data set, or each training data set depends on at least one measurement data set.

[0068] In particular, within the scope of training or generating a training data set based on the measurement data, semantic information relating to at least one object and / or anatomical feature shown in the respective representation can be determined and, for example, can be assigned as a desired result of a training function as part of the training data set.

[0069] Generally speaking, the training function maps the cognitive functions that humans associate with other human brains. Through training based on training data (machine learning), the trained function has the ability to adapt to new environments and can discover and infer patterns.

[0070] Generally speaking, the parameters of the training function can be adjusted by training. In particular, supervised learning, semi-supervised learning, unsupervised learning, reinforcement learning and / or active learning can be used. In addition, representation learning (also called "feature learning") can also be used. In particular, the parameters of the training function can be repeatedly adjusted through multiple training steps.

[0071] For example, the training function may include a neural network, a support vector machine (SVM), a decision tree and / or a Bayesian network, and / or the training function may be based on mean clustering, Q-learning, a genetic algorithm and / or an association rule. In particular, the neural network may be a deep neural network, a convolutional neural network (CNN) or a deep CNN. In addition, the neural network may also be an adversarial network, a deep adversarial network and / or a generative adversarial network (GAN).

[0072] The training function can be arranged to enrich the medical image data with semantic information and / or segmentation of the medical image data. Thus, the semantic information or segmentation information extracted from the measurement data set is not used directly or at least not completely for the segmentation or labeling of the medical image data, but is used indirectly by the trained algorithm. This can be used, for example, to at least partially transfer the knowledge or understanding of a reference person or a particularly preferred group of reference persons to the trained function.

[0073] Determining the setpoint value from the measurement data set may include applying a machine-trained function to the measurement data set and / or to the processed data determined therefrom. Possible designs of such trained functions or their training have been described previously. In addition or as an alternative to the above-described designs of trained functions, in particular transformers, in particular visual transformers, may be used as trained functions for deep learning architectures, forming training functions or subfunctions of training functions.

[0074] Algorithms trained by machine learning can be used in particular to at least partially reconstruct or approximate representations based on a measurement data set and / or to obtain at least one piece of semantic information related to the image content, such as the described object or anatomical feature. In the article by Yu Takagi et al. cited at the beginning and in the article by Zijiao Chen et al. also cited at the beginning, we already know implementation plans for the representation of everyday objects or their semantic recognition or classification. It is recognized that the methods described in detail in these articles can be applied to medical imaging by using suitable training data.

[0075] For example, a suitable training dataset may be provided by an expert through manual classification of objects and / or anatomical features seen in a corresponding representation or base medical image dataset.

[0076] Alternatively or additionally, training can also be performed by repeatedly comparing the image viewed by a reference person with the image reconstructed based on the measurement data of the reference person and minimizing the deviations within the scope of training. This can be achieved, for example, by error feedback or generative adversarial networks.

[0077] Furthermore, the present invention relates to a computer-implemented method for presenting medical image data to an observer, wherein the detection of the medical image data and / or the processing of the medical image data to provide a processing result to be displayed and / or a representation of the medical image data and / or the processing result are parameterized respectively by at least one control parameter, wherein the rated values ​​used for the respective control parameters are set by means of the computer-implemented method according to the present invention in order to determine a corresponding rated value for at least one control parameter.

[0078] In addition to the method according to the invention, the invention also relates to a processing device which is designed to implement a computer-implemented method according to the invention for determining a corresponding setpoint value for at least one control parameter and / or is designed to implement a computer-implemented method according to the invention for illustrating image data for an observer.

[0079] The processing device can be provided in particular by a suitable program of a programmable data processing device. As an alternative, however, a hard-wired processing device can also be used.

[0080] The processing device can be integrated in the medical imaging apparatus or can be constructed independently of the medical imaging apparatus, for example. For example, the processing device can be implemented as a workstation computer, a server or a cloud solution.

[0081] Furthermore, the present invention relates to a computer program comprising program instructions, the implementation of which is designed to, when it is executed in a data processing device, implement a computer-implemented method according to the present invention to determine a corresponding rated value for at least one control parameter and / or implement a computer-implemented method according to the present invention to illustrate medical image data for an observer. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Furthermore, the invention relates to a data carrier comprising the computer program according to the invention.

[0083] The features explained above for the individual inventive subjects can be transferred to other inventive subjects as desired with the advantages mentioned.

[0084] Further advantages and details of the invention are apparent from the exemplary embodiments described below and with reference to the accompanying drawings. Here, schematically:

[0085] Figure 1 An embodiment of a processing device according to the invention and the application of the processing device in medical imaging are shown,

[0086] Figures 2 to 4 In each case, an exemplary embodiment of a computer-implemented method according to the invention for determining a corresponding desired value for at least one control parameter is shown in conjunction with an exemplary embodiment of a method according to the invention for displaying medical image data, and

[0087] Figures 5 and 6 Possible configurations of trained functions are shown, which can be used in the method according to the invention. DETAILED DESCRIPTION

[0088] Figure 1 A medical image is shown, wherein medical image data 39 and processing results 47 determined therefrom are displayed to an observer 58 via a processing device 59 .

[0089] It is thus shown purely by way of example that quasi-continuous imaging with a relatively high imaging rate, such as fluorescence imaging, is used to determine the diagram 42, wherein the medical image data 39 are provided directly by the medical imaging device 62. However, the method described below can also be transferred to other methods for medical imaging, such as any two-dimensional or three-dimensional and / or functional imaging, such as medical imaging based on X-ray radiation, magnetic resonance, positron radiation, etc. In this case, the processing can also include the reconstruction of three-dimensional image data. In addition or as an alternative, it is also possible that the diagram 42 is based on medical image data 39, which are clearly detected before the processing and, for example, read from a database.

[0090] In medical imaging, it is generally possible to parameterize the detection and / or processing and / or display of medical image data 39 by means of a plurality of control parameters. In order to determine the desired values ​​for at least some of the control parameters, the processing device 59 is designed to evaluate the brain activity of at least one reference person 41 viewing the display 42. In this embodiment, the reference person 41 is identical to the observer 58, but can also be a different person.

[0091] The processing device 59 first controls a display device 67 for outputting a medical image data set 43 or a graphical representation 42 of a processing result of the image data set 43 detected in this embodiment as part of the medical image data 39. Control parameters 47 (e.g. for adjusting brightness or contrast) for controlling the display device 67 and / or Figure 1 In this case, a control parameter (not shown) in relation to the detection or processing of the image data set 43 initially predetermines a reference value which is, for example, permanently predefined or determined in a previous iteration as a desired value.

[0092] Subsequently, a measurement data set 40 is detected by a sensor 65, which is abstractly shown in this embodiment as a helmet worn by a reference person 41, and which relates to the brain activity of the reference person while viewing the illustration 42. The sensor 65 can be implemented to provide the measurement data set 40, in particular to perform brain functional imaging based on magnetic resonance and / or to measure brain currents. In an alternative embodiment, it is also possible to use a chip implanted in the brain of the reference person 41 or another computer-brain interface to detect the measurement data set 40.

[0093] Based on the measurement data set 40, a corresponding setpoint value for at least one control parameter can then be predefined, which can then be used for the detection or processing or representation of medical image data detected or to be detected after the medical image data set 43. This is expedient because, as explained in detail above in the description, it is possible to detect, for example, based on the brain activity or the measurement data set 40 whether and how clearly the reference person 41 recognizes the object 50 or the anatomical feature 51 in the representation 42, or whether the attention of the reference person 41 is mainly focused on the representation 42.

[0094] Some specific embodiments for presetting the rated value are described below. Each embodiment combines various advantageous further developments of the method generally explained above, so that individual features or implementation details of the following embodiments may also be omitted, or features of the various embodiments explained may be combined.

[0095] Figure 2 A flow chart of a method is shown in which medical image data 39 are acquired and displayed to an observer 58 .

[0096] As part of the initialization, in step S1, reference values ​​44, 45, 46 are first predefined for the various control parameters 36, 37, 38 used in the method. In the exemplary embodiment, the reference value 44 specifies the control parameter 36, which parameterizes the acquisition of the medical image data 39. The reference value 45 specifies the control parameter 37, which parameterizes the processing of the medical image data 39. The reference value 46 specifies the control parameter 38, which parameterizes the display of the medical image data 39.

[0097] In step S2, the medical image data 39, for example, is collected by Figure 1The imaging device 64 is shown for detection. In this case, in the exemplary embodiment, an imaging sequence is used in which medical image data 39 are repeatedly detected for the same examination object 49, wherein in the exemplary embodiment, a medical image data set 43 is respectively detected in step S2 by corresponding iterations. The detection of the medical image data 39 or medical image data set 43 is parameterized by control parameters 36, which in the exemplary embodiment specify the X-ray dose radiated during the detection of the respective image data set 43, for example by specifying the voltage and / or current of the X-ray tube used and / or by adjusting the exposure time.

[0098] In step S3, the corresponding image data set 39 is processed to provide a processing result 47. The processing is parameterized by control parameters 37 which in this exemplary embodiment define the degree of noise suppression, for example by parameterization of the filter kernel used for this purpose.

[0099] In step S4 , the processing result 47 and thus the medical image data 39 are made available to an observer 58 . In this case, the representation is parameterized by control parameters 38 , which can control, for example, a display device 67 in order to adjust the brightness and / or contrast of the representation 42 .

[0100] In this exemplary embodiment, the value of the control parameter 36 and thus the parameterization of the acquisition of the medical image data 43 acquired in a subsequent iteration of the method is to be automatically adjusted according to the requirements of the observer 58. The observer thus serves as a reference person 41 for the adjustment.

[0101] For simplicity, Figure 2 In the embodiment shown, the values ​​of the control parameters 37, 38 are not adjusted between iterations. However, as described in the description section or shown in other embodiments, the control parameters 37, 38 may be adjusted accordingly.

[0102] Since the subsequent steps S5 to S11 are used to predetermine the desired value of the control parameter, they can therefore be regarded independently of the previously described steps as a computer-implemented method for determining the desired value of the control parameter.

[0103] In step S5, a measurement data set 40 is detected, which is obtained based on the sensory detection of the corresponding brain activity of the reference person 41 while the reference person is viewing the illustration 42. In this embodiment, the sensor 65 is abstractly shown as a helmet for detecting the measurement data set. Technically, brain activity can be recorded by methods such as measuring brain currents and / or functional imaging (such as functional magnetic resonance imaging).

[0104] If Figure 1 As shown, real-time imaging was used during the procedure, and as Figure 2As shown, the setpoint value 33 is iteratively adjusted in the measurement sequence, and the sensor 65 should continue to allow sufficient freedom of movement for the reference person 41. On the other hand, if movement of the reference person 41 is not required, a magnetic resonance scanner or the like can also be used to record brain activity more accurately, in which case the reference person 41 lies down while the corresponding diagram 42 is shown to them.

[0105] As explained in the description section, information about the image content seen or recognized by the reference person 41 in the illustration, as well as the robustness and clarity of such recognition, can be obtained based on the measurement data set 40 by a suitable algorithm, in particular an algorithm trained by machine learning. Complete details can also be obtained from the previously cited literature of Yu Takagi et al. and Zijiao Chen et al.

[0106] In this exemplary embodiment, the evaluation of the measurement data set 40 is primarily used to reduce the intensity of the X-rays incident on the examination object 49 during the imaging process if the reference person 41 is not focusing on the representation 42. For this purpose, in this exemplary embodiment, an anatomical feature 51 in the displayed area of ​​the examination object 49 is selected, from which it can be seen that the reference person 41 can recognize this anatomical feature well when the reference person 41 focuses his attention on the representation 42 over the entire permissible adjustment range of the control parameter 36 and thus the incident X-ray dose.

[0107] In step S6, a check is then performed for the anatomical feature 51 to determine whether a perception condition 52 is satisfied, the satisfaction of which depends on the corresponding measurement data set 40 and indicates the perception of the anatomical feature 51 by the reference person 41 in the corresponding representation 42. For this purpose, it can be checked, for example, whether the semantic information extracted based on the measurement data set and relating to the viewed representation content represents the anatomical feature 51.

[0108] If the perception condition 52 is not satisfied in step S6, it indicates that the reference person 41 is not currently focusing his attention on the illustration 42. With respect to other methods of attention recognition, such as eye tracking, it is also possible to recognize that the illustration is out of focus of the reference person 41.

[0109] Since the reference person 41 is not currently using the illustration 42 in this case or at most only paying little attention to it, he can already recognize, without having to know the details of the operation or other procedures involving imaging, that the incident X-ray dose and thus the stress on the examination subject 49 can be reduced at least temporarily.

[0110] For this purpose, in step S7, a setpoint value 33 can be preset for the control parameter 36, which achieves the minimum radiation exposure. However, in order to achieve a self-use increase in the X-ray dose and thus the image quality at a later time, in this exemplary embodiment, the setpoint value 33 is selected such that the reference person 41 can further recognize the anatomical feature 51 when the reference person focuses his attention on the illustration 42, so that the perception condition 52 can be met in a later iteration as soon as the reference person 41 focuses his attention on the illustration 42 again.

[0111] If, on the other hand, the perception condition 52 is not met, then in step S8 a further perception condition 53 is checked, the satisfaction of which indicates that the reference person 41 has perceived an object 50, for example a microcatheter to be positioned. If the imaging is used, for example, to support the reference person 41 in the positioning of a microcatheter, it is essential that the reference person 41 always recognizes the microcatheter when his or her attention is focused on the representation 42. Therefore, if the perception condition 53 is not met, in step S9 the setpoint value 33 for the control parameter 36 is preset in such a way that the ideal image quality and thus a robust recognition of the object 50 by the reference person 41 is achieved in advance.

[0112] In order to avoid an unnecessary increase in the X-ray dose even when the reference person 41 has already focused his attention on the representation 42 and therefore requires a good recognition of the object 50 in this exemplary embodiment, an image criterion 48 is determined in step S10 for the representation 42 when the perception condition 53 is met. For this purpose, it can be evaluated, for example, with what degree of unambiguity the semantic content of the representation 42 can be determined, that is to say, for example, how much the determined probability of the selected classification differs from the probability of the next most likely classification, and / or with what robustness the semantic content can be recognized, for example, how much the probability of the selected classification exceeds a limit value from which the recognition is completed.

[0113] The desired value 33 can then be determined in step S11 by superimposing an offset, which can be positive or negative, selected according to the quality measure 48 to the previously used desired value 33 for the control parameter 36 or, in the first iteration, to the reference value 44. The quality measure 48 can thus be adjusted to a predefined value or to a predefined interval, thereby ensuring, on the one hand, a robust recognition of the object 50 by the reference person 41 and, on the other hand, avoiding an unnecessary high radiation dose to the examination object 49.

[0114] The method is performed in Figure 1The method is performed by a processing device 60, which is formed by a suitably programmed data processing device 59. In this exemplary embodiment, the data processing device 59 includes a processor 61, which can be formed by a microprocessor, a microcontroller, an FPGA, a graphics processor, etc. A computer program 63 is stored in an associated memory 62, whose instructions execute the method. In addition or alternatively, the method described in detail below or the methods and method features described in other parts of the description can be executed by using a suitable computer program 63.

[0115] Figure 3 An alternative method for determining the corresponding setpoint values ​​34, 35 for the individual control parameters 37, 38 is shown, wherein in this exemplary embodiment, for the sake of simplicity, it is provided that the control parameters 37, 38 are to relate only to the processing and display of the medical image data 39. In this exemplary embodiment, a desired image impression 66, which can also be referred to as a "style", is to be selected from a plurality of possibilities for the image display, which image impression enables the best possible recognition of the image content. For each desired image impression 66, a setpoint value 45, 46 for the control parameter 37, 38 is hereby assigned, the use of which achieves the desired image impression 66.

[0116] Unlike the previously discussed embodiments, Figure 3 The selection of the median setpoint values ​​34 , 35 is not optimized for a single reference person, but rather a group study is carried out, wherein the brain activity of a plurality of reference persons 41 when viewing representations 42 with different desired image impressions 66 is to be taken into account.

[0117] For this purpose, at least one medical image dataset 43 is predefined in step S12, for example by detecting the image dataset 43 or reading the image dataset 43 from a database of previously detected image datasets. For the sake of simplicity, it is understood in the following description that only a single image dataset 43 is processed.

[0118] In step S13, a reference value 45 for a control parameter 37 and a reference value 46 for a control parameter 38 are respectively predefined for each possible image impression 66. For the sake of simplicity, it is provided that the processing and the representation of the image data set 43 are parameterized by exactly one control parameter 37, 38 in each case. However, it is also possible to use a plurality of control parameters in each case, for which corresponding reference values ​​are predefined as part of the desired image impression 66, or it is also possible that only the processing or only the representation is parameterized by predefining the desired image impression.

[0119] In step S14, the image data set 43 is then processed as a function of the control parameters 37 in order to generate a corresponding processing result 47 for each possible image impression 66 and thus for each reference value 45. In addition to providing individual processing results 42 for a plurality of individual reference values ​​45, the processing and reference values ​​45 are also processed in a manner that is consistent with the processing results 47 of the image data set 43. Figure 2 The processing is the same as that described in step S3.

[0120] In step S15, different representations 42 of the image data set 43 or its processing results 47 are then output to each reference person 41, wherein a plurality of representations 42 are output for each processing result 47 by using different reference values ​​46 of the control parameter 38 in each case. Thus, a representation 42 for each possible image impression 66 is output for each reference person 41. The representations according to the control parameter 38 have been referenced to the image data set 43. Figure 2 Step S4 is described in detail.

[0121] In step S16, for each pair of a corresponding representation 42 and a corresponding reference person 41, a measurement data set 40 is detected, which relates to the brain activity of the corresponding reference person 41 while viewing the corresponding representation 42. The detection of such a measurement data set 40 has been described with reference to Figure 2 Step S5 is described in detail.

[0122] In step S17 , an algorithm, in this exemplary embodiment a training function 66 , is used to determine corresponding image metrics 48 for each representation 42 based on the respective sets of measurement data sets 40 , which relate to the sensory detection of brain activity of different reference persons 41 when viewing the same representation 42 .

[0123] To this end, in the simplest case, an individual quality measure can be determined for each measurement data set 40, as already described with reference to Figure 2 As explained in step S10 in FIG. 4 , an averaging or median calculation or, for example, a weighted sum calculation can then be performed in order to determine a common quality measure 48 for the individual representations 42 .

[0124] As an alternative, the algorithm or training function 66 can also be configured to jointly process a plurality of measurement data sets which relate to different reference persons 41 viewing the same illustration. The method of determining the quality measure 48 using the training function has already been discussed in the description section.

[0125] In this exemplary embodiment, the representation with the highest value of the quality criterion 48 is selected in step S18. In step S19, the reference parameters 45, 46 used to generate the selected representation 42 can then be directly provided as setpoint values ​​34, 35 for the control parameters 37, 38 for processing the medical image data 39 to be processed subsequently. In this way, a suitable image impression 66 or "style" is automatically selected.

[0126] In an alternative embodiment of the method, a plurality of representations 42 are selected. For example, a subgroup of representations 42 can be selected, which includes the representation 42 with the highest quality value 48 and other representations 42 whose quality value 48 is below the highest quality value 48 by a predetermined limit value. The setpoint value 34, 35 should be determined, for example, as the average value or median or weighted sum of the respectively assigned reference values ​​45, 46 of the representations of the subgroup.

[0127] Additionally or alternatively, it is also possible to carry out the method based on a plurality of image data sets 43 provided in step S12. In this case, for example, all quality values ​​48 based on the same parameterization of control parameters 37, 38 (i.e., assigned to the same possible image impression 66) are added together in order to then select a suitable parameterization based on the sum, etc.

[0128] The setpoint values ​​34, 35 determined in step S19 can subsequently be used to process and display the medical image data 39 (which are in particular further image data sets) provided in step S20 in the same manner in step S21 as explained above for the image data set 43, wherein the control parameters 37, 38 are set to the setpoint values ​​34, 35. This makes it possible to provide a presetting for the processing and display of the medical image data, which presetting, based on a study of the reference person 41, predictively enables a large number of observers 58 to easily and robustly recognize relevant image contents.

[0129] exist Figure 3 In the method shown, the parameterization determined in step S19 is also used to generate a representation of image data set 43 used as image data 39, or the representation selected in step S18 can be output to observer 58. This allows the best image impression to be selected for current image data 39.

[0130] In addition to generating multiple graphs 42 in parallel, it is also possible to generate multiple graphs 42 step by step, similar to the reference Figure 2 To this end, the reference can be improved, for example, as follows Figure 2The described method, i.e. the detection of the image data set 43 or the detection of the image data 39, is only performed in the first iteration or is not part of the method. Subsequently, the setpoint values ​​34, 35 can be preset for the control parameters 37 and / or 38 instead of the setpoint value 33 for the control parameter 36. This can be used, for example, to dynamically find the best representation of a specific reference person 41 for the detected medical image data 39.

[0131] In principle, it is also possible to use reference Figure 3 The method described determines a setpoint value 33 for a control parameter 36 which parameterizes the acquisition of medical image data 39. For this purpose, for example, a plurality of image data sets 43 can be acquired with different parameterizations for the same examination object 49, in particular a phantom, in order to select the best parameterization.

[0132] In addition or alternatively to the aforementioned fields of application, the detection of brain activity during viewing of an image data set or medical image data by a reference person 41 can also be used to enrich a medical image data set with semantic information or to determine a segmentation. Figure 1 In the diagram 42 shown, objects 50 and / or anatomical features 51 can be segmented and annotated with semantic information, i.e., labels. A possible method for this is described below with reference to Figure 4 Elaborate in detail.

[0133] Here, the segmentation and semantic supplementation of medical image data 39 are first discussed directly with reference to steps S22 to S26, wherein the medical image data are used as a medical image data set 43 to be displayed. It is then explained with reference to steps S27 to S30 how this medical image data set is used for training a training function 56, which can then also segment medical image data 39 that differ from the image data set 43 or supplement it with semantic information.

[0134] In step S22 , an image data set 43 is firstly provided, for example, by being captured by an imaging device 64 or being retrieved from a database.

[0135] In step S23, the image data set 43 or a representation 42 of the result of the processing of the image data set 43 is transmitted to a reference person 41 or preferably to a group of reference persons 41. Possible ways of forming a corresponding image 42 have already been explained above.

[0136] In step S24 , a corresponding measurement data set 40 is detected, which relates to the brain activity of the corresponding reference person while viewing the representation 42 .

[0137] In step S25 , semantic information about the objects recognized or at least viewed by the respective reference person 41 in the representation 42 is subsequently extracted by an algorithm (in the exemplary embodiment, a training function 66 ) based on the measurement data set 40 or the measurement data sets 40 .

[0138] As described in detail above in the description or in the cited specialist literature, semantic information can be directly extracted based on the measured brain activity or, for example, an image reconstruction can be performed based on the corresponding measurement data set 40, based on which the displayed object can then be recognized or classified.

[0139] After the object has been successfully identified for at least part of the measurement data set 40, Figure 1 The depicted representation 42 determines, for example, as semantic information 54 that the microcatheter and the anatomical feature 51 as the object 50 were recognized or at least seen by the reference person 41 or at least a part of the reference person 41 .

[0140] In step S26, a segmentation 55 is then determined for the representation 42 or, in general, for the image data set 43. A known segmentation algorithm is used here, which can, however, also use the semantic information 54 and thus also be related to the at least one measurement data set 40. As also described in the description, the location of the recognized semantic features in the representation 42 can also be carried out based on the measurement data set 40, so that a rough segmentation can already be predetermined by the measurement data set 40, which can then preferably also be refined by using an iteratively processed segmentation method, etc.

[0141] The robustness of the recognized semantic information 54 or the segmentation 55 achieved can be improved on the one hand by detecting, as described above, the measurement data sets 40 for a plurality of reference persons 41. On the other hand, a plurality of different representations 42 can also be generated by varying, as described above, the control parameters 37, 38 for the processing and / or representation of the image data sets 43 in order to be able to evaluate a greater number of different measurement data sets 40 overall.

[0142] If the semantic information 54 and / or the segmentation 55 are directly regarded as control parameters 37 for processing the medical image data set 43, the medical image data set can be confirmed by the medical image data 39, and the method for determining the rated value of the control parameter 37 used to control the processing of the medical image data 39 is thus completed in step S25 or step S26.

[0143] Preferably, steps S22 to S26 are also used to provide a training data set, which in step S27 trains the training function 56 by machine learning. For this purpose, as described above, the segmentation 55 and / or the semantic information 54 of the plurality of different image data sets 43 or the representation 42 formed by the plurality of image data sets is supplemented, whereby a training data set 57 can be provided respectively, which includes the medical image data set 43 or, in this embodiment, the representation 42 as input data and the semantic information 54 and / or the segmentation 55 as a nominal result (used within the scope of training).

[0144] If a sufficiently large number of training data sets 57 are available, the training function 56 can be parameterized or trained in step S28 using conventional methods of supervised learning (e.g. using error feedback) so that it can perform the determination of semantic information 54 and segmentation 55 for a large amount of medical image data 39. The control parameters 37 for the processing of the medical image data 39 (for which the setpoint values ​​34 are preset) are in this case the parameters of the training function 56, which are learned by machine learning, for example the node weights in a neural network.

[0145] After the training function 56 is trained, the medical image data 39 to be segmented or enriched with semantic information can be provided in step S29, and then the training function 56 can be applied to the medical image data 39 or the illustration generated by the image data 39 in step S30 to determine the semantic information 54 and the segmentation 55.

[0146] As described above, the basic method of using measurement data based on the sensory detection of respective brain activities of respective reference persons viewing the representation of the medical image data set to parameterize the acquisition and / or processing and / or representation of medical image data can be used in various ways to solve various related problems in medical imaging. As can be seen from the general part of the description, both the basic idea and the further embodiments of the invention are not limited to the specific embodiments mentioned, and the features of the explained embodiments can be combined in various ways.

[0147] Figure 5 and Figure 6 Training functions, such as training functions 56 and / or 66, that may be used in the method are illustrated.

[0148] Figure 5 An embodiment of an artificial neural network 1 is shown. The artificial neural network 1 is expressed in English as "artificial neural network", "neural network", "artificial neural net" or "neural net".

[0149] The artificial neural network 1 includes nodes 6 to 18 (nodes) and edges 19 to 21 (edges), wherein each edge 19 to 21 is a directed connection from a first node 6 to 18 to a second node 6 to 18. Generally speaking, the first node 6 to 18 and the second node 6 to 18 are different nodes 6 to 18, and the first node 6 to 18 and the second node 6 to 18 may also be the same. For example, in Figure 5 , edge 19 is a directed connection from node 6 to node 9, and edge 21 is a directed connection from node 16 to node 18. Edge 19 to 21 from the first node 6 to 18 to the second node 6 to 18 is also referred to as an "ingoing edge" of the second node 6 to 18 and an "outgoing edge" of the first node 6 to 18.

[0150] In the present embodiment, the nodes 6 to 18 of the artificial neural network 1 can be arranged in layers 2 to 5, wherein the layers can have an internal hierarchy introduced by edges 19 to 21 between the nodes 6 to 18. In particular, edges 19 to 21 may only exist between adjacent layers of nodes 6 to 18. In the illustrated embodiment, there is an input layer 2 which comprises only nodes 6, 7, 8, each without any incoming edges. The output layer 5 comprises only nodes 17, 18, in each case without outgoing edges, and there are additional hidden layers 3 and 4 between the input layer 2 and the output layer 5. In general, the number of hidden layers 3, 4 can be selected arbitrarily. The number of nodes 6, 7, 8 in the input layer 2 usually corresponds to the number of input values ​​of the neural network 1, while the number of nodes 17, 18 in the output layer 5 usually corresponds to the number of output values ​​of the neural network 1.

[0151] In particular, nodes 6 to 18 of the neural network 1 can be assigned a (real number). (n) i represents the value of the i-th node 6 to 18 of the n-th layer 2 to 5. The values ​​of the nodes 6, 7, 8 of the input layer 2 correspond to the input values ​​of the neural network 1, and the values ​​of the nodes 17, 18 of the output layer 5 correspond to the output values ​​of the neural network 1. In addition, each edge 19, 20, 21 can be assigned a weight in the form of a real number. The weight is especially a real number in the interval [-1, 1] or in the interval [0, 1,]. Here, w (m,n) i,j represents the weight of the edge between the i-th node 6 to 18 in the m-th layer 2 to 5 and the n-th node 6 to 18 in the 2 to 5 layers. Representation weight

[0152] In order to calculate the output value of the neural network 1, the input value is propagated through the neural network. In particular, the values ​​of nodes 6 to 18 of the (n+1)th layer 2 to 5 can be calculated based on the values ​​of nodes 6 to 18 of the nth layer 1 to 5 by the following formula:

[0153]

[0154] In this case, f is a transfer function, also called an activation function. Known transfer functions are step functions, sigmoid functions (e.g., logistic functions, generalized logistic functions, hyperbolic tangent functions, inverse tangent functions, error functions, smoothed step functions), or rectifier functions. Transfer functions are mainly used for normalization.

[0155] Specifically, the values ​​are propagated layer by layer through the neural network 1, where the values ​​of the input layer 2 are given by the input data of the neural network 1. The values ​​of the first hidden layer 3 can be calculated based on the values ​​of the input layer 2 of the neural network 1, while the values ​​of the second hidden layer 4 can be calculated based on the values ​​of the first hidden layer 3, and so on.

[0156] In order to be able to specify the values ​​of edges 19 to 21 The neural network 1 must be trained using training data. The training data includes in particular training input data and training output data, hereinafter referred to as t i In a training step, the neural network 1 is applied to training input data in order to determine calculated output data. In particular, the training output data and the calculated output data comprise a number of values, which are determined by the number of nodes 17 , 18 in the output layer 5 .

[0157] In particular, the comparison between the calculated output data and the training output data is used to recursively adjust the weights of the neural network 1 (back propagation algorithm). The weights can be changed in accordance with the following formula, in which γ is the learning rate and Can be calculated recursively

[0158]

[0159] for

[0160]

[0161] When the (n+1)th layer is not the output layer 5, based on and

[0162]

[0163] When the (n+1)th layer is the output layer 5, where f' is the first derivative of the activation function, and is the comparative training value of the j-th node 17 and 18 of the output layer 5.

[0164] The following reference Figure 6An example of a convolutional neural network (CNN) is also given. It should be noted that the term "layer" used here is slightly different from that used in classical neural networks. For classical neural networks, the term "layer" refers only to the set of nodes that make up a layer, that is, a specific generation of nodes. In convolutional neural networks, "layer" usually refers to an object that actively modifies data, in other words, a collection of nodes of the same generation, and a collection of incoming or outgoing edges.

[0165] Figure 6 An embodiment of a convolutional neural network 22 is shown. In the embodiment shown, the convolutional neural network 22 includes an input layer 23, a convolutional layer 24 (ConvolutionalLayer), a pooling layer 25, a fully connected layer 26 and an output layer 27. In an alternative design, the convolutional neural network 22 can contain multiple convolutional layers 24, multiple pooling layers 25 and multiple fully connected layers 26, just like other types of layers. The order of the layers can be selected arbitrarily, wherein the fully connected layer 26 usually constitutes the last layer before the output layer 27.

[0166] In particular, the nodes 28 to 32 of the 23rd to 27th layers in the convolutional neural network 22 can be understood as a d-dimensional matrix or a d-dimensional image. In particular, especially in the two-dimensional case, the values ​​of the nodes 28 to 32 with indices i, j in the nth layer 23 to 27 can be represented as x (n) [i,j]. It should be noted that the arrangement of nodes 28 to 31 in layers 23 to 27 has no effect on the computation of the convolutional neural network 22 itself, since these effects are entirely determined by the structure and weights of the edges.

[0167] The convolutional layer 24 is characterized in that the structure and weights of the input edges constitute a convolution operation based on a specific number of kernels. In particular, the structure and weights of the input edges are selected so that the value of the node 29 of the convolutional layer 24 Based on the value x of node 28 in the previous layer 23 (n-1) is calculated as a convolution The convolution is defined in two dimensions as

[0168]

[0169] Here, the kth kernel K kis a d-dimensional matrix, in this embodiment a two-dimensional matrix, which is usually small compared to the number of nodes 28 to 32, for example a 3×3 matrix or a 5×5 matrix. This means, in particular, that the weights of the incoming edges are not independent, but are selected for them to derive the convolution equation. Taking the kernel forming a 3x3 matrix as an example, regardless of the number of nodes 28 to 32 in the corresponding layer 23 to 27, there are only 9 independent weights (one independent weight for each entry in the kernel matrix). In particular, for a convolutional layer 24, the number of nodes 29 in the convolutional layer is equal to the number of nodes 28 in the previous layer 23 multiplied by the number of convolution kernels.

[0170] When the nodes 28 of the previous layer 23 are arranged as a d-dimensional matrix, the use of multiple kernels can be interpreted as adding an additional dimension (also called the "depth" dimension), so that the nodes 29 of the convolution layer 24 are arranged as a (d+1)-dimensional matrix. If the nodes 28 of the previous layer 23 are already arranged as a (d+1)-dimensional matrix with a depth dimension, then the use of multiple convolution kernels can be understood as an expansion along the depth dimension, so that the nodes 29 of the convolution layer 24 are also arranged into a (d+1)-dimensional matrix. The size of the (d+1)-dimensional matrix in the depth dimension is larger than that of the previous layer 23, and its coefficient is determined by the number of kernels.

[0171] The advantage of using convolutional layers 24 is that it is able to exploit spatially local correlations in the input data by enforcing local connectivity patterns between nodes in adjacent layers, in particular each node has only connections to a small region of nodes in the previous layer.

[0172] In the illustrated embodiment, the input layer 23 includes 36 nodes 28, which are arranged in a two-dimensional 6×6 matrix. The convolution layer 24 includes 72 nodes 29, which are arranged in two two-dimensional 6×6 matrices, where each of the two matrices is the result of convolution of the values ​​of the input layer 23 with the convolution kernel. In the same way, the nodes 29 of the convolution layer 24 can be understood as being arranged in a three-dimensional matrix of 6x6x2, the last dimension being the depth dimension.

[0173] The advantage of the pooling layer 25 is that the structure and weights of the input edges and the activation functions of its nodes 30 define a pooling operation based on a nonlinear pooling function f. For example, in the two-dimensional case, the value x of the node 30 of the pooling layer 25 can be calculated based on the value x(n+1) of the node 29 of the previous layer 24. (n+1) , as shown below:

[0174] x (n) [i,j]=f(x (n-1) [id1,jd2],…,x (n-1) [id1+d1-1,jd2+d2-1])

[0175] Calculation. In other words, using a pooling layer 25, the number of nodes 29, 30 can be reduced by replacing the number of d1xd2 neighboring nodes 29 in the previous layer 24 with a single node 30, the number of which is calculated from the numerical value of the referenced number of neighboring nodes 29. The pooling function f can in particular be a maximum function, a mean value or an L2 norm. In particular, the weights of the incoming edges can be specified for the pooling layer 25 and cannot be modified by training.

[0176] The advantage of using a pooling layer 25 is that the number of nodes 29, 30 and the number of parameters are reduced. This reduces the computational load required by the convolutional neural network 22, thereby controlling overfitting.

[0177] In the exemplary embodiment shown, the pooling layer 25 is a maximum pooling layer, in which four adjacent nodes are replaced by only one node, the value of which is formed by the maximum value of the four adjacent node values. The maximum pooling operation is applied to each d-dimensional matrix of the previous layer; in the present exemplary embodiment, the maximum pooling operation is applied to each of the two two-dimensional matrices, so the number of nodes is reduced from 72 to 18.

[0178] The fully connected layer 26 may be characterized in that most, in particular all, edges exist between the nodes 30 of the previous layer 25 and the nodes 31 of the fully connected layer 26, wherein the weight of each edge can be adjusted individually. In the present embodiment, the nodes 30 in the previous layer 25 of the fully connected layer 26 are represented both as a two-dimensional matrix and additionally as non-continuous nodes (represented as node rows, wherein the number of nodes is reduced to improve representability). In the present embodiment, the number of nodes 31 in the fully connected layer 26 is equal to the number of nodes 30 in the previous layer 25. In alternative embodiments, the number of nodes 30, 31 may be different.

[0179] Furthermore, in this embodiment, the values ​​of the nodes 32 of the output layer 27 can be determined by further determining the values ​​of the nodes 31 of the previous layer 26 by applying a softmax function. By applying the softmax function, the sum of the values ​​of all nodes 32 of the output layer is 1, and all values ​​of all nodes 32 of the output layer are real numbers between 0 and 1. When the convolutional neural network 22 is used to classify input data, the values ​​of the output layer 27 can be interpreted in particular as the probability that the input data belongs to one of the different categories.

[0180] The convolutional neural network 22 may also include a ReLU layer, where ReLU is an acronym for "rectified linear unit". In particular, the number of nodes and the node structure of the ReLU layer are the same as the number of nodes and the node structure of the previous layer. In particular, the value of each node in the ReLU layer is calculated by applying a rectification function to the value of the corresponding node in the previous layer. Examples of rectification functions are f(x)=max(0,x), a hyperbolic tangent function, or a sigmoid function.

[0181] The convolutional neural network 22 can be trained in particular according to a back-propagation algorithm. To avoid overfitting, regularization methods can be used, such as the elimination of individual nodes 28 to 32, random pooling, the use of artificial data, weight decay based on L1 or L2 norms, or maximum norm constraints.

[0182] Although the details of the present invention have been illustrated and described in detail through the preferred embodiments, the present invention is not limited to the disclosed examples, and those skilled in the art may derive other different variations therefrom without departing from the scope of protection of the present invention.

Claims

1. A computer-implemented method for determining a corresponding setpoint value (33-35) of at least one control parameter (36-38) for controlling the detection and / or processing and / or display of medical image data (39), the method comprising the steps of: - during the viewing of at least one representation (42) by a reference person (41), at least one measurement data set (40) is obtained, the measurement data set being based on sensory detection of a corresponding brain activity of the corresponding reference person (41), wherein: Each illustration (42) is based on a corresponding medical image dataset (43), and - determining a setpoint value (33-35) based on the measurement data set (40).

2. The computer-implemented method of claim 1, wherein: The individual diagrams (42) are dependent on reference values ​​(44-46) for the individual control parameters (34-36) assigned to the individual diagrams (42), wherein the corresponding setpoint values ​​(33-35) for the individual control parameters (36-38) are also determined as a function of the reference values ​​(44-46) for the individual control parameters (36-38), the reference values ​​being assigned to the diagram (42) or to at least one of the diagrams (42).

3. The computer-implemented method of claim 2, wherein: The individual illustrations (42), i.e. the detection of the medical image data set (43) and / or the processing of the medical image data set (43) for providing a processing result (4) to be illustrated and / or the illustration of the medical image data set (43) and / or the processing result (47), depend on reference values ​​(44-46) for the individual control parameters (36-38) assigned to the individual illustrations (42).

4. A computer-implemented method according to any one of the preceding claims, characterized in that Based on individual measurement data sets (40) or on groups of measurement data sets (40), a corresponding quality criterion (48) is determined for each illustration (42), wherein the rated value (33-35) or at least one rated value (33-35) is determined according to the quality criterion (48), wherein the measurement data sets are based on sensory detection of brain activity of different reference persons (41) when viewing the same illustration (42).

5. The computer-implemented method of claim 4, wherein: Control parameters (36-38) or at least one control parameter (36-38) are used to control the processing and / or representation of medical image data (39), wherein a plurality of mutually different representations (42) are generated based on respective medical image data sets (43), in that the respective medical image data sets (43) are processed and represented by the same method as for the medical image data (39), wherein for providing the respective representations (42) reference values ​​(44-46) respectively assigned to the representations (42) are used to predetermine the respective control parameters (36-38), wherein: Different diagrams (42) differ from each other at least in terms of a reference value (44-46) of at least one control parameter (36-38), wherein a subgroup of diagrams (42) including one of the diagrams (42) or not all of the diagrams (42) is selected based on a quality criterion (48) determined for each diagram (42), whereby the setpoint value (33-35) of each control parameter (36-38) is set based on the selected diagram (42) or the corresponding reference value (44-46) assigned to the corresponding selected diagram (42).

6. A computer-implemented method according to any one of the preceding claims, characterized in that The medical image data set (43) on which the illustration (42) is based is a medical image data set (43) determined within the scope of an imaging sequence, wherein the imaging sequence includes repeated or continuous detection of medical image data (39) of the same examination object (49), wherein rated values ​​(33-35) are used to control the detection and / or processing and / or illustration of at least part of the medical image data (39), which is detected within the scope of the same imaging sequence after detecting the medical image data set (43).

7. A computer-implemented method according to any one of the preceding claims, characterized in that A check is performed for at least one object (50) and / or at least one anatomical feature (51) to determine whether a perception condition (52, 53) is satisfied, the satisfaction of which depends on the respective measurement data set (40) and indicates the perception of the respective object (50) or anatomical feature (51) by the respective reference person (41) in the respective illustration (42), wherein the rated value (33-35) or at least one rated value (33-35) is determined as a function of the satisfaction of the respective perception condition (52, 53).

8. A computer-implemented method according to any one of the preceding claims, characterized in that The control parameter (36-38) or each control parameter (36-38) specifies an X-ray dose radiated onto the examination object (49) for detecting medical image data (39) and / or specifies an imaging rate for detecting medical image data (39).

9. A computer-implemented method according to any one of the preceding claims, characterized in that The control parameters (36-38) or at least one control parameter (36-38) for which a preset rated value (33-35) controls the enrichment of medical image data (39) using semantic information (554), and / or controls the segmentation (55) of the medical image data (39) or of a processing result determined based on the medical image data (39), wherein the enrichment and / or segmentation is part of processing the medical image data (39).

10. A computer-implemented method according to any one of the preceding claims, characterized in that A training function (56) is trained, which is used for processing medical image data (39) or for use in processing medical image data (39), wherein control parameters (36-38) are parameters of the training function (56), whose rated values ​​(33-35) are preset by machine learning in the case of supervised learning, wherein a plurality of training data sets (57) are used for supervised learning, wherein the training data sets each include at least one measurement data set (40) or each depend on at least one measurement data set (40).

11. A computer-implemented method according to any one of the preceding claims, characterized in that The training function (56) is designed to perform enrichment of the medical image data (39) with semantic information (54) and / or segmentation (55) of the medical image data (39).

12. A computer-implemented method according to any one of the preceding claims, characterized in that Determining the setpoint value (33-35) from the measurement data set (40) includes applying a machine-trained function (66) to the measurement data set (40) and / or processed data determined therefrom.

13. A computer-implemented method for displaying medical image data to a viewer (58), wherein: The detection of the medical image data (39) and / or the processing of the medical image data (39) to provide a processing result (47) to be displayed and / or a representation of the medical image data (39) and / or the processing result (47) are parameterized respectively by at least one control parameter (36-38), wherein the rated value (33-35) used for the respective control parameter (36-38) is preset by a computer-implemented method according to any of the above claims.

14. A processing device, characterized in that: The processing device is designed to carry out a computer-implemented method according to any of the preceding claims.

15. A computer program comprising program instructions which are designed to carry out a computer-implemented method according to any one of claims 1 to 13 when the program instructions are run in a data processing device (60).

16. A data carrier comprising a computer program (63) according to claim 15.

Images