Methods, devices, systems, products and media for detecting out-of-distribution conditions and training uncertainty models

Abstract

The embodiment of the invention relates to a method, equipment, a system, a product and a medium for detecting an out-of-distribution condition and training an uncertainty model. The invention relates to a computer-implemented method for detecting out-of-distribution conditions, comprising: receiving, by an input interface of a data processing device (12), medical imaging data (22) comprising voxels, where the medical imaging data (22) represents an anatomical region (16) comprising a set of objects, the set of objects comprising at least one anatomical object (14); applying, by the computing unit, a cognitive Bayesian uncertainty model (38) to the medical imaging data (22) to determine cognitive uncertainty information (52) describing a cognitive uncertainty of assignment information (54), the assignment information (54) describing an assignment of a respective voxel to a respective anatomical object (14); applying a scoring program to the cognitive uncertainty information (52) to determine scoring information (36); and providing, by the output interface, a warning signal if the scoring information (36) satisfies a predefined out-of-distribution condition.

Description

Technical Field

[0001] The present invention relates to a computer-implemented method for detecting out-of-distribution cases, a computer-implemented method for training an epistemic Bayesian uncertainty model, a data processing apparatus configured to perform a computer-implemented method for determining out-of-distribution cases and / or a computer-implemented method for training an epistemic Bayesian uncertainty model, an imaging system (particularly a medical imaging system including a data processing apparatus), a computer program product, and a computer-readable medium. Background Technology

[0002] In radiotherapy, contour mapping is an essential process for accurately depicting the target treatment area, including organs at risk, tumor volume, and dose calculation regions. However, significant challenges arise when using deep learning models for this purpose, particularly with out-of-distribution (OOD) data. While these models are efficient, they can generate unreliable outputs when faced with scenarios not included in their training data. This unreliability introduces safety risks, especially when clinicians may inadvertently use systems beyond their intended specifications without fully recognizing their limitations. Figure 1 As shown, instances such as the presence of femoral implants, brachytherapy dressings, or hydrogel rectal spacers are just a few examples of challenging scenarios that, if not included during the model's training phase, can lead to inaccuracies in contour drawing. It is crucial to recognize that these are representative cases, and many other OOD instances may arise. The primary problem this invention aims to address is the detection of OOD data in contour drawing for radiotherapy. By identifying such instances, the system provides a vital warning to clinicians about potential inaccuracies and reduces the risk of using unreliable contours in treatment planning. This OOD detection is essential for ensuring safety and effectiveness in radiotherapy because it helps prevent the accidental use of the system in scenarios where the system output is unreliable, thereby mitigating the risks associated with applying AI-driven solutions outside their effective operational scope.

[0003] Highly efficient contour mapping models exist suitable for detecting over 153 organs and anatomical structures at risk. These models have been successfully deployed in scanners, workstations, and the cloud at over 600 locations worldwide. Despite these advances, traditional methods have primarily focused on improving contour accuracy rather than explicitly quantifying or utilizing measures of uncertainty, such as cognitive uncertainty in the contour mapping process.

[0004] Given the current level of technology, there is a lack of overall uncertainty management: in the context of radiotherapy, uncertainty is acknowledged, typically concerning anatomical variations and image quality. However, these uncertainties are often addressed through rigorous training, standardized protocols, and manual review processes, rather than through systematic, quantifiable methods.

[0005] Given the current state of technology, there is a lack of utilization of cognitive uncertainty. There is a significant gap in explicitly using cognitive uncertainty to mitigate the risks in radiotherapy profiling. While some profiling models in medical imaging may implicitly account for uncertainty, they typically do not provide clear, quantifiable measures of cognitive uncertainty that can be directly used to inform clinical decisions or to label potential out-of-distribution cases.

[0006] Recent advances in AI have begun to explore the integration of uncertainty measures, but these have not yet been widely adopted in clinical practice, particularly in the specific context of radiotherapy profiling. The utilization of cognitive uncertainty, especially for detecting out-of-distribution scenes, represents a novel approach in this field.

[0007] The current state of the technology is described in the following documents:

[0008] US11275976B2 describes medical image evaluation with classification uncertainty. A medical image can be classified by receiving a first medical image. The medical image can be applied to a machine learning classifier, which can be trained on a second medical image. Labels and a measure of uncertainty for the medical image can be generated. The measure of uncertainty can be compared to a threshold. When the measure of uncertainty is within the threshold, the first medical image and its label can be output.

[0009] US11185231B2 describes intelligent multiscale medical image landmark detection. Intelligent multiscale image resolution, while searching for anatomical landmarks, uses an artificial agent to determine the optimal size of each observation at a given time point.

[0010] US10600185B2 discloses automatic liver segmentation using adversarial image-to-image networks on three-dimensional medical images of patients. Summary of the Invention

[0011] The purpose of this invention is to provide a method that allows for the detection of out-of-distribution conditions in the analysis of medical imaging data.

[0012] This objective is achieved through the subject matter of the independent claims. Further implementations and preferred embodiments are the subject matter of the dependent claims.

[0013] This invention seeks to bridge this gap by integrating cognitive uncertainty assessment into AI tools to enhance safety and effectiveness in radiotherapy profiling.

[0014] A first aspect of the invention relates to a computer-implemented method for detecting out-of-distribution conditions in medical imaging data. Medical imaging data, comprising voxels, is received via an input interface of a data processing device. The medical imaging data refers to a digital representation of an anatomical region obtained through an imaging modality such as computed tomography (CT), magnetic resonance imaging (MRI), or positron emission tomography (PET). The medical imaging data includes a set of two-dimensional images reconstructed to form a three-dimensional volume, wherein each volume element or voxel contains information about underlying tissue properties such as density, intensity, or contrast. Therefore, the medical imaging data includes voxels representing anatomical regions. Because at least one predefined anatomical object is located within the anatomical region, at least some voxels represent at least one predefined anatomical object. The anatomical region may be a predefined body region. At least one predefined anatomical object may be a specific organ located within the anatomical region. At least one anatomical object is, for example, a specific organ for which an organ outline will be provided, such that it can be highlighted in the visualization of the anatomical region.

[0015] It can be stipulated that assignment information is determined for each voxel, assigning the voxel to the anatomical object represented by the voxel. The assignment information can be generated using one of the contour determination models described at the beginning. However, due to cognitive uncertainty, assignment can be prone to error. Therefore, it is necessary to determine the cognitive uncertainty of the assignment in order to estimate the reliability of the assignment and to determine whether out-of-distribution situations exist.

[0016] This method involves applying a cognitive Bayesian uncertainty model to medical imaging data to determine cognitive uncertainty information, which describes the cognitive uncertainty of the assignment of each voxel to its corresponding anatomical object. This step is performed by the computational unit of the data processing device. The cognitive Bayesian uncertainty model comprises a deep ensemble model with base learners, where each base learner determines corresponding weak assignment information for the corresponding voxel in the corresponding forward pass. The weak assignment information may include values ​​that provide the assignment of a voxel to its corresponding anatomical object, or values ​​that provide the probability of such assignment.

[0017] Each base learner in the deep ensemble model is trained on a corresponding subset of the training data and includes a Monte Carlo dropout layer. In other words, each base learner in the deep ensemble model is trained independently on a separate subset of the training data. The training data can represent different samples of an anatomical region, where assignment information is used to label voxels in the anatomical region belonging to the corresponding anatomical object. Because each subset includes different samples, the base learners provide different outputs for the same input. The uncertainty model is constructed as a deep ensemble model consisting of a predetermined number of base learners, where each base learner is trained on a corresponding subset of the training dataset using a machine learning algorithm to generate weak assignment information that describes the assignment of the corresponding voxel to the corresponding anatomical object in the corresponding forward propagation. In other words, each base learner in the deep ensemble model determines a prediction about the affiliation of each voxel in the medical imaging data with the corresponding predefined anatomical object.

[0018] The cognitive Bayesian uncertainty model also includes a corresponding Monte Carlo dropout implementation applied to the Monte Carlo dropout layer of the base learner during each forward propagation. In other words, the cognitive Bayesian uncertainty model is applied to the medical imaging data by the computational unit. Each base learner within the deep ensemble model includes a Monte Carlo dropout layer. Monte Carlo dropout is used to approximate the Bayesian neural network, allowing uncertainty to be estimated by averaging over multiple random forward propagations via the neural network. In this context, the cognitive Bayesian uncertainty model is designed to perform a corresponding Monte Carlo dropout in the Monte Carlo dropout layer during each forward propagation via the base learner. This means that for each base learner in the deep ensemble model, the Monte Carlo dropout layer is activated during the corresponding forward propagation, and a random subset of neurons is dropped. Multiple forward propagations can be performed to account for the random variations introduced by the Monte Carlo dropout layer. A key advantage of using Monte Carlo dropout layers within the base learners is that they enable the estimation of cognitive uncertainty by capturing the variability across multiple random forward propagations. This allows for the assessment of cognitive uncertainty related to the voxel-to-anatomical object assignment in medical imaging data.

[0019] In the next step, based on the variance of the predicted weak assignments of the corresponding voxels, the corresponding cognitive uncertainty of the affiliation between the voxel and the corresponding predefined anatomical object is determined. In other words, to quantify the cognitive uncertainty in the predicted affiliation of each voxel with the predefined anatomical object, the cognitive uncertainty is determined for each voxel based on the variance of the predicted weak assignments of the different base learners across the deep ensemble model. Specifically, for each voxel, predicted weak assignments from all base learners are collected. The mean and / or standard deviation of this output set can then be determined to provide the cognitive uncertainty of the predicted assignment for each voxel.

[0020] In the next step, a scoring procedure is applied to the cognitive uncertainty information to generate a score. If this score meets predefined out-of-distribution conditions related to out-of-distribution situations, a warning signal is generated by the output interface of the data processing device. In other words, the scoring procedure involves evaluating the cognitive uncertainty information to generate a score. This score reflects the confidence or reliability of the voxel-to-anatomical object assignment in medical imaging data based on cognitive uncertainty.

[0021] Out-of-distribution situations refer to cases where medical imaging data deviates significantly from the distribution of the training data used to build and train deep ensemble models. Out-of-distribution situations can occur when medical imaging data represents anatomical regions or includes features or patterns not adequately represented in the training data, leading to higher uncertainty in voxel-to-anatomical object assignment. This typically occurs when an implant is present in an anatomical region, or when a specific abnormality is present in an anatomical region that was not represented in any training data during training.

[0022] To demonstrate the existence of out-of-distribution situations, the scoring information can be checked to see if it meets predefined out-of-distribution conditions. Such out-of-distribution situations can indicate that the cognitive uncertainty in the allocation of voxels to anatomical objects is significantly higher than expected, thus suggesting a potential out-of-distribution situation. If these out-of-distribution conditions are met, the output interface generates a warning signal to alert the user to the potential out-of-distribution situation with the medical imaging data.

[0023] The advantage of this invention is that it allows for the detection of possible out-of-distribution situations in the analysis of medical imaging data.

[0024] Unless otherwise stated, all steps of the computer-implemented method according to the first aspect of the invention can be performed by a data processing device. Specifically, the data processing device is configured or adapted to perform the steps of the computer-implemented method. For this purpose, the data processing device may, for example, store a computer program including instructions that, when executed by the data processing device, cause the data processing device to perform the computer-implemented method.

[0025] According to another embodiment of the invention, the computer-implemented method includes determining allocation information describing the assignment of corresponding voxels in medical imaging data. The allocation information is determined based on the mean of the weak allocation information provided by the base learners in the deep ensemble model for the corresponding voxels during their respective forward propagations. Once all base learners have completed their forward propagations and provided their corresponding weak allocation information, the cognitive Bayesian uncertainty model provides the mean of the weak allocation information for each voxel. This mean serves as the basis for the allocation information, or allocation information, describing the assignment of a particular voxel to its corresponding anatomical object. Using a mean-based approach to determine the allocation information allows for a more robust and reliable representation of voxel-to-anatomical object assignments because it reduces the influence of outliers or individual base learners who may be overconfident or underconfident in their weak allocation information. An advantage of this embodiment is that the allocation information and the cognitive uncertainty information are provided by the same model.

[0026] According to another embodiment of the invention, the computer-implemented method includes determining allocation information describing the assignment of corresponding voxels in the medical imaging data by applying a predefined contour drawing model to the medical imaging data. This process is performed by a computing unit and involves assigning the corresponding voxels to their corresponding anatomical objects using the predefined contour drawing model. A contour drawing model (also called a segmentation or delineation model) refers to a set of rules or algorithms that define how to identify and delineate the boundaries of a specific anatomical object within the medical imaging data. Contour drawing models can be based on various techniques, such as threshold-based methods, region growing, or machine learning methods (e.g., deep neural networks).

[0027] According to another embodiment of the invention, the computer-implemented method includes generating a visualization of a predefined region of interest (ROI) of an anatomical region based on medical imaging data. For each identified anatomical object within the predefined ROI, a corresponding anatomical object contour is generated. Based on voxels assigned to the anatomical object in question, the contour encloses a volume representing the corresponding anatomical object. The predefined ROI may be a layer passing through the anatomical region, thus providing a cross-section through the anatomical region and the anatomical object. In this case, the anatomical object contour may be a 2D contour defining a cross-sectional region of the anatomical object. The anatomical object contour may be generated based on voxels assigned to the corresponding anatomical object located within the layer. The predefined ROI may also be a predefined 3D region. This embodiment allows visualization of individual anatomical objects within the ROI, thereby facilitating more accurate diagnosis and treatment planning. Finally, this embodiment includes generating review data for diagnosis. The review data includes a visualization of the predefined ROI, the anatomical object contour of each identified anatomical object, and at least one piece of cognitive uncertainty information or scoring information. These visualization and review features allow for the interpretation of the medical imaging data.

[0028] According to another embodiment of the present invention, the computer-implemented method includes: post-processing the contour of the corresponding anatomical object based on cognitive uncertainty information of at least some voxels assigned to the corresponding anatomical object, and / or based on predefined morphological conditions.

[0029] In other words, the method includes applying post-processing to refine the generated anatomical object contours based on cognitive uncertainty information or predefined morphological conditions. In this embodiment, the method processes the corresponding anatomical object contours according to cognitive uncertainty information associated with at least one voxel assigned to the corresponding anatomical structure. Cognitive uncertainty information allows the identification of voxels or regions between neighboring anatomical objects where the depiction may be unclear or ambiguous. By incorporating this information during post-processing, the method can adjust the contours to better align with actual anatomical boundaries, thereby enhancing diagnostic accuracy and confidence. It is possible to remove voxels with cognitive uncertainty information exceeding a predetermined threshold at the boundaries of the anatomical object contours. Additionally, the method post-processes the corresponding anatomical object contours according to predefined morphological conditions. These conditions refer to a set of rules or constraints based on known characteristics of typical anatomical structures, such as shape, size, volume, or spatial relationships between neighboring structures. By applying these morphological conditions during post-processing, the anatomical object contours can be adjusted and refined to provide a more accurate match to real-world anatomical object shapes.

[0030] According to another embodiment of the invention, the scoring procedure includes: for each contour of each identified anatomical object, determining corresponding contour uncertainty score information based on cognitive uncertainty information associated with voxels representing that particular anatomical object within a volume. Alternatively, cognitive uncertainty information from voxels located within volumes covering all detected anatomical objects can be used to determine total uncertainty score information. Determining contour uncertainty score information for each individual contour may involve analyzing the cognitive uncertainty of voxels corresponding to that particular anatomical object within a volume. By evaluating these uncertainties, potential errors or inconsistencies in the contour definition can be better understood, leading to improvements in overall accuracy and reliability. In addition to contour uncertainty score information, a total uncertainty score can also be calculated by examining cognitive uncertainty information of voxels representing each anatomical object across all volumes. This aggregated metric provides a comprehensive overview of the overall reliability and accuracy across multiple segmented structures, allowing for more efficient quality control and evaluation during medical image analysis. By considering both local contour uncertainty and the global total uncertainty score, out-of-distribution conditions can be examined against corresponding out-of-distribution conditions to identify out-of-distribution situations.

[0031] According to another embodiment of the invention, Monte Carlo dropout layers are arranged after each residual block layer of the corresponding base learner in the deep ensemble model. This configuration achieves robust regularization, which helps prevent overfitting and improves the model's ability to generalize across a wide range of anatomical and medical imaging datasets. In this embodiment, each base learner includes multiple residual block layers responsible for learning and extracting relevant features from the input training data. The Monte Carlo dropout layer follows each residual block layer to introduce randomness during training by randomly dropping a certain proportion of activations in the network. This randomization technique effectively prevents over-reliance on specific connections or features, thereby promoting more diverse and comprehensive learning. Using Monte Carlo dropout layers after each residual block layer within the base learner enhances the overall robustness and accuracy of the cognitive Bayesian uncertainty model. In summary, arranging Monte Carlo dropout layers after each residual block layer within the base learner represents an effective strategy for improving robustness and generalization ability.

[0032] According to another embodiment of the invention, the computer-implemented method includes receiving request information describing a set of objects via a request interface of a data processing device. The request information describes the set of objects to be analyzed. Objects typically include one or more anatomical objects of particular interest in a given medical context. By explicitly specifying the set of anatomical objects, the user can ensure that subsequent analysis focuses on relevant and significant aspects of the medical imaging data. The request information may also include regions of interest for providing visualization. The request interface allows for seamless interaction between the user and the computer-implemented method, thereby facilitating efficient communication and reducing potential obstacles to accurate diagnostic results. Users can easily input their desired set of objects using a graphical user interface or other suitable input methods, thereby simplifying the overall process and ensuring that the output of the method meets their specific needs and expectations.

[0033] A second aspect of the invention relates to a computer-implemented training method for training a cognitive Bayesian uncertainty model.

[0034] The first step involves receiving training medical imaging data through a first input interface of a data processing device. Relevant samples of the training medical imaging data include voxels representing anatomical regions containing a training object set, which includes at least one training anatomical object. This data provides the basis for training input data during a computer-implemented training method to train a cognitive Bayesian uncertainty model.

[0035] The next step involves receiving training assignment information for the training data via a second input interface of the data processing device. This training assignment information describes the allocation of each voxel within the training medical imaging data to its corresponding training anatomical object. By incorporating this training assignment information as the training output, the cognitive Bayesian uncertainty model can learn relationships between different anatomical structures, including the anatomical object and its corresponding voxel.

[0036] The next step involves partitioning the training data into subsets according to a given specification, such that each subset is assigned to a corresponding base learner in the deep ensemble model of cognitive Bayesian uncertainty. This separation allows for parallelization and optimization of the learning process, as each base learner focuses on understanding and interpreting a different aspect of the training data. In other words, each base learner is trained on a subset of the total samples, rather than receiving access to the entire set during the learning process. This partitioning approach allows the corresponding models to develop independently, focusing on different aspects of the input data, which in turn fosters diversity among the base learners and enhances the overall performance and robustness of the deep ensemble model. By exposing each learner to a unique subset of the samples, the system can better capture uncertainty and variation in the data, thereby improving generalization ability and more accurate predictions of out-of-distribution or novel situations.

[0037] After dividing the training data into subsets, the computational units of the data processing equipment are used to train the base learners based on the corresponding subsets. By providing a personalized training experience for each base learner, the cognitive Bayesian uncertainty model can form a more comprehensive and detailed understanding of the anatomical structures present within the training medical imaging data.

[0038] Finally, a cognitive Bayesian uncertainty model is provided through the output interface of the data processing device. This cognitive Bayesian uncertainty model can be provided to the data processing device to execute the method according to the first aspect of the invention.

[0039] According to another embodiment of the invention, the computer-implemented method incorporates a specific data separation strategy when processing training data in the context of a request for diagnostic analysis of a specified set of objects. In this configuration, the training data is divided into eight subsets, such that each subset is assigned to a base learner in a deep ensemble.

[0040] The training data separation process follows strict guidelines designed to ensure a uniform distribution and minimize redundancy between different subsets. Each base learner processes at most N / 2+1 cases from the N cases in the total training data, such that each case is included in exactly four of the eight subsets. This approach ensures that each base learner receives a unique but overlapping subset of the training data, thus promoting more comprehensive learning and better generalization across a wide range of anatomical structures and medical imaging scenarios. Furthermore, the overlap between any pair of subsets is limited to at most N / 4+1 cases. This constraint further prevents overfitting and fosters robustness in deep ensemble models by preventing individual base learners from becoming overly specialized or dependent on specific patterns within their respective subsets. The use of this data separation strategy contributes to more accurate diagnostic results by promoting diversity, comprehensiveness, and robustness within the computer-implemented approach.

[0041] Generally, trained cognitive Bayesian uncertainty models and / or trained profiling models mimic the cognitive function of humans associating with other human thoughts. Specifically, through training on training data, trained functions can adapt to new environments and detect and infer patterns.

[0042] Generally, the parameters of a trained model can be adapted through training. Specifically, supervised training, semi-supervised training, unsupervised training, reinforcement learning, and / or active learning can be used. Further, representation learning (an alternative term is "feature learning") can be used. Specifically, the parameters of a trained function can be iteratively adapted through several training steps.

[0043] Specifically, the trained model may include neural networks, support vector machines, decision trees, and / or Bayesian networks, and / or the trained model may be based on k-means clustering, Q-learning, genetic algorithms, and / or association rules. Specifically, the neural network may be a deep neural network, a convolutional neural network, or a convolutional deep neural network. Further, the neural network may be an adversarial network, a deep adversarial network, and / or a generative adversarial network.

[0044] According to a third aspect of the invention, a data processing apparatus is provided, which is configured to perform a computer-implemented method according to a first aspect of the invention and / or a method according to a second aspect of the invention.

[0045] A data processing device can be specifically understood as a data processing device that includes processing circuitry. Therefore, a data processing device can specifically process data to perform computational operations. This can also include operations such as performing index access on data structures (e.g., look-up tables (LUTs)) and data processing procedures implemented in hardware.

[0046] Specifically, the data processing device may include one or more computers, one or more microcontrollers, and / or one or more integrated circuits, such as one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), and / or one or more systems on a chip (SoCs). The data processing device may also include one or more processors, such as one or more microprocessors, one or more central processing units (CPUs), one or more graphics processing units (GPUs), and / or one or more signal processors (specifically, one or more digital signal processors (DSPs)). The data processing device may also include a physical or virtual cluster of computers or other said units.

[0047] In various embodiments, the data processing device includes one or more hardware and / or software interfaces and / or one or more memory units.

[0048] Memory cells can be implemented as volatile data memory, such as dynamic random access memory (DRAM) or static random access memory (SRAM), or as non-volatile data memory, such as read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or flash EEPROM, ferroelectric random access memory (FRAM), magnetoresistive random access memory (MRAM), or phase-change random access memory (PCRAM).

[0049] According to a fourth aspect of the present invention, a medical imaging system is provided. The medical imaging system includes a data processing device according to a third aspect of the present invention and a medical imaging device. The medical imaging device is configured to generate raw imaging data representing anatomical regions, and to generate medical imaging data depending on the raw imaging data.

[0050] Medical imaging equipment can be, for example, CT equipment, CBCT (cone-beam CT) equipment, MRI equipment, or X-ray equipment.

[0051] According to a fifth aspect of the invention, a computer program product including instructions is provided. When executed by a data processing device, the instructions cause the data processing device to perform a computer-implemented method according to the first and / or second aspects of the invention. The instructions may be provided, for example, as program code. The program code may be provided, for example, as binary code or assembler and / or as source code of a programming language (e.g., C) and / or as a program script (e.g., Python).

[0052] According to a sixth aspect of the invention, a computer-readable storage medium for storing a computer program according to the invention is provided, specifically a tangible and / or non-transient computer-readable storage medium. The computer program and the computer-readable storage medium are corresponding computer program products including instructions. These instructions cause a data processing apparatus to perform a computer-implemented method according to the first and / or second aspects of the invention.

[0053] Further features and combinations of features of the invention can be obtained from the accompanying drawings, their description, and the claims. Specifically, further implementations of the invention may not necessarily include all the features of any one of the claims. Further implementations of the invention may include features or combinations of features not recited in the claims.

[0054] The invention will be explained in detail below with reference to specific exemplary implementations and corresponding schematic diagrams. In the drawings, identical or functionally identical elements may be denoted by the same reference numerals. Descriptions of identical or functionally identical elements are not necessarily repeated for different drawings. Attached Figure Description

[0055] In the attached diagram,

[0056] Figure 1 A schematic diagram of a medical imaging system is shown;

[0057] Figure 2 A schematic diagram of a cognitive Bayesian uncertainty model is shown.

[0058] Figure 3 A schematic diagram illustrating the computer-implemented method is shown;

[0059] Figure 4 A schematic diagram of a computer-implemented training method is shown;

[0060] Figure 5 A schematic diagram of the post-processing is shown;

[0061] Figure 6 A schematic diagram of the workflow for post-processing of cognitive uncertainty graphs is shown;

[0062] Figure 7 A schematic diagram of the outline drawing of a cognitive uncertainty graph using a polygonization procedure is shown;

[0063] Figure 8 A schematic diagram of a drawing including the outline of cognitive uncertainty and the outline of an anatomical object is shown;

[0064] Figure 9 An embodiment of an artificial neural network is shown. Alternative terms for "artificial neural network" are "neural network," "artificial neural network," or "neural network"; and

[0065] Figure 10 An example of a convolutional neural network is shown. Detailed Implementation

[0066] Figure 1 A schematic diagram of a medical imaging system is shown. The medical imaging system 10 may include a data processing device 12, which may be configured to execute a computer-implemented method to detect a predefined anatomical object 14 of an anatomical region 16. The medical imaging system 10 may also include a medical imaging device 18, which may be, for example, a CT device or an MR device. Optionally, the medical imaging system 10 includes a patient table 20, etc. The medical imaging device 18 is configured to generate medical imaging data 22 representing the anatomical region 16 of the body on the patient table 20. The medical imaging data 22 may include a three-dimensional representation of the anatomical region 16.

[0067] Medical imaging data 22 may include voxels. Anatomical region 16 may include a set of objects, including at least one anatomical object 14. Medical imaging system 10 may include data processing device 12. Data processing device 12 may include a request interface 24 configured to receive request information 26 provided by a user describing the set of objects. In other words, request information 26 may describe the anatomical object 14 of the set of objects. Anatomical object 14 may include organs, bones, or implants located within anatomical region 16. Request information 26 may include a request for visualization 28 of a predefined region of interest (ROI) of anatomical region 16 based on medical imaging data 22. Visualization 28 may visualize a 2D plane in a cross-section through anatomical region 16. Visualization 28 may include an anatomical object outline 30 of at least one anatomical object 14, the anatomical object outline 30 enclosing a volume representing the anatomical object 14 within the ROI. The volume of at least one anatomical object 14 may be determined based on voxels assigned to the respective anatomical object 14 and the ROI.

[0068] The data processing device 12 can be configured to generate visualization data 28, which includes visualization 28 of a predefined region, anatomical outline 30 of at least one anatomical object 14, and at least one cognitive uncertainty information 52, 34 and / or rating information 36.

[0069] The data processing device 12 can be configured to determine allocation information 54 describing the assignment of corresponding voxels to corresponding anatomical objects 14. In other words, the medical imaging data 22 may include voxels, some of which represent corresponding anatomical objects 14.

[0070] The corresponding voxel allocation information 54 can be determined by the data processing device 12 by applying a predefined contour drawing model to the medical imaging data 22 through the computing unit. The allocation information 54 describes the allocation of the corresponding voxels to the anatomical object 14.

[0071] The data processing device 12 can be configured to determine a corresponding anatomical object outline 30 in the region of interest that encloses the volume of the anatomical object 14 for at least one anatomical object 14.

[0072] However, there may be uncertainties related to the allocation of the corresponding voxels to the corresponding anatomical objects 14.

[0073] Uncertainty can be based on cognitive uncertainty. Cognitive uncertainty can be based on out-of-distribution situations. An out-of-distribution situation may occur when the anatomical region 16 represented in the medical imaging data 22 is not adequately trained during the training of the predefined contour drawing model.

[0074] In this situation, unreliable results may occur, giving incorrect contours of the predefined anatomical object 14. It may be necessary to identify out-of-distribution situations.

[0075] Therefore, the data processing device 12 can be configured to apply a cognitive Bayesian uncertainty model 38 to the medical imaging data 22. The cognitive Bayesian uncertainty model 38 can be configured to determine cognitive uncertainty information 52, 34 to describe the cognitive uncertainty of allocation information 54, which describes the allocation of a corresponding voxel to a corresponding anatomical object 14. In other words, the cognitive Bayesian uncertainty model 38 is configured to determine the cognitive uncertainty of allocation information 54 for a corresponding voxel, which describes the allocation of the corresponding voxel to the corresponding anatomical object 14.

[0076] Out-of-distribution conditions can be identified by applying a scoring procedure to the cognitive uncertainty information 52, 34 to determine scoring information 36 indicating the presence of an out-of-distribution condition. The scoring procedure may include determining the scoring information 36 related to voxels assigned to the corresponding anatomical object 14, voxels assigned to the contour of the corresponding anatomical object 14, and / or all voxels. The scoring information 36 may include, for example, the average value of the assignment information 54 and / or burst information of the assignment information 54. The medical imaging system 10 may be configured to provide a warning signal via an output interface when the scoring information 36 meets predefined out-of-distribution conditions. Out-of-distribution conditions may include a threshold value for the scoring information 36. The warning signal may be provided via the output interface of the data processing device 12. The warning signal may be directed to the display device of the medical imaging system 10 to display a visual warning on the display device.

[0077] The cognitive Bayesian uncertainty model 38 may include a deep ensemble or more models comprising base learners 40. Each base learner 40 is configured to determine weak assignment information 54, which describes the assignment of a corresponding voxel to a corresponding anatomical object 14 in a corresponding forward propagation. In other words, the deep ensemble model includes base learners 40 that can be trained on corresponding subsets of training data. Therefore, base learners 40 can be trained on different subsets. Medical imaging data 22 may be provided to each base learner 40 to determine corresponding weak assignment information 54 describing the assignment of a corresponding voxel to a corresponding anatomical object 14 in a corresponding forward propagation. In other words, each base learner 40 is configured to provide a weak assignment of a corresponding voxel to a corresponding anatomical object 14 in a corresponding forward propagation.

[0078] Each base learner in the base learner 40 includes a Monte Carlo dropout layer 42, wherein the cognitive Bayesian uncertainty model 38 is configured to perform a corresponding Monte Carlo dropout in the Monte Carlo dropout layer 42 during the corresponding forward propagation. In other words, medical imaging data 22 can be provided to the corresponding base learner 40 during the corresponding forward propagation. In each forward propagation, Monte Carlo dropout can be performed on the Monte Carlo dropout layer 42 of the base learner 40. Therefore, the Monte Carlo dropout layer 42 can be different in each forward propagation. Thus, each forward propagation can provide corresponding weak assignment information 54 describing the assignment of the corresponding voxel to the corresponding anatomical object 14.

[0079] It is possible that the deep ensemble model includes a base learner 40. It may be intended to perform four forward passes on each base learner 40 using Monte Carlo dropout. Therefore, by applying a deep ensemble model combined with Monte Carlo dropout, thirty-two distinct weak assignment information 54 describing the assignment of corresponding voxels to corresponding anatomical objects 14 can be provided.

[0080] To determine cognitive Bayesian uncertainty, data processing device 12 can determine corresponding voxel-based cognitive uncertainty information 52, 34, which is the weak assignment information 54, 50 provided by the base learner 40 for the corresponding voxels in the corresponding forward propagation. In other words, for each voxel, data processing device 12 provides corresponding weak assignment information 54, 50 for different forward propagations. Data processing device 12 can determine the variance of the weak assignment information 54, 50 in order to determine cognitive uncertainty information 52, 34 for the corresponding voxels. Data processing device 12 can be configured to provide such cognitive uncertainty information 52, 34 for at least some voxels, or such cognitive uncertainty information 52, 34 for voxels within the contour surrounding the volume representing the anatomical object 14, to a scoring procedure to provide scoring information 36.

[0081] The data processing device 12 can be configured to perform a post-processing procedure on the corresponding anatomical object 14 based on cognitive uncertainty information 52, 34 of at least some voxels, and / or based on predefined morphological conditions. It is possible that, based on the corresponding cognitive uncertainty information 52, 34, voxels at the boundaries of the contour can be removed from the allocation to the corresponding object. It is also possible that, when some voxels meet predefined morphological conditions, they can be removed from the contour.

[0082] Figure 2 A schematic diagram of the cognitive Bayesian uncertainty model 38 is shown.

[0083] The cognitive Bayesian uncertainty model 38 can be applied to medical imaging data 22. The medical imaging data 22 can be provided to the base learner 40 of the cognitive Bayesian uncertainty model 38.

[0084] The base learner 40 within the cognitive Bayesian uncertainty model 38 consists of layers 42, including convolutional (Conv) layers 42, batch normalization (BatchNorm) layers 42, and rectified linear unit (ReLU) activation functions, forming the basis for processing medical imaging data 22. To account for cognitive uncertainty in the model's predictions, Monte Carlo dropout layers 44, 42 are included after each ResBlock, enabling stochastic variability during training. This approach cultivates an understanding of the model's confidence in its predictions and helps estimate underlying uncertainties during segmentation. Additionally, the base learner 40 incorporates pooling layers 46, 42 for downsampling spatial dimensions and / or upsampling layers 42 for restoring resolution as the network hierarchy moves. The final layer 42 of each base learner 40 includes a softmax function 48 to output weak assignment information 54, 50 describing the assignment probability of each voxel belonging to a specific anatomical object 14. During each forward propagation, Monte Carlo dropout is applied to the corresponding Monte Carlo dropout layers 44, 42 to generate multiple hypotheses about voxel-to-object assignment. This approach introduces randomness into the model, thereby achieving better quantification of cognitive uncertainty and prompting more informed decision-making during segmentation. The input to each base learner 40 consists of medical imaging data 22, which undergoes convolution, batch normalization, activation, pooling or upsampling, and softmax operations to produce weak assignment information 54, 50, ultimately supporting a robust and adaptive method for automating the segmentation of anatomical objects 14 in the medical imaging data 22.

[0085] The generation of cognitive uncertainty information 52, 34 for each voxel within the medical imaging data 22 relies on weak assignment information 54, 50 generated by the base learner 40 in the cognitive Bayesian uncertainty model 38. During multiple forward propagations, Monte Carlo dropout layers 44, 42 introduce stochastic variability into the model, leading to different assumptions about voxel-to-object assignments in each iteration. By collecting and analyzing the outputs of this weak assignment information 54, 50 from several forward propagations with varying stochasticity, the cognitive uncertainty associated with a particular voxel can be quantified. The degree of variation in the predicted probabilities across multiple runs is used as an indicator of the model's confidence in assigning voxels to a specific anatomical object 14. Higher variability suggests lower confidence, thus implying higher cognitive uncertainty, while lower variability indicates greater certainty in the assignment. This process involves calculating the entropy, or variance, of the weak assignment probabilities across various forward propagations, thereby generating a measure of cognitive uncertainty for each voxel. This metric reflects the model's confidence and allows users to better understand potential errors, inconsistencies, or ambiguities during segmentation.

[0086] The assignment information 54, describing the allocation of each corresponding voxel to a specific anatomical object 14, is generated from weak assignment information 54, 50 taken from the base learner 40. During this process, the base learner 40 generates multiple weak assignment hypotheses for each voxel through Monte Carlo dropout layers 44, 42, thus producing probabilistic predictions about their affiliation with various anatomical objects 14. To transform this weak assignment information 54, 50 into assignment information 54, a decision rule is applied to incorporate information from several forward propagations. This can be achieved by selecting the anatomical object 14 with the highest probability for each voxel as its final assignment across all iterations. In cases where multiple anatomical objects 14 have similar probabilities, a threshold can be used to determine whether a voxel should be assigned or labeled as uncertain, thus reflecting the ambiguity in the segmentation.

[0087] Figure 3 A schematic diagram of the computer-implemented method is shown.

[0088] Step S1: The method begins with the input interface receiving medical imaging data 22, including voxels, where each voxel represents an anatomical region 16 comprised of at least one anatomical object 14 within a set of objects. These voxels are used to construct detailed images of the internal body structures for further analysis and segmentation. The medical imaging data 22 can be obtained from various modalities such as computed tomography (CT), magnetic resonance imaging (MRI), or positron emission tomography (PET) scans.

[0089] Step S2: The computing unit applies a predefined contour rendering model to the medical imaging data 22 to establish allocation information 54 describing which voxels belong to the corresponding anatomical objects 14. Subsequently, based on these allocated voxels and defined regions of interest, the corresponding anatomical object contours 30 are generated. The contour rendering model can be a machine learning algorithm or a deep neural network designed to accurately segment the anatomical objects 14 in the medical imaging data 22.

[0090] Step S3: In this step, the computational unit applies a cognitive Bayesian uncertainty model 38 to the medical imaging data 22 to assess the cognitive uncertainty information 52, 34 associated with the assignment information 54. Cognitive uncertainty stems from a lack of knowledge or understanding regarding correct segmentation and can be quantified using statistical methods. This uncertainty information 52 helps determine the confidence level of the contour drawing model in assigning specific voxels to their corresponding anatomical objects 14.

[0091] Step S4: Following the cognitive uncertainty assessment, for some or all of the assigned voxels and predefined morphological conditions, the corresponding anatomical object contour 30 is post-processed based on the cognitive uncertainty information 52, 34. Post-processing may include various techniques such as smoothing, hole filling, and size filtering to ensure the accuracy and consistency of the contour representation.

[0092] Step S5: This step involves performing a scoring procedure that includes determining a corresponding contour uncertainty score for each anatomical object contour 30 based on cognitive uncertainty information 52, 34 of voxels representing the corresponding anatomical object 14 within the volume. Additionally, a total uncertainty score can be calculated based on the cognitive uncertainty information 52, 34 of all anatomical objects 14 in the medical imaging data 22. This scoring procedure provides an overall assessment of segmentation quality and highlights areas where further improvement or human intervention may be necessary.

[0093] Step S6: If the scoring information 36 meets the predefined out-of-distribution conditions, a warning signal is provided to the user interface through the output interface. This alert informs the user of potential problems with the segmentation results and reminds them to exercise caution when interpreting or using these results.

[0094] Step S7: Finally, the user interface visually displays uncertainty information 52 by highlighting areas of high uncertainty and combining them with simple decision points (such as traffic lights or individual scores indicating the confidence level of the predicted segmentation). This visual representation helps users quickly assess segmentation quality, thereby enabling informed decision-making and effective use of medical imaging data 22 for diagnostic or treatment planning purposes.

[0095] Figure 4 A schematic diagram of a computer-implemented training method is shown.

[0096] Step T1: The computer-implemented training method begins by receiving training medical imaging data 22 at the first input interface. The training data consists of voxels representing anatomical regions 16 containing the training object set. This training data is used to guide and optimize the cognitive Bayesian uncertainty model 38 for accurate segmentation. The training medical imaging data 22 can be obtained from various modalities such as computed tomography (CT), magnetic resonance imaging (MRI), or positron emission tomography (PET) scans.

[0097] Step T2: The second input interface on the data processing device 12 receives training allocation information 54 for the training data. This information describes which voxels in the training medical imaging data 22 belong to each training anatomical object 14, thus providing the ground truth required for model training.

[0098] Step T3: The training data is partitioned into subsets based on the given specifications. Each subset is then assigned to a corresponding base learner 40 of the deep ensemble model, which forms part of the cognitive Bayesian uncertainty model 38. This separation process increases the robustness and accuracy of the model by allowing each base learner 40 to specialize in different aspects or features of the training data.

[0099] Step T4: The computing unit of data processing device 12 trains the base learner 40 using a corresponding subset of the training data of the base learner 40. Training involves optimizing the performance of each base learner 40 to minimize the difference between its predictions and the baseline ground truth training assignment information 54. This process is typically implemented using iterative techniques such as backpropagation and stochastic gradient descent.

[0100] Step T5: Once the base learner 40 has been trained, the output interface of the data processing device 12 provides a cognitive Bayesian uncertainty model 38. This complete model can then be used to evaluate segmentation uncertainty in new medical imaging data 22, thereby providing valuable insights into the reliability and confidence of its predictions.

[0101] Figure 5 A schematic diagram of post-processing is shown.

[0102] Figure 5A visualization 28 of the cognitive uncertainty map based on cognitive uncertainty information 52, 34 of anatomical region 16 is shown. The upper visualization 28 of anatomical region 16 shows the cognitive uncertainty map based on the raw values ​​of cognitive uncertainty information 52, 34 before post-processing. The area around the hip implant shows higher cognitive uncertainty. This cognitive uncertainty may be due to the lack of or insufficient number of images with hip implants in the training data. Other areas show high cognitive uncertainty at the edges of anatomical object 14. These edges may be due to boundary effects rather than cognitive uncertainty. Therefore, it may be desirable to remove these edges.

[0103] The lower part of anatomical region 16, visualized in visualization 28, shows a post-processed cognitive uncertainty map. In this drawing, the edges have been removed. However, the area around the hip implant remains, which is the desired effect.

[0104] Generally, cognitive uncertainty maps may contain artifacts and boundary regions with high cognitive uncertainty, which can impair the accuracy and reliability of contour generation. To address this issue, morphological operations are performed during post-processing of the cognitive uncertainty map before contour extraction. These operations can remove voxels assigned to anatomical objects 14 or fill gaps in regions assigned to these objects.

[0105] Morphological dilation expands the boundaries of high-intensity regions in the cognitive uncertainty map, thereby incorporating adjacent pixels and filling gaps. This operation is particularly useful for sparse or noisy datasets, where small gaps may occur between voxels belonging to the same object.

[0106] Morphological erosion shrinks high-intensity areas by removing external voxels, eliminating small artifacts, and preserving key structures. This operation ensures a more accurate representation of the anatomical object 14 by removing foreign noise or minor inconsistencies from the data.

[0107] Combining operations such as morphological opening (erosion followed by dilation) and morphological closing (dilation followed by erosion) offer additional benefits in refining cognitive uncertainty maps. Opening removes small objects or bridges, thus smoothing the contours without altering the main structure, while closing fills gaps and connects discontinuous areas, thereby improving uniformity.

[0108] Morphological thinning reduces the thickness of elongated objects in cognitive uncertainty maps while preserving their topological structure. By identifying and removing foreign voxels from anatomical object 14, thinning improves contour accuracy without altering the overall shape or structure, making it particularly useful for complex structures such as blood vessels or neural networks.

[0109] Post-processing can be performed prior to the scoring procedure to determine the contour uncertainty score of the corresponding contour in the object based on cognitive uncertainty information 52, 34 from voxels representing at least one anatomical object 14 within the volume. This post-processing prior to the scoring procedure ensures that all regions containing voxels undergo the same pre-processing steps, resulting in consistent and standardized contour uncertainty scores. Removing voxels at boundaries ensures that the influence of boundaries on voxel mapping uncertainty can be filtered out.

[0110] Figure 6 A schematic diagram of the workflow for post-processing cognitive uncertainty graphs is shown.

[0111] The workflow for post-processing the cognitive uncertainty map is initiated by receiving input data in step P1, which includes cognitive uncertainty information 52, 34 representing each voxel in the volume of the anatomical object 14.

[0112] In the first step P2, thresholding is applied to manage cognitive uncertainty. Figure 2 Value-based approach. A predefined threshold is set, and all cognitive uncertainty information 52 and 34 below this level are set to 0, while all cognitive uncertainty information 52 and 34 equal to or higher than this threshold are set to 1. This process distinguishes between high-uncertainty and low-uncertainty regions within the volume.

[0113] The second step, P3, involves performing morphological dilation on the thresholded cognitive uncertainty map. By adding voxels to the boundaries of connected components in the binary image, the size of high-uncertainty regions is increased, thus ensuring that neighboring regions with high uncertainty values ​​are included in the final analysis.

[0114] Following dilation, morphological opening is performed in the third step, P4. Opening combines erosion (shrinking object boundaries) and dilation (expanding object boundaries), effectively eliminating small unwanted regions (such as noise or outliers) from the binary image while preserving the overall shape of high-uncertainty regions within the volume.

[0115] The fourth step, P5, involves applying a masking operation, where the cognitive uncertainty map generated in the morphological open is used as a mask for the original input data. This process ensures that only voxels with high uncertainty values ​​are included in the final result, while low uncertainty regions are filtered out.

[0116] In the final step, P6, a post-processed cognitive uncertainty map is displayed, showing detailed information about high-uncertainty regions within the volume. The processed data can then be used to determine scores for the corresponding anatomical subject 14.

[0117] Figure 7 A schematic diagram of the outline drawing of a cognitive uncertainty graph using a polygonization procedure is shown.

[0118] It is possible Figure 6 The post-processing shown generates a cognitive uncertainty map. A contour drawing procedure can be performed on the post-processed cognitive uncertainty map to generate cognitive uncertainty contours 56 around the areas of cognitive uncertainty. Contours can be provided only for the areas associated with the corresponding anatomical object 14.

[0119] Figure 8 A schematic diagram of a drawing including the cognitive uncertainty contour 56 and the anatomical object contour 30 is shown.

[0120] Users can be provided with the option to perform a polygonization procedure on the regions of voxels assigned to their respective anatomical objects 14 during post-processing. This process converts the regions of voxels assigned to anatomical objects 14 into 2D mesh representations (polygons), thereby allowing for easier visualization and analysis of complex structures 28.

[0121] Polygonization procedures include the Marching Cube algorithm, which is widely used in scientific visualization and computer graphics.28 The Marching Cube algorithm generates a polygonal mesh from a set of voxels representing the boundary surface of an object by examining each cell (cube) formed by eight adjacent voxels and creating triangles based on the voxel values ​​within that cell.

[0122] Another example is the Surface Net method, which uses spheres centered at each voxel location to approximate the surface of an object to generate a 2D mesh. Compared to traveling cubes, this technique offers better control over the smoothing and preservation of the shape of complex structures.

[0123] Ball Pivoting is another polygonization procedure that creates a mesh from a set of voxels using spheres of varying radii. By adjusting the sphere radii, this method can handle different levels of detail in the input data and provide a more accurate representation of anatomical structures.

[0124] Performing a polygonization procedure on the region after post-processing ensures that any noise, artifacts, or gaps within the assigned voxel regions are resolved before converting it to a 2D mesh. This approach enhances the quality of visualization28 and downstream analysis by producing a clearer and more accurate representation of the anatomical structure in polygonal form.

[0125] Using established methods such as the traveling cube, surface mesh, or rolling ball algorithms, these procedures enable easier visualization and analysis of complex structures, while ensuring high-quality representations by addressing noise, artifacts, and gaps during post-processing. Users can also execute these algorithms on corresponding cognitive uncertainty graphs. Users can request... Figure 8 The combination of contours shown in visualization 28

[0126] Figure 9 An example of an artificial neural network is shown. Alternative terms for "artificial neural network" are "neural network," "artificial neural network," or "neural network."

[0127] The trained function can include an artificial neural network 100.

[0128] The artificial neural network 100 includes nodes 120, ..., 132 and edges 140, ..., 142, where each edge 140, ..., 142 is a directed connection from a first node 120, ..., 132 to a second node 120, ..., 132. Generally, the first nodes 120, ..., 132 and the second nodes 120, ..., 132 are different nodes 120, ..., 132; however, it is also possible that the first nodes 120, ..., 132 and the second nodes 120, ..., 132 are the same. For example, in... Figure 1 In the diagram, edge 140 is a directed connection from node 120 to node 123, while edge 142 is a directed connection from node 130 to node 132. Edges 140, ..., 142 from the first node 120, ..., 132 to the second node 120, ..., 132 are also represented as the "input edges" of the second node 120, ..., 132 and the "output edges" of the first node 120, ..., 132.

[0129] In this embodiment, nodes 120, ..., 132 of the artificial neural network 100 can be arranged in layers 110, ..., 113, wherein these layers may include an inherent order introduced by edges 140, ..., 142 between nodes 120, ..., 132. Specifically, edges 140, ..., 142 can only exist between adjacent layers of nodes. In the illustrated embodiment, input layer 110 includes only nodes 120, ..., 122 without input edges, output layer 113 includes only nodes 131, 132 without output edges, and hidden layers 111, 112 are located between input layer 110 and output layer 113. Generally, the number of hidden layers 111, 112 can be arbitrarily chosen. The number of nodes 120, ..., 122 in input layer 110 is typically related to the number of input values ​​of the neural network, while the number of nodes 131, 132 in output layer 113 is typically related to the number of output values ​​of the neural network.

[0130] Specifically, (real) numbers can be assigned as values ​​to each node 120, ..., 132 of the neural network 100. Here, x (n) i This represents the value of the i-th node 120, ..., 132 in the nth layer 110, ..., 113. The values ​​of nodes 120, ..., 122 in the input layer 110 are equivalent to the input values ​​of the neural network 100, and the values ​​of nodes 131, 132 in the output layer 113 are equivalent to the output values ​​of the neural network 100. Furthermore, each edge 140, ..., 142 may include a weight as a real number; specifically, the weight is a real number within the interval [-1, 1] or the interval [0, 1]. Here, w... (m,n) i,j This represents the weight of the edge between the i-th node (120, ..., 132) of layer m (110, ..., 113) and the j-th node (120, ..., 132) of layer n (110, ..., 113). Further, the abbreviation is w. (n) i,j This is the weight w (n,n+1) i,j And defined.

[0131] Specifically, to calculate the output value of neural network 100, the input values ​​are propagated through the neural network. Specifically, the values ​​of nodes 120, ..., 132 in layer (n+1) 110, ..., 113 can be calculated based on the values ​​of nodes 120, ..., 132 in layer (n) using the following formula.

[0132] Here, the function f is the transfer function (another term is the "activation function"). Known transfer functions are step functions, sigmoid functions (e.g., logic functions, generalized logic functions, hyperbolic tangent functions, arctangent functions, error functions, smooth step functions), or rectifier functions. Transfer functions are primarily used for normalization purposes.

[0133] Specifically, these values ​​are propagated layer by layer through a neural network, wherein the value of the input layer 110 is given by the input of the neural network 100, the value of the first hidden layer 111 can be calculated based on the value of the input layer 110 of the neural network, the value of the second hidden layer 112 can be calculated based on the value of the first hidden layer 111, and so on.

[0134] To set the value w of the edge (m,n) i,j Training data is required to train the neural network 100. Specifically, the training data includes training input data and training output data (denoted as t). i For the training step, neural network 100 is applied to the training input data to generate computational output data. Specifically, the training data and the computational output data include a number of values ​​equal to the number of nodes in the output layer.

[0135] Specifically, the comparison between the calculated output data and the training data is used to recursively adapt the weights within the neural network 100 (backpropagation algorithm). Specifically, the weights change according to the following formula: Where γ is the learning rate, and if the (n+1)th layer is not an output layer, it can be based on δ. (n+1) j The quantity δ (n) j Recursively calculate as If the (n+1)th layer is the output layer 113, then the number δ can be reduced. (n) j Calculated as Where f' is the first derivative of the activation function, y (n+1) j It is the comparison training value of the j-th node of the output layer 113.

[0136] Figure 10 An example of a convolutional neural network is shown.

[0137] The trained function can include a convolutional neural network 200.

[0138] In the shown embodiment, the convolutional neural network 200 includes an input layer 210, convolutional layers 211, pooling layers 212, fully connected layers 213, and an output layer 214. Alternatively, the convolutional neural network 200 may include several convolutional layers 211, several pooling layers 212, and several fully connected layers 213, as well as other types of layers. The order of the layers can be chosen arbitrarily; typically, the fully connected layer 213 is used as the last layer before the output layer 214.

[0139] Specifically, within the convolutional neural network 200, the nodes 220, ..., 224 of layer 210, ..., 214 can be considered as arranged as a d-dimensional matrix or a d-dimensional image. Specifically, in the two-dimensional case, the value of node 220, ..., 224 in the nth layer 210, ..., 214, indexed by i and j, can be represented as x. (n) [i,j]. However, the arrangement of nodes 220, ..., 224 in layer 210, ..., 214 has no effect on the computation performed within such a convolutional neural network 200, because these are given solely by the structure and weights of the edges.

[0140] Specifically, the convolutional layer 211 is characterized by the structure and weights of the input edges that form the convolution operation based on a specific number of kernels. Specifically, the structure and weights of the input edges are chosen such that the value x of node 221 of the convolutional layer 211... (n) k The value x is calculated based on node 220 of the previous layer 210. (n-1) convolution Convolution In the two-dimensional case, it is defined as

[0141] Here, the k-th kernel K k This is a d-dimensional matrix (a two-dimensional matrix in this embodiment), which is typically small compared to the number of nodes 220, ..., 224 (e.g., a 3×3 or 5×5 matrix). Specifically, this implies that the weights of the input edges are not independent, but are chosen such that they produce the convolution equation. Specifically, for a kernel that is a 3×3 matrix, there are only 9 independent weights (each entry in the kernel matrix corresponds to one independent weight), regardless of the number of nodes 220, ..., 224 in the corresponding layers 210, ..., 214. Specifically, for convolutional layer 211, the number of nodes 221 in the convolutional layer is equivalent to the number of nodes 220 in the previous layer 210 multiplied by the number of kernels.

[0142] If the nodes 220 of the previous layer 210 are arranged as a d-dimensional matrix, then using multiple kernels can be interpreted as adding a further dimension (denoted as the "depth" dimension) so that the nodes 221 of the convolutional layer 221 are arranged as a d+1-dimensional matrix. If the nodes 220 of the previous layer 210 have already been arranged as a d+1-dimensional matrix including the depth dimension, then using multiple kernels can be interpreted as extending along the depth dimension so that the nodes 221 of the convolutional layer 221 are also arranged as a d+1-dimensional matrix, where the size of the d+1-dimensional matrix relative to the depth dimension is a larger factor than the number of kernels in the previous layer 210.

[0143] The advantage of using convolutional layers 211 is that by implementing local connectivity patterns between nodes in neighboring layers, specifically by connecting each node to only a small region of nodes in the previous layer, the spatial local correlation of the input data can be utilized.

[0144] In the shown embodiment, the input layer 210 includes 36 nodes 220 arranged as a two-dimensional 6×6 matrix. The convolutional layer 211 includes 72 nodes 221 arranged as two two-dimensional 6×6 matrices, each of which is the result of the convolution of the input layer values ​​with the kernel. Equivalently, the nodes 221 of the convolutional layer 211 can be interpreted as being arranged as a three-dimensional 6×6×2 matrix, where the last dimension is the depth dimension.

[0145] Pooling layer 212 can be characterized by the structure and weights of the input edges that form the pooling operation based on the nonlinear pooling function f, as well as the activation function of its nodes 222. For example, in the two-dimensional case, the value x of node 222 of pooling layer 212 is... (n) It can be based on the value x of node 221 of the previous layer 211. (n-1) And calculated as

[0146] In other words, by using pooling layer 212, the number of nodes 221 and 222 can be reduced, that is, by replacing the number d1·d2 of adjacent nodes 221 in the previous layer 211 with a single node 222, which is calculated as a function of the value of the number of adjacent nodes in the pooling layer. Specifically, the pooling function f can be a maximum function, an average function, or an L2 norm function. Specifically, for pooling layer 212, the weights of the input edges are fixed and are not modified through training.

[0147] The advantage of using pooling layer 212 is that it reduces the number of nodes 221 and 222 and the number of parameters. This results in a reduction in the computational cost of the network and controls overfitting.

[0148] In the illustrated embodiment, pooling layer 212 is a max pooling that replaces the max pooling of four adjacent nodes with only one node, where the value is the maximum of the four adjacent node values. Max pooling is applied to each d-dimensional matrix of the previous layer; in this embodiment, max pooling is applied to each of two two-dimensional matrices, thereby reducing the number of nodes from 72 to 18.

[0149] The fully connected layer 213 can be characterized by the fact that there are most (specifically all) edges between nodes 222 of the previous layer 212 and nodes 223 of the fully connected layer 213, and the weight of each edge can be adjusted individually.

[0150] In this embodiment, the nodes 222 of the preceding layer 212 of the fully connected layer 213 are displayed as a two-dimensional matrix and are also displayed as unrelated nodes (indicated as a row of nodes, where the number of nodes is reduced for better presentation). In this embodiment, the number of nodes 223 in the fully connected layer 213 is equal to the number of nodes 222 in the preceding layer 212. Alternatively, the number of nodes 222 and 223 may be different.

[0151] Furthermore, in this embodiment, the value of node 224 in the output layer 214 is determined by applying a Softmax function to the value of node 223 in the preceding layer 213. By applying the Softmax function, the sum of the values ​​of all nodes 224 in the output layer is 1, and all values ​​of all nodes 224 in the output layer are real numbers between 0 and 1. Specifically, if the input data is classified using the convolutional neural network 200, the value of the output layer can be interpreted as the probability that the input data falls into one of the different categories.

[0152] Convolutional neural networks (CNNs) can also include ReLU (Rectified Linear Unit) layers. Specifically, the number and structure of nodes in a ReLU layer are the same as those in the previous layer. Specifically, the value of each node in a ReLU layer is calculated by applying a rectified function to the value of the corresponding node in the previous layer. Examples of rectified functions are f(x) = max(0,x), the hyperbolic tangent function, or the sigmoid function.

[0153] Specifically, the convolutional neural network 200 can be trained based on the backpropagation algorithm. To prevent overfitting, regularization methods can be used, such as dropping nodes 220, ..., 224, random pooling, using artificial data, weight decay based on L1 or L2 norm, or maximum norm constraint.

Claims

1. A computer-implemented method for detecting out-of-distribution situations, comprising: - Medical imaging data (22) including voxels is received by the input interface of the data processing device (12), wherein the medical imaging data (22) represents an anatomical region (16) including a set of objects, the set of objects including at least one anatomical object (14). - The computing unit applies a cognitive Bayesian uncertainty model (38) to the medical imaging data (22) to determine cognitive uncertainty information (52) describing the allocation information (54) which describes the allocation of the corresponding voxels to the corresponding anatomical objects (14). - Apply a scoring procedure to the cognitive uncertainty information (52) to determine the scoring information (36); - If the scoring information (36) meets the predefined out-of-distribution conditions, a warning signal is provided by the output interface; in - The cognitive Bayesian uncertainty model (38) includes a deep ensemble model, which includes base learners (40), wherein each base learner (40) is configured to determine weak assignment information (50), which describes the assignment of the corresponding voxel to the corresponding anatomical object (14) in the corresponding forward propagation. - Each of the base learners (40) is trained on a corresponding subset of training data. - Each of the base learners (40) includes a Monte Carlo dropout layer (44), wherein the cognitive Bayesian uncertainty model (38) is configured to perform a corresponding Monte Carlo dropout in the Monte Carlo dropout layer (44) during the corresponding forward propagation; and - The cognitive Bayesian uncertainty model (38) is configured to determine the cognitive uncertainty information (52) of the corresponding voxel based on the variance of the weak assignment information (50) provided by the base learner (40) for the corresponding voxel in the corresponding forward propagation.

2. The computer-implemented method according to claim 1, comprising: Based on the mean of the weak allocation information (50) provided by the base learner (40) for the corresponding voxel in the corresponding forward propagation, the allocation information (54) describing the allocation of the corresponding voxel is determined.

3. The computer-implemented method according to claim 1, wherein the determination of the allocation information (54) includes: - The computing unit applies a predefined contour drawing model to the medical imaging data (22) to determine the allocation information (54) describing the allocation of the corresponding voxels to the anatomical object (14).

4. A computer-implemented method according to any one of the preceding claims, comprising: - Based on the medical imaging data (22), a visualization (28) of a predefined region of interest of the anatomical region (16) is generated. - Based on the voxels assigned to the corresponding anatomical object (14) and the region of interest, a corresponding anatomical object contour (30) is generated for the at least one anatomical object (14), the corresponding anatomical object contour (30) enclosing the volume representing the anatomical object (14) in the region of interest; - Generate review data including the following: the visualization (28) of the predefined region, the anatomical outline (30) of the at least one anatomical object (14), and at least one piece of cognitive uncertainty information (34, 52) and / or the rating information (36).

5. A computer-implemented method according to any one of the preceding claims, comprising: - The outline (30) of the corresponding anatomical object is post-processed based on the cognitive uncertainty information (52) of at least some of the voxels assigned to the corresponding anatomical object (14) and / or based on predefined morphological conditions.

6. The computer-implemented method according to any one of the preceding claims, wherein: - The scoring procedure includes: determining the corresponding contour uncertainty scoring information of the corresponding anatomical object contour (30) of the corresponding at least one anatomical object (14) based on the cognitive uncertainty information (52) of the voxels representing the corresponding anatomical object (14) within the volume, and / or determining the total uncertainty scoring information based on the cognitive uncertainty information (34, 52) of the voxels representing the corresponding anatomical object (14) within the volume.

7. The computer-implemented method according to any one of the preceding claims, wherein... - The discard layer (44) is arranged after each residual block layer of the corresponding base learner (40).

8. A computer-implemented method according to any one of the preceding claims, comprising: - The request information (26) describing the set of objects is received by the request interface (24) of the data processing device (12).

9. A computer-implemented training method for training a cognitive Bayesian uncertainty model (38), comprising: - Training medical imaging data (22) is received by the first input interface of the data processing device (12), the training medical imaging data (22) comprising voxels, wherein the training medical imaging data (22) represents an anatomical region (16) comprising a training object set, the training object set comprising at least one training anatomical object (14). - Training allocation information (54) of the training data is received by the second input interface of the data processing device (12), the training allocation information (54) describing the allocation of the corresponding voxels of the training medical imaging data (22) to the corresponding training anatomical objects (14); - The training data is divided into subsets according to a given specification, wherein each subset of the training data is assigned to a corresponding base learner (40) of the deep ensemble model of the cognitive Bayesian uncertainty model (38). - The computing unit of the data processing device (12) trains the base learner (40) based on the corresponding subset of the training data. - The cognitive Bayesian uncertainty model (38) is provided by the output interface of the data processing device (12).

10. The computer-implemented training method according to claim 9, comprising: - The training data was divided into eight subsets as described; - The training data is divided into segments such that Each of the subset base learners (40) includes at most N / 2+1 cases from the N cases of the training data, wherein each case is included in exactly four of the eight subsets; The overlap of samples between any pair of said subsets is limited to a maximum of N / 4+1 cases.

11. A data processing apparatus (12) configured to perform the method according to any one of claims 1 to 8 or the method according to any one of claims 9 to 10.

12. A medical imaging system (10) comprising an imaging device (18) according to claim 11 and a data processing device (12).

13. A computer program comprising instructions, which, when executed by a computer, causes the computer to perform the method according to any one of claims 1 to 8 or the method according to any one of claims 9 to 10.

14. A computer-readable medium comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 8 or the method according to any one of claims 9 to 10.

Images