Correcting the segmentation of medical images using history-corrected statistical analysis
By statistically analyzing and transforming the manually generated correction data, combined with machine learning algorithms, the problem of insufficient reliability measurement in medical image segmentation is solved, and the reliability of the segmentation algorithm and the correction effect in low-reliability areas are improved.
Patent Information
- Application Number
- CN201980078812.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-12-20
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2039-12-20
AI Technical Summary
The lack of reliability measurement and consideration of low reliability areas in the corrective medical image segmentation process makes it difficult to quantify the uncertainty of the segmentation algorithm.
By reading manually generated correction data, transforming it into a non-patient-specific reference system, performing statistical analysis, and applying the results to patient images to train machine learning algorithms for automatic segmentation, and using statistical analysis results to improve the reliability of segmentation .
The reliability improvement of the medical image segmentation process is achieved. Through the combination of statistical analysis and machine learning algorithms, the reliability evaluation and improvement of the segmentation algorithm is provided, especially strong correction in low-reliability areas.
Smart Images

Figure CN113316803B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a computer-implemented method of determining a correction distribution for correcting a segmentation of medical image data, a method for correcting a segmentation using the determined correction distribution, a corresponding computer program, a computer-readable storage medium storing such a program, a computer executing such a program and a system comprising an electronic data storage device and the aforementioned computer. Background Art
[0002] It is known to manually generate a correction of the image segmentation and transform the correction in position into a non-patient-specific coordinate space, for example for reasons of data security. However, during the correction process, the correction itself does not provide a reliable measure for quantifying the reliability of the applied segmentation algorithm and does not provide any consideration for low reliability areas of the segmentation algorithm. Summary of the invention
[0003] The object of the present invention is to provide an improved method for correcting image segmentation.
[0004] The following discloses various aspects, examples and exemplary steps of the invention and its embodiments.Different exemplary features of the invention can be combined according to the invention whenever technically suitable and feasible.
[0005] In the following, a brief description of specific features of the present invention is given, which should not be understood as limiting the present invention to only the features or feature combinations described in this section.
[0006] The method of the present disclosure comprises reading a plurality of corrections for image segmentation, where the corrections themselves may have been manually generated, transforming these corrections to a non-patient-specific reference frame (such as an atlas reference frame), performing a statistical analysis of the corrections, and applying the re-transformed results of the statistical analysis to the patient images. The results of the statistical analysis may also be used to appropriately train a machine learning algorithm for automatically segmenting the patient images. Application of such a trained machine learning algorithm is also part of the present disclosure.
[0007] In this summary, a description of the general features of the invention is given, for example, by reference to possible embodiments of the invention.
[0008] In general, the present invention achieves the above-mentioned object by providing, in a first aspect, a computer-implemented method (e.g., a medical method) for determining a correction distribution for correcting a segmentation of medical image data. The method according to the first aspect comprises executing the following exemplary steps on at least one processor of at least one computer (e.g., at least one computer as part of a navigation system) performed by at least one processor.
[0009] In a (e.g. first) exemplary step of the method according to the first aspect, initial segmented patient image data are acquired, which describe an initial segmentation of digital medical images of body anatomical parts of a first number of patients. The digital medical images are, for example, two-dimensional images (such as radiographs) or three-dimensional images (i.e. tomography, such as computer tomography or magnetic resonance tomography or ultrasound scans).
[0010] In a (e.g. second) exemplary step of the method according to the first aspect, segmentation correction data is obtained, which describes a correction of the initial segmentation for each patient in the first number of patients. For example, the segmentation correction data is obtained based on user input, such as the segmentation correction data has been generated by manually creating a correction.
[0011] In a (eg third) exemplary step of the method according to the first aspect, atlas data are acquired, the atlas data describing an image-based model of a body anatomical region.
[0012] In an (e.g., fourth) exemplary step of the method according to the first aspect, correction transformation data is determined based on the segmented correction data and the atlas data, wherein the correction transformation data describes a transformation (e.g., a mapping) for transforming the correction into a reference system for describing positions in an image-based model.
[0013] In a (eg fifth) exemplary step of the method according to the first aspect, transformed correction data are determined based on the segmented correction data and the correction transformation data, by applying a transformation to the correction to determine the transformed correction.
[0014] In an (e.g. sixth) exemplary step of the method according to the first aspect, combined correction data are determined based on the transformed correction data, wherein the combined correction data describes the results of a statistical analysis of the correction. For example, the results of the statistical analysis constitute a corrected statistical distribution for the first number of patients. For example, the statistical analysis is an average analysis.
[0015] In a second aspect, the present invention is directed to a computer-implemented method (e.g., a medical method) for determining a correction for correcting a segmentation of medical image data. The method according to the second aspect comprises executing the following exemplary steps according to the method of the first aspect and performed by at least one processor of at least one computer (e.g., at least one computer as part of a navigation system).
[0016] In a (e.g. first) exemplary step of the method according to the second aspect, segmented individual patient image data are acquired, the segmented individual patient image data describing a segmentation of an anatomical part of a patient's body in a digital individual medical image. The individual patient image has been generated, for example, using the same imaging modality as the digital medical image from which the initial segmentation has been generated.
[0017] In an (e.g. second) exemplary step of the method according to the second aspect, correction retransformation data are determined based on the atlas data and the segmented individual patient image data. The correction retransformation data describes a transformation (e.g. a mapping) for transforming the results of the corrected statistical analysis into a reference system for describing locations in the digital individual medical image.
[0018] In a (eg third) exemplary step of the method according to the second aspect, retransformed correction data are determined based on the combined correction data and the correction retransformed data.
[0019] In a (e.g., fourth) exemplary step of the method according to the second aspect, corrected segmented patient image data are determined based on the segmented individual patient image data and the re-transformed correction data, wherein the corrected segmented patient image data describes the result of applying the correction to the segmentation of the patient's body anatomical part in the digital individual medical image. For example, the result of applying the correction to the segmentation of the patient's body anatomical part in the digital individual medical image is determined by applying the correction according to a feature quantity of the result of the statistical analysis. For example, the feature quantity is a mean value (e.g., an arithmetic mean) or a variance.
[0020] In a third aspect, the present invention is directed to a computer-implemented method (e.g., a medical method) for training a learning algorithm for determining segmentation of a digital medical image. The method according to the third aspect comprises executing the following exemplary steps according to the method of the first aspect or the second aspect and performed by at least one processor on at least one computer (e.g., at least one computer as part of a navigation system).
[0021] In a (eg first) exemplary step of the method according to the third aspect, training patient image data are acquired, the patient image data describing digital medical images of anatomical structures of a second number of patients.
[0022] In a (eg second) exemplary step of the method according to the third aspect, label data are acquired, the label data describing positions of labels assigned to training patient image data, wherein the positions of the labels are defined in a reference system defining positions in the training patient image data.
[0023] In an (e.g., third) exemplary step of a method according to the third aspect, transformed statistical distribution data is determined based on the training patient image data and the combined correction data, wherein the transformed statistical distribution data describes the result of transforming the statistical distribution into a reference system defining positions in the training patient image data.
[0024] In a (eg fourth) exemplary step of the method according to the third aspect, statistical assignment data is determined based on the transformed statistical distribution data and the label data, wherein the statistical assignment data describes the assignment of the results of the statistical analysis to the labels.
[0025] In an (e.g., fifth) exemplary step of the method according to the third aspect, label relationship data is determined, which describes model parameters (e.g., weights) of a learning algorithm for establishing a relationship between the position of an anatomical structure in each medical image of a second plurality of patients and the position of an associated label of the patient, wherein the label relationship data is determined by inputting training patient image data and label data and transformed statistical distribution data into a function that establishes the relationship. For example, the combined correction data is used as a basis for determining weights, which are, for example, input into a cost or loss function that compares the established relationship with the label data. This allows the quality of the training results to be evaluated while focusing on areas to be strongly corrected in medical image segmentation.
[0026] In an example of the method according to the third aspect, the relationship between the position of the anatomical structure in each medical image of a second plurality of patients and the position of the patient's associated label is defined as the relative position between the position of the anatomical structure in each medical image of the second plurality of patients and the position of the patient's associated label, and the learning algorithm determines the relative position based on the transformed statistical distribution data.
[0027] In an example of the method according to the third aspect, the learning algorithm comprises or consists of a machine learning algorithm.
[0028] In an example of the method according to the third aspect, the learning algorithm comprises or consists of a convolutional neural network.
[0029] In an example of the method according to the third aspect, the model parameters define learnable parameters of the learning algorithm, such as weights of the learning algorithm.
[0030] In a fourth aspect, the present invention is directed to a computer-implemented method (e.g., a medical method) of determining a relationship between an image representation of an anatomical part of a body in a medical image of an individual and a label to be associated with the image representation of the anatomical part of the body in the medical image of the individual. The method according to the fourth aspect includes executing the following exemplary steps performed by at least one processor of at least one computer (e.g., at least one computer as part of a navigation system) on at least one processor.
[0031] In a (e.g. first) exemplary step of the method according to the fourth aspect, individual patient image data are acquired, the individual patient image data describing a digital individual medical image describing an anatomical part of a patient's body. The individual patient image has been generated, for example, using the same imaging modality as the digital medical image from which the initial segmentation has been generated.
[0032] In an (e.g. second) exemplary step of the method according to the fourth aspect, segmented individual patient image data describing a relationship between an image representation of an anatomical part of a body in an individual medical image and a label to be associated with the image representation of the anatomical part of the body in the individual medical image is determined. The segmented individual patient image data is determined by inputting the individual patient image data into a function that establishes a relationship between the image representation of the anatomical part of the body in the individual medical image and the label to be associated with the image representation of the anatomical part of the body in the individual medical image. The function is part of a learning algorithm that has been trained by executing the method according to the third aspect.
[0033] In an example of the method according to the fourth aspect, a learning algorithm has been trained by executing the method according to the third aspect, wherein the relationship between the position of the anatomical structure in each medical image of the second plurality of patients and the position of the patient's associated label is defined as the relative position between the position of the anatomical structure in each medical image of the second plurality of patients and the position of the patient's associated label, and the learning algorithm determines the relative position based on the transformed statistical distribution data. Then, the relationship between the image representation of the body anatomical part in the individual medical image and the label to be associated with the image representation of the body anatomical part in the individual medical image is the relative position between the position of the image representation of the body anatomical part in the individual medical image and the position of the label to be associated with the image representation of the body anatomical part in the individual medical image, and the learning algorithm determines the relative position based on the transformed statistical distribution data.
[0034] In one example of the method according to the third and fourth aspects, the learning algorithm may be a random forest algorithm. See Antonio Criminisi, Jamie Shotton and Ender Konukoglu, Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning, 2011, available at h ttps: / / www.microsoft.com / zh-us / research / wp-content / uploads / 2016 / 02 / CriminisiFore sts_FoundTrends_2011.pdf, the explanation of random forest is as follows:
[0035] Random forests or ensemble learning methods for classification or regression work by constructing a large number of decision trees during training and outputting the pattern of classes (classification) or the average prediction of the individual trees (regression).
[0036] The basic building block of a random forest is a single decision tree. A decision tree is a set of questions organized in a hierarchical manner and represented graphically as a tree. A decision tree estimates an unknown property ("label") of an object by asking consecutive questions about its known properties (so-called "features"). The next question to be asked depends on the answer to the previous question, and this relationship is represented graphically as a path through the tree that the object follows. Decisions are then made based on the terminal nodes on the path (so-called "leaf nodes"). Each question corresponds to an internal node of the tree (a so-called "split node").
[0037] Each split node has a so-called test function associated with it. The test function at a split node j is formulated as a function with binary output:
[0038]
[0039] Among them, 0 and 1 can be interpreted as "false" and "true" respectively. Represents the parameters of the test function at the jth split node.
[0040] v is therefore the vector represents the current object ("data point"), where the component x i Represents certain attributes (features) of data points, all of which form a feature space
[0041] In its simplest form, the test function is a linear model that selects a feature axis in the feature space and classifies each data point according to whether the value of the respective feature is below or above a learnable threshold. For other more complex forms, non-linear test functions are possible.
[0042] To train a decision tree, a set of training data points is used, for which the features and the required labels are known. The goal of the training is to automatically learn a suitable test function at all split nodes that is best suited to determine the label from the features of the data point. Later, this trained decision tree can be evaluated for new data points with unknown labels by sending the data points through a tree trained on its features.
[0043] To understand the training process, it is helpful to represent subsets of training points as being associated with different branches. For example, S1 represents the subset of training points that reach node 1 (nodes are numbered starting from 0 in breadth-first order to the root F), while Represents a subset of the left and right children going to node 1.
[0044] Training is responsible for selecting a test function h(v,θ) associated with each split node (indexed by j) by optimizing a chosen objective function defined on the available training set. j ) types and parameters.
[0045] The optimization of the split function is performed in a greedy manner. At each node j, according to the subset S of the training set introduced j , learn to convert S j The "best" split is and This problem is formulated as the maximization of the objective function at the node:
[0046]
[0047] in
[0048]
[0049]
[0050]
[0051] As mentioned above, the symbol S j , represents the set of training points before and after segmentation. The objective function is in abstract form here. Its precise definition and the meaning of "best" depend on the task at hand (e.g., whether it is supervised, continuous or discrete output). For example, for binary classification, the term "best" can be defined as the segmentation of the training subset S j , so that the resulting child nodes are as pure as possible, that is, only contain training points of a single class. In this case, the objective function can be defined as information gain, for example.
[0052] During training, the structure (shape) of the tree also needs to be optimized. Training starts from the root node j=0, where the optimal split parameters are found as described before. Therefore, two child nodes are constructed, each receiving a different disjoint subset of the training set. This process is then applied to all newly constructed nodes, and the training phase continues. The structure of the tree depends on how and when to decide to stop growing the various branches of the tree. Different stopping criteria can be applied. For example, the tree is usually stopped when a maximum number of levels D is reached. Alternatively, a maximum value can be applied. In other words, it stops when the attributes of the training points in the leaf nodes are similar to each other. The tree growth may also stop when the node contains too few training points. In terms of generalization, avoiding growing the entire tree has been shown to have a positive effect.
[0053] During training, randomness is injected into the tree: when training at the jth node, instead of optimizing over the entire parameter space of the test function, only a small random subset of the parameter values is used. Therefore, under the randomness model, the training tree is completed by optimizing each split node j:
[0054]
[0055] Due to this random setting, it is later possible to train multiple decision trees in parallel, each utilizing a different set of attributes from the data points.
[0056] At the end of the training phase, we obtain: (i) the (greedy) optimal weak learner associated with each node; (ii) the learned tree structure; and (iii) a different set of training points at each leaf.
[0057] After training, each leaf node is still associated with a subset of the (labeled) training data. During testing, a previously unseen point traverses the tree until it reaches a leaf. Since the split nodes act on features, an input test point may end up in a leaf associated with a training point similar to itself. Therefore, it is reasonable to assume that the associated label must also be similar to the labels of the training points in that leaf. This justifies using the label statistics collected in that leaf to predict the label associated with the input test point.
[0058] In the most general sense, leaf statistics can be gathered using the posterior distribution:
[0059] p(c|v) and p(y|v)
[0060] where c and y represent discrete or continuous labels, respectively. v is the data point tested in the tree, and the conditional representation distribution depends on the fact that the particular leaf node is reached by the test point. Different leaf predictor variables can be used. For example, in the discrete case, the maximum a posteriori (MAP) estimate can be obtained as c*=argmax c p(c|).
[0061] Building on the construction principles of decision trees described above, we can now move on to decision forests, also known as random forests:
[0062] A random decision forest is a set of randomly trained decision trees. The key aspect of the forest model lies in the fact that its constituent trees are randomly different from each other. This leads to decorrelation between individual tree predictions, which in turn leads to improved generalization and robustness.
[0063] In a forest of T trees, use the variable to index each constituent tree. All trees are trained independently (and possibly in parallel). During testing, each test point v is pushed into all trees simultaneously (starting from the root) until it reaches the corresponding leaf. Tree testing can also usually be done in parallel, thus achieving high computational efficiency on modern parallel CPU or GPU hardware. All tree predictions can be merged into a single forest prediction by a simple averaging operation. For example, in classification:
[0064]
[0065] where p t (c|v) represents the posterior distribution obtained by the tth tree. Alternatively, the tree outputs can be multiplied together (although the trees are not statistically independent):
[0066]
[0067] The partition function Z is used to ensure that the probabilities are normalized.
[0068] In one example of the method according to the first aspect, the second aspect and the third aspect, the learning algorithm may be a convolutional neural network. Figure 1 A convolutional neural network is explained as an example of a machine learning algorithm used in conjunction with the presently disclosed invention.
[0069] A convolutional network (also called a convolutional neural network, or CNN) is an example of a neural network for processing data with a known grid-like topology. Examples include time series data (which can be viewed as a one-dimensional grid sampled at regular time intervals) and image data (which can be viewed as a two-dimensional or three-dimensional grid of pixels). The name "convolutional neural network" indicates that the network employs the mathematical operation of convolution. Convolution is a linear operation. Convolutional networks are simple neural networks that use convolution instead of general matrix multiplication in at least one of their layers. There are many variations of the convolution function, which is widely used in neural networks in practice. In general, the operation used in convolutional neural networks does not correspond exactly to the definition of convolution used in other fields (such as engineering or pure mathematics).
[0070] The main component of a convolutional neural network is an artificial neuron. Figure 1 is an example of a single neuron depicted in Figure 1. The node in the middle represents the neuron, which accepts all inputs (x1,…,x n ), and multiply them by their specific weights (w1,…,wn ). The importance of the input depends on its weight value. The addition of these calculated values is called the weighted sum, which will be plugged into the activation function. The weighted sum z is defined as:
[0071]
[0072] The bias b is a value that is independent of the input and modifies the boundaries of the threshold. The resulting value is processed by the activation function, which decides whether to pass the input to the next neuron.
[0073] CNNs usually take 1st-order or 3rd-order tensors as their input, for example, an image with H rows, W columns, and 3 channels (R, G, B color channels). However, CNNs can process higher-order tensor inputs in a similar way. The input then continues to undergo a series of processing. A processing step is usually called a layer, which can be a convolutional layer, a pooling layer, a normalization layer, a fully connected layer, a loss layer, etc. The details of these layers are described in the following sections.
[0074]
[0075] Equation 5 above explains how CNN operates layer by layer in the forward pass. The input is x 1 , usually an image (1st or 3rd order tensor). The parameters involved in the first layer of processing are collectively referred to as tensor w 1 The output of the first layer is x 2 , which also serves as the input for the second layer of processing. This process continues until all layers in the CNN are processed, and its output x L . But add an additional layer for backward error propagation, which is a method for learning good parameter values in CNN. Suppose the problem at hand is an image classification problem with class C. A common strategy is to convert x L Output as a C-dimensional vector whose i-th entry encodes the prediction (x 1 The posterior probability of comes from the i-th class). In order to make x L becomes a probability mass function, which can set the processing in the (L-1)th layer to x L-1 Softmax transformation (comparing distance measures and data transformation nodes). In other applications, the output x L can have other forms and interpretations. The last layer is the loss layer. Assume t is the input x 1 The corresponding target value (ground truth) of , then the cost or loss function can be used to measure the CNN prediction x L The difference between t and the target t. It should be noted that some layers may not have any parameters, i.e., for some i, w i May be empty.
[0076] In the example of CNN, ReLu is used as the activation function of the convolutional layer, while the softmax activation function provides information to give a classification output. The following section will explain the purpose of the most important layers.
[0077] The input image is input to the feature learning part which includes layers of convolution and ReLu, followed by layers including pooling, which are followed by further pairwise repetitions of layers of convolution and ReLu and pooling. The output of the feature learning part is input to the classification part which includes layers for flattening, fully connected and maximum softening.
[0078] In a convolutional layer, multiple convolution kernels are usually used. Assuming that D kernels are used and the spatial span of each kernel is H×W, all kernels are denoted as f. f is Similarly, using index variables 0≤i<H, 0≤j<W, 0≤d l <D l and 0≤d<D to determine the specific elements in the kernel. It should also be noted that the kernel set f is similar to the symbol w above L The same objects are referenced. The notation is changed slightly to simplify the derivation. It is also clear that the kernel remains the same even when the mini-batch strategy is used.
[0079] As long as the convolution kernel is larger than 1×1, the spatial amplitude of the output is smaller than the spatial amplitude of the input. Sometimes it is necessary for the input and output images to have the same height and width, and a simple padding trick can be used. For each input channel, if padding (i.e., insertion) is done above the first row rows, fill below the last row (i.e., insert) Rows and fill Column to the left of the first column, fill Column to the right of the last column, the size of the convolution output will be H l ×W l ×D, i.e., has the same spatial magnitude as the input. is a floor function. The elements of padded rows and columns are usually set to 0, but other values are possible.
[0080] Stride is another important concept in convolution. The kernel is convolved with the input at every possible spatial amplitude, which corresponds to a stride of s = 1. However, if s>1, each movement of the kernel skips s-1 pixel positions (i.e., a convolution is performed every s pixels in the horizontal and vertical directions).
[0081] In this section, we consider the simple case where the stride is 1 and no padding is used. Therefore, There is y (or x l+1 ), where H l+1 =H l-H+1,W l+1 =W l -W+1, and D l+1 = D. In precise mathematics, the convolution process can be expressed as an equation:
[0082]
[0083] For all 0≤d≤D=D l+1 , and satisfy 0≤i l+1 <H l –H+1=H l+1 , 0≤j l+1 <W l –W+1=W l+1 Any spatial position (i l+1, j l+1 ) Repeat Equation 2. In this equation, Refers to the triple (i l+1 +i,j l+1 +j,d l ) indexed by x l element. Usually the bias term b d Add to In order to make the description clearer, this term will not be described in detail.
[0084] Pooling functions replace the network output at a certain location with summary statistics of nearby outputs. For example, the max pooling operation reports the maximum output within a rectangular neighborhood of a table. Other popular pooling functions include the average of a rectangular neighborhood, the L2 norm of a rectangular neighborhood, or a weighted average based on the distance to the center pixel. In all cases, pooling helps make the representation approximately invariant to small translations of the input. Translation invariance means that if the input is translated by a small amount, the value of the pooled output will not change.
[0085] Since pooling aggregates responses over an entire neighborhood, it is possible to use fewer pooling units than detector units by reporting summary statistics for regions that are k pixels apart instead of one pixel. This improves the computational efficiency of the network because the next layer has approximately k times fewer inputs to process.
[0086] Assume that the CNN model w has been learned 1 ,…,w L-1 If we know all the parameters of , we can use the model to make predictions. Prediction involves only running the CNN model in the forward direction, that is, in the direction of the arrow in Equation 1. Take the image classification problem as an example. From the input x 1 To begin, pass it through the first layer (with parameter w 1 ) and obtain x 2 . In turn, x 2Pass it to the second layer, and so on. Finally, after receiving Its estimated x 1 The posterior probability belongs to class C. The CNN prediction output can be:
[0087]
[0088] The question at this point is: how to learn the model parameters?
[0089] As in many other learning systems, the parameters of the CNN model are optimized to minimize the loss z, i.e., we want the predictions of the CNN model to match the ground truth labels. Suppose we are given a training example x 1 The training process involves running the CNN network in two directions. First, the network is run in a forward pass to obtain x L , to achieve predictions using the current CNN parameters. Instead of outputting predictions, we need to compare the predictions with the corresponding 1 The target t is compared, that is, the forward pass is continued until the last loss layer. Finally, the loss z is obtained. The loss z is a supervisory signal that guides how to correct (update) the parameters of the model.
[0090] There are several algorithms for optimizing loss functions, and CNNs are not limited to a specific algorithm. An example algorithm is called stochastic gradient descent (SGD). This means that the parameters are updated by using gradients estimated from a (usually) small subset of the training examples.
[0091]
[0092] In Equation 4, the ← symbol implicitly indicates the parameter w (of layer i) i Update from time t to t+1. If the time index t is used explicitly, the equation will be written as:
[0093]
[0094] In Equation 4, the partial derivative Measure z relative to w i The growth rate of the different dimensions of . This partial derivative vector is called a gradient in mathematical optimization. Therefore, in w i In a small local area near the current value of , move w in the direction determined by the gradient i The target value z will be increased. To minimize the loss function, w should be updated in the opposite direction of the gradient i . This update rule is called gradient descent.
[0095] However, if you move too far in the direction of the negative gradient, the loss function may increase. Therefore, in each update, the parameters are only changed by a small fraction of the negative gradient (controlled by η (learning rate)). η>0 is usually set to a small number (such as η=0.001). If the learning rate is not too high, then the loss function is based on x 1 An update of will make the loss for this particular training example smaller. However, it is likely to make the loss for some other training examples larger. Therefore, all training examples need to be used to update the parameters. When all training examples have been used to update the parameters, it is said that a learning cycle has been processed. Typically, a learning cycle will reduce the average loss of the training set until the learning system fits the training data. Therefore, the gradient descent update learning cycle can be repeated and terminated at a certain point to obtain the CNN parameters (for example, it can be terminated when the average loss of the validation set increases).
[0096] The partial derivatives of the last layer are easy to compute. L In the parameter w L is directly connected to z under the control of , so it is easy to calculate Only when w L This step is only required if it is not empty. Similarly, it is easy to calculate For example, if we use the squared L2 loss, then is empty, and
[0097] In fact, for each layer, two sets of gradients are computed: with respect to the layer parameters w i The partial derivative of z, and the input x of this layer i As shown in Equation 4, the term Can be used to update the parameters of the current (i-th layer). Can be used to update parameters backwards, for example, to update to the (i-1)th layer. The intuitive explanation is: x i is the output of the (i-1)th layer, and Should change x i In order to reduce the loss function. Therefore, it can be regarded as layer by layer is considered as part of the “wrong” supervision information propagated back from z to the current layer. Therefore, the backpropagation process can be continued and used The error is back-propagated to the (i-1)th layer. This layer-by-layer backward update procedure can greatly simplify learning CNN.
[0098] Take the i-th layer as an example. When updating the i-th layer, the back propagation process of the (i+1)-th layer must have been completed. That is, the term and Both are stored in memory and are available for use. The task at this point is to calculate and Using the chain rule, we get:
[0099]
[0100]
[0101] Since it has been calculated and store it in memory, so only matrix reshaping operations (vec) and an additional transpose operation are required to obtain It is the first term on the right side (RHS) of both equations. As long as we can calculate and The expected value (the left side of both equations) can be easily obtained.
[0102] and Compared with direct calculation and Much easier, because x i By taking the parameter w i The function of x i+1 Directly related.
[0103] In the context of neural networks, activations act as transfer functions between neuron inputs and outputs. They define under which conditions a node is activated, i.e., an input value is mapped to an output which in turn serves as one of the inputs to subsequent neurons in the hidden layer. There are a large number of different activation functions with different characteristics.
[0104] The loss function quantifies how well the algorithm models the given data. In order to learn from the data and change the weights of the network, the loss function must be minimized. In general, a distinction can be made between regression loss and classification loss. In classification, one tries to predict an output from a limited set of categorical values (class labels), while regression, on the other hand, deals with predicting continuous values.
[0105] In the following mathematical formula, the following parameters are defined as:
[0106] n is the number of training examples;
[0107] i is the i-th training example in the dataset;
[0108] ·y i is the ground truth label of the ith training example;
[0109] · is the prediction for the ith training example.
[0110] The most common setting for classification problems is the cross entropy loss. It increases as the predicted probability deviates from the actual label. The logarithm of the actual predicted probability is multiplied by the ground truth category. The important aspect is that the cross entropy loss severely penalizes confident but wrong predictions. The mathematical formula can be described as:
[0111]
[0112] A typical example of regression loss is the mean squared error or L2 loss. As the name suggests, the mean squared error is the average of the squared differences between the predicted values and the actual observed values. It only concerns the average error magnitude, regardless of their direction. However, due to the squaring, predictions far from the actual values suffer from severe flaws compared to predictions with smaller deviations. In addition, MSE has nice mathematical properties that make it easier to calculate the gradient. Its formula is as follows:
[0113]
[0114] For information on the capabilities of convolutional neural networks, see the following papers:
[0115] I. Goodfellow, Y. Bengio, and A. Courville, Deep learning, chapter convolutional networks, 2016, available at http: / / www.deeplearningbook.org ;
[0116] J. Wu, Introduction to convolutional neural networks, see https: / / pdfs.semanticscholar.org / 450c / a19932fcef1ca6d0442cbf52fec38fb9d1e5.p df ;
[0117] "Common loss functions in machine learning", please see https: / / towardsdatascience.com / common-loss-functions-in-machine-learning- 46af0ffc4d23 , last visited: 2019-08-22;
[0118] Imagenet classification with deep convolutional neural networks by Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton, see http: / / papers.nips.cc / paper / 4824-imagenet-classification-with-deep-convolutional- neur al-networks.pdf ;
[0119] S. Ren, K. He, R. Girshick, and J. Sun, Faster r-CNN: Towards real-time object detection with region proposal networks, available at https: / / arxiv.org / pdf / 1506.01497.pdf ;
[0120] S.-E. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh, Convolutional posemachines, available at https: / / arxiv.org / pdf / 1602.00134.pdf ;
[0121] Jonathan Long, Evan Shelhamer and Trevor Darrell "Fully convolutional networks for semantic segmentation", please see https: / / www.cv-foundation.org / openaccess / content_cvpr_2015 / papers / Long_Fully_Convolutional_Networks_2015_ CVPR_paper.pdf .
[0122] In a fifth aspect, the present invention is directed to a program which, when run on a computer or loaded onto a computer, causes the computer to execute the method steps of the method according to the first, second, third or fourth aspect, and / or a program storage medium storing the program, or a program storage medium storing data defining model parameters and an architecture of a learning algorithm that has been trained by executing the method according to the third aspect, and / or a data carrier signal carrying the above-mentioned program, and / or a data carrier signal carrying data defining model parameters and an architecture of a learning algorithm that has been trained by executing the method according to the third aspect, and / or a data stream carrying the above-mentioned program, and / or a data stream carrying data defining model parameters and an architecture of a learning algorithm that has been trained by executing the method according to the third aspect, and / or at least one computer comprising at least one processor and a memory, wherein the above-mentioned program runs on at least one processor or is loaded into the memory of the computer.
[0123] The present invention may alternatively or additionally relate to a signal wave (e.g., physically generated by technical means, such as electricity) carrying information representing a program, such as the above-mentioned program, such as a digital signal wave, such as an electromagnetic carrier wave, which, for example, includes a code mechanism suitable for executing any or all steps of the method according to the first aspect. In one example, the signal wave is a data carrier signal carrying the above-mentioned computer program. The computer program stored on the disk is a data file, which, when read and transmitted, becomes a data stream in the form of a (e.g., physically generated by technical means, such as electricity) signal. The signal can be implemented as a signal wave, such as the electromagnetic carrier wave described herein. For example, the signal (e.g., signal wave) is constructed to be transmitted through a computer network (e.g., LAN, WLAN, WAN, mobile network (e.g., Internet)). For example, the signal (e.g., signal wave) is constructed to be transmitted through optical or acoustic data transmission. Therefore, the present invention may alternatively or additionally relate to a data stream representing the above-mentioned program (i.e., including the program) according to the second aspect.
[0124] In a sixth aspect, the present invention is directed to a medical data processing system comprising:
[0125] a) at least one computer according to the fifth aspect, wherein the program causes the computer to perform the method steps according to the method of the first aspect or the second aspect;
[0126] b) at least one electronic data storage device storing the initial segmented patient image data and the segmented correction data; and
[0127] c) the program storage medium according to the fifth aspect,
[0128] wherein at least one computer is operably coupled to at least one electronic data storage device to retrieve initial segmented patient image data and segmented correction data from the at least one electronic data storage device and to store at least the combined correction data in the at least one electronic data storage device, and insofar as the program causes the computer to perform the method steps of the method according to the second aspect, namely to store the corrected segmented patient image data in the at least one electronic data storage device.
[0129] In a seventh aspect, the present invention is directed to a medical data processing system comprising:
[0130] a) at least one computer according to the fifth aspect, wherein the program causes the computer to perform the method steps according to the method of the fourth aspect;
[0131] b) at least one electronic data storage device storing individual patient image data;
[0132] c) the program storage medium according to the fifth aspect,
[0133] Wherein at least one computer is operably coupled to:
[0134] - at least one electronic data storage device to retrieve the individual patient image data from the at least one electronic data storage device and store at least the segmented individual patient image data in the at least one electronic data storage device; and
[0135] - a program storage medium to obtain data defining model parameters and a learning algorithm architecture from the program storage medium.
[0136] definition
[0137] This section provides definitions of certain terms used in this disclosure, which also constitute a part of this disclosure.
[0138] The method according to the invention is, for example, a computer-implemented method. For example, all or only some of the steps (i.e., less than the total number of steps) of the method according to the invention can be performed by a computer (e.g., at least one computer). An embodiment of the method implemented by a computer is a purpose of a computer for performing a data processing method. An embodiment of the method implemented by a computer is a method involving computer operation, such that the computer is operated to perform one, more or all of the steps of the method.
[0139] The computer comprises, for example, at least one processor and, for example, at least one memory, in order to (technically) process data, for example electronically and / or optically process data. The processor is, for example, made of a semiconductor substance or composition, for example, at least partially n-type and / or p-type doped semiconductor, for example, at least one of type II, type III, type IV, type V, type VI semiconductor materials, for example (doped) silicon arsenide and / or gallium arsenide. The described calculation steps or determination steps are, for example, performed by a computer. The determination step or calculation step is, for example, a step of determining data within the framework of a technical method (for example, within the framework of a program). The computer is, for example, any type of data processing device, for example, an electronic data processing device. The computer can be a device that is generally regarded as a computer, such as a desktop personal computer, a laptop, a netbook, etc., but can also be any programmable device, such as a mobile phone or an embedded processor. The computer can, for example, include a "sub-computer" system (network), in which each sub-computer represents its own computer. The term "computer" includes cloud computers, such as cloud servers. The term "computer" includes server resources. The term "cloud computer" includes a cloud computer system, which includes, for example, a system of at least one cloud computer, and, for example, includes a plurality of operably interconnected cloud computers, such as a server cluster. Such a cloud computer is preferably connected to a wide area network such as the World Wide Web (WWW) and is located in a so-called cloud of computers all connected to the World Wide Web. Such infrastructure is used for "cloud computing", which describes those computing, software, data access and storage services that do not require the end user to know the physical location and / or configuration of the computer providing a particular service. For example, the term "cloud" is used metaphorically for the Internet (World Wide Web). For example, the cloud provides a computing infrastructure as a service (IaaS). A cloud computer can be used as a virtual host for an operating system and / or a data processing application for executing the method of the present invention. A cloud computer is, for example, provided by Amazon Web Services. TM) provided by Elastic Compute Cloud (EC2). The computer, for example, includes an interface to receive or output data and / or perform analog-to-digital conversion. The data, for example, represents physical properties and / or is generated from technical signals. The technical signal is generated, for example, by a (technical) detection device (for example, a device for detecting a marking device) and / or a (technical) analysis device (for example, a device for performing a (medical) imaging method), wherein the technical signal is, for example, an electrical signal or an optical signal. The technical signal, for example, represents data received or output by the computer. The computer is preferably operably coupled to a display device that allows information output by the computer to be displayed to, for example, a user. An example of a display device is a virtual reality device or an augmented reality device (also known as virtual reality glasses or augmented reality glasses), which can be used as "goggles" for navigation. A specific example of such augmented reality glasses is Google Glass (Google Glass, a trademark brand of Google, Inc.). The augmented reality device or virtual reality device can be used to input information into the computer through user interaction and to display information output by the computer. Another example of a display device is a standard computer monitor, for example comprising a liquid crystal display, operatively connected to a computer for receiving display control data from a computer for generating signals for displaying the image information content on the display device. A specific embodiment of such a computer monitor is a digital light box. An example of such a digital light box is the product 1000x1000 from Brainlab AG. The monitor may also be a handheld portable device such as a smart phone or a personal digital assistant or a digital media player, for example.
[0140] The present invention also relates to a computer program, which includes instructions, when the program is executed by a computer, these instructions cause the computer to perform one or more methods described herein, such as one or more steps of the method, and / or a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) storing the program, and / or a computer including the above-mentioned program storage medium, and / or a signal wave (physical, such as electrical, such as technically generated) carrying information representing a program (e.g., the above-mentioned program), such as a digital signal wave, such as an electromagnetic carrier wave, the program for example including a code mechanism suitable for performing any or all of the method steps described herein. In one example, the signal wave is a data carrier signal carrying the above-mentioned computer program. The present invention also relates to a computer, which includes at least one processor and / or the above-mentioned computer-readable storage medium and, for example, a memory, wherein the program is executed by a processor.
[0141] Within the framework of the present invention, a computer program element may be embodied by hardware and / or software (this includes firmware, resident software, microcode, etc.). Within the framework of the present invention, a computer program element may take the form of a computer program product, which may be implemented by a computer-usable, e.g. computer-readable data storage medium, which includes computer-usable, e.g. computer-readable program instructions, the "code" or "computer program" embodied in the data storage medium being used on or in conjunction with an instruction execution system. Such a system may be a computer; a computer may be a data processing device comprising means for executing a computer program element and / or a program according to the present invention, e.g. a data processing device comprising a digital processor (central processing unit or CPU) executing the computer program element, and optionally a data processing device comprising a volatile memory (e.g. random access memory or RAM) for storing data used for and / or generated by executing the computer program element. Within the framework of the present invention, a computer-usable, e.g. computer-readable data storage medium may be any data storage medium that may contain, store, communicate, propagate or transmit programs for use on or in conjunction with those instruction execution systems, devices or apparatuses. Computer-usable, for example, computer-readable data storage media can be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or apparatuses, or communication media such as the Internet. Computer-usable or computer-readable data storage media can even be, for example, paper or other suitable media on which the program can be printed, because the program can be captured electronically, for example, by optically scanning the paper or other suitable media, and then compiled, decoded or otherwise processed in an appropriate manner. The data storage medium is preferably a non-volatile data storage medium. The computer program product described herein and any software and / or hardware form various mechanisms for performing the functions of the present invention in the example embodiments. The computer and / or data processing device can, for example, include a guidance information device, which includes a mechanism for outputting guidance information. The guidance information can, for example, be output to the user visually by a visual indication mechanism (e.g., a monitor and / or a light) and / or audibly by an auditory indication mechanism (e.g., a speaker and / or a digital voice output device) and / or tactilely by a tactile indication mechanism (e.g., a vibration element or a vibration element incorporated into an instrument). For the purpose of this document, a computer is a technical computer, for example comprising technical components such as tangible components, for example mechanical components and / or electronic components. Any device mentioned in this document is a technical device and is for example a tangible device.
[0142] The expression "obtaining data" for example includes (within the framework of the computer-implemented method) a scenario in which data is determined by a computer-implemented method or program. Determining data for example includes measuring a physical quantity and transforming the measured value into data, such as digital data, and / or calculating (such as outputting) the data with the aid of a computer and for example within the framework of the method according to the invention. The "determining" step as described herein for example includes or consists of issuing a command to perform the determination described herein. For example, the step includes or consists of issuing a command to cause a computer (such as a remote computer, such as a remote server, such as in the cloud) to perform the determination. Alternatively or additionally, the "determining" step as described herein for example includes or consists of the following steps: receiving the result data of the determination described herein, such as receiving the result data from a remote computer (such as from a remote computer that causes it to perform the determination). The meaning of "obtaining data" also includes, for example, the following scenario: receiving or retrieving data by (such as inputting) a method or program implemented by a computer, such as from another program, a previous method step or a data storage medium, such as for further processing by a method or program implemented by a computer. The generation of the data to be acquired can, but does not have to be, part of the method according to the invention. Therefore, the expression "obtaining data" can also, for example, mean waiting to receive data and / or receiving data. The received data can be input, for example, via an interface. The expression "obtaining data" can also mean that a method or program implemented by a computer performs some steps in order to (actively) receive or retrieve data from a data source such as a data storage medium (e.g., ROM, RAM, database, hard disk drive, etc.) or via an interface (e.g., from another computer or network). The data obtained by the method or device of the present disclosure, respectively, can be obtained from a database located in a data storage device, which is operably connected to a computer for data transmission between the database and the computer, such as data transmission from a database to a computer. The computer obtains data for use as input in the "determine data" step. The determined data can be output to the same or another database for storage for subsequent use. The database or the database for implementing the disclosed method can be located in a network data storage device or a network server (e.g., a cloud data storage device or a cloud server) or a local data storage device (e.g., a mass storage device operably connected to at least one computer performing the disclosed method). The data can be "ready" by performing an additional step before the acquisition step. According to this additional step, data is generated for acquisition. For example, the data are detected or captured (eg, by an analysis device). Alternatively or additionally, the data are input according to an additional step, such as via an interface. For example, the generated data may be input (eg, into a computer).According to an additional step (which is performed before the acquisition step), the data can also be provided by performing an additional step of storing the data on a data storage medium (such as a ROM, RAM, CD and / or hard disk drive), so that the data are ready within the framework of the method or program according to the invention. Therefore, the step of "acquiring data" can also involve commanding the device to acquire and / or provide the data to be acquired. In particular, the acquisition step does not involve an invasive step, which represents a substantial physical interference with the body, requiring professional medical measures, and even if the required professional care and measures are taken when performed, the body may be exposed to significant health risks. In particular, the step of acquiring data, such as determining data, does not involve a surgical step, in particular does not involve a step of treating the human or animal body with surgery or therapy. In order to distinguish between the different data used by the present method, the data are denoted (i.e. referred to as) "XY data" etc. and are defined according to the information they describe, and then preferably referred to as "XY information" etc.
[0143] Preferably, atlas data describing (e.g. defining, more particularly representing and / or serving as) the general three-dimensional shape of a body anatomical part is obtained. Thus, the atlas data represents an atlas of a body anatomical part. The atlas is typically composed of a plurality of object general models, wherein these object general models together form a composite structure. For example, the atlas constitutes a statistical model of a patient's body (e.g. a part of the body), which has been generated based on anatomical information collected from a plurality of human bodies, for example, based on medical image data containing images of these human bodies. Thus, in principle, the atlas data represents the results of a statistical analysis of such medical image data of a plurality of human bodies. This result can be output as an image - thus the atlas data contains or is equivalent to the medical image data. This comparison can be performed, for example, by applying an image fusion algorithm, wherein the image fusion algorithm performs image fusion between the atlas data and the medical image data. The comparison result can be a measure of similarity between the atlas data and the medical image data. The atlas data includes image information (e.g., position image information) that can be matched with image information (e.g., position image information) contained in, for example, medical image data (e.g., by applying an elastic or rigid image fusion algorithm) so that, for example, the atlas data is compared with the medical image data in order to determine the position of anatomical structures in the medical image data that correspond to anatomical structures defined by the atlas data.
[0144] Multiple human bodies (whose anatomical structures are used as inputs to generate atlas data) advantageously share common features, such as at least one of gender, age, race, physical measurements (e.g., height and / or weight), and pathological conditions. Anatomical information describes, for example, human anatomical structures and is extracted, for example, from medical image information about the human body. For example, an atlas of the femur may include the femoral head, femoral neck, body, greater trochanter, lesser trochanter, and lower limbs as objects that together constitute a complete structure. For example, an atlas of the brain may include the telencephalon, cerebellum, diencephalon, pons, midbrain, and medulla oblongata as objects that together constitute a complex structure. One application of such an atlas is in medical image segmentation, where the atlas is matched to medical image data and the image data is compared to the matched atlas so that points (pixels or voxels) of the image data are assigned to objects of the matched atlas, thereby segmenting the image data into objects.
[0145] For example, the atlas data includes information about the anatomical part of the body. The information is, for example, at least one of patient-specific, non-patient-specific, indication-specific, or non-indication-specific. Therefore, the atlas data describes, for example, at least one of a patient-specific, non-patient-specific, indication-specific, or non-indication-specific atlas. For example, the atlas data includes movement information indicating the freedom of movement of the anatomical part of the body relative to a given reference (e.g., another anatomical part of the body). For example, the atlas is a multimodal atlas that defines atlas information for multiple (i.e., at least two) imaging modalities and includes mappings between atlas information under different imaging modalities (e.g., mappings between all modalities), so that these atlases can be used to transform medical image information from its image depiction under a first imaging modality to its image depiction under a second imaging modality different from the first imaging modality, or to compare different imaging modalities with each other (e.g., matching or registration).
[0146] In the medical field, imaging methods (also referred to as imaging modalities and / or medical imaging modalities) are used to generate image data (e.g. two-dimensional or three-dimensional image data) of human anatomical structures (e.g. soft tissue, bones, organs, etc.). The term "medical imaging method" is to be understood as meaning (advantageously device-based) imaging methods (e.g. so-called medical imaging modalities and / or radiographic imaging methods), such as computed tomography (CT) and cone beam computed tomography (CBCT, such as volumetric CBCT), x-ray tomography, magnetic resonance tomography (MRT or MRI), conventional x-ray, ultrasound examination and / or ultrasound examination and positron emission tomography. For example, the medical imaging method is performed by an analysis device. Examples of medical imaging modalities applied by the medical imaging method are: X-ray radiography, magnetic resonance imaging, medical ultrasound or ultrasound, endoscopy, elastography, tactile imaging, thermography, medical photography and nuclear medicine functional imaging techniques, such as positron emission tomography (PET) and single-photon emission computed tomography (SPECT), see the Wikipedia entries.
[0147] The image data thus generated is also referred to as "medical imaging data". The analysis device is used, for example, to generate image data in an imaging method based on an apparatus. The imaging method is used, for example, for medical diagnosis of the analysis of the anatomical structure of the body to generate an image described by the image data. The imaging method is also used, for example, to detect pathological changes in the human body. However, some changes in the anatomical structure, such as pathological changes in the structure (tissue), may not be detected and may not be visible, for example, in an image generated by the imaging method. A tumor represents an example of a change in the anatomical structure. If a tumor grows, it can be considered to represent an expanded anatomical structure. Such an expanded anatomical structure may not be detected, for example, only a part of the expanded anatomical structure can be detected. For example, early / late brain tumors are usually visible in MRI scans when a contrast agent is used to infiltrate the tumor. MRI scanning represents an example of an imaging method. In the case of an MRI scan of such a brain tumor, it is considered that the signal enhancement in the MRI image (caused by the contrast agent infiltrating the tumor) represents a solid tumor mass. Therefore, the tumor is detectable and can be distinguished, for example, in an image generated by the imaging method. In addition to these tumors, known as "enhancing" tumors, it is believed that approximately 10% of brain tumors are not discernible in scans and are not visible, for example, to a user viewing an image generated by an imaging method.
[0148] The mapping describes a transformation (e.g., a linear transformation) that transforms elements (e.g., pixels or voxels), e.g., the positions of elements, of a first data set in a first coordinate system into elements (e.g., pixels or voxels), e.g., the positions of elements, of a second data set in a second coordinate system (whose basis may be different from that of the first coordinate system). In one embodiment, the mapping is determined by comparing (e.g., matching) color values (e.g., grayscale values) of the respective elements by means of an elastic or rigid fusion algorithm. The mapping is embodied, for example, as a transformation matrix (such as a matrix defining an affine transformation). BRIEF DESCRIPTION OF THE DRAWINGS
[0149] In the following, the present invention is described with reference to the accompanying drawings, which give a background description of the present invention and show specific embodiments of the present invention. However, the scope of the present invention is not limited to the specific features disclosed in the context of the accompanying drawings, in which:
[0150] Figure 1 showing neurons of a (convolutional) neural network;
[0151] Figure 2 The basic flow of the method according to the first aspect is shown;
[0152] Figure 3 The basic flow of the method according to the second aspect is shown;
[0153] Figure 4 The basic flow of the method according to the third aspect is shown;
[0154] Figure 5 The basic flow of the method according to the fourth aspect is shown;
[0155] Figure 6 Possible applications of the methods according to the first to fourth aspects are shown;
[0156] Figure 7 is a schematic diagram of a system according to the fifth and sixth aspects. DETAILED DESCRIPTION
[0157] Figure 1 Shows the structure of a neuron as part of a (convolutional) neural network, where inputs are assigned certain weights for processing by an activation function that generates the neuron's output.
[0158] Figure 2 The basic flow of the method according to the first aspect is described, which starts from step S21, obtaining initial segmented patient image data, continues to step S22, which includes obtaining segmented correction data, and then continues to obtain atlas data in step S23. On this basis, step S24 calculates correction transformation data, and then determines the transformed correction data in step S25. Finally, combined correction data is determined in step S26.
[0159] Figure 3 The basic steps of the method according to the second aspect are shown, wherein step S31 comprises performing the method according to the first aspect, and step S32 involves obtaining segmented individual patient image data. Step S33 comprises determining corrected retransformed data. Then, step S34 follows, which determines the retransformed corrected data, and step 35 then determines the corrected segmented patient image data.
[0160] Figure 4 The basic steps of the method according to the third aspect are shown, wherein step S41 comprises performing the method according to the first aspect or the second aspect, and step S42 involves acquiring training patient image data. A subsequent step S43 acquires label data, followed by determining transformed statistics in step S44. Then, step S45 determines statistical distribution data, and finally step S46 determines label relationship data.
[0161] Figure 5 The basic steps of the method according to the fourth aspect are shown, where step S51 comprises acquiring individual patient image data, and a subsequent step S52 determines segmenting the individual patient image data.
[0162] Figure 6 An overview of the application of the method according to the first aspect to the third aspect is given. In the first (correction) stage, the following steps are performed: the patient image data 1 is initially segmented by an algorithm 2 to generate segmented patient image data 3, for example to generate labels indicating the contours of organs in the patient image data, and then the segmented patient image data or the labels are corrected 4, for example by a user. The correction is stored in the form of a spatial correction map. The spatial correction map is registered to a common model (atlas) 5 and aggregated (combined), for example in a central repository storing combined correction data 6. In the second (learning) stage, the following steps are performed: the combined correction data 6 is then input as a weight 7 into, for example, a cost or loss function, which compares the predicted segmented patient image data 3 used as the cost or loss function with a label 8 that is part of a segmentation learning algorithm 2, for example the same segmentation algorithm as used in the first stage. In addition, the patient data 1 and the label 8 (for example a manually generated organ contour) are input.
[0163] Figure 7 is a schematic diagram of a medical system 61 according to the fifth or sixth aspect. The system is generally labeled with reference numeral 61 and comprises a computer 62 and an electronic data storage device (such as a hard disk) 63, respectively for storing data stored by the system according to the fifth or sixth aspect. The components of the medical system 1 respectively have the functions and characteristics described above in relation to the fifth and sixth aspects of the present disclosure.
Claims
1. A computer-implemented method for determining a correction for correcting a segmentation of medical image data, the method comprising the following steps: A) performing a computer-implemented method of determining a correction distribution for correcting a segmentation of medical image data, comprising: a) acquiring initial segmented patient image data describing an initial segmentation of digital medical images of body anatomical regions of a first plurality of patients; b) acquiring segmentation correction data describing a correction of the initial segmentation for each patient of the first number of patients; c) acquiring atlas data, wherein the atlas data describes the three-dimensional shape of the anatomical part of the body; d) determining correction transformation data based on the segmented correction data and the atlas data, wherein the correction transformation data describes a transformation for transforming the correction into a reference system for describing positions in the three-dimensional shape; e) determining transformed correction data based on said segmented correction data and said correction transformed data by applying said transformation to said correction to determine the transformed correction; f) determining combined correction data based on the transformed correction data, wherein the combined correction data describes a statistical analysis result of the correction; Wherein, the computer-implemented method of determining a correction for correcting the segmentation of medical image data further comprises the following steps: B) acquiring segmented individual patient image data describing segmentation of anatomical parts of a patient's body in a digital individual medical image; C) determining correction retransformation data based on the atlas data and the individual patient image data, wherein the correction retransformation data describes a transformation for transforming the result of the corrected statistical analysis into a reference system for describing locations in the digital individual medical image; D) determining re-transformed corrected data based on the combined corrected data and the corrected re-transformed data; and E) determining corrected segmented patient image data based on the individual patient image data and the retransformed correction data, wherein the corrected segmented patient image data describes the result of applying the correction to a segmentation of an anatomical part of a patient's body in the digital individual medical image.
2. The method according to claim 1, wherein: The result of applying the correction to the segmentation of the anatomical part of the patient's body in the digital individual medical image is determined by applying the correction according to the feature quantity of the result of the statistical analysis.
3. The method according to claim 1, wherein: The results of the statistical analysis constitute the corrected statistical distribution for the first number of patients.
4. The method according to claim 1, wherein: The statistical analysis was a mean analysis.
5. The method according to claim 2, wherein: The characteristic amount is an average value or a variance.
6. The method according to claim 1, wherein: The segmented correction data is obtained based on user input.
7. A computer-implemented method for training a learning algorithm for determining corrections for digital medical image segmentation, the method comprising the steps of: A) performing a computer-implemented method of determining a correction distribution for correcting a segmentation of medical image data, comprising: a) acquiring initial segmented patient image data describing an initial segmentation of digital medical images of body anatomical regions of a first plurality of patients; b) acquiring segmentation correction data describing a correction of the initial segmentation for each patient of the first number of patients; c) acquiring atlas data, wherein the atlas data describes the three-dimensional shape of the anatomical part of the body; d) determining correction transformation data based on the segmented correction data and the atlas data, wherein the correction transformation data describes a transformation for transforming the correction into a reference system for describing positions in the three-dimensional shape; e) determining transformed correction data based on said segmented correction data and said correction transformed data by applying said transformation to said correction to determine the transformed correction; f) determining combined correction data based on the transformed correction data, wherein the combined correction data describes a statistical analysis result of the correction; Wherein, the computer-implemented method of determining a correction for correcting the segmentation of medical image data further comprises the following steps: B) acquiring training patient image data describing digital medical images of anatomical structures of a second plurality of patients; C) acquiring label data describing positions of labels assigned to the training patient image data, wherein the positions of the labels are defined in a reference frame defining positions in the training patient image data; D) determining transformed statistical distribution data based on the training patient image data and the combined correction data, wherein the transformed statistical distribution data describes the result of transforming the statistical distribution into a reference system defining locations in the training patient image data; E) determining statistical allocation data based on the transformed statistical distribution data and the label data, wherein the statistical allocation data describes the allocation of the result of the statistical analysis to the label; F) determining label relationship data, wherein the label relationship data describes model parameters of a learning algorithm for establishing a relationship between the position of an anatomical structure in each medical image of the second plurality of patients and the position of an associated label of the patient, wherein the label relationship data is determined by inputting the training patient image data and the label data and the transformed statistical distribution data into a function for establishing the relationship.
8. The method according to claim 7, wherein: The relationship between the position of the anatomical structure in each medical image of the second plurality of patients and the position of the relevant label of the patient is defined as the relative position between the position of the anatomical structure in each medical image of the second plurality of patients and the position of the relevant label of the patient, wherein the learning algorithm determines the relative position based on the transformed statistical distribution data.
9. The method according to claim 7, wherein: The learning algorithm includes a machine learning algorithm or consists of a machine learning algorithm.
10. The method according to claim 7, wherein: The learning algorithm includes or consists of a convolutional neural network.
11. The method according to claim 7, wherein: The model parameters define the learnable parameters of the learning algorithm.
12. A computer-implemented method of determining a relationship between an image representation of an anatomical part of a body in a medical image of an individual and a label to be associated with the image representation of the anatomical part of the body in the medical image of the individual, the method comprising the steps of: a) acquiring individual patient image data, the individual patient image data describing a digital individual medical image, the digital individual medical image describing an anatomical part of a patient's body; and b) determining segmented individual patient image data describing a relationship between an image representation of an anatomical part of a body in the individual medical image and a label to be associated with the image representation of the anatomical part of the body in the individual medical image, wherein the segmented individual patient image data is determined by inputting the individual patient image data into a learning algorithm that establishes a relationship between the image representation of the anatomical part of the body in the individual medical image and the label to be associated with the image representation of the anatomical part of the body in the individual medical image, wherein the learning algorithm has been trained by performing a method according to any one of claims 7 to 11.
13. The method according to claim 12, in, The learning algorithm has been trained by executing the method according to claim 8, wherein the relationship between the image representation of the body anatomical part in the individual medical image and the label to be associated with the image representation of the body anatomical part in the individual medical image is a relative position between the position of the image representation of the body anatomical part in the individual medical image and the position of the label to be associated with the image representation of the body anatomical part in the individual medical image, and Wherein, the learning algorithm determines the relative position based on the transformed statistical data.
14. A non-transitory computer-readable storage medium storing instructions, which when executed on at least one computer cause the at least one computer to perform the method steps of the method according to any one of claims 7 to 11.
15. A medical data processing system, comprising: a) at least one computer having at least one processor and a memory, wherein the memory stores instructions, which, when executed by the at least one processor of the at least one computer, cause the at least one processor of the computer to perform the method steps of any one of claims 1 to 6; b) at least one electronic data storage device storing initial segmented patient image data and segmented correction data; as well as c) The storage medium according to claim 14, wherein the at least one computer is operably coupled to the at least one electronic data storage device to retrieve the initial segmented patient image data and the segmented correction data from the at least one electronic data storage device and store at least the combined correction data in the at least one electronic data storage device, and store the corrected segmented patient image data in the at least one electronic data storage device as long as the instructions cause the computer to perform the method steps of the method according to claim 2.
16. A medical data processing system, comprising: a) at least one computer having at least one processor and a memory, wherein the memory stores instructions which, when executed by the at least one processor of the at least one computer, cause the at least one processor of the computer to perform the method steps of the method according to any one of claims 12 and 13; as well as b) at least one electronic data storage device storing individual patient image data; c) The storage medium according to claim 14, Wherein, the at least one computer is operably coupled to: the at least one electronic data storage device to retrieve the individual patient image data from the at least one electronic data storage device and store at least the segmented individual patient image data in the at least one electronic data storage device; as well as The storage medium is used to obtain data defining model parameters and learning algorithm architecture from the storage medium.
