Method and system for identifying image segmentation quality using neural networks

Abstract

Disclosed is the use of a neural network to identify image segmentation quality. Specifically, comparison logic compares boundaries of features of one or more images based at least in part on identifying the boundaries, and indication logic coupled to the comparison logic is to indicate whether the boundaries differ by at least a first threshold. The boundaries can include a first label mask representing a boundary of an object in an image (which is a boundary in a segmentation determined by a segmentation process) and a second label mask from a shape evaluation process applied to the first label mask. The indication logic can be configured to compare the first label mask and the second label mask to determine a quality of the segmentation. The neural network can perform the segmentation. The shape evaluation using the first label mask as input and the second label mask as output can be performed by a variational autoencoder. A graphics processing unit (GPU) can be used for the segmentation and / or the autoencoder.

Description

Background Technology

[0001] In image processing, segmentation refers to identifying the boundaries of image elements, such as dividing an image into regions based on the boundaries of objects depicted within it. Segmentation can be performed automatically using trained neural networks for boundary detection. Segmentation is useful in medical image processing and other fields. However, automatic segmentation may fail when encountering rare images that are difficult to segment. Sometimes, additional training of the neural network with extra training data can help with some automatic segmentation processes, but in rare cases where there may be no corresponding elements in the training data, it may still fail. Therefore, it is difficult to completely prevent segmentation failures.

[0002] One solution is to also have an automated quality assessment process. Applying Bayesian neural networks can capture the uncertainty of the segmentation results and perform quality assessment based on that uncertainty. However, this is also susceptible to rare events, as the segmentation process may make mistakes, but we should still have confidence in the accuracy of errors on some rare events. Attached Figure Description

[0003] Various technologies are described with reference to the accompanying drawings, in which:

[0004] Figure 1 An example of segmentation according to an embodiment is shown;

[0005] Figure 2 A diagram of a system for training a neural network to evaluate the segmentation of an input image, according to an embodiment, is shown.

[0006] Figure 3 A diagram of a system for verifying a neural network to be verified, according to an embodiment, is shown;

[0007] Figure 4 A diagram is shown of a variational autoencoder (VAE) to be trained for a truth label mask according to an embodiment;

[0008] Figure 5 The process for training a segmenter to obtain a label mask for a segmented image in an embodiment is illustrated, as well as the process for training a VAE to evaluate the quality of the label mask output by the segmenter against ground truth label encoding.

[0009] Figure 6 A diagram is shown of a system according to an embodiment for processing an image, generating a label mask for the image, applying the label mask to an autoencoder, comparing the label mask with the output of the autoencoder, and taking action based on the comparison result;

[0010] Figure 7 An autoencoder is shown according to an embodiment of which encodes an input label mask into a latent representation and decodes the latent representation into an output label mask.

[0011] Figure 8 An example of a parallel processing unit (“PPU”) according to an embodiment is shown;

[0012] Figure 9 An example of a general-purpose processing cluster (“GPC”) according to one embodiment is shown;

[0013] Figure 10 An example of a memory partition unit according to one embodiment is shown;

[0014] Figure 11 An example of a streaming multiprocessor according to one embodiment is shown;

[0015] Figure 12 A computer system that can implement various examples according to one embodiment is shown. Detailed Implementation

[0016] Segmentation is the process of determining the boundaries of elements depicted in an image, and possibly labeling those objects. Humans can usually easily identify the start location of one object and the end location of other objects in an image, but this is a difficult process for computer systems to perform. Furthermore, in certain types of images, even untrained humans may struggle to discern object boundaries or the presence of objects in the image. For example, computed tomography (“CT”) scans can image parts of the human body, and professional radiologists may spend considerable time observing, identifying, and labeling the boundaries of objects in CT scan data. Therefore, having an automated process capable of performing these tasks is beneficial, even if there are some signs, indicators, or alarms in the rare cases where the automated segmentation process fails.

[0017] Figure 1 An example of segmentation in the embodiments is shown, specifically segmentation dataset 102, which includes image 104 and label mask 106. Each of these terms is represented in memory as stored computer-readable data and / or in transit as computer-readable data. In one embodiment, image 104 is taken as a two-dimensional slice of a three-dimensional CT scan of a patient's abdomen, and label mask 106 is obtained via a user interface by manually inputting pancreatic boundaries determined by a professional radiologist and adding labels to identify the boundaries as pancreatic. In more complex images, in one embodiment, there are more than one bounded element and corresponding label from the CT scan, and multiple objects with incomplete boundaries and other features are included.

[0018] In one embodiment, a "segmentation dataset" refers to a data structure containing images and their associated label masks. In one embodiment, a goal of the segmentation system is to generate at least the boundaries of objects in an image in a computer-generated manner, so that humans do not have to manually input the object boundaries into the image. In one embodiment, the collection of segmentation datasets includes one or more segmentation datasets, each including images and associated label masks. In a particular embodiment, the label mask includes the boundaries of the objects, an object number that uniquely identifies the closed boundaries in the label mask, a reference to the image background, and the name of the object.

[0019] In one embodiment, the set (X, Y) of segmentation datasets comprises a set X of images in computer-readable data form and a set Y of corresponding label masks, wherein each label mask indicates a partial or complete segmentation of its corresponding image. In some embodiments, the set includes labels for the segmentations of the label masks, while in other embodiments, the label masks include segmentation boundaries without segmentation labels. If the label masks of the images are created by a person examining the images or through known good processes, they are often referred to as "ground truth" or "GT" label masks. In one embodiment, the set (X, Y) includes a training subset (X... T Y T ) and the verification subset (X) V Y V The training subset has a label mask that serves as a truth label mask, which can be used to train the neural network. In one embodiment, a validation subset is used to test the neural network, and the validation subset also has a label mask that serves as a truth label mask; the validation subset can be used to validate the training of the neural network.

[0020] In one embodiment, as an example, if F() refers to the segmentation process, then F(X) is verified by comparing the individual segmented datasets of the subset. V ) and Y V This is used to test the quality of the segmentation process. In one embodiment, the input to the trained neural network is an image X, and the output can be F(X). If the corresponding ground truth label mask Y is unknown, F(X) can be used as a label mask for image X.

[0021] In one embodiment, despite training, the neural network may still output a label mask F(X) that is not a good segmentation of image X, possibly because the content of image X is sparse or unexpected. In one embodiment, by using different processes to determine Y, this determination can be compared with F(X) to determine whether the segmentation process F() is applied to image X. If the boundaries of objects in image X are correctly identified, the output of the individual process will be close to F(X), which is the output of the neural network as part of the trained segmenter. In one embodiment, when there is a significant difference between the identified boundaries (greater than an acceptable threshold for the output of the trained segmenter), an alert is provided to the user of the system (e.g., a notification or change in the interface).

[0022] In one embodiment, a first neural network is used to process the image to determine a first segmentation of the image (e.g., a label mask that at least indicates the boundaries of some objects in the image). In one embodiment, the first segmentation is the output of a neural network other than the neural network itself, but in many of the examples described herein, the first segmentation is the output of the neural network. The label mask is passed through a second neural network that effectively maps the label mask to a feature space. Then, in one embodiment, the output of the second neural network is used to predict or estimate the quality of the segmentation in the label mask.

[0023] In one embodiment, the second neural network includes a variational autoencoder (“VAE”) that takes a label mask as its input and outputs a label mask. An autoencoder is a neural network that includes an encoder that encodes its input data into a latent representation of the input data in a compressed form, and the neural network also includes a decoder that decodes the latent representation and the original input data as closely as possible, wherein training the autoencoder involves reducing the difference between the encoder input and the decoder output. In a VAE, during the learning process, the VAE makes strong assumptions about the distribution of variables in the latent representation. In other embodiments, more generally, this document describes the VAE as using a fully convolutional neural network type, as in the case where it is being used.

[0024] In one embodiment, a first segment generated by a first neural network is compared with a second segment generated by a VAE. In one embodiment, this is done using a loss function. In another embodiment, this is done using a different method. In one embodiment, the second neural network is trained using a known good image segmentation. If the comparison shows that the first and second segments are very close, the first neural network is likely working well. If they are significantly different, it is likely problematic. In this case, if the segmentation is part of a system with a user interface, labels, warnings, alerts, etc., can be output to alert the user that the segmentation is questionable.

[0025] Regarding strong assumptions about the distribution of variables in the latent representation, in one embodiment, one approach is to project the segmentation results into a low-dimensional feature space, and then use a VAE to learn a classifier and / or regressor in that feature space to predict the quality of the segmentation results. In one embodiment, the feature space uses shape features to predict the quality of segmentation results given different segmentation algorithms on different datasets, and these shape features have strong prior information shared across the different data.

[0026] In one embodiment, the shape features of the segmentation result are captured using the value of the loss function, and the segmentation result is tested using a VAE trained only with a ground truth mask (i.e., a label mask that is a ground truth label mask). Therefore, bad segmentation may result in bad shape, which is a rare event for the VAE and will lead to a huge loss value.

[0027] Various techniques have been described in the preceding and following sections. For ease of explanation, specific configurations and details have been outlined to provide a comprehensive understanding of the possible approaches to implementing these techniques. However, it is evident that the techniques described below can be practiced in different configurations without specific details. Furthermore, well-known features may be omitted or simplified to avoid obscuring the described techniques.

[0028] Figure 2 A diagram of a system 200 according to one embodiment is shown, in which one or more arithmetic logic circuits at least partially cause a neural network to be trained to evaluate segmentation of an input image. In one embodiment, a computational entity 202 (e.g., a computer system or computer server) processes a training set of images and label masks and generates a segmentation (e.g., in the form of label masks). In one embodiment, the computational entity 202 is or includes a computer system or processing unit, such as a combination of... Figures 8-12The described one or more computer systems or processing units. In one embodiment, computing entity 202 is a distributed system comprising multiple computer servers that operate together as partitioned computing entities. In one embodiment, computing entity 202 is a server computer system, server cluster, virtual computer system, virtual server, virtual runtime environment, container environment, serverless execution environment, or service-hosted system. In one embodiment, computing entity 202 is or includes an application-specific integrated circuit (ASIC) microchip designed for a particular purpose or application. In one embodiment, computing entity may include special-purpose circuitry for sparse activation, low-precision computation, low-latency operation (e.g., in a real-time operating system (RTOS) computing environment), low-power hardware, and combinations thereof. In one embodiment, computing entity 202 has associated memories in a memory hierarchy, such that there are regions of faster memory (e.g., registers and L1 cache) and regions of slower memory (e.g., main memory).

[0029] In one embodiment, computational entity 202 is used to train a segmenter for a segmentation system and includes memory for a training set 204 and memory for training an output label mask set 206. In one embodiment, computational entity 202 implements segmenter 210, for example, by executing program code having program instructions that, when executed, perform the role of a neural network that can be trained on training set 204. In one embodiment of the operation of computational entity 202, training set 204 includes a plurality of images and a corresponding ground truth label mask for each image, wherein the ground truth label mask of an image in training set 204 represents a known good segmentation of that image and may also represent some or all of the segmentation labels. In one embodiment, segmenter 210 is trained such that its output for training images is a label mask that approximates the ground truth label mask of that image. When the output training label mask does not match the ground truth label mask well, training feedback can be provided to segmenter 210 to improve the output.

[0030] In one embodiment, the various other elements in the figure may be one or more arithmetic logic circuits, computer systems, or computer servers, or combinations thereof. Figures 8-12 This can be implemented using one or more such processing units. For example, in one embodiment, Figures 3-7The components are implemented by computing entities that have memory and are capable of executing program instructions, such as a distributed system comprising multiple computer servers that operate together as computing entities that generate segments or perform training. In embodiments, computing entities include server computer systems, server clusters, virtual computer systems, virtual servers, virtual runtime environments, container environments, serverless execution environments, service hosting systems, application-specific integrated circuit (ASIC) microchips designed for a particular purpose or application, and / or special-purpose circuitry for sparse activation, low-precision computing, low-latency operation (e.g., in a real-time operating system (RTOS) computing environment), low-power hardware, and combinations thereof.

[0031] Figure 3 A diagram of a system 300 according to one embodiment is shown, in which one or more arithmetic logic circuits at least partially enable the validation of a trained neural network. In the illustrated embodiment, computational entity 302 implements a validation set 304 for segmenting a dataset, wherein images of the validation set 304 are applied as input to a trained segmenter 310, which outputs a segmentation of the images of the validation set 304. In one embodiment, comparator 312 compares the output segmentation of the image with label masks from the validation set 304, wherein these label masks are ground truth label masks, and outputs an evaluation 314 of the quality of the output segmentation, which can be used to fine-tune the trained segmenter.

[0032] Figure 4 A diagram of system 400 in an embodiment is shown, in which one or more arithmetic logic circuits at least partially enable a variational autoencoder (VAE) to be trained against a truth label mask 404. In the illustrated embodiment, computational entity 402 includes a VAE 408 that takes the truth label mask 404 as its input and outputs an output label mask 406. In one embodiment, comparator 412 compares the truth label mask 404 and the output label mask 406 to determine the training feedback required for the VAE 408. In one embodiment, training results in adjusting the weights and connections between layers of the VAE 408 to optimize or minimize the loss function.

[0033] Figure 5An illustrative example of process 502 in one embodiment is shown, which is used to train a segmenter for segmenting an image to obtain a label mask for the image, and to train a VAE for ground truth label masks to evaluate the quality of the label mask output by the segmenter. In one embodiment, the process begins by applying a training label mask set (step 504), which includes ground truth label masks or other label masks known to be good examples of segmentation. In one embodiment, this training label mask set is applied to an autoencoder (e.g., VAE 408 referenced herein) as input to the autoencoder. As explained herein, in one embodiment, the autoencoder encodes its input into a latent representation and then decodes the latent representation into the autoencoder's output. Next, in step 506, in one embodiment, the segmenter system under training compares the output label mask output by the autoencoder with the input label mask input to the autoencoder. In step 508, the autoencoder is tuned to reduce the difference between the input label mask and the output label mask (which can be measured by a loss function). Once completed, in the embodiment, the autoencoder receives the input label mask and outputs the output label mask.

[0034] Then, in step 510, in this embodiment, the training set of the segmentation dataset (images and their corresponding label masks) is applied to the segmenter. Since label masks are available for these segmentation datasets, the label masks of the images can be compared with the output of the segmenter (step 512). In one embodiment, the segmenter system under training can then check (step 514) whether the label masks in the training set match the output of the segmenter sufficiently. If the image is not an anomalous image, a well-trained segmenter can output label masks of images in the segmentation dataset that are similar to the label masks of the segmentation dataset, so this comparison is a valid test of whether the segmenter is trained.

[0035] In one embodiment, if the segmenter is not adequately tuned, feedback can be provided (step 520), and the process returns to step 510 for additional training. In another embodiment, if the segmenter is adequately tuned, the process can move to step 516, where a validation set is applied to the segmenter, and further training can be provided if needed (step 518), and the training process is complete (step 522).

[0036] Figure 6A diagram of system 600 in an embodiment is shown, in which one or more arithmetic logic circuits at least partially process an input image, generate a label mask of the input image, apply the label mask to an autoencoder, compare the label mask with the output of the autoencoder, and take an action based on the comparison result. In the illustrated embodiment, computational entity 602, such as a computer system or computer server, includes a trained segmenter 604, an autoencoder 606, a comparator 608, memory for the input image 610, memory for the label mask 612, and memory for a quality indicator 614. In one embodiment, autoencoder 606 includes an encoder 620, memory for a latent representation 622, and a decoder 624. In one embodiment, autoencoder 606 is a variational autoencoder trained on shape features. In one embodiment, computational entity 602 can be used for multiple images, some of which may be three-dimensional images and some of which may be two-dimensional images. In one embodiment, computational entity 602 is or includes a computer system or processing unit, such as a combination of... Figures 8-12 The one or more computer systems or processing units described.

[0037] In this embodiment, system 600 follows... Figure 5 The training process described herein is performed, during which the trained segmenter 604 receives the input image 610 without requiring labels or ground truth about the objects in the input image 610. In one embodiment, the trained segmenter 604 outputs a label mask 612 as the output of system 600. In one embodiment, the label mask 612 informs the user of the objects present in the input image 610 and the boundaries of those objects. In one embodiment, the label mask 612 is applied to the input layer of an autoencoder, which in turn outputs an autoencoder output 630 in the form of a label mask. In one embodiment, a comparator 608 compares the autoencoder output 630 with the label mask 612 and outputs a quality indicator 614. In one embodiment, the quality indicator 614 may be an alert, flag, color, and / or value displayed to the user as part of the user interface to indicate whether the label mask 612 is likely a good segmentation of the input image 610. For poor segmentations, the input and output of the autoencoder 606 are more likely to be mismatched, but for good segmentations, they are more likely to match. In one embodiment, when the autoencoder 606 is trained on shape features, the latent representation in the autoencoder 606 is likely to represent the shape features.

[0038] In one embodiment, comparator 608 includes logic, a general-purpose ALU, fixed-function hardware, an FPGA, etc. Although in Figure 6While shown as a separate component, in one embodiment, comparator 608 may be implemented in combination with a trained segmenter 604 and an autoencoder 606. For example, dedicated hardware components may process the input image to segment it and form a label mask, process the label mask by the autoencoder, and compare the label mask with the autoencoder output.

[0039] Figure 7 A diagram of an autoencoder 702 in an embodiment is shown, wherein one or more arithmetic logic circuits at least partially encode an input label mask into a latent representation and decode the latent representation into an output label mask. In the illustrated embodiment, the autoencoder 702 is a neural network comprising several layers, including an input layer 704 coupled to a first hidden layer 706, the first hidden layer 706 coupled to a latent layer 708, the latent layer 708 coupled to a second hidden layer 710, and the second hidden layer 710 coupled to an output layer 712. In one embodiment, the autoencoder 702 is or includes a computer system or processing unit, such as a combination of... Figures 8-12 The one or more computer systems or processing units described.

[0040] Specific example implementation

[0041] In the specific examples described in this section, a system is described in various embodiments in which a variational autoencoder is trained with a set of known good label masks (label masks), and the variational autoencoder is trained to encode each input label mask into a latent representation and then decode the output label mask from that latent representation, wherein the training is designed to reduce the loss from encoding and decoding. The latent representation represents shape features.

[0042] Using shape features in object segmentation often provides stable shapes under various conditions, especially in volumetric images (i.e., data corresponding to voxel values ​​in a 3D image). Shape features can provide strong prior information for judging the quality of segmentation results; poor segmentation in the label mask often has poor shapes, and vice versa. As used in this paper, "prior" such as "shape prior" refers to data representing some assumptions made, such as a plausible prior concept of shape in machine learning or a prior concept of shape determined in other ways.

[0043] Modeling shape priors from the segmentation mask space is faster and computationally simpler than doing so in the image space, and shape priors can be shared across different datasets, where features such as image intensity are influenced by many factors. This means that shape features can handle not only rare events but also diverse data distributions in the image space, offering good generalization potential in transitive learning. In one embodiment, a VAE is used to capture shape features. In one embodiment, the VAE is trained against a ground truth label mask, and then the value of the loss function is computed as the shape feature of the segmentation result when the VAE network is tested. In one embodiment, poor segmentation results with poor shapes are rare events for the VAE after it has been trained because it is trained only with ground truth labels (known good label masks under a normal shape distribution). Therefore, they will have larger loss values. In this sense, the segmentation system takes advantage of the fact that the learning process performs poorly on rare events. Formally, during training, the loss function (called the variational lower bound) can be optimized to approximate the function logP(Y). Therefore, after training, the value of the loss function given the segmentation result is... near Therefore, it is a good definition of shape characteristics.

[0044] In one embodiment, the VAE-based quality inspection system outputs to an alarm system to signal poor segmentation. In another embodiment, further steps are taken in response to the detection of poor segmentation. These systems can be used to predict the quality of segmentation results quite well. In one embodiment, to verify the effectiveness of the alarm system, it is tested on multiple segmentation processes. These segmentation processes can be trained on one dataset and tested on several other datasets to simulate situations where rare events occur.

[0045] To define the logic of a portion of the process, the dataset is represented as (X, Y), where X is the image set and Y is the label mask of X, i.e., the label mask of known good (ground truth) segmentation of the images in X. The dataset (X, Y) can be divided into a training set (X... T Y T ) and verification set (X) V Y V Here, F() refers to the segmentation process trained on X. In one embodiment, Y is used. V To evaluate F in X V While performance is not guaranteed, as explained herein, in one embodiment, the segmentation system can operate without requiring Y. V The evaluation is performed under these conditions. Formally, the training system attempts to find a function L that satisfies Equation 1.

[0046] Λ(F(X),Y)=L(F,X;ω) (Equation 1)

[0047] In Equation 1, Λ is a function used to calculate the similarity of the segmentation result F(X) with respect to the ground truth Y, i.e., the quality of F(X). Failure may occur when X is a rare event. However, due to the complexity of the image spatial structure, it is difficult to detect whether the image X is within the distribution range of the training data, which is precisely what F is trained to learn. The properties of F can be encoded by sampling the parameters of F and calculating the uncertainty of the output. This uncertainty does contribute to the prediction quality, but its performance is strongly dependent on F. It requires F to have a Bayesian structure, but a well-trained F does not require such a constraint. The uncertainty is mainly distributed on the boundaries of the segmentation prediction. Adding this constraint changes Equation 1 to Equation 2.

[0048] Λ(F(X),Y)=L(F(X);ω) (Equation 2)

[0049] In one embodiment, this constraint is added to utilize information from F and X, but not directly. A specific process can be implemented using a two-step approach, where the first step is to encode the segmentation result F(X) into a feature space, and the second step is to learn from the feature space to predict the quality of F(X). Shape features can be captured from F(X), denoted here as S(F(X); θ). In this case, Equation 2 becomes Equation 3.

[0050] Λ(F(X),Y)=L(S(F(X);θ);ω) (Equation 3)

[0051] Shape features of variational autoencoders

[0052] In one embodiment, shape features are derived from a truth mask Y∈Y T The trained VAE captures and segments the mask shape as its distribution in volume form. Assuming the normal label Y follows a certain distribution P(Y), for the predicted mask... Its quality should be consistent with In one embodiment, the segmentation system estimates the function P(Y). Because a VAE is used, an estimation function Q(z) is found that minimizes the difference between Q(z) and P(z|Y), where z is the variable encoded by Y into the latent space, as shown in Equation 4 for optimization.

[0053] KΛ[Q(z)||P(z|Y)]=E z～Q [log Q(z)-log P(z|Y)](Equation 4)

[0054] In equation 4, KΛ is the Kullback-Leibler divergence. Replacing Q(z) with Q(z|Y) and rearranging the terms, we finally derive the core equation of the VAE, as shown in equation 5. log P(Y) - KΛ[Q(z|Y)||P(z|Y)]=E z～Q [log P(Y|z)]-KΛ[Q(z|Y)||P(z)] (Equation 5)

[0055] In Equation 5, P(z) is the prior distribution chosen for z, typically a Gaussian distribution, and in this embodiment, Q(z|Y) and P(Y|z) correspond to the encoder and decoder, respectively. Once Y is given, log P(Y) is a constant. Therefore, by optimizing the variational lower bound of log P(Y) on the right-hand side of Equation 5, this optimizes KΛ[Q(z)||P(z|Y)]. In one embodiment, the attention is P(Y). By swapping the second term on the left-hand side of Equation 5 with all the terms on the right-hand side of Equation (5), we obtain Equation 6, which shows that the training process minimizes the number there.

[0056] E Y～YT KΛ[Q(z|Y)||P(z|Y)]

[0057] =E Y～YT log P(Y)–E z～Q [log P(Y|z)]+KΛ[Q(z|Y)||P(z)] (Equation 6)

[0058] =E Y～YT log P(Y)–S(Y;θ)

[0059] In equation 6, for the sake of simplicity, E z～Q [log P(Y|z)]+KΛ[Q(z|Y)||P(z)] is represented as S(Y;θ). As shown in Equation 6, in one embodiment, the training process is actually learning a function to best fit log P(Y) on the distribution of Y. In one embodiment, after training the VAE, S(Y;θ) becomes a natural approximation of log P(Y). Therefore, in one embodiment, S(Y;θ) can be used as a representation of shape features.

[0060] In one embodiment, the Dice loss function, which is useful for medical segmentation tasks, is used when training the VAE. In one embodiment, the final form of S is shown in Equation 7.

[0061]

[0062] In one embodiment, the encoder μ, ∑, and decoder g of Equation 7 are controlled by θ, and λ is a coefficient balancing these two terms. The first term is the coefficient of Dice between Y and g(z), ranging from 0 to 1, and equal to 1 if Y = g(z). In one embodiment, the shape feature S(Y; θ) representing Equation 7 indicates the probability of predicting the mask after training the VAE with data having only a normal shape. If a smaller reconstruction error can be obtained and the prediction mask is closer to the prior distribution in the latent space, then the prediction mask is... It tends to be more likely to be in a normally shaped distribution because it always maintains... On the other hand, for high But low In such cases, it will impose a significant penalty on the objective function (see Equation 6), and this is less likely to occur for well-trained VAEs.

[0063] Predicting quality using shape features

[0064] In one embodiment, the shape features are typically good enough for reliable quality assessment. One reason is that for a segmentation result F(X), the higher log P(F(X)), the better the shape F(X), and therefore the higher L(F(X),Y). In one embodiment, formally, utilizing the shape features captured from the VAE, the segmentation system can predict the quality of the segmentation by fitting a function L according to Equation 8.

[0065] L(F(X),Y)=L(S(F(X);θ);ω) (Equation 8)

[0066] In one embodiment, by using training data Y T The VAE is trained using labels to learn the parameters θ in Equation 8, and then fixed in the second step. In one embodiment, function L is chosen as a simple linear model, in which case the energy function for optimizing the segmentation system is shown in Equation 9.

[0067] E(S(F(X);θ);a,b)=||aS(F(X);θ)+bL(F(X),Y)|| 2 (Equation 9)

[0068] In one embodiment, a linear regression model is used, and a strong linear correlation is shown between shape features and the quality of the segmentation results. In equations 8 and 9, L is the coefficient of Dice, as shown in equation 10.

[0069]

[0070] Training process

[0071] In one embodiment, in the first step of the training process, the VAE is trained using only the labels from the training data. Then, in another embodiment, θ is fixed in the second step of the training process. In one embodiment, to learn a and b, one approach is to optimize the energy function of Equation 9 using the segmentation results on the training data, as shown in Equation 11.

[0072]

[0073] In one embodiment, the segmentation system uses a segmentation process F to learn a and b, which is referred to herein as the "preparation process". In one embodiment, if F is in X T If trained on top of this, the quality of F(X) is expected to always be high, thus providing less information for regressing a and b. To overcome this, in one embodiment, the segmentation system trains X... T The F-axis uses a jackknifing training strategy, where not all X values ​​are trained using this strategy. T Both are used at the beginning. In one embodiment, the process includes first setting X... T Divided into X T 1 and X T 2 Then in X T \X T 1 and X T \X T 2 Two versions of F are trained, such as F1 and F2. In one embodiment, the optimization function is then changed to the optimization function of Equation 12.

[0074]

[0075] In one embodiment, using the above process, the segmentation system can address the aforementioned problem by simulating the performance of F on a test set. In one embodiment, an accurate approach is to perform "leave-one-out" training on F, but if the time consumption is unacceptable, two-fold separation (X) can be used. T Dividing the data into two equal-sized subsamples (one for training and one for validation) is sufficiently effective. In one embodiment, when training is complete, the trained segmenter can be tested using the segmentation algorithm F and the data X to predict quality using the quality metric of Equation 13.

[0076] Q = aS(F(X); θ) + b (Equation 13)

[0077] As now described, in one embodiment, the segmentation system obtains a reliable prediction of the quality of the segmentation results. In one embodiment, when testing the segmentation process on other datasets, an alarm system can be used, which has good quality assessment capabilities and transferability compared to uncertainty-based methods, direct regression methods, and other methods.

[0078] Images from public medical datasets were tested, including 3D abdominal CT images of the portal venous phase, each with full annotation of the pancreatic region. The CT scans were performed at a resolution of 512 × 512 × h voxels, with varying voxel sizes. In one embodiment, other images were used instead, or other methods were employed, and the procedures described herein can also be used with images other than medical images. Public medical datasets used for testing included the NIH Pancreatic CT Dataset (NIH), the Medical Segmentation Decathlon (MSD), and the Multi-Map Labelling Challenge (MLC).

[0079] The NIH Pancreatic CT Dataset (NIH) comprises 82 abdominal 3D CT scans from 53 male and 27 female participants (Roth et al., 2015). Participants were selected by radiologists from patients without significant abdominal pathology or pancreatic cancer lesions. The Medical Segmentation Decathlon (MSD) collected 420 abdominal 3D CT scans (281 training scans and 139 test scans) from Memorial Sloan Kettering Cancer Center. Each participant had a cancerous lesion within the pancreatic region. The Multi-Atlas Labelling Challenge (MLC) provided 50 abdominal CT scans (30 training, 20 test) randomly selected from a combination of ongoing colorectal cancer chemotherapy trials and retrospective abdominal hernia studies.

[0080] In one embodiment, the example segmentation process may include V-Net (Milletari et al., 2016), 3DCoarse2Fine (Zhu et al., 2018), Deeplabv3 (Chen et al., 2018), and 3DCoarse2Fine with a Bayesian structure (Kwon et al., 2018). The first two are based on 3D networks, while Deeplab is based on 2D. In one embodiment, the 3D Coarse2Fine with a Bayesian structure is used for comparison with uncertainty-based methods and then represented as a Bayesian neural network (BNN).

[0081] For data preprocessing, since the voxel size varies for each case, which affects the prediction of pancreatic shape and segmentation, the test included a step of resampling all CT scans and annotation masks to 1mm cubic voxels. To train the VAE, a simple alignment was applied to the annotation mask. A cubic bounding box, large enough to contain the entire pancreatic region and centered at the pancreatic centroid, was used. The volume and label masks were cropped and resized to a fixed size of 128×128×128 voxels. This simple alignment was effective for these test cases because human posture is typically fixed during CT scans, preventing organ rotation or severe deformation. For segmentation prediction, the test also cropped the prediction foreground, resized it to 128×128×128 voxels, and fed it into the VAE to capture shape features, similar to... Figure 6 As shown. As used in this article, "annotation mask" can refer to or be used instead of a label mask containing truth details. Truth details can be obtained from manual input after examining the image.

[0082] In one embodiment, during training, rotations of -10 degrees, 0 degrees, and 10 degrees along the x, y, and z axes, respectively, and random translations of less than 5 voxels on the annotation mask are used as data augmentation. This slight perturbation can enhance the data distribution while preserving the alignment properties of the annotation mask. In one embodiment, a suitable dimension for the latent space containing the latent representations in the VAE could be 128. VAEs with latent spaces of different dimensions may have different capabilities in terms of quality evaluation. In one embodiment, the hyperparameter λ in the objective function of the VAE is set to λ = 2. -5 To balance the small value of the Dice loss and the large KL divergence. In the test instance, the network was trained using an SGD optimizer with a batch size of 4, and the learning rate for training the VAE was fixed at 0.1. In one embodiment, the framework and other baseline models can be built using TensorFlow, and experiments can be run on an nVidiaTesla V100 GPU. In one example, the first training step was completed in 20,000 iterations, taking approximately 5 hours.

[0083] In one embodiment, after training, the segmentation system can be used for segmentation, and the output of the trained segmenter is provided to the VAE for quality evaluation. The VAE can predict the quality of the segmentation results without using a ground truth mask. Shape features help predict the quality of the segmentation results. To capture shape features, in one embodiment, the VAE is trained using only a ground truth mask and relies on rare events to obtain a larger loss function value, and successfully detects out-of-distribution shapes based on the loss function value during test time.

[0084] As described above, the segmentation system in the embodiments includes a trained segmenter that receives an image and determines a segmentation, which can be output as a label mask. In one embodiment, an autoencoder trained on shape features takes the label mask as its input and outputs a resulting label mask formed as the autoencoder encodes the label mask into a latent representation and then decodes that latent representation. In one embodiment, there is less loss during autoencoding when segmentation is good and more loss when segmentation is poor, where the quality of segmentation represents how well the boundaries of objects in the image are correctly identified. In one embodiment, a comparator checks whether the boundaries between objects in the image are correctly identified by the autoencoder's output to its input. If the boundaries of objects in the image are correctly identified, these boundaries will be close to the autoencoder's output. When significant differences between the boundaries identified from the autoencoder's output exceed an acceptable threshold, an alert can be provided to the system's user (e.g., a notification or change in the interface).

[0085] In a particular embodiment, the image is processed using a first neural network to determine a first segmentation of the image (i.e., an output indicating the boundaries of objects in the image), while in other embodiments, it is used for segmentation. In one embodiment, the segmentation is passed through a variational autoencoder (VAE) to obtain a second segmentation. In one embodiment, the first and second segments are compared, which can be done using a loss function. If the comparison shows that the first and second segments are very close, then the segmenter is likely working well. If they are significantly different, then there may be a problem. In one embodiment, the autoencoder is trained using the segmentation of an image known to be well-segmented. Other embodiments may be apparent upon reading this description.

[0086] Figure 8A parallel processing unit (“PPU”) 800 according to one embodiment is illustrated. In one embodiment, the PPU 800 is configured with machine-readable code, which, if executed by the PPU, causes the PPU to perform some or all of the processes and techniques described herein. In one embodiment, the PPU 800 is a multi-threaded processor implemented on one or more integrated circuit devices and utilizes multithreading as a latency-hiding technique, designed to process computer-readable instructions (also referred to as machine-readable instructions or simply instructions) on multiple threads in parallel. In one embodiment, a thread refers to a thread of execution and is an instance of a set of instructions configured to be executed by the PPU 800. In one embodiment, the PPU 800 is a graphics processing unit (“GPU”) configured to implement a graphics rendering pipeline for processing three-dimensional (“3D”) graphics data to generate two-dimensional (“2D”) image data for display on a display device, such as a liquid crystal display (LCD) device. In one embodiment, the PPU 800 is used to perform computations such as linear algebra operations and machine learning operations. Figure 8 An exemplary parallel processor is shown for illustrative purposes only and should be interpreted as a non-limiting example of a processor architecture contemplated within the scope of this disclosure, and any suitable processor may be used to complement and / or replace this processor.

[0087] In one embodiment, one or more PPUs are configured to accelerate high-performance computing (HPC), data center, and machine learning applications. In one embodiment, PPU 800 is configured to accelerate deep learning systems and applications, including, but not limited to, the following non-limiting examples: autonomous vehicle platforms, deep learning, high-precision speech, image, and text recognition systems, intelligent video analytics, molecular simulation, drug discovery, disease diagnosis, weather forecasting, big data analytics, astronomy, molecular dynamics simulation, financial modeling, robotics, factory automation, real-time language translation, online search optimization, and personalized user recommendations.

[0088] In one embodiment, PPU 800 includes an input / output (“I / O”) unit 806, a front-end unit 810, a scheduler unit 812, a job allocation unit 814, a hub 816, a crossbar (“Xbar”) 820, one or more general-purpose processing clusters (“GPCs”) 818, and one or more partitioning units 822. In one embodiment, PPU 800 is connected to a host processor or other PPU 800 via one or more high-speed GPU interconnects 808. In one embodiment, PPU 800 is connected to a host processor or other peripheral device via interconnect 802. In one embodiment, PPU 800 is connected to local memory including one or more memory devices 804. In one embodiment, local memory includes one or more dynamic random access memory (“DRAM”) devices. In one embodiment, one or more DRAM devices are configured and / or configurable as a high-bandwidth memory (“HBM”) subsystem, wherein multiple DRAM dies are stacked within each device.

[0089] The high-speed GPU interconnect 808 can refer to a wire-based multi-channel communication link used by the system to extend and include one or more PPUs 800 in conjunction with one or more CPUs, supporting cache coherency between the PPUs 800 and the CPU, as well as CPU master control. In one embodiment, data and / or commands are sent from / from the high-speed GPU interconnect 808 to other units of the PPU 800 via a hub 816, such as one or more copy engines, video encoders, video decoders, power management units, and other components. Figure 8 Components that may not be explicitly shown.

[0090] In one embodiment, the I / O unit 806 is configured to access the host processor via the system bus 802. Figure 8 (Not shown) Sending and receiving communications (e.g., commands, data). In one embodiment, I / O unit 806 communicates directly with the host processor via system bus 802 or via one or more intermediate devices (e.g., memory bridges). In one embodiment, I / O unit 806 may communicate with one or more other processors (e.g., one or more PPUs 800) via system bus 802. In one embodiment, I / O unit 806 implements a Peripheral Component Interconnect Fast (“PCIe”) interface for communication via the PCIe bus. In one embodiment, I / O unit 806 implements an interface for communication with external devices.

[0091] In one embodiment, I / O unit 806 decodes packets received via system bus 802. In one embodiment, at least some packets represent commands configured to cause PPU 800 to perform various operations. In one embodiment, I / O unit 806 sends the decoded commands to various other units of PPU 800 specified by the commands. In one embodiment, the commands are sent to front-end unit 810 and / or to hub 816 or other units of PPU 800, such as one or more copy engines, video encoders, video decoders, power management units, etc. Figure 8 (Not explicitly shown). In one embodiment, I / O unit 806 is configured to route communication between and within the various logical units of PPU 800.

[0092] In one embodiment, a command stream is encoded in a program buffer executed by the host processor, and the buffer provides the workload to the PPU 800 for processing. In one embodiment, the workload includes instructions and data to be processed by these instructions. In one embodiment, the buffer is an area in memory that can be accessed (e.g., read / write) by both the host processor and the PPU 800, and the host interface unit can be configured to access the buffer in system memory connected to the system bus 802 via memory requests sent through the system bus 802 via the I / O unit 806. In one embodiment, the host processor writes a command stream to the buffer and then sends a pointer to the start of the command stream to the PPU 800, such that the front-end unit 810 receives pointers to one or more command streams and manages one or more command streams, reads commands from the streams, and forwards the commands to the respective units of the PPU 800.

[0093] In one embodiment, front-end unit 810 is coupled to scheduler unit 812, which configures various GPCs 818 to handle tasks defined by one or more flows. In one embodiment, scheduler unit 812 is configured to track status information associated with various tasks managed by scheduler unit 812, wherein the status information may indicate which GPC 818 a task is assigned to, whether the task is active or inactive, the priority associated with the task, etc. In one embodiment, scheduler unit 812 manages the execution of multiple tasks on one or more GPCs 818.

[0094] In one embodiment, scheduler unit 812 is coupled to job allocation unit 814, which is configured to assign tasks for execution on GPC 818. In one embodiment, job allocation unit 814 tracks multiple scheduled tasks received from scheduler unit 812 and manages a pending task pool and an active task pool for each GPC 818. In one embodiment, the pending task pool may include several time slots (e.g., 32 time slots) containing tasks assigned to a specific GPC 818 for processing; the active task pool may include several time slots (e.g., 4 time slots) for tasks being actively processed by GPC 818, such that when GPC 818 completes the execution of a task, the task is evicted from the active task pool of GPC 818, and one of the other tasks from the pending task pool is selected and scheduled for execution on GPC 818. In one embodiment, if an active task is idle on GPC 818, for example while waiting for data dependencies to be resolved, the active task is evicted from GPC 818 and returned to the task pool, while another task in the task pool is selected and scheduled to be executed on GPC 818.

[0095] In one embodiment, the work allocation unit 814 communicates with one or more GPCs 818 via XBar 820. In one embodiment, XBar 820 is an interconnect network that couples a plurality of units of PPU 800 to other units of PPU 800, and can be configured to couple the work allocation unit 814 to a specific GPC 818. Although not explicitly shown, one or more other units of PPU 800 may also be connected to XBar 820 via hub 816.

[0096] These tasks are managed by scheduler unit 812 and dispatched to GPC 818 by work allocation unit 814. GPC 818 is configured to process tasks and generate results. Results can be consumed by other tasks within GPC 818, routed to different GPC 818 via XBar 820, or stored in memory 804. Results can be written to memory 804 via partitioning unit 822, which implements a memory interface for reading / writing data to / from memory 804. Results can be sent to another PPU 804 or CPU via high-speed GPU interconnect 808. In one embodiment, PPU 800 includes U partitioning units 822, which is equal to the number of independent and different memory devices 804 coupled to PPU 800. The following will be combined with... Figure 10 Partition unit 822 is described in more detail.

[0097] In one embodiment, the host processor executes a driver kernel that implements an application programming interface (“API”) that enables one or more applications executing on the host processor to schedule operations for execution on the PPU 800. In one embodiment, the PPU 800 executes multiple computing applications simultaneously, and the PPU 800 provides isolation, quality of service (“QoS”), and independent address spaces for the multiple computing applications. In one embodiment, an application generates instructions (e.g., in the form of API calls) that cause the driver kernel to generate one or more tasks for the PPU 800 to execute, and the driver kernel outputs the tasks to one or more streams being processed by the PPU 800. In one embodiment, each task includes one or more associated thread groups, which may be referred to as a warp. In one embodiment, a warp includes multiple associated threads that can be executed in parallel (e.g., 32 threads). In one embodiment, a cooperating thread may refer to multiple threads that include instructions for executing tasks and instructions for exchanging data via shared memory. According to one embodiment, in conjunction with... Figure 10 Threads and cooperative threads are described in more detail.

[0098] Figure 9 A GPC 900 according to one embodiment is shown, for example. Figure 8 The PPU 800 illustrates a GPC. In one embodiment, each GPC 900 includes multiple hardware units for processing tasks, and each GPC 900 includes a pipeline manager 902, a pre-raster operation unit (“PROP”) 904, a raster engine 908, a job distribution cross unit (“WDX”) 916, a memory management unit (“MMU”) 918, one or more data processing clusters (“DPC”) 906, and any suitable combination thereof. It should be understood that... Figure 9 The GPC 900 may be replaced or added to Figure 9 Other hardware units of the unit shown.

[0099] In one embodiment, the operation of GPC 900 is controlled by pipeline manager 902. Pipeline manager 902 manages the configuration of one or more DPCs 906 for processing tasks assigned to GPC 900. In one embodiment, pipeline manager 902 configures at least one of one or more DPCs 906 to implement at least a portion of the graphics rendering pipeline. In one embodiment, DPC 906 is configured to execute vertex shader procedures on a programmable streaming multiprocessor (“SM”) 914. In one embodiment, pipeline manager 902 is configured to route packets received from job assignments to appropriate logic units within GPC 900, and some packets may be routed to fixed-function hardware units in PROP 904 and / or raster engine 908, while other packets may be routed to DPCs 906 for processing by primitive engine 912 or SM 914. In one embodiment, pipeline manager 902 configures at least one of one or more DPCs 906 to implement neural network models and / or computation pipelines.

[0100] In one embodiment, PROP unit 904 is configured to route data generated by raster engine 908 and DPC 906 to a raster operation (“ROP”) unit in a memory partition unit, as described in detail above. In one embodiment, PROP unit 904 is configured to perform color blending optimization, organize pixel data, perform address translation, etc. In one embodiment, raster engine 908 includes several fixed-function hardware units configured to perform various raster operations, and raster engine 908 includes a setup engine, a coarse raster engine, a culling engine, a clipping engine, a fine raster engine, and a tile aggregation engine, and any suitable combination of the above engines. In one embodiment, the setup engine receives transformed vertices and generates plane equations associated with the geometric primitives defined by the vertices, which are sent to the coarse raster engine to generate primitive overlay information (e.g., x, y overlay masks of tiles); the output of the coarse raster engine is sent to the culling engine, where fragments associated with primitives that fail the z-test are culled, and to the clipping engine, where fragments located outside the view frustum are clipped. In one embodiment, the fragments remaining after clipping and culling can be passed to a fine raster engine to generate properties of the pixel fragments based on the plane equations generated by the setup engine. In one embodiment, the output of the raster engine 908 includes fragments processed by any suitable entity, such as fragments processed by the fragment shader implemented in DPC 906.

[0101] In one embodiment, each DPC 906 included in the GPC 900 includes an M-Pipeline Controller (“MPC”) 910, a Primitive Engine 912, one or more SMs 914, and any suitable combination thereof. In one embodiment, the MPC 910 controls the operation of the DPC 906, routing packets received from the Pipeline Manager 902 to the appropriate units within the DPC 906. In one embodiment, packets associated with vertices are routed to the Primitive Engine 912, which is configured to retrieve vertex attributes associated with vertices from memory; conversely, packets associated with shader programs may be sent to the SMs 914.

[0102] In one embodiment, the SM 914 includes a programmable streaming processor configured to process tasks represented by multiple threads. In one embodiment, the SM 914 is multithreaded and configured to execute multiple threads (e.g., 32 threads) simultaneously from a specific thread group and implements a SIMD (Single Instruction, Multiple Data) architecture, where each thread in a group of threads (thread bundles) is configured to process different datasets based on the same instruction set. In one embodiment, all threads in the thread group execute the same instructions. In another embodiment, the SM 914 implements a SIMT (Single Instruction, Multiple Threads) architecture, where each thread in the thread group is configured to process different datasets based on the same instruction set, but allows individual threads in the thread group to diverge during execution. In one embodiment, a program counter, call stack, and execution state are maintained for each thread bundle, enabling concurrency between the thread bundle and serial execution within the thread bundle when threads diverge. In yet another embodiment, a program counter, call stack, and execution state are maintained for each individual thread, thereby achieving equal concurrency among all threads within and between thread bundles. In one embodiment, an execution state is maintained for each individual thread, and threads executing the same instructions can be converged and executed in parallel for maximum efficiency. In one embodiment, SM 914 is described in more detail below.

[0103] In one embodiment, the MMU 918 provides an interface between the GPC 900 and the memory partitioning unit, and provides virtual address to physical address translation, memory protection, and memory request arbitration. In one embodiment, the MMU 918 provides one or more Translation Backing Buffers (“TLBs”) for performing virtual address to physical address translation in memory.

[0104] Figure 10A memory partitioning unit of a PPU according to one embodiment is illustrated. In one embodiment, the memory partitioning unit 1000 includes a raster operation (“ROP”) unit 1002, a secondary (“L2”) cache 1004, a memory interface 1006, and any suitable combination thereof. The memory interface 1006 is coupled to memory. The memory interface 1006 may implement a 32, 64, 128, 1024-bit data bus, etc., for high-speed data transfer. In one embodiment, the PPU includes U memory interfaces 1006, one memory interface 1006 per pair of partitioning units 1000, wherein each pair of partitioning units 1000 is connected to a corresponding memory device. For example, the PPU may be connected to up to Y memory devices, such as high-bandwidth memory stacks or graphics dual data rate, version 5, synchronous dynamic random access memory (“GDDR5 SDRAM”).

[0105] In one embodiment, memory interface 1006 implements an HBM2 memory interface, and Y is equal to half of U. In one embodiment, the HBM2 memory stack and PPU reside on the same physical package, providing significant power and area savings compared to conventional GDDR5 SDRAM systems. In one embodiment, each HBM2 stack comprises four memory dies and Y equals 4, wherein each HBM2 stack includes two 128-bit channels per die, for a total of eight channels and a 1024-bit data bus width.

[0106] In one embodiment, the memory supports single error correction, double error detection (“SECDED”) error correction codes (“ECC”) to protect data. ECC provides enhanced reliability for data corruption-sensitive computing applications. Reliability is particularly critical in large-scale cluster computing environments where the PPU handles very large datasets and / or long-running applications.

[0107] In one embodiment, the PPU implements a multi-level memory hierarchy. In one embodiment, the memory partitioning unit 1000 supports unified memory to provide a single, unified virtual address space for the CPU and PPU memories, thereby enabling data sharing between virtual memory systems. In one embodiment, the frequency of PPU accesses to memory located on other processors is tracked to ensure that memory pages are moved to the physical memory of the PPU that accesses pages more frequently. In one embodiment, the high-speed GPU interconnect 808 supports address translation services, allowing the PPU to directly access the CPU's page tables and providing full access to the CPU's memory by the PPU.

[0108] In one embodiment, the replication engine transfers data between multiple PPUs or between a PPU and a CPU. In one embodiment, the replication engine can generate a page fault for an address not mapped to a page table, then the memory partitioning unit 1000 servicing the page fault, mapping the address to a page table, and then the replication engine performs the transfer. In one embodiment, multiple replication engines operating on fixed memory (i.e., non-pageable) across multiple processors substantially reduce available memory. In one embodiment, as a hardware page fault occurs, the address can be passed to the replication engine regardless of whether the memory page resides, and the replication process is transparent.

[0109] According to one embodiment, from Figure 8 Data from memory or other system memory is retrieved by memory partitioning unit 1000 and stored in an L2 cache 1004 located on-chip and shared among the various GPCs. In one embodiment, each memory partitioning unit 1000 includes at least a portion of an L2 cache 960 associated with a corresponding memory device. In one embodiment, lower-level caches are implemented in individual cells within a GPC. In one embodiment, each of the SM1040s may implement a Level 1 (“L1”) cache, wherein the L1 cache is a dedicated memory for a particular SM 1040, and data from the L2 cache 1004 is retrieved and stored in each L1 cache for processing in a functional unit of the SM1040. In one embodiment, the L2 cache 1004 is coupled to memory interface 1006 and XBar 820.

[0110] In one embodiment, ROP unit 1002 performs graphic raster operations related to pixel color, such as color compression, pixel blending, etc. In one embodiment, ROP unit 1002, together with raster engine 1025, performs depth testing, receiving the depth of sample locations associated with pixel fragments from the culling engine of raster engine 1025. In one embodiment, the depth of the sample location associated with the fragment is tested relative to the corresponding depth in the depth buffer. In one embodiment, if the fragment passes the depth test for the sample location, ROP unit 1002 updates the depth buffer and sends the result of the depth test to raster engine 1025. It should be understood that the number of partition units 1000 may differ from the number of GPCs; therefore, in one embodiment, each ROP unit 1002 may be coupled to each GPC. In one embodiment, ROP unit 1002 tracks packets received from different GPCs and determines which GPC the result generated by ROP unit 1002 is routed to via Xbar.

[0111] Figure 11 A streaming multiprocessor according to one embodiment is shown, for example... Figure 9A streaming multiprocessor. In one embodiment, the SM 1100 includes: an instruction cache 1102; one or more scheduler units 1104; a register file 1108; one or more processing cores 1110; one or more special function units (“SFUs”) 1112; one or more load / store units (“LSUs”) 1114; an interconnect network 1116; a shared memory / L1 cache 1118; and any suitable combination thereof. In one embodiment, a work allocation unit assigns tasks to be executed on a GPC of the PPU, and each task is assigned to a specific DPC within the GPC, and if the task is associated with a shader program, the task is assigned to the SM 1100. In one embodiment, the scheduler unit 1104 receives tasks from the work allocation unit and manages instruction scheduling for one or more thread blocks assigned to the SM 1100. In one embodiment, the scheduler unit 1104 schedules the thread blocks for execution into thread bundles of parallel threads, wherein each thread block is assigned at least one thread bundle. In one embodiment, each thread bundle executes a thread. In one embodiment, scheduler unit 1104 manages multiple different thread blocks, assigns thread bundles to different thread blocks, and then dispatches instructions from multiple different cooperative groups to various functional units (e.g., core 1110, SFU 1112, and LSU 1114) in each clock cycle.

[0112] Collaboration groups can refer to a programming model used to organize groups of communicating threads, allowing developers to express the granularity at which threads are communicating, enabling richer and more efficient parallel decompositions. In one embodiment, a collaboration-initiated API supports synchronization between blocks of threads to execute parallel algorithms. In another embodiment, a conventional programming model provides a single, simple structure for synchronizing collaborative threads: a barrier (e.g., the `syncthreads()` function) across all threads in a block. However, programmers often want to define thread groups at a granularity smaller than that of a single thread block and synchronize within those defined groups, enabling greater performance, design flexibility, and software reuse in the form of a collective group-wide function interface. Collaboration groups allow programmers to explicitly define thread groups at both sub-block (i.e., as small as a single thread) and multi-block granularity, and perform collective operations (such as synchronization) on the threads within the collaboration group. This programming model supports clean composition across software boundaries, so libraries and utility functions can be safely synchronized in their local context without assumptions about convergence. Collaborative group primitives support new modes of collaborative parallelism, including producer-consumer parallelism, opportunistic parallelism, and global synchronization across the entire thread block grid.

[0113] In one embodiment, dispatch unit 1106 is configured to send instructions to one or more functional units, and scheduler unit 1104 includes two dispatch units 1106, enabling the dispatch of two different instructions from the same thread bundle during each clock cycle. In one embodiment, each scheduler unit 1104 includes a single dispatch unit 1106 or additional dispatch units 1106.

[0114] In one embodiment, each SM 1100 includes a register file 1108 that provides a set of registers for functional units of the SM 1100. In one embodiment, the register file 1108 is partitioned among each functional unit such that each functional unit is allocated a dedicated portion of the register file 1108. In one embodiment, the register file 1108 is partitioned among different thread bundles being executed by the SM 1100, and the register file 1108 provides temporary storage for operands in the data paths connected to the functional units. In one embodiment, each SM 1100 includes a plurality of L processing cores 1110. In one embodiment, the SM 1100 includes a large number (e.g., 128 or more) of different processing cores 1110. In one embodiment, each core 1110 includes a fully pipelined, single-precision, double-precision, and / or mixed-precision processing unit that includes floating-point arithmetic logic units and integer arithmetic logic units. In one embodiment, the floating-point arithmetic logic unit implements the IEEE 754-2008 standard for floating-point arithmetic. In one embodiment, core 1110 includes 64 single-precision (32-bit) floating-point cores, 64 integer cores, 32 double-precision (64-bit) floating-point cores, and 8 tensor cores.

[0115] According to an embodiment, the tensor core is configured to perform matrix operations. In one embodiment, one or more tensor cores are included in core 1110. In one embodiment, the tensor core is configured to perform deep learning matrix algorithms, such as convolution operations for neural network training and inference. In one embodiment, each tensor core performs operations on a 4x4 matrix and performs matrix multiplication and accumulation operations D = A × B + C, where A, B, C, and D are 4x4 matrices.

[0116] In one embodiment, matrix multiplication inputs A and B are 16-bit floating-point matrices, and accumulation matrices C and D are either 16-bit or 32-bit floating-point matrices. In one embodiment, the Tensor Core operates on the 16-bit floating-point input data and the 32-bit floating-point accumulation. In one embodiment, the 16-bit floating-point multiplication requires 64 operations and produces a full-precision product, which is then accumulated using 32-bit floating-point addition with other intermediate products used for 4x4x4 matrix multiplication. In one embodiment, the Tensor Core is used to perform operations on larger two-dimensional or higher-dimensional matrices constructed from these smaller elements. In one embodiment, an API (such as the CUDA 9 C++ API) exposes specialized matrix loading, matrix multiplication, and accumulation operations, as well as matrix storage operations, to efficiently utilize the Tensor Core from CUDA-C++ programs. In one embodiment, at the CUDA level, the thread bundle-level interface assumes a 16×16 matrix spanning all 32 threads of the thread bundle.

[0117] In one embodiment, each SM 1100 includes M SFUs 1112 that perform special functions (e.g., attribute evaluation, inverse square root, etc.). In one embodiment, an SFU 1112 includes a tree traversal unit configured to traverse a hierarchical tree data structure. In one embodiment, an SFU 1112 includes a texture unit configured to perform texture map filtering operations. In one embodiment, a texture unit is configured to load a texture map (e.g., a 2D texture pixel array) from memory and sample the texture map to produce sampled texture values ​​for use in a shader program executed by the SM 1100. In one embodiment, the texture map is stored in shared memory / L1 cache. According to one embodiment, the texture unit implements texture operations, such as filtering operations using mip maps (e.g., texture maps with different levels of detail). In one embodiment, each SM 1100 includes two texture units.

[0118] In one embodiment, each SM 1100 includes N LSUs 1054 that implement load and store operations between the shared memory / L1 cache 1006 and the register file 1108. In one embodiment, each SM 1100 includes an interconnect network 1116 that connects each functional unit to the register file 1108 and connects LSUs 1114 to the register file 1108 and the shared memory / L1 cache 1118. In one embodiment, the interconnect network 1116 is a crossbar switch that can be configured to connect any functional unit to any register in the register file 1108 and to connect LSUs 1114 to memory locations in the register file and the shared memory / L1 cache 1118.

[0119] In one embodiment, the shared memory / L1 cache 1118 is an on-chip memory array that allows data storage and communication between the SM1100 and the primitive engine, as well as between threads within the SM1100. In one embodiment, the shared memory / L1 cache 1118 includes 128KB of storage capacity and is located on the path from the SM1100 to the partition unit. In one embodiment, the shared memory / L1 cache 1118 is used for cache reads and writes. One or more of the shared memory / L1 cache 1118, the L2 cache, and memory are used for backup storage.

[0120] In one embodiment, combining data caching and shared memory functionality into a single memory block provides improved performance for both types of memory access. In one embodiment, this capacity can be used or can be used as a cache by programs that do not use shared memory; for example, if shared memory is configured to use half its capacity, texture and load / store operations can use the remaining capacity. According to an embodiment, integration within the shared memory / L1 cache 1118 enables the shared memory / L1 cache 1118 to operate as a high-throughput pipeline for streaming data, while providing high-bandwidth and low-latency access to frequently reused data. When configured for general-purpose parallel computing, a simpler configuration can be used compared to graphics processing. In one embodiment, a fixed-function graphics processing unit is bypassed, creating a simpler programming model. In one embodiment, in a general-purpose parallel computing configuration, the work allocation unit directly assigns and allocates thread blocks to the DPC. According to one embodiment, threads within a block execute the same program, using unique thread IDs in the computation to ensure each thread produces a unique result. The program is executed and computations are performed using an SM 1100, a shared memory / L1 cache 1118 is used for communication between threads, and an LSU 1114 is used to read and write global memory via the shared memory / L1 cache 1118 and memory partitioning units. In one embodiment, when configured for general-purpose parallel computing, the SM 1100 can write commands to the scheduler unit to initiate new work on the DPC.

[0121] In one embodiment, the PPU is included in or coupled to a desktop computer, laptop computer, tablet computer, server, supercomputer, smartphone (e.g., wireless, handheld device), personal digital assistant (“PDA”), digital camera, vehicle, head-mounted display, handheld electronic device, etc. In one embodiment, the PPU is disposed on a single semiconductor substrate. In one embodiment, the PPU is included in a system-on-a-chip (“SoC”) together with one or more other devices, such as an additional PPU, memory, a reduced instruction set computer (“RISC”) CPU, a memory management unit (“MMU”), a digital-to-analog converter (“DAC”), etc.

[0122] In one embodiment, the PPU may be included on a graphics card that includes one or more memory devices. The graphics card may be configured to interface with a PCIe slot on the motherboard of a desktop computer. In yet another embodiment, the PPU may be an integrated graphics processing unit (“iGPU”) included in the chipset of the motherboard.

[0123] Figure 12 A computer system 1200, according to one embodiment, is shown, on which various architectures and / or functions may be implemented. In one embodiment, the computer system 1200 is configured to implement the various processes and methods described throughout this disclosure.

[0124] In one embodiment, the computer system 1200 includes at least one central processing unit 1202 connected to a communication bus 1210 implemented using any suitable protocol, such as PCI (Peripheral Component Interconnect), PCI-Express, AGP (Accelerated Graphics Port), HyperTransport, or any other bus or point-to-point communication protocol. In one embodiment, the computer system 1200 includes main memory 1204, control logic (e.g., implemented in hardware, software, or a combination thereof), and data stored in main memory 1204, which may be in the form of random access memory (“RAM”). In one embodiment, a network interface subsystem 1222 provides interfaces to other computing devices and networks for receiving data from and sending data to other systems within the computer system 1200.

[0125] In one embodiment, the computer system 1200 includes an input device 1208, a parallel processing system 1212, and a display device 1206, which can be implemented using conventional CRT (cathode ray tube), LCD (liquid crystal display), LED (light-emitting diode), plasma display, or other suitable display technologies. In one embodiment, user input is received from the input device 1208 (such as a keyboard, mouse, touchpad, microphone, etc.). In one embodiment, each of the above modules can reside on a single semiconductor platform to form a processing system.

[0126] In this specification, a single semiconductor platform may refer to an integrated circuit or chip based on a single semiconductor. It should be noted that the term "single semiconductor platform" can also refer to a multi-chip module with increased connectivity for analog on-chip operation, representing a substantial improvement over implementations using a conventional central processing unit ("CPU") and bus. Of course, depending on the user's needs, the modules may also be placed separately or in various combinations of semiconductor platforms.

[0127] In one embodiment, a computer program in the form of machine-readable executable code or computer control logic algorithms is stored in main memory 1204 and / or secondary memory. According to one embodiment, if the computer program is executed by one or more processors, the system 1200 is enabled to perform various functions. Memory 1204, memory, and / or any other memory are possible examples of computer-readable media. Secondary memory can refer to any suitable storage device or system, such as hard disk drives and / or removable storage drives, representing floppy disk drives, magnetic tape drives, optical disk drives, digital versatile disc (“DVD”) drives, recording devices, Universal Serial Bus (“USB”) flash memory.

[0128] In one embodiment, the architectures and / or functions of the various prior figures may be implemented in the context of the central processing unit 1202, the parallel processing system 1212, an integrated circuit capable of having at least a portion of the capabilities of both the central processing unit 1202 and the parallel processing system 1212, a chip set (e.g., an integrated circuit group designed to operate as a unit performing related functions and sold, etc.), and any suitable combination of integrated circuits.

[0129] In one embodiment, the various architectures and / or functions of the prior drawings can be implemented in the context of general-purpose computer systems, circuit board systems, game console systems for entertainment purposes, special-purpose systems, etc. In one embodiment, computer system 1200 can take the form of: desktop computer, laptop computer, tablet computer, server, supercomputer, smartphone (e.g., wireless, handheld device), personal digital assistant (“PDA”), digital camera, vehicle, head-mounted display, handheld electronic device, mobile phone device, television, workstation, game console, embedded system, and / or any other type of logic.

[0130] In one embodiment, the parallel processing system 1212 includes multiple PPUs 1214 and associated memory 1216. In one embodiment, the PPUs are connected to a host processor or other peripheral device via interconnect 1218 and switch 1220 or multiplexer. In one embodiment, the parallel processing system 1212 distributes computational tasks across parallelizable PPUs 1214, for example, as part of a computational task distribution across multiple GPU thread blocks. In one embodiment, the memory is shared and accessible (e.g., for read and / or write access) on some or all of the PPUs 1214, although such shared memory may incur performance penalties due to the use of local memory and registers residing on the PPU. In one embodiment, the operation of the PPUs 1214 is synchronized using a command such as __syncthreads(), which requires all threads in a block (e.g., executing across multiple PPUs 1214) to reach a certain point of execution in the code before proceeding.

[0131] Therefore, the specification and drawings should be considered illustrative rather than restrictive. However, it will be apparent that various modifications and changes can be made thereto without departing from the broader spirit and scope of the invention as set forth in the claims.

[0132] Other variations are within the spirit of this disclosure. Therefore, while the disclosed technology is susceptible to various modifications and alternative constructions, certain illustrated embodiments are shown in the accompanying drawings and have been described in detail above. However, it should be understood that the invention is not intended to be limited to one or more specific forms disclosed, but rather is intended to cover all modifications, alternative constructions, and equivalents that fall within the spirit and scope of the invention as defined in the appended claims.

[0133] In the context of describing the disclosed embodiments (particularly in the context of the following claims), the terms “a” and “an”, as well as “the” and similar designations, should be interpreted as including both singular and plural, unless otherwise stated herein or explicitly contradicted by the context. Unless otherwise stated, the terms “comprising,” “having,” “containing,” and “including” should be interpreted as open-ended terms (i.e., meaning “including but not limited to”). The term “connection,” when unmodified and referring to a physical connection, should be interpreted as partially or wholly contained in, attached to, or connected together, even if something is involved. Unless otherwise stated herein, the enumeration of ranges of values ​​herein is merely a simplified expression of each individual value within the range as a separate reference, and each individual value is incorporated into the specification as if individually enumerated herein. Unless otherwise stated or contradicted by the context, the use of the terms “set” (e.g., “item set”) or “subset” should be interpreted as a non-empty set containing one or more members. Furthermore, unless otherwise stated or contradicted by the context, the term “subset” for a corresponding set does not necessarily refer to an appropriate subset of the corresponding set, but the subset and the corresponding set may be equivalent.

[0134] Conjunctions (such as phrases like "at least one of A, B, and C" or "at least one of A, B, and C"), unless explicitly stated otherwise or clearly contradicted by the context, should be understood to mean that the context in which items, terms, etc., are generally used can be A, B, or C, or any non-empty subset of the set A, B, and C. For example, in an example of a set with three members, the conjunctive phrases "at least one of A, B, and C" and "at least one of A, B, and C" refer to any one of the following sets: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Therefore, such conjunctions generally do not imply that some embodiments require the presence of at least one of A, at least one of B, and at least one of C. Furthermore, unless explicitly stated or contradicted by the context, the term "multiple" indicates a plural state (e.g., "multiple items" means multiple items). The number of items in multiple items is at least two, but can be more when explicitly stated or indicated by the context. Furthermore, unless otherwise stated or the context clearly indicates otherwise, the phrase “based on” means “at least partially based on” rather than “based on only”.

[0135] Unless otherwise stated herein or otherwise explicitly contradicted by the context, the operations of the processes described herein may be performed in any suitable order. In one embodiment, processes such as those described herein (or variations and / or combinations thereof) are executed under the control of one or more computer systems configured with executable instructions and are implemented as code (e.g., executable instructions, one or more computer programs, or one or more application programs) that executes jointly on one or more processors via hardware or a combination thereof. In one embodiment, the code is stored, for example, in the form of a computer program comprising a plurality of instructions executable by one or more processors. In one embodiment, the machine-readable medium or computer-readable storage medium is a non-transitory machine-readable medium or computer-readable storage medium that does not include transient signals (e.g., propagating transient electrical or electromagnetic transmissions) but includes non-transitory data storage circuitry (e.g., buffers, cache memories, and queues) within transceivers of transient signals. In one embodiment, code (e.g., executable code or source code) is stored on a combination of one or more non-transitory computer-readable storage media, having executable instructions (or other memory storing executable instructions) stored thereon, which, when executed by one or more processors of a computer system (e.g., as a result of execution), cause the computer system to perform the operations described herein. In one embodiment, the combination of one or more non-transitory computer-readable storage media or machine-readable media includes information that, if used by one or more computer instructions, configures one or more processors of the computer system and further causes one or more processors to perform the operations described herein. In one embodiment, the combination of non-transitory computer-readable storage media includes multiple non-transitory computer-readable storage media and one or more individual non-transitory storage media lacking all the code, while multiple non-transitory computer-readable storage media collectively store all the code. In one embodiment, execution of the executable instructions causes different instructions to be executed by different processors; for example, non-transitory computer-readable storage media store instructions and the main CPU executes some instructions, while the graphics processing unit executes other instructions. In one embodiment, different components of the computer system have separate processors, and the different processors execute different subsets of instructions.

[0136] Therefore, in one embodiment, the computer systems are configured to implement one or more services that individually or collectively perform the operations of the processes described herein, and these computer systems are configured with suitable hardware and / or software capable of performing the operations. Furthermore, the computer systems implementing embodiments of this disclosure are single devices, and in another embodiment, they are distributed computer systems comprising multiple devices operating differently, such that the distributed computer systems perform the operations described herein and that a single device does not perform all operations.

[0137] The use of any and all examples or exemplary language provided herein (e.g., “for example”) is merely for the purpose of better illustrating embodiments of the invention and does not limit the scope of the invention unless otherwise required. No language in the specification should be construed as representing any unstated element essential to the implementation of the invention.

[0138] The embodiments described herein include the best modes known to the inventors for carrying out the invention. Variations of these embodiments will be apparent to those skilled in the art upon reading the foregoing description. The inventors expect those skilled in the art to appropriately employ such variations, and the inventors intend to implement the embodiments of this disclosure in ways other than those specifically described herein. Therefore, the scope of this disclosure includes all modifications and equivalents of the subject matter permitted by applicable law as set forth in the appended claims. Furthermore, unless otherwise stated herein or expressly contradicted by the context, any combination of the foregoing elements in all their possible variations is included within the scope of this disclosure.

[0139] All references cited in this article (including publications, patent applications and patents) are incorporated herein by reference to the same extent as each reference individually and explicitly stated to be incorporated by reference is listed in full herein.

[0140] In the specification and claims, the terms “coupled” and “connected” and their derivatives may be used. It should be understood that these terms may not be synonyms for each other. Rather, in specific examples, “connected” or “coupled” may be used to indicate that two or more elements are in direct or indirect physical or electrical contact with each other. “Coupled” may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

[0141] Unless otherwise specified, it is understood that throughout this specification, terms such as “processing,” “calculating,” “measuring,” “determining,” etc., refer to the actions and / or processes of a computer or computing system or similar electronic computing device that process and / or convert physical quantities (such as electronic quantities) in the registers and / or memory of the computing system into other data similarly represented as physical quantities in the memory, registers, or other such information storage, transmission, or display devices of the computing system.

[0142] Similarly, the term "processor" can refer to any device or part of a device that processes electronic data from registers and / or memory and converts that electronic data into other electronic data that can be stored in registers and / or memory. As a non-limiting example, a "processor" can be a central processing unit (CPU) or a graphics processing unit (GPU). A "computing platform" can include one or more processors. As used herein, a "software" process can include, for example, software and / or hardware entities that perform work over time, such as tasks, threads, and intelligent agents. Furthermore, each process can refer to multiple processes for executing instructions sequentially or in parallel, continuously or intermittently. The terms "system" and "method" are used interchangeably herein, provided that a system can embody one or more methods and those methods can be considered a system.

[0143] This document describes the process of acquiring, obtaining, receiving, or inputting analog or digital data into a subsystem, computer system, or computer-implemented machine. The process of acquiring, obtaining, receiving, or inputting analog and digital data can be accomplished in various ways, such as by receiving data as a parameter to a function call or an application programming interface (API) call. In some implementations, the process of acquiring, obtaining, receiving, or inputting analog or digital data can be accomplished by transmitting data via a serial or parallel interface. In another implementation, the process of acquiring, obtaining, receiving, or inputting analog or digital data can be accomplished by transmitting data from a providing entity to an acquiring entity via a computer network. Reference can also be made to providing, outputting, sending, emitting, or presenting analog or digital data. In various examples, the process of providing, outputting, sending, emitting, or presenting analog or digital data can be implemented by using data as an input or output parameter to a function call, an API call, or an inter-process communication mechanism.

[0144] While the foregoing discussion illustrates exemplary implementations of the technology, other architectures can be used to implement the functionality, and are intended to be within the scope of this disclosure. Furthermore, although specific assignments of responsibilities have been defined above for the purposes of discussion, various functions and responsibilities may be assigned and divided in different ways depending on the specific circumstances.

[0145] Furthermore, although the subject matter has been described in language specific to structural features and / or methodological behavior, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or behaviors described. Rather, the specific features and behaviors disclosed are exemplary forms for implementing the claims.

Claims

1. A processor comprising: comparison logic to compare a first boundary of one or more objects in one or more images generated by a neural network to one or more auto-encoding versions of the first boundary generated by another neural network, wherein the first boundary and the one or more auto-encoding versions are represented by label masks; and indicating logic, coupled to the comparison logic, to indicate whether the first boundary and the one or more auto-encoding versions of the first boundary differ by at least a first threshold.

2. The processor of claim 1, wherein the first boundary and one or more autoencoding versions of the first boundary comprise: a first label mask representing a boundary of an object in an image, the boundary being a boundary in a segmentation determined from a first segmentation process, and a second label mask representing an output of a shape assessment process, wherein the first label mask is an input to the shape assessment process, and wherein the indicating logic is configured to compare the first label mask and the second label mask to determine a quality of the segmentation.

3. The processor of claim 2, further comprising a first trained neural network that performs the first segmentation process to output the first label mask in response to obtaining a representation of the image.

4. The processor of claim 3, further comprising a second trained neural network that performs the shape assessment process using the first label mask as its input and outputs the second label mask.

5. The processor of claim 4, wherein the second trained neural network is an autoencoder with internal layers that maps its input to a latent representation in a feature space, wherein the features in the feature space are shape features.

6. The processor of claim 5, wherein the autoencoder is a variational autoencoder.

7. The processor of claim 5, further comprising logic to train the second trained neural network using a training sub-set of a segmentation data set, wherein the training sub-set of the segmentation data set includes training images and corresponding training label masks.

8. The processor of claim 7, wherein the logic to train the second trained neural network further uses a validation sub-set of a segmentation data set.

9. The processor of claim 1, wherein the processor comprises a graphics processing unit (GPU).

10. A method of processing an image using a processor comprising one or more arithmetic logic units (ALUs), comprising: comparing a first boundary of one or more objects in one or more images generated by a neural network to one or more auto-encoding versions of the first boundary generated by another neural network, wherein the first boundary and the one or more auto-encoding versions are represented by label masks; and indicating whether the first boundary and the one or more auto-encoding versions of the first boundary differ by at least a first threshold.

11. The method of claim 10, further comprising: generating a segmentation of the image, wherein the segmentation represents a processor-determined boundary set of an object depicted in the image; inputting the segmentation to a neural network previously trained on a set of training segmentations; comparing the segmentation to an output of the neural network; and determining a score for the segmentation, wherein the score is a function of a difference between the segmentation and the output of the neural network.

12. The method of claim 11, wherein the neural network is a variational autoencoder that takes the segmentation as its input, wherein the variational autoencoder maps features of its input to a reduced feature space from which the segmentation can be approximately reproduced from features in the reduced feature space.

13. The method of claim 12, further comprising: training the variational autoencoder with the set of training segmentations, wherein the set of training segmentations are represented by label masks that are ground truth label masks because the segmentations of these label masks were previously determined to be good segmentations of the images.

14. The method of claim 12, further comprising: training the segmenter to generate segmentations of the images by applying a set of segmentation data sets to the segmenter, wherein each segmentation data set of the set of segmentation data sets comprises a training image and a corresponding training label mask that is a ground truth label mask for the training image because the segmentation in the corresponding training label mask was previously determined to be a good segmentation of the training image.

15. The method of claim 11, further comprising: performing a first segmentation process using a first trained neural network to output a first label mask in response to obtaining the image.

16. The method of claim 11, further comprising: performing a shape assessment process using a second trained neural network with the first label mask as input and outputting a second label mask.

17. The method of claim 16, further comprising: mapping the input to the second trained neural network to a latent representation in a feature space, wherein the features in the feature space are shape features.

18. The method of claim 17, wherein the second trained neural network is a variational autoencoder.

19. A system comprising a memory storing instructions that, when executed by one or more processors, cause the system to: train one or more neural networks as a variational autoencoder (VAE) using, at least in part, a set of one or more training segmentations that are ground truth label masks for images because the segmentations of the training segmentations in the set of one or more training segmentations were previously determined to be segmentations that are good segmentations of the images that depict objects; and train a segmenter to generate label masks that represent processor-determined segmentations of images from images, wherein the training includes applying a set of segmentation data sets to the segmenter, wherein each segmentation data set of the set of segmentation data sets comprises a training image and a corresponding training label mask that is a ground truth label mask for the training image because the segmentation in the corresponding training label mask was previously determined to be a good segmentation of the training image.

20. The system of claim 19, wherein the instructions further cause the system to: generate a segmentation of an input image, wherein the segmentation represents a set of boundaries of an object depicted in the input image as determined by the processor; input the segmentation as a VAE input to the VAE; compare the VAE input to a VAE output of the VAE; and determine a score for the segmentation, wherein the score is a function of a difference between the VAE input and the VAE output.

21. The system of claim 20, further comprising: output the score; determine whether the score is within a predetermined range; and if the score is within the predetermined range, output an alert signal.

Images