Quantification method of target contour uncertainty error based on conditional constraint probability generation

Abstract

This case involves a target contour uncertainty error quantification method based on conditional constrained probability generation, which is used to solve the problem that existing technologies cannot provide intuitive explanations of nodule ultrasound diagnostic results, and that existing technologies find it difficult to simultaneously learn clinical opinions from multiple experts. The method proposes a target contour uncertainty error quantification method based on conditional constrained probability generation. It captures the semantic and structural coding information in nodule ultrasound images by establishing a deep learning model, maps the coding information to the latent space distribution, and then performs sampling. Based on the sampling results, the target contour segmentation results and uncertainty quantification are obtained. The method uses the model to learn the annotation distribution from multiple experts, outputs the uncertainty quantification of its own prediction results, effectively improves the interpretability of the model, and helps doctors identify high-risk areas of nodules, thereby enhancing the transparency and reliability of the method in actual clinical applications.

Description

Technical Field

[0001] The present disclosure relates to technical fields such as computer vision and artificial intelligence, medical image processing, and in particular to a method for quantifying target contour uncertainty errors based on conditional constraint probability generation. Background Art

[0002] Medical image segmentation plays a vital role in medical diagnosis and treatment. By accurately segmenting organs and tissues, it provides powerful support to doctors, thereby improving diagnostic efficiency and saving medical resources. Compared to natural images, the segmentation targets in medical images often have complex shapes, blurred edges, and tissue variability. Noise and artifacts in medical images further increase the difficulty of segmentation, making medical image segmentation more challenging. In addition, medical images often do not have unique labels like natural images. Due to differences in anatomical knowledge, diagnostic experience, and interpretation standards, different doctors may give different annotation results for the same case. In clinical practice, gathering the opinions of multiple experts can improve diagnostic capabilities and help enhance the reliability and accuracy of diagnoses. However, most current deep learning segmentation models are often unable to learn annotations from multiple experts and can only provide a single segmentation result in a "black box" format. Incorrect segmentation results can lead to serious consequences. Therefore, it is particularly important to be able to estimate the uncertainty of the segmentation results. It can not only effectively help doctors evaluate the model's confidence in the segmentation results, but also help identify high-risk areas and improve the interpretability of the model, making the application of the method in medical scenarios more transparent and credible, thereby better meeting actual clinical needs. Summary of the Invention

[0003] The purpose of this case is to propose a target contour uncertainty error quantification method based on conditional constraint probability generation through this study, which can effectively realize the segmentation of lesions in ultrasound images, output the uncertainty quantification information of its own segmentation results and visualize the uncertainty information, so as to effectively help doctors evaluate the reliability of segmentation results and identify high-risk areas, improve the interpretability of the method, and have important application value in the field of computer-aided diagnosis.

[0004] On the first aspect, this case proposes a method for quantifying target contour uncertainty errors based on conditional constraint probability generation, and the steps of the method include: constructing a priori network based on a pre-trained first visual Transformer encoder, constructing a posterior network using a second visual Transformer encoder with the same structural parameters as the first visual Transformer encoder, and constructing a target contour segmentation network based on the Unet network; inputting nodule ultrasound image samples into the priori network to obtain a priori distribution, concatenating the nodule ultrasound image samples and their corresponding labels, and inputting them into the posterior network after convolution dimensionality reduction to obtain a posterior distribution, and calculating the KL divergence based on the prior distribution and the posterior distribution. ; Input the nodule ultrasound image sample and prior distribution into the target contour segmentation network, output the prediction result, and calculate the cross entropy loss between the prediction result and the true value ; Input the nodule ultrasound image sample and posterior distribution into the target contour segmentation network, output the prediction result, and calculate the cross entropy loss between the prediction result and the true value ; Calculate the total training loss , ; Input the ultrasonic image of the nodule to be segmented into the trained prior network to obtain the prior distribution, input the obtained prior distribution and its corresponding ultrasonic image of the nodule to be segmented into the trained target contour segmentation network, output the segmentation result of the nodule ultrasonic image, and then obtain the target contour confidence.

[0005] In one embodiment of the above technical solution, the pre-training is to divide the nodule ultrasound image into image blocks, and use the visual Transformer encoder to achieve self-supervised learning through masking and reconstruction.

[0006] In one embodiment of the above technical solution, the target contour segmentation network takes the nodule ultrasound image as the input of its encoder and injects the latent variable into the bottleneck layer; the latent variable is sampled from the prior distribution or the posterior distribution during training, and is sampled from the prior distribution when the training is completed; the injection is to expand the latent variable to the same dimension as the bottleneck layer feature, and then splice it with the feature output by the encoder, and then pass it to the decoder after dimensionality reduction.

[0007] In one implementation of the above technical solution, the prior distribution and the posterior distribution are both Gaussian distributions.

[0008] In one implementation of the above technical solution, the target contour confidence is the standard deviation of each pixel.

[0009] In one implementation of the above technical solution, the method visualizes the target contour confidence through color.

[0010] On the second aspect, this case proposes a computer-readable storage medium storing a computer program that can be loaded by a processor and execute any of the above methods.

[0011] In the third aspect, a target contour segmentation system comprises a prior network, a posterior network, and a target contour segmentation network; wherein: the prior network is configured to use a pre-trained first visual Transformer encoder to obtain a first coding feature of a nodule ultrasound image, and obtain a prior distribution corresponding to the first coding feature; the target contour segmentation network is configured to use a Unet network to obtain a nodule segmentation result based on the nodule ultrasound image and its corresponding probability distribution, and then obtain a target contour confidence, wherein the probability distribution is a prior distribution or a posterior distribution; the posterior network is configured to be used in training, and uses a pre-trained second visual Transformer encoder to obtain a second coding feature of the nodule ultrasound image, while using the expert-annotated label as a conditional variable, based on the conditional variable and the posterior distribution corresponding to the second coding feature; the loss of the system in training is calculated as , , is the KL divergence of the posterior network and the prior network; is the cross entropy loss between the predicted result and the true value based on the posterior network; It is the cross entropy loss between the predicted result and the true value corresponding to the prior network; when the trained system is used, the ultrasound image of the nodule to be segmented is used as the input of the prior network to obtain the prior distribution, and then the obtained prior distribution and the corresponding ultrasound image of the nodule to be segmented are input into the trained target contour segmentation network, and the segmentation result of the nodule ultrasound image is output, thereby obtaining the target contour confidence.

[0012] In one embodiment of the above technical solution, the system further includes a calculation module; the calculation module is configured to calculate the target contour confidence based on the segmentation result of the ultrasound image of the nodule to be segmented.

[0013] The beneficial technical effects of this case are: (1) By introducing a pre-training strategy for image masking and reconstruction, and using the visual Transformer encoder to enhance the global image representation capability of the method, combined with the target contour segmentation network, accurate segmentation of ultrasound nodule images is achieved, providing a reliable auxiliary diagnosis solution for clinical diagnosis and treatment; (2) The annotation distribution of multiple experts is introduced as a condition, and then a probability generation model based on conditional constraints is used to learn the probability distribution of data, so that the method can generate multiple predicted segmentation results that conform to the real data distribution, thereby obtaining the uncertainty quantification result of the method, providing a quantitative basis for doctors to evaluate the reliability of the model segmentation results, helping to identify high-risk areas, and effectively improving the interpretability of the model and its application value in medical scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0015] Figure 1 A schematic structural diagram of an ultrasonic target profile uncertainty error quantification method based on conditional constraint probability generation in one embodiment.

[0016] Figure 2 Schematic diagram of the visualization effect of the segmentation results of thyroid ultrasound image nodules in one embodiment.

[0017] Figure 3 A schematic diagram illustrating the visualization of uncertainty errors in target contours of thyroid ultrasound images in one embodiment. DETAILED DESCRIPTION

[0018] Based on the medical image analysis process and ultrasound diagnostic guidelines, this case utilizes image processing techniques such as pre-training, feature extraction, and uncertainty error quantification to establish a method for quantifying target contour uncertainty errors based on conditional constrained probability generation. This method has important application value and prospects for 2D ultrasound image segmentation and assisted clinical diagnosis. The uncertainty error quantification method outputs a confidence map of the predicted segmentation results, quantifying the method's confidence in the segmentation results.

[0019] The following, combined with the accompanying drawings, provides a clear and complete description of how the technical solution of this case is implemented. Obviously, the described implementation methods are only part of the implementation methods of this case, not all of the implementation methods. Based on the implementation methods of this case, all other implementation methods obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0020] 1. Extraction of nodule ultrasound image features

[0021] In this case, a pre-trained visual Transformer encoder is used to extract features from nodule ultrasound images.

[0022] Pre-training is accomplished through self-supervised learning via masking and reconstruction. The pre-trained visual Transformer encoder can be transferred to a variety of downstream tasks and perform well, effectively improving the training speed and accuracy of the entire method in downstream tasks.

[0023] Specifically, in the pre-training stage, it is assumed that the original input nodule ultrasound image is , which is divided into Image blocks: , through random mask operation, the image is randomly divided into a set of visible blocks and mask block set , mask rate .

[0024] Use the visual Transformer encoder as the encoder , only process visible blocks Since the visual Transformer encoder itself does not have the ability to process the order or spatial position of the input data, in order to recover the position information, a linear projection is applied to add position embedding to the visible image block: , is the embedded position information. The image block embedded with position information is input into the visual Transformer encoder for encoding, and the corresponding feature representation is output. , .

[0025] In one embodiment, the visual Transformer encoder specifically includes 12 Transformer layers, each of which includes the following two sublayers:

[0026] Multi-Head Self-Attention (MHSA):

[0027]

[0028] in is a linear transformation from The query, keys, and values ​​obtained from .

[0029] Feed-Forward Network (FFN):

[0030]

[0031] in , is the weight matrix, , is the bias, is the activation function.

[0032] Use residual connections and layer normalization (LayerNorm) between each Transformer layer:

[0033]

[0034] In the above formula, For the The output of the layer, is the intermediate result of applying MHSA and residual connection, For the The output of the layer.

[0035] The image mask based pre-training architecture is an asymmetric encoder-decoder architecture, where the decoder is only used to perform the image reconstruction task during pre-training but not for downstream tasks.

[0036] The decoder uses 4 Transformer blocks. Through this asymmetric design, the entire method architecture is made lighter and the pre-training time can be greatly reduced.

[0037] Decoder Receive encoder output and a learnable mask token placed at the mask block location , reconstruct the original image block : .

[0038] Instead of reconstructing the entire image, the decoder only predicts and reconstructs the masked image blocks, which has been shown to achieve better results. The goal of reconstruction is to minimize the difference between the intermediate image and the original image, usually using the mean square error loss (L2 loss):

[0039]

[0040] in, is the masked block in the original image, is the reconstructed block output by the decoder, is the number of mask blocks.

[0041] Since the output of the encoder is used to reconstruct the complete image, this network design encourages the encoder to understand and extract global features from partial image information, thereby improving performance in image reconstruction and downstream tasks. The weights of the above encoder will be passed to the training stage.

[0042] By reconstructing the mask image, the network is encouraged to aggregate information from the context to infer the masked target. Contextual information is crucial for medical image analysis. Therefore, introducing the image mask-based pre-training architecture into medical image segmentation can effectively improve accuracy.

[0043] This case introduces an image mask-based pre-training architecture into the technical solution, aiming to learn the global features of images more quickly and accurately through pre-training strategies. In this image mask-based pre-training architecture, the self-attention mechanism of the visual Transformer encoder is utilized to directly model the global dependencies between image patches.

[0044] This enables the next uncertainty module to learn a distribution that is more consistent with the data characteristics, thereby generating more valuable samples.

[0045] 2. Quantification of target contour uncertainty error

[0046] The quantification of target contour uncertainty error is achieved through a visual Transformer encoder and a probabilistic generation model based on conditional constraints. The visual Transformer encoder is used to obtain image features, while the probabilistic generation model encodes image features into a distribution in the latent space.

[0047] Before introducing the probability generation model based on conditional constraints, let’s first introduce the probability generation model based on variational inference.

[0048] The probabilistic generative model based on variational inference is a generative model whose core idea is to encode the image features obtained based on the visual Transformer encoder into a probability distribution in the latent space rather than a single point, so as to generate meaningful new data.

[0049] The training process of the probabilistic generation model based on variational inference is to first encode the image features obtained based on the visual Transformer encoder into a distribution in the latent space, then sample the latent variables from the distribution, and then input the latent variables into the decoder to reconstruct the input data.

[0050] The training loss function of the probability generation model based on variational inference includes minimizing the KL divergence between the encoding distribution and the standard normal distribution and the reconstruction error, that is, maximizing the variational lower bound (ELBO), so that the latent space has good regularity and is close to the standard normal distribution. Points randomly sampled from the latent space can decode meaningful new data. The formula of ELBO is expressed as:

[0051]

[0052] in, is the input data, is a hidden variable, is the encoder network (i.e., the posterior distribution), is the decoder network, is the prior distribution (usually a normal distribution), represents the KL divergence.

[0053] The probabilistic generative model based on conditional constraints introduces conditional variables on the basis of the probabilistic generative model based on variational inference, allowing the model to generate data under given conditions. The conditional variables can be labels, features, or other auxiliary information.

[0054] During the training process of a probabilistic generative model based on conditional constraints, the original data and the conditional variables are input into the encoder to generate the posterior distribution. The latent variables are sampled from the posterior distribution and input into the decoder to obtain the reconstruction result. At the same time, the original data is input into the encoder of the same architecture to generate the prior distribution. During inference, only the prior distribution is used, and the data is input into the prior encoder, and encoding, sampling, and decoding are performed to obtain the result.

[0055] The training goal of the probabilistic generative model based on conditional constraints is also to maximize the variational lower bound. By minimizing the KL divergence and reconstruction error of the prior distribution and the posterior distribution, the two distributions are made as similar as possible, and useful data can be sampled from the prior distribution during inference.

[0056] The optimization objective of the probability generation model based on conditional constraints can be expressed as:

[0057]

[0058] in, is a conditional variable, is the posterior encoder, is the a priori encoder.

[0059] In this case, two visual Transformer encoders with the same architecture and the same pre-trained weights are used. and encoder Construct the prior network and the posterior network respectively.

[0060] The prior network is configured to use a pre-trained first visual Transformer encoder to obtain a first encoding feature of the nodule ultrasound image and obtain a prior distribution corresponding to the first encoding feature.

[0061] In the prior network, all blocks of the image are input to The first image encoding feature is obtained: , and the first image encoding feature Mapping to the prior distribution , the prior distribution is a standard Gaussian distribution: .

[0062] The posterior network is configured to be used in training to assist the training of the prior network, and uses the pre-trained second visual Transformer encoder to obtain the second encoding features of the nodule ultrasound image. At the same time, the label annotated by the expert is used as a conditional variable. Based on the posterior distribution corresponding to the conditional variable and the second encoding feature, the posterior distribution and the corresponding nodule ultrasound image are input into the target contour segmentation network to obtain the segmentation result of the nodule ultrasound image.

[0063] In the posterior network, use the label as the conditional variable , used to guide data generation, the image blocks and labels are spliced ​​and input into the posterior network The second image encoding feature is obtained: , and encode the second image features Mapping to the posterior distribution , the posterior distribution has a mean of , the standard deviation is Gaussian distribution.

[0064] During training, latent variables are sampled from the posterior distribution , this process requires the use of reparameterization techniques to allow gradients to propagate and avoid sampling directly from complex distributions:

[0065]

[0066] in is noise sampled from a standard normal distribution.

[0067] This case constructs a priori network based on the visual Transformer encoder architecture, which can not only better learn the representation of the image, but also obtain a distribution that is more consistent with the characteristics of the data, thereby providing more valuable sampling data. In order to enable the priori network to generate a more accurate feature distribution, auxiliary training is performed through the posterior network. The posterior network is constructed based on the visual Transformer encoder with the same structure and parameters, and introduces expert-annotated labels as conditional variables to better capture the correlation information between the input data and the labels. When there is a one-to-many relationship, that is, when one image corresponds to multiple true values, the distribution of the true values ​​can be better learned, so that the samples generated in the latent space are more consistent with the distribution of the true values. Moreover, the introduction of conditional variables can increase the diversity of generated samples and improve the overall performance and effect of the generation model.

[0068] (3) Target contour segmentation

[0069] This case uses the Unet model as the object contour segmentation network. The Unet model is a multi-layer segmentation network model with an encoder-decoder structure, where the encoder and decoder are connected by a bottleneck layer. The Unet model gradually extracts and reduces feature dimensions through multiple convolutional and pooling layers. It then restores the feature maps to the resolution of the original input image through transposed convolutional layers and skip connections. Skip connections help preserve feature information at different levels, improving segmentation accuracy and detail preservation.

[0070] In one embodiment, the encoder in the object contour segmentation network includes 4 layers of operations, each of which includes double convolution and downsampling operations to extract feature representations of different scales. , after the encoder The layer operations are:

[0071]

[0072] in, For the The feature map of the layer, It is the maximum pooling operation.

[0073] The decoder gradually restores the original resolution through upsampling operations and concatenates the features of the corresponding layers of the encoder. The layer operations are:

[0074]

[0075] in, For decoder The feature map of the layer, is the concatenated feature map, is the deconvolution operation.

[0076] Finally, the last layer of the decoder is output , through a The convolution layer adjusts the number of channels to the number of categories and through Operation to get the prediction result:

[0077]

[0078] In this case, the original input image Input into the target contour segmentation network and extract the Upsampling to obtain latent variables ,Will Injecting latent variables into the bottleneck layer of the object contour segmentation network involves expanding z to the same dimension as the bottleneck layer features, concatenating them, and performing dimensionality reduction through a Conv layer and a ReLU layer. Because the bottleneck layer highly abstracts and compresses features and passes them to the decoder, injecting latent variables into this layer can enhance the diversity of the model's generated results.

[0079] (4) Training

[0080] The prior network, posterior network, and target contour segmentation network are trained, and the training loss function consists of three parts.

[0081] First, calculate the KL divergence between the posterior distribution and the prior distribution:

[0082]

[0083] Second, the latent variables sampled from the posterior distribution Inject it into the target contour segmentation network to obtain the cross entropy loss between the predicted result and the true value:

[0084]

[0085] Finally, in order to enhance the predictive power of the prior distribution, we also sample from the prior distribution and inject it into the target contour segmentation network to obtain the predicted value and calculate the cross entropy loss. .

[0086] Therefore, the overall loss function is:

[0087]

[0088] (V) Prediction and Reasoning Stage

[0089] Only the prior network is used in the inference phase To make a prediction, the image is input into the prior network, encoded and sampled, and injected into the target contour segmentation network to finally obtain the predicted image.

[0090] The number of samples in this case Configurable, record the prediction result each time as , first calculate the mean of the prediction results:

[0091]

[0092] Then calculate the standard deviation of each pixel as the confidence:

[0093]

[0094] The larger the standard deviation, the higher the uncertainty of the prediction result at that point, and vice versa.

[0095] The above method is used for segmentation of thyroid ultrasound image nodules. The visualization of the segmentation results can be found in Figure 2 , where the first row and the first column are the original images, the second column is the real segmentation label annotated by the doctor, and the number of sampling times The second and third rows are the segmentation results obtained by sampling 10 times. It can be seen that the segmentation results are basically consistent with the true value and can show a certain diversity. The contour confidence visualization effect of the segmentation result can be seen in Figure 3 , shows two examples on a thyroid ultrasound dataset. The first column shows the ultrasound image and the ground truth, while the second column shows the results of a single sampling, along with uncertainty quantification obtained by calculating the variance of the segmentation results. Colors closer to red indicate higher uncertainty, while colors closer to blue indicate lower uncertainty. The example on the left shows a segmentation result that is close to the ground truth, with relatively low overall uncertainty, except for slightly higher uncertainty at the edges. The example on the right, while producing a poorer segmentation result, has high uncertainty in the incorrectly segmented regions, indicating that the model performs poorly in that region.

[0096] In summary, this method can segment lesions in medical images and output the uncertainty quantification of its own prediction results, thereby improving the interpretability of the method and helping doctors identify high-risk areas.

[0097] Through the description of the above embodiments, those skilled in the art can clearly understand that the present disclosure proposes a target contour uncertainty error quantification method based on conditional constraint probability generation, which includes a target contour segmentation network training method and a target contour uncertainty quantification method; wherein:

[0098] The target contour segmentation network training method comprises the following steps:

[0099] A priori network is constructed based on the pre-trained first visual Transformer encoder, a posterior network is constructed using the second visual Transformer encoder with the same structural parameters as the first visual Transformer encoder, and an object contour segmentation network is constructed based on the Unet network;

[0100] The nodule ultrasound image samples are input into the prior network to obtain the prior distribution. The nodule ultrasound image samples and their corresponding labels are concatenated and input into the posterior network after convolution dimensionality reduction to obtain the posterior distribution. The KL divergence is calculated based on the prior distribution and the posterior distribution. ;

[0101] The nodule ultrasound image sample and prior distribution are input into the target contour segmentation network, the prediction result is output, and the cross entropy loss between the prediction result and the true value is calculated. ;

[0102] The nodule ultrasound image sample and posterior distribution are input into the target contour segmentation network, the prediction result is output, and the cross entropy loss between the prediction result and the true value is calculated. ;

[0103] Calculate the total training loss , .

[0104] The target profile uncertainty quantification method comprises the following steps:

[0105] The ultrasound image of the nodule to be segmented is input into the trained prior network to obtain the prior distribution, the obtained prior distribution and its corresponding ultrasound image of the nodule to be segmented are input into the trained target contour segmentation network, the segmentation result of the nodule ultrasound image is output, and then the target contour confidence is obtained.

[0106] Through the description of the above embodiments, those skilled in the art can clearly understand that according to the method disclosed herein, a target contour segmentation network training method can be obtained, and the method includes the following steps:

[0107] A priori network is constructed based on the pre-trained first visual Transformer encoder, a posterior network is constructed using the second visual Transformer encoder with the same structural parameters as the first visual Transformer encoder, and an object contour segmentation network is constructed based on the Unet network;

[0108] The nodule ultrasound image samples are input into the prior network to obtain the prior distribution. The nodule ultrasound image samples and their corresponding labels are concatenated and input into the posterior network after convolution dimensionality reduction to obtain the posterior distribution. The KL divergence is calculated based on the prior distribution and the posterior distribution. ;

[0109] The nodule ultrasound image sample and prior distribution are input into the target contour segmentation network, the prediction result is output, and the cross entropy loss between the prediction result and the true value is calculated. ;

[0110] The nodule ultrasound image sample and posterior distribution are input into the target contour segmentation network, the prediction result is output, and the cross entropy loss between the prediction result and the true value is calculated. ;

[0111] Calculate total loss , .

[0112] Through the description of the above embodiments, technical personnel in the relevant field can clearly understand that the present disclosure proposes a target contour segmentation system, which includes a prior network, a posterior network, and a target contour segmentation network; wherein: the prior network is configured to use a pre-trained first visual Transformer encoder to obtain a first coding feature of the nodule ultrasound image, and obtain a prior distribution corresponding to the first coding feature; the target contour segmentation network is configured to use a Unet network to obtain a nodule segmentation result based on the nodule ultrasound image and its corresponding probability distribution, and then obtain the target contour confidence, and the probability distribution is a prior distribution or a posterior distribution; the posterior network is configured to be used in training, and uses a pre-trained second visual Transformer encoder to obtain a second coding feature of the nodule ultrasound image, and at the same time uses the expert-annotated label as a conditional variable, based on the conditional variable and the posterior distribution corresponding to the second coding feature; the loss of the system in training is calculated as , , is the KL divergence of the posterior network and the prior network; is the cross entropy loss between the predicted result and the true value based on the posterior network; The prior distribution is the cross entropy loss between the predicted result and the true value corresponding to the prior network. When the trained system is used, the ultrasound image of the nodule to be segmented is used as the input of the prior network to obtain a prior distribution. The obtained prior distribution and the corresponding ultrasound image of the nodule to be segmented are then input into the trained target contour segmentation network, and the segmentation result of the nodule ultrasound image is output. In some embodiments, the system further includes a calculation module; the calculation module is configured to calculate the target contour confidence based on the segmentation result of the ultrasound image of the nodule to be segmented.

[0113] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that a target contour uncertainty error quantification method based on conditional constraint probability generation according to the present disclosure is implemented as a target contour uncertainty error quantification system based on conditional constraint probability generation, and the method steps are implemented in the system through programs or program modules.

[0114] Through the description of the above embodiments, those skilled in the art can clearly understand that the method or system of the present disclosure can be implemented by means of software plus necessary general-purpose hardware, and of course can also be implemented by means of dedicated hardware including dedicated integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. In general, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the present disclosure, in most cases, software program implementation is a better implementation method. Therefore, those skilled in the art can implement a target profile uncertainty error quantification method based on conditional constraint probability generation using a software program (computer program), and can store the software program in a storage medium and record and execute it in a processor.

[0115] Although the embodiments of the present disclosure have been described above with reference to the accompanying drawings, the present disclosure is not limited to the specific embodiments and application areas described above. The specific embodiments described above are merely illustrative and instructive, and not restrictive. A person of ordinary skill in the art, guided by this specification and without departing from the scope of protection of the claims of the present disclosure, may devise various other forms, all of which fall within the scope of protection of the present disclosure.

Claims

1. A target contour uncertainty error quantification method based on conditional constraint probability generation, characterized in that: The method comprises the following steps: An asymmetric encoder-decoder architecture is used for pre-training. In the pre-training stage, it is assumed that the original input nodule ultrasound image is , which is divided into Image blocks: , through random mask operation, the image is randomly divided into a set of visible blocks and mask block set , add position embeddings for visible image patches: , is the embedded position information; the image block embedded with the position information is input into the visual Transformer encoder, and the decoder is used to predict and reconstruct the masked image block; The first visual Transformer encoder is pre-trained using the above-mentioned pre-training method, a priori network is constructed based on the pre-trained first visual Transformer encoder, a posterior network is constructed using a second visual Transformer encoder having the same structural parameters as the first visual Transformer encoder, and an object contour segmentation network is constructed based on a Unet network; wherein the priori network is capable of encoding image features obtained by the first visual Transformer encoder into a priori distribution in a latent space, and the posterior network is capable of encoding image features obtained by the second visual Transformer encoder into a posterior distribution in a latent space; In the training phase, the nodule ultrasound image samples are input into the prior network to obtain the prior distribution, and the corresponding labels of the nodule ultrasound image samples are used as conditional variables. The nodule ultrasound image samples and their corresponding labels are concatenated and input into the posterior network after convolution dimensionality reduction to obtain the posterior distribution. The KL divergence is calculated based on the prior distribution and the posterior distribution. ; The nodule ultrasound image sample and prior distribution are input into the target contour segmentation network, the prediction result is output, and the cross entropy loss between the prediction result and the true value is calculated. ; Among them, the latent variables sampled from the prior distribution are injected into the bottleneck layer of the target contour segmentation network; The nodule ultrasound image sample and posterior distribution are input into the target contour segmentation network, the prediction result is output, and the cross entropy loss between the prediction result and the true value is calculated. ; Among them, the latent variables sampled from the posterior distribution are injected into the bottleneck layer of the target contour segmentation network; Calculate the total training loss , ; In the inference stage, the ultrasound image of the nodule to be segmented is input into the trained prior network to obtain the prior distribution, the obtained prior distribution and its corresponding ultrasound image of the nodule to be segmented are input into the trained target contour segmentation network, and the segmentation result of the nodule ultrasound image is output, and then the target contour confidence is obtained, which is the standard deviation of each pixel point.

2. The method according to claim 1, wherein: The target contour segmentation network takes the nodule ultrasound image as input to its encoder and injects the latent variable into the bottleneck layer; The latent variables are sampled from the prior distribution or the posterior distribution during training, and are sampled from the prior distribution after training and application. The injection is to expand the latent variable to the same dimension as the bottleneck layer feature, then concatenate it with the feature output by the encoder, and then pass it to the decoder after dimensionality reduction.

3. The method according to claim 1, characterized in that The prior distribution and the posterior distribution are both Gaussian distributions.

4. The method according to claim 1, wherein The method visualizes the confidence of the target contour through color.

5. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 4.

6. A target contour segmentation system, characterized in that: The system includes a pre-training module, a priori network, a posterior network, a target contour segmentation network and a calculation module; wherein: The pre-training module is configured to use an asymmetric encoder-decoder architecture for pre-training. In the pre-training stage, it is assumed that the original input nodule ultrasound image is , which is divided into Image blocks: , through random mask operation, the image is randomly divided into a set of visible blocks and mask block set , add position embeddings for visible image patches: , is the embedded position information; the image block embedded with the position information is input into the visual Transformer encoder, and the decoder is used to predict and reconstruct the masked image block; The prior network is configured to pre-train the first visual Transformer encoder using the following pre-training method, obtain a first encoding feature of the nodule ultrasound image using the pre-trained first visual Transformer encoder, and obtain a prior distribution corresponding to the first encoding feature; The target contour segmentation network is configured to obtain a nodule segmentation result using a Unet network based on the nodule ultrasound image and its corresponding probability distribution, and then obtain a target contour confidence, wherein the probability distribution is a prior distribution or a posterior distribution; The posterior network is configured to be used in training, and is constructed using a second visual Transformer encoder having the same structural parameters as the pre-trained first visual Transformer encoder to obtain a second encoded feature of the nodule ultrasound image, while using the expert-annotated label as a conditional variable, and a posterior distribution corresponding to the conditional variable and the second encoded feature; During training, the system inputs nodule ultrasound image samples into the prior network to obtain a prior distribution, uses the corresponding label of the nodule ultrasound image sample as a conditional variable, concatenates the nodule ultrasound image sample and its corresponding label, and then inputs the convolutional dimension reduction into the posterior network to obtain a posterior distribution. The KL divergence is calculated based on the prior distribution and the posterior distribution. ; Input the nodule ultrasound image sample and prior distribution into the target contour segmentation network, output the prediction result, and calculate the cross entropy loss between the prediction result and the true value ; Among them, the latent variables sampled from the prior distribution are injected into the bottleneck layer of the target contour segmentation network; the nodule ultrasound image sample and the posterior distribution are input into the target contour segmentation network, the prediction result is output, and the cross entropy loss between the prediction result and the true value is calculated ; Among them, the latent variables sampled from the posterior distribution are injected into the bottleneck layer of the target contour segmentation network; the total loss is calculated as , ; When the trained system is used, the ultrasound image of the nodule to be segmented is used as the input of the prior network to obtain the prior distribution, and then the obtained prior distribution and the corresponding ultrasound image of the nodule to be segmented are input into the trained target contour segmentation network, and the segmentation result of the ultrasound image of the nodule is output; The calculation module is configured to calculate the target contour confidence based on the segmentation result of the ultrasound image of the nodule to be segmented, and the target contour confidence is the standard deviation of each pixel.

7. The system according to claim 6, characterized in that The injection is to expand the latent variable to the same dimension as the bottleneck layer feature, then concatenate it with the feature output by the encoder, and then pass it to the decoder after dimensionality reduction.

Images