A single sample image segmentation method based on double-layer coupled learning

Abstract

The present application belongs to the field of image processing and computer vision, and particularly relates to a single sample image segmentation method based on double-layer coupled learning. The present application introduces a registration task, performs spatial transformation learning from a source domain to a target domain, so as to obtain a plurality of pseudo-label data pairs that are well registered. A learning method based on double-layer optimization is adopted, a coupling dependency relationship between the registration and segmentation tasks is jointly modeled, and an optimal response feedback solving algorithm based on implicit gradient heuristic is designed. Based on the solving strategy of implicit gradient and outer product approximation, the influence of segmentation on the registration task can be dynamically and real-timely fed back, and finally efficient and stable learning is realized.

Description

Technical Field

[0001] This invention belongs to the field of image processing and computer vision. It involves using image data with only a single sample annotation and aims to establish a framework for coupled learning of collaborative registration and segmentation tasks. It adopts a two-layer optimized mathematical modeling paradigm to construct a single-sample image segmentation algorithm, specifically a single-sample medical image segmentation method based on two-layer coupled learning. Background Technology

[0002] With the continuous advancement of computer vision and deep learning technologies, image segmentation has attracted increasing attention. The goal of image segmentation is to subdivide an image into multiple regions with different structures. Traditional learning-based segmentation models typically require a large number of labeled images as training data. However, manually labeling these images is time-consuming and prone to errors, resulting in high human costs. In recent years, the rise of single-sample learning techniques has brought new hope to image segmentation. This technique allows models to learn using only a very small amount of labeled data. This research aims to explore the methods and applications of single-sample image segmentation.

[0003] Image segmentation tasks often face the challenge of limited labeled data, lacking sufficient labeled samples for single-sample segmentation. To address this challenge, a direct approach is to introduce a registration task to find the spatial transformation between an unlabeled image and a template image with applied atlas labels. This registration process allows atlases to be applied to the unlabeled image through spatial transformation, thus augmenting the data. Therefore, the accuracy of registration is crucial to segmentation performance. Furthermore, the annotation of anatomical structures can provide additional structural constraints to guide the registration process. Consequently, there is a high degree of coupling between registration and segmentation tasks in registration-based image segmentation. However, relying solely on naive learning strategies based on alternating training often leads to training instability and model collapse.

[0004] Several techniques for semantic segmentation based on few-shot data have emerged, but problems such as unstable training and poor performance still exist. For example, patent CN115546474A proposes a few-shot semantic segmentation method based on a learner ensemble strategy. This method introduces an additional auxiliary branch to identify base class targets on top of the meta-learner and estimates the differences between input images by introducing an adjustment factor, thereby promoting the model to generate accurate segmentation masks. In addition, patent CN112686850A proposes a few-shot segmentation method and system for CT images based on spatial location and prototype networks. This method uses local average pooling operations to obtain prototype vectors representing organ masks to evaluate similarity metrics and generate 2D / 3D segmentation results. The goal of this invention is to explore how to perform segmentation using a single labeled sample, learn spatial transformations through coupled registration tasks to generate pseudo-labels, and design an efficient and stable solution strategy based on a two-layer optimization learning framework to obtain robust and accurate segmentation results. Summary of the Invention

[0005] This invention aims to overcome the shortcomings of existing technologies by providing a single-sample image segmentation method based on a two-layer optimization paradigm. Addressing the challenge of insufficient training data, this technique introduces a registration task, performing spatial transformation learning from the source domain to the target domain. By utilizing multiple deformable fields in the output, it enables spatial transformation of the source image and a single labeled atlas, thereby creating pseudo-label data pairs. Through a two-layer optimization-based learning method, this technique jointly models the registration and segmentation tasks and designs an optimal response feedback algorithm based on implicit gradient heuristics, enabling dynamic real-time feedback of the impact of segmentation on the registration task.

[0006] The technical solution of the present invention:

[0007] A single-sample image segmentation method based on two-layer coupled learning, the specific steps of which are as follows:

[0008] 1) Prepare single-sample labeled data pairs and multiple target images.

[0009] 2) Perform unsupervised pre-training registration tasks. Based on the source and target images as training data, obtain multiple deformable fields from the source image to the target image.

[0010] 3) Construct a registration network to perform registration from the source image to the target image. Utilize the learned deformable field to perform spatial transformations on the source image and the label respectively to obtain multiple registered pseudo-label data pairs.

[0011] 4) Construct the leader's energy function Φ based on the registration task. r We designed a training loss based on the registration task as leader energy, and introduced dynamic semantic loss and intensity-based regular registration loss.

[0012] 5) Perform a supervised segmentation learning process. Construct a segmentation network, input the deformed source image and target image into the segmentation network to obtain the segmentation results of the deformed source image and the target image.

[0013] 6) Construct a loss based on the segmentation task as the follower energy function Φ s By using the probabilistic map of the predicted segmentation to generate a boundary-preserving reweighted map, the sensitivity to boundary noise is reduced, while emphasizing the optimization of the dissected boundary.

[0014] 7) Construct a mathematical model for bi-level optimization; define the leader function Φ. r and follower function Φ s The learning process of the registration network and the segmentation network can be represented as a two-layer optimization learning paradigm:

[0015]

[0016]

[0017] Where w r and w s These are the parameters for the registration network and the segmentation network, respectively; Let j represent the source image and the target image, and j represent the j-th target image sample. For the lower-level function with respect to the fixed variable w r Get information about variable w s The solution set, For the best response; and These represent the source image x s and tag z s The deformed source image and the deformed label obtained by performing a spatial transformation;

[0018] 8) The network training process corresponds to solving a two-layer optimization problem, and the following numerical solution method is proposed:

[0019] 8-1) First calculate Φ r Regarding variable w r gradient:

[0020]

[0021] Will Defined as G c , is represented as:

[0022]

[0023] 8-2) Calculate the response gradient of the optimal response with respect to the variable. as follows:

[0024]

[0025] in Represents the variable w s The second gradient, Represents the variable w s and w r The second-order gradient;

[0026] 8-3) Calculate the outer product approximation of the second-order gradient to the first-order gradient:

[0027]

[0028]

[0029] 8-4) Calculate G c Approximate representation of the first-order gradient:

[0030]

[0031] 9) Repeat step 8) until training converges and output the segmentation result.

[0032] The beneficial effects of this invention are:

[0033] This invention designs an image processing technique that allows the generation of pseudo-label data through registration tasks to support single-sample segmentation. A two-layer optimized modeling method is employed to accurately characterize the multi-task learning process of joint registration and segmentation tasks, and coupled dependency constraints are introduced. By adopting a solution strategy based on implicit gradients and outer product approximation, efficient and stable learning is ultimately achieved. This invention has the following characteristics:

[0034] (1) It reduces the dependence on training data, requiring only a single labeled sample, thereby reducing the cost of manual annotation;

[0035] (2) The program is simple and easy to implement;

[0036] (3) A two-level optimization paradigm was adopted to perform interpretable modeling of the joint task, which enhanced the understandability of the algorithm;

[0037] (4) A stable and efficient training process was achieved by using the first-order gradient feedback method;

[0038] (5) The program has a complete structure, supports multi-threaded operation, and has strong robustness;

[0039] (6) This technique can be transferred to different single-sample noisy data pairs, showing good generalization ability. Attached Figure Description

[0040] Figure 1 This is a flowchart of a single-sample image segmentation method based on two-layer coupled learning.

[0041] Figure 2 This is the final segmented image in the embodiment. Detailed Implementation

[0042] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0043] The present invention provides a single-sample image segmentation method based on two-layer coupled learning, the process of which is as follows: Figure 1 As shown.

[0044] In this embodiment, the registration network uses a shared dual encoder as a feature extraction module to extract features from the source and target images, and inputs these features into a single decoder to obtain a deformable field. The segmentation network uses a shared Unet network to perform a segmentation learning process on the transformed source and target images. The method includes the following steps:

[0045] 1) Prepare a single labeled data pair, including the source image x s and tag z s and target image set Where j belongs to the target image data set J.

[0046] 2) Perform unsupervised pre-training registration tasks and construct a registration network. The parameter is w r ;Transfer the source image x s and target image Input to registration network And generate deformable fields

[0047] 3) Generate pseudo-label data pairs by performing the registration learning process. The specific steps are as follows: x from the source image s and tag z s Based on the deformation field Perform a spatial transformation to obtain the deformed source image. and the deformed label This process can be expressed by the following formula: Among the symbols This represents an affine transformation of space.

[0048] 4) Construct the loss based on the registration task as the leader energy function Φ r :

[0049]

[0050]

[0051] in Represents a measure of category similarity. This represents the dynamic semantic consistency loss. This represents the cross-correlation strength similarity loss, used to ensure the source image after deformation. and target image Consistency of strength; This indicates smoothness loss, ensuring the deformation field. Smoothness; λ ncc and λ smo The introduced loss weight hyperparameter; dynamic semantic consistency loss. The definition is as follows:

[0052]

[0053] in λ dsc represents the introduced loss weight hyperparameter, and KL represents the KL divergence.

[0054] 5) Perform a supervised segmentation learning process to construct a segmentation network. The parameter is w s Specifically, it involves transforming the source image... and target image The segmentation network input to the shared encoder Output segmentation mask and the mask of the target image

[0055] 6) Construct a loss based on the segmentation task as the follower energy function Φ s :

[0056]

[0057] in, It is the cross-entropy loss of multiple anatomical structures. For dynamic semantic consistency loss, The loss is based on mismatch resistance mapping.

[0058] for By using the Softmax function, a probability map is defined for the transformed mask, and then the Laplacian operator (LAP) is introduced to compute the boundary weight map τ. MRM ,Right now

[0059]

[0060] in Let represent the Laplacian kernel, and Conv3d be a 3D convolution function. We utilize an MRM to reweight the Dice loss, focusing on individual voxels in the boundary regions of each class:

[0061]

[0062] Where η represents the pixel position in space. and These represent the predicted segmentation mask and the true label, respectively.

[0063] 7) Set the leader function Φ r and follower function Φ s This can be redefined as a two-layer optimization learning paradigm:

[0064]

[0065]

[0066] in For the lower-level function with respect to the fixed variable w r Get information about variable w s The solution set, This represents the optimal response. The above equation explicitly describes the coupling dependency between the registration and segmentation tasks, passing the optimal feedback from the lower layer regarding segmentation learning to the upper layer registration task.

[0067] 8) Solve the lower-level problem using multi-step gradient descent to obtain an approximate optimal response. Feed the optimal response back to the upper-level energy. Solve the upper-level problem and calculate the gradients of the upper-level registration variables. This process includes the leader update process for calculating the coupled gradients of the partitioned energy with respect to the registration variables.

[0068] Training and registration network (parameter is w) r and segmentation networks (parameter is w) s The following numerical solution method is proposed:

[0069] 8-1) First calculate Φ r Regarding variable w r gradient:

[0070]

[0071] In the above formula Reflects the registration variable w r For the registration energy Φ r The direct gradient; Defined as G c This reflects the segmentation variable Φ s For the registration variable wr The coupled gradient response.

[0072] 8-2) Obtain the coupled gradient response G using the chain rule. c Represented as:

[0073]

[0074] Previous two-level optimization algorithms solved G in an explicit manner. c Because it involves a second-order Hessian matrix, it is usually computationally complex and inefficient.

[0075] 8-3) Using the implicit differential theorem, we obtain Thus, the optimal response gradient with respect to the variable is obtained. as follows:

[0076]

[0077] in Represents the variable w s The second gradient, Represents the variable w s and w r The second-order gradient.

[0078] 8-4) According to the Gauss-Newton formula, calculate the approximate expression of the outer product from the second gradient to the first gradient:

[0079]

[0080]

[0081] 8-5) The outer product approximation of the second-order gradient to the first-order gradient:

[0082]

[0083] The quotient of the two scalars within the parentheses represents the feedback gradient of the segmentation energy relative to the registration parameters. This approximation decomposes the second-order Hessian matrix into a product of multiple first-order derivatives, thus significantly reducing computational complexity.

[0084] 9) Repeat step 8) above until training converges and output the segmentation result.

[0085] Through this two-layer coupled learning paradigm, this invention characterizes the inherent constraints between registration and segmentation, thereby achieving a collaborative optimization process that stably converges to the optimal result.

[0086] like Figure 2As shown, based on a single-sample labeled brain dataset, the input source image and target image are used as training sets to carry out a single-sample segmentation learning process. This invention can obtain fine segmentation results in complex terminal brain regions and different local brain structural regions.

Claims

1. A single-sample image segmentation method based on two-layer coupled learning, characterized in that, The specific steps are as follows: 1) Prepare single-sample labeled data pairs and multiple target images; 2) Perform unsupervised pre-training registration tasks; based on source and target images as training data, obtain multiple deformable fields from source and target images; 3) Construct a registration network to perform registration from the source image to the target image. Utilize the learned deformable field to perform spatial transformations on the source image and the label respectively to obtain multiple registered pseudo-label data pairs. 4) Constructing the leader's energy function Φ based on the registration task r ; We design a registration task-based training loss as leader energy and introduce dynamic semantic loss and intensity-based regular registration loss. 5) Implement a supervised segmentation learning process; Construct a segmentation network, input the deformed source image and target image into the segmentation network to obtain the segmentation results of the deformed source image and the segmentation results of the deformed target image; 6) Construct a loss based on the segmentation task as the follower energy function Φ s A reweighted graph that preserves the boundaries is generated using the probabilistic graph of the predicted segmentation, thereby reducing sensitivity to boundary noise while emphasizing the optimization of the dissected boundaries. 7) Construct a mathematical model for bi-level optimization; define the leader function Φ. r and follower function Φ s The learning process of the registration network and the segmentation network can be represented as a two-layer optimization learning paradigm: Where w r and w s These are the parameters for the registration network and the segmentation network, respectively; Let j represent the source image and the target image, and j represent the j-th target image sample. For the lower-level function with respect to the fixed variable w r Get information about variable w s The solution set, For the best response; and These represent the source image x s and tag z s The deformed source image and the deformed label obtained by performing a spatial transformation; 8) The network training process corresponds to solving a two-layer optimization problem, and the following numerical solution method is proposed: 8-1) First calculate Φ r Regarding variable w r gradient: Will Defined as G c , represented as: 8-2) Calculate the response gradient of the optimal response with respect to the variable. as follows: in Represents the variable w s The second gradient, Represents the variable w s and w r The second-order gradient; 8-3) Calculate the outer product approximation of the second-order gradient to the first-order gradient: 8-4) Calculate G c Approximate representation of the first-order gradient: 9) Repeat step 8) until training converges and output the segmentation result.

Images