Semi-supervised medical image segmentation method based on two-way feature alignment

Abstract

The invention relates to a semi-supervised medical image segmentation method based on double-path feature alignment, and belongs to the field of medical image processing. Aiming at the problems that medical image annotation is scarce, high-frequency information is sensitive and feature fusion is not accurate in an existing method, a three-branch network architecture is provided, a main network extracts original image features, and an auxiliary network processes high-frequency and low-frequency images subjected to discrete wavelet decomposition respectively; cross-branch feature deformation correction and fusion are realized through a wavelet feature alignment module (encoder) and a simple feature alignment module (decoder); and unsupervised loss dynamic training is carried out in combination with supervised loss and pseudo-label, so that label dependence is relieved. According to the method, the segmentation precision and the boundary detail retention capability are effectively improved, the noise robustness is enhanced, the annotation data dependence is remarkably reduced, meanwhile, the cross-branch fusion efficiency is improved through a feature alignment mechanism, and the method has high clinical application value.

Description

Technical Field

[0001] The present invention belongs to the field of medical image processing and relates to a semi-supervised medical image segmentation method based on two-way feature alignment. Background Art

[0002] Medical image segmentation, a core technology for computer-aided diagnosis, aims to accurately separate target regions (such as lesions and organs) from complex medical images (such as CT and MRI) to provide clinicians with clear anatomical visualization support. With breakthroughs in deep learning technology, fully supervised learning methods have made significant progress in segmentation accuracy, but they are highly dependent on large-scale, high-quality annotated data. However, medical image annotation faces the following inherent challenges:

[0003] 1) The labeling process requires the reliance of experienced radiologists, which is labor-intensive and time-consuming;

[0004] 2) Imaging noise (such as CT ray scattering and motion artifacts) causes blurred image boundaries, exacerbating annotation ambiguity;

[0005] 3) Differences in doctors’ subjective judgments may introduce labeling inconsistencies and affect the generalization ability of the model.

[0006] To alleviate the scarcity of labeled data, semi-supervised learning frameworks have been widely introduced, improving model performance by synergistically leveraging a small amount of labeled data with a large amount of unlabeled data. Existing methods often employ multi-branch network structures, extracting complementary features from heterogeneous branches and implementing knowledge transfer through consistency constraints. However, these approaches still face two major technical bottlenecks:

[0007] First, there is a flaw in the sensitivity to high-frequency information. For example, when the typical algorithm XNet lacks high-frequency components (such as edges and textures) in the input image, the segmentation accuracy drops significantly because its feature extraction module does not explicitly model the robustness of frequency domain information.

[0008] Secondly, there is the problem of inaccurate feature fusion across branches. The feature maps output by different branches are spatially misaligned due to differences in receptive fields or spatial deformation. Direct splicing or addition operations cause feature conflicts, especially in small objects or edge areas, resulting in loss of detailed information.

[0009] Current feature alignment methods often employ attention mechanisms or simple deformable convolutions, but these methods fail to fully consider the decoupling of frequency-domain features and the dynamic generation of cross-modal deformation fields. For example, while traditional wavelet transforms can separate high- and low-frequency components, fixed-threshold fusion strategies struggle to adapt to the dynamic distribution of frequency-domain features in complex medical images. Furthermore, the transfer of pseudo-label noise is a prominent issue in existing semi-supervised frameworks. In particular, when the low-frequency sub-network's predictions are inconsistent with those of the main network, misaligned features can exacerbate pseudo-label ambiguity, leading to unstable model convergence.

[0010] To address the above problems, there is an urgent need for a semi-supervised segmentation method that can achieve adaptive alignment of frequency domain features across branches, while reducing the dependence on annotations and improving the model's robustness to missing frequency domain information and its ability to restore details. Summary of the Invention

[0011] In view of this, the purpose of the present invention is to provide a semi-supervised medical image segmentation method based on two-way feature alignment to solve the problem of limited model performance caused by the scarcity of medical image annotation data.

[0012] In order to achieve the above object, the present invention provides the following technical solutions:

[0013] A semi-supervised medical image segmentation method based on two-way feature alignment includes the following steps:

[0014] S1: Data preparation stage: obtain medical image dataset, slice the 3D data into 2D slices and unify the size, and divide the dataset into training set and validation set;

[0015] S2: Three-branch network feature extraction stage, the main network extracts the features of the original image, and the two auxiliary networks extract the features of the high-frequency decomposition image and the low-frequency decomposition image respectively;

[0016] S3: Downsampling feature alignment and fusion stage, using the wavelet feature alignment module to align and fuse the features of the low-frequency sub-network and the main network across branches;

[0017] S4: Upsampling feature alignment and fusion stage, using a simple feature alignment module to align and fuse the decoder features of the high-frequency sub-network and the main network;

[0018] S5: During the model training phase, the total loss of the supervised loss and the pseudo-label unsupervised loss is calculated, and the network parameters are updated until convergence.

[0019] Furthermore, the S1 specifically includes:

[0020] The three-dimensional medical images were sliced into two-dimensional cross sections and normalized to a size of 512×512×3. The dataset was divided into a training set and a validation set in proportion, and the training set was further divided into labeled data and unlabeled data. The original image was decomposed into a high-frequency image H by discrete wavelet transform. image With the low-frequency image L image ,in:

[0021] H image = IDWT(DWT(I image )-L)

[0022] L image =IDWT(L)

[0023] Among them Iimage is the original image, DWT is discrete wavelet transform, IDWT is inverse discrete wavelet transform, and L is the low-frequency signal.

[0024] Furthermore, the encoder of the three-branch network in S2 uses ResNet-50 as the backbone network, and the main network, high-frequency sub-network and low-frequency sub-network extract features of the original image, high-frequency image and low-frequency image respectively; the first convolutional layer of each branch contains 3 bottleneck residual blocks, and the extracted features are down-sampled by maximum pooling to obtain the first layer feature map I Main , I High , I Low .

[0025] Furthermore, the down-sampling feature alignment and fusion in S3 includes:

[0026] The features of the main network and the low-frequency sub-network are subjected to 1×1 convolution dimensionality reduction, and then concatenated and subjected to two 3×3 convolutions to generate the feature deformation field. use The low-frequency sub-network features are spatially deformed and aligned; the deformation features and the main network features are decomposed by wavelet transform respectively, and the high and low frequency components are stacked respectively. Then the low frequency components are convolved by 3×3, and the high frequency components are convolved by 1×1, and then restored to the fusion feature I by inverse wavelet transform. Output .

[0027] Furthermore, the generation of the characteristic deformation field satisfies:

[0028]

[0029] Among them I Main_down and I Low_down are the main network and low-frequency sub-network features after dimensionality reduction.

[0030] Furthermore, the up-sampling feature alignment and fusion in S4 includes:

[0031] The decoder features of the main network and high-frequency sub-network are upsampled and 1×1 convolution is used for dimensionality reduction. After splicing, two 3×3 convolutions are performed to generate the feature deformation field. use The high-frequency sub-network features are spatially deformed and aligned, and the deformed features are concatenated with the main network features and fused through a 3×3 convolution.

[0032] Furthermore, the generation of the characteristic deformation field satisfies:

[0033]

[0034] Among them I Main_Dr and I High_Dr are the main network and high-frequency sub-network features after dimensionality reduction.

[0035] Furthermore, the total loss Ltotal in S5 is the weighted sum of the supervised loss Lsup and the unsupervised loss Lunsup:

[0036] L total =L sup +λL unsup

[0037] in λ max is the preset maximum value, epoch is the current training round number, epoch max is the total number of rounds.

[0038] Furthermore, the supervision loss Lsup is the sum of the three-branch network prediction result and the Dice loss of the label:

[0039]

[0040] in are the prediction results of the high-frequency sub-network, the main network, and the low-frequency sub-network, respectively. i For label.

[0041] Furthermore, the unsupervised loss Lunsup is the sum of the Dice losses of the three-branch prediction results that are pseudo-labels of each other:

[0042]

[0043] in and Represent the bidirectional pseudo-label losses of high-frequency-main network and low-frequency-main network respectively.

[0044] The beneficial effects of the present invention are as follows: the present invention realizes the coordinated optimization of high-frequency details and low-frequency semantics through a three-branch network architecture, and uses a wavelet feature alignment module to perform deformation correction and cross-domain fusion on low-frequency features, effectively solving the problem of feature offset caused by local deformation interference and modal differences in medical images; the high-frequency enhancement mechanism based on simple feature alignment significantly improves the restoration accuracy of lesion edges and textures, and overcomes the segmentation fuzziness defect caused by the sensitivity of traditional methods to high-frequency information; combined with a dynamic weighted pseudo-label training strategy, while reducing label dependence, the model's feature mining ability for unlabeled data is enhanced to ensure generalization performance in semi-supervised scenarios; the cross-branch feature alignment mechanism optimizes the feature fusion path through bidirectional constraints, avoids multi-scale information loss, and achieves more efficient context modeling. This method shows excellent stability and adaptability in complex lesion area segmentation and small sample scenarios, providing a reliable solution for multimodal medical image analysis.

[0045] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0047] Figure 1 : is a network structure diagram of the medical image segmentation method based on two-way feature alignment in the present invention;

[0048] Figure 2 This is a structural diagram of the wavelet feature alignment module in the present invention;

[0049] Figure 3 This is a structural diagram of a simple feature alignment module in the present invention. DETAILED DESCRIPTION

[0050] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0051] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.

[0052] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0053] See also Figures 1 to 3 , Figure 1 The algorithm network structure diagram of the present invention includes the following steps:

[0054] Step 1: Data preparation: Obtain a medical image dataset. If the images in the dataset are 3D data, perform 2D slicing and unify the size. Then, divide the dataset into a training set and a validation set. The specific steps are as follows:

[0055] Step 101: Slice the three-dimensional volume medical image into two-dimensional images along the cross section;

[0056] Step 102: Normalize the segmented image and unify the image size to 512×512×3;

[0057] Step 103: Divide the images in step 102 into a training set and a validation set according to the ratio.

[0058] Step 104: Decompose the data divided in S12 into high-frequency and low-frequency images after discrete wavelet transform and discrete inverse wavelet transform, and then send them into the network for feature extraction according to batch random sampling. image , the low-frequency signal decomposed by wavelet transform is L, then the generation of high- and low-frequency images can be expressed as:

[0059] H image = IDWT(DWT(I image )-L) (1)

[0060] L image =IDWT(L) (2)

[0061] Step 2: Three-branch network feature extraction stage: The algorithm network structure of the present invention is a three-branch architecture. Its main network is used to extract all features of the original image, while the two auxiliary networks are used to receive the high-frequency and low-frequency images decomposed from the original image, and then perform targeted feature extraction on the two decomposed images. The specific steps are as follows:

[0062] Step 201: The training set data in step 103 is input into a three-branch encoder with ResNet-50 as the backbone network in batches for feature extraction. First, the encoder performs feature extraction on the original image and high-frequency and low-frequency images respectively. The specific feature extraction steps are as follows:

[0063] T1=ReLU(BN(Conv 1×1 (T0))) (3)

[0064] T2=ReLU(BN(Conv 3×3 (T1))) (4)

[0065] T3=ReLU(BN(Conv 1×1 (T2))+T0) (5)

[0066] Where T0 is the input feature of the bottleneck residual block, Conv 1×1 (·) and Conv 3×3 (·) represents the two-dimensional convolution operation with the convolution kernel size of 1×1 and 3×3, respectively. BatchNorm(·) represents the batch normalization operation. ReLU(·) is the activation function used by the network. T1, T2, and T3 represent the features after the first 1×1 convolution, the features after the next 3×3 convolution, and the final output features after the second 1×1 convolution and jump line connection, respectively.

[0067] Step 202: After the three different input data are fed into the three-branch encoder, they pass through the first convolutional layer of the encoder, i.e., three connected bottleneck residual blocks, and then undergo a downsampling operation to obtain the first-layer feature maps of the high-frequency branch network, the low-frequency branch network, and the main network. The specific steps are as follows:

[0068] I High =MaxPool 2×2 (ConvLayer H_1 (H image )) (6)

[0069] I Main =MaxPool 2×2 (ConvLayer M_1 (I image )) (7)

[0070] I Low =MaxPool 2×2 (ConvLayer L_1 (L image )) (8)

[0071] Among them, I High , I Low They represent the feature maps of the high and low frequency images input by the network in S13 after being output by the first layer of different branch network encoders. 2×2 (·) represents the maximum pooling operation with a size of 2×2 and a stride of 1. In addition, ConvLayer M_1 (·), ConvLayer L_1 (·), ConvLayer M_1 (·) represent the high-frequency branch network, the low-frequency branch network and the first convolutional layer of the main network encoder, respectively, and this convolutional layer consists of three bottleneck residual blocks in the ResNet50 network.

[0072] Step 3: Downsampling Feature Alignment and Fusion: A wavelet feature alignment module is used to enable the features of the low-frequency sub-network to cross branches and guide the overall feature extraction of the main network. In this process, the wavelet feature alignment module can help the network effectively complete feature alignment and then fully fuse the encoder features from different branches.

[0073] Step 301: Since the first layer of the encoder of the three-branch network will obtain the three features extracted in step 2, only the low-frequency sub-network output features and the main network output features are needed at this stage. Its main function is to use the features of the low-frequency sub-network to enhance the features of the main network, thereby guiding the main network to learn more critical information. This process involves feature alignment and feature fusion operations. The two dimensionality-reduced features are then concatenated and sent to two 3×3 convolutional layers with a stride of 1. The specific steps are as follows:

[0074] I Main_down =Conv 1×1 (I Main ) (9)

[0075] I Low_down =Conv 1×1 (I Low ) (10)

[0076]

[0077] First, 1×1 convolution is used to reduce the dimensionality of the features of the low-frequency sub-network and the main network. This is beneficial to reduce the number of network parameters and the overall amount of computation when generating the feature deformation field later, and maintain the linear transmission of information. 1×1 (·) represents a convolution operation with a kernel size of 1×1 and a stride of 1. Main_down and I Low_downRepresents the main network and low-frequency sub-network features after channel number adjustment. Concat(·) is a feature concatenation operation used to combine I Main_down and I Low_down The two features are concatenated in the channel dimension, and ×2 means that the operation is performed twice. is the result of the convolutional network output, that is, the feature deformation field required for the feature alignment operation. Among them, E represents the Encoder. The subscript l = 1 indicates that the feature deformation field is located in the first layer of the encoder. In addition, The spatial dimension of is the same as the input feature, but the channel dimension is 2, which describes the offset of the pixel in the horizontal and vertical directions respectively.

[0078] Step 302: Calculate the characteristic deformation field After that, it is necessary to use it to deform the low-frequency sub-network features to achieve alignment with the main network features. Then, the low-frequency sub-network features that have completed feature deformation and alignment are subjected to discrete wavelet transform with the main network features to obtain features in both high-frequency and low-frequency dimensions. The specific steps are as follows:

[0079]

[0080] I Warp_LL ,I Warp_Concat(LH,HL,HH) =DWT(I Low_warp ) (13)

[0081] I Main_LL ,I Main_Concat(LH,HL,HH) =DWT(I Main_down ) (14)

[0082] Among them, I Low_warp (i,j) and I Low_down (i, j) is the pixel representation of the deformed feature and the undeformed feature at the position (i, j), and and Characteristic deformation field The horizontal and vertical offsets at position (i, j). Warp_LL and I Warp_Concate(LH,HL,HH) They are the characteristics of deformation completion I Low_warp The high and low frequency information is decomposed. Warp_Concat(LH,HL,HH) Concat(LH,HL,HH) in indicates that the feature is composed of three high-frequency information decomposed by wavelet transform. Main_LL and I Main_Concat(LH,HL,HH) They are respectively the high-frequency and low-frequency information decomposed from the main network features.

[0083] Step 303: Recombining and fusing the high and low frequency signals of the two features, that is, concatenating the high and low frequency sub-signals corresponding to the features, and then extracting the most important feature information through 3×3 or 1×1 convolution. L and I H As the high- and low-frequency information in the same feature map, the discrete inverse wavelet transform is used to restore it back to a complete feature in the spatial domain, thus completing the entire process from feature alignment to feature fusion of the two features. Finally, since the encoder has a total of four convolutional layers, the above operation is repeated four times in the encoder stage to complete the feature alignment and fusion operation of the encoder part. The specific steps are as follows:

[0084] I L =ReLU(BN(Conv 3×3 (Concat(I Warp_LL ,I Main_LL )))) (15)

[0085] I H =ReLU(BN(Conv 3×3 (Concat(I Warp_Concat(LH,HL,HH) ,I Main_Concat(LH,HL,HH) )))) (16)

[0086] I Output =IDWT(I L ,I H ) (17)

[0087] Among them, I L and I H They are respectively composed of high-frequency and low-frequency information decomposed from two different features in S34, and the high-frequency and low-frequency information are fused after extracting features with 3×3 and 1×1 convolution kernels and adjusting the number of channels. Output That is the final fusion feature, and IDWT(·) represents the operation of discrete inverse wavelet transform.

[0088] Step 4: Upsampling feature alignment and fusion stage: Using a simple feature alignment module, the decoder features from the high-frequency sub-network can be used to guide the decoder of the main network to extract features and restore details, which helps the network better grasp the restoration of detailed information and accelerate the convergence of the model.

[0089] Step 401: Upsample the encoder downsampled features to restore the resolution. The upsampled features of the high-frequency sub-network decoder are then used to enhance the upsampled features of the main network decoder, thereby guiding the network to better restore image detail information during the decoder phase. Two 1×1 convolution kernels are then used to reduce the dimensionality of the upsampled features and the upsampled features of the high-frequency sub-network, unifying their dimensions and reducing the number of network parameters. The specific steps are as follows:

[0090] I up =Ups(I Encoder ) (18)

[0091] I Main_Dr =Conv 1×1 (I Main_up ) (19)

[0092] I High_Dr =Conv 1×1 (I High_up ) (20)

[0093] Among them, I up is the feature with increased spatial scale after upsampling. Encoder It is the global semantic structure feature obtained by gradually downsampling in the encoder stage, which is the output result of the last layer of the encoder in S37. Ups(·) is the upsampling method to restore the feature resolution. Main_up and I Main_Dr Represent the up-sampled features of the main network and the up-sampled dimensionality reduction features of the main network respectively. Similarly, I High_up and I High_Dr It is the upsampling feature of the high-frequency sub-network and its upsampling dimension reduction feature.

[0094] Step 402: After obtaining the two dimensionality reduction features, they are concatenated and fed into two 3×3 convolutional layers with a step size of 1 to generate the feature deformation field required for feature alignment. After that, it is necessary to use it to deform the low-frequency sub-network features to achieve alignment with the main network features. After obtaining the deformed aligned features, the two features to be fused are stacked and sent to a 3×3 convolutional layer for simple feature fusion. The specific steps are as follows:

[0095]

[0096]

[0097] I output =ReLU(BN(Conv 3×3 (Concat(I Hig_warp ,I Main_up )))) (twenty three)

[0098] Among them, Concat(I Main_Dr ,I High_Dr ) represents the main network decoder feature I Main_Dr and high-frequency sub-network decoder features I High_Dr Perform splicing in the channel dimension. is the characteristic deformation field of the first layer of the decoder stage. Where D represents the decoder. Similarly, The spatial dimension of is the same as the input feature, and its channel dimension is 2, which describes the offset of the pixel in the horizontal and vertical directions respectively. Hig_warp (i,j) and I Hig_down (i, j) is the pixel representation of the deformed feature and the undeformed feature at the position (i, j), and and Characteristic deformation field The horizontal and vertical offsets at position (i, j). output The output features of this module have the same size as the input data, keeping the spatial and channel dimensions unchanged.

[0099] Step 5: Model training phase: Dice Loss is used as the loss function. Simultaneously, the supervised loss consisting of the three-branch network and the label is calculated, along with the unsupervised loss consisting of the three prediction results acting as pseudo-labels, to obtain the total loss of the entire network. The total loss is used to update the parameters of the three-branch network until the model converges. Here, the sum of the supervised and unsupervised Dice Losses is used as the loss function, as shown in the following formula:

[0100] L total =L sup +λL unsup (twenty four)

[0101] Among them, L sup is the supervision loss in the network, and L unsup = is the unsupervised loss in the network, and λ is used as a hyperparameter to balance the ratio between supervised loss and unsupervised loss. Since this study completes the model training by gradually adding unlabeled data to the input data during neural network training, the size of λ will also increase with the number of training rounds. The calculation formula of λ is:

[0102]

[0103] In addition, the total supervision loss L sup The calculation process can be expressed as:

[0104]

[0105] in Represents the prediction results of the high-frequency sub-network, main network, and low-frequency sub-network output, and y i is the label of the annotated data.

[0106] Unsupervised loss L unsupIt can be expressed as:

[0107]

[0108] in It is explained that the prediction results output by the high-frequency sub-network and the prediction results output by the main network are pseudo-labels of each other, so as to calculate the unsupervised loss. Similarly. And both can be expressed as:

[0109]

[0110]

[0111] in represents the pseudo labels generated by the high-frequency sub-network, the main network, and the low-frequency sub-network. L(*) is the Dice loss, which can be calculated as follows:

[0112]

[0113] Among them, N represents the total number of pixels, p represents the predicted segmentation value, g represents the true label value, and s is to avoid division by zero.

[0114] like Figure 2 As shown, the wavelet feature alignment module processing process in the present invention is:

[0115] Step 1: Record the input original image data as and Where B, C, W, and H represent the batch size (Batch), the number of channels (Channel), and the width (Width) and height (Height) of the feature map, respectively. First, a 1×1 convolution is used to reduce the dimensionality of the features of the low-frequency sub-network and the main network. This helps reduce the number of network parameters and the overall computational effort when subsequently generating the feature deformation field, maintaining linear information transfer. The two reduced features are then concatenated and fed into two 3×3 convolutional layers with a stride of 1. The processing is expressed as follows:

[0116] I Main_down =Conv 1×1 (I Main ) (31)

[0117] I Low_down =Conv 1×1 (I Low ) (32)

[0118]

[0119] Among them, Conv 1×1(·) represents a convolution operation with a kernel size of 1×1 and a stride of 1. Main_down and I Low_down Represents the main network and low-frequency sub-network features after channel number adjustment. Concat(·) is a feature concatenation operation used to combine I Main_down and I Low_down The two features are concatenated in the channel dimension, and ×2 means that the operation is performed twice. is the result of the convolutional network output, that is, the feature deformation field required for the feature alignment operation. Among them, E represents the Encoder. The subscript l = 1 indicates that the feature deformation field is located in the first layer of the encoder. In addition, The spatial dimension of is the same as the input feature, but the channel dimension is 2, which describes the offset of the pixel in the horizontal and vertical directions respectively.

[0120] Step 2: Calculate the characteristic deformation field After that, it is necessary to use it to deform the low-frequency sub-network features to achieve alignment with the main network features. Then, the low-frequency sub-network features that have completed feature deformation and alignment are subjected to discrete wavelet transform with the main network features to obtain features in both high-frequency and low-frequency dimensions. The specific steps are as follows:

[0121]

[0122] I Warp_LL ,I Warp_Concat(LH,HL,HH) =DWT(I Low_warp ) (35)

[0123] I Main_LL ,I Main_Concat(LH,HL,HH) =DWT(I Main_down ) (36)

[0124] Among them, I Low_warp (i,j) and I Low_down (i, j) is the pixel representation of the deformed feature and the undeformed feature at the position (i, j), and and Characteristic deformation field The horizontal and vertical offsets at position (i, j). Warp_LL and I Warp_Concate(LH,HL,HH) They are the characteristics of deformation completion I Low_warp The high and low frequency information is decomposed. Warp_Concat(LH,HL,HH) Concat(LH,HL,HH) in indicates that the feature is composed of three high-frequency information decomposed by wavelet transform. Main_LL and I Main_Concat(LH,HL,HH) They are respectively the high-frequency and low-frequency information decomposed from the main network features.

[0125] Step 3: Recombine and fuse the high and low frequency signals of the two features, that is, concatenate the high and low frequency sub-signals corresponding to the features, and then extract the most important feature information through 3×3 or 1×1 convolution. L and I H As the high- and low-frequency information in the same feature map, the discrete inverse wavelet transform is used to restore it back to a complete feature in the spatial domain, thus completing the entire process from feature alignment to feature fusion of the two features. Finally, since the encoder has a total of four convolutional layers, the above operation is repeated four times in the encoder stage to complete the feature alignment and fusion operation of the encoder part. The specific steps are as follows:

[0126] I L =ReLU(BN(Conv 3×3 (Concat(I Warp_LL ,I Main_LL )))) (37)

[0127] I H =ReLU(BN(Conv 3×3 (Concat(I Warp_Concat(LH,HL,HH) ,I Main_Concat(LH,HL,HH) )))) (38)

[0128] I Output =IDWT(I L ,I H ) (39)

[0129] Among them, I L and I H They are respectively composed of high-frequency and low-frequency information decomposed from two different features in S34, and the high-frequency and low-frequency information are fused after extracting features with 3×3 and 1×1 convolution kernels and adjusting the number of channels. Output That is the final fusion feature, and IDWT(·) represents the operation of discrete inverse wavelet transform.

[0130] like Figure 3 As shown, the processing process of the simple feature alignment module in the present invention is:

[0131] Step 1: Record the input original image data as and Where B, C, W, and H represent the batch size (Batch), the number of channels (Channel), and the width (Width) and height (Height) of the feature map, respectively. First, the features downsampled by the encoder are upsampled to restore the resolution. Then, two 1×1 convolution kernels are used to reduce the dimensionality of the upsampled features and the upsampled features of the high-frequency sub-network to unify the dimensions and reduce the number of network parameters. The specific steps are as follows:

[0132] I up =Ups(I Encoder ) (40)

[0133] I Main_Dr =Conv 1×1 (I Main_up ) (41)

[0134] I High_Dr =Conv 1×1 (I High_up ) (42)

[0135] Among them, I up is the feature with increased spatial scale after upsampling. Encoder It is the global semantic structure feature obtained by gradually downsampling in the encoder stage, which is the output result of the last layer of the encoder in S37. Ups(·) is the upsampling method to restore the feature resolution. Main_up and I Main_Dr Represent the up-sampled features of the main network and the up-sampled dimensionality reduction features of the main network respectively. Similarly, I High_up and I High_Dr It is the upsampling feature of the high-frequency sub-network and its upsampling dimension reduction feature.

[0136] Step 2: After obtaining the two dimensionality reduction features, they are concatenated and sent to two 3×3 convolutional layers with a step size of 1 to generate the feature deformation field required for feature alignment. After that, it is necessary to use it to deform the low-frequency sub-network features to achieve alignment with the main network features. After obtaining the deformed aligned features, the two features to be fused are stacked and sent to a 3×3 convolutional layer for simple feature fusion. The specific steps are as follows:

[0137]

[0138]

[0139] I output =ReLU(BN(Conv 3×3 (Concat(I Hig_warp ,I Main_up )))) (45)

[0140] Among them, Concat(I Main_Dr ,I High_Dr ) represents the main network decoder feature I Main_Dr and high-frequency sub-network decoder features I High_Dr Perform splicing in the channel dimension. is the characteristic deformation field of the first layer of the decoder stage. Where D represents the decoder. Similarly, The spatial dimension of is the same as the input feature, and its channel dimension is 2, which describes the offset of the pixel in the horizontal and vertical directions respectively. Hig_warp (i,j) and I Hig_down (i, j) is the pixel representation of the deformed feature and the undeformed feature at the position (i, j), and and Characteristic deformation field The horizontal and vertical offsets at position (i, j). output The output features of this module have the same size as the input data, keeping the spatial and channel dimensions unchanged.

[0141] Example 1: Multimodal Data Preprocessing and Feature Alignment Based on Wavelet Decomposition

[0142] Workflow:

[0143] (1) Data input: Liver CT three-dimensional volume data (512 × 512 × 300) were obtained, and the cross-sectional slices were 512 × 512 × 3 two-dimensional images, totaling 3000 images.

[0144] (2) Wavelet decomposition: Perform Haar wavelet transform on each image to generate high-frequency components H image (edge texture) and low-frequency component L image (organ outline), stored as H5PY file.

[0145] (3) Dataset division: The training set (2400 images) and the validation set (600 images) are divided into 8:2. The labeled data in the training set accounts for 20% (480 images), and the rest are unlabeled data.

[0146] (4) Feature alignment training:

[0147] Encoder processing: the main network inputs the original image, and the low-frequency sub-network inputs L image After extracting features through the first layer of ResNet-50, execute step S3:

[0148] to I Main (channel number 256) and I Low (Number of channels 256) 1×1 convolution is performed to reduce the dimension to 64 channels.

[0149] After concatenation, it is generated by two 3×3 convolutions (deformation field), using bilinear interpolation to I Low_down Perform spatial offset.

[0150] The aligned features are decomposed into LL / LH / HL / HH components by DWT, concatenated with the decomposition features of the main network, and restored by IDWT after 3×3 convolution fusion.

[0151] (5) Output: Encoder outputs fusion feature I Output , the Dice coefficient on the validation set reaches 0.892, which is 3.3% higher than the unaligned baseline (0.863).

[0152] Example 2: Real-time liver lesion segmentation guided by high-frequency details

[0153] Workflow:

[0154] (1) Decoder initialization: Load the pre-trained encoder, and the main network and high-frequency sub-network decoders use transposed convolution upsampling.

[0155] (2) Detail alignment reasoning:

[0156] Feature upsampling: Main network decoder features I Main_up (128×128×128) is upsampled to 256×256 using bilinear interpolation.

[0157] High-frequency guidance: High-frequency sub-network output I High_up (256×256×64) is reduced to 32 channels by 1×1 convolution and concatenated with the main network features.

[0158] Deformation Fusion: Using to I High_Dr Perform pixel-level offset, and I Main_up After splicing, 3×3 convolution outputs the final features.

[0159] (3) Semi-supervised training:

[0160] Loss calculation: The supervised loss uses the Dice loss of the labeled data, and the unsupervised loss λ increases linearly from 0.1 (epoch = 0) to 0.8 (epoch = 200).

[0161] Pseudo-label generation: The prediction results of the main network are processed and used as pseudo-labels for the high-frequency / low-frequency sub-networks.

[0162] (4) Results: On the test set of liver lesions smaller than 10 mm, the boundary segmentation HD95 index was reduced from 3.2 mm to 2.6 mm, and the inference speed reached 15 FPS (GPU: Tesla V100).

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A semi-supervised medical image segmentation method based on two-way feature alignment, characterized by: The following steps are involved: S1: Data preparation stage: obtain medical image dataset, slice the 3D data into 2D slices and unify the size, and divide the dataset into training set and validation set; S2: Three-branch network feature extraction stage, the main network extracts the features of the original image, and the two auxiliary networks extract the features of the high-frequency decomposition image and the low-frequency decomposition image respectively; S3: Downsampling feature alignment and fusion stage, using the wavelet feature alignment module to align and fuse the features of the low-frequency sub-network and the main network across branches; S4: Upsampling feature alignment and fusion stage, using a simple feature alignment module to align and fuse the decoder features of the high-frequency sub-network and the main network; S5: During the model training phase, the total loss of the supervised loss and the pseudo-label unsupervised loss is calculated, and the network parameters are updated until convergence.

2. The semi-supervised medical image segmentation method based on two-way feature alignment according to claim 1, characterized in that: Said S1 specifically includes: The three-dimensional medical images were sliced into two-dimensional cross sections and normalized to a size of 512×512×3. The dataset was divided into a training set and a validation set in proportion, and the training set was further divided into labeled data and unlabeled data. The original image was decomposed into a high-frequency image H by discrete wavelet transform. image With the low-frequency image L image ,in: H image =IDWT(DWT(I image )-L) L image =IDWT(L) Among them I image is the original image, DWT is discrete wavelet transform, IDWT is inverse discrete wavelet transform, and L is the low-frequency signal.

3. The semi-supervised medical image segmentation method based on two-way feature alignment according to claim 1, characterized in that: The encoder of the three-branch network in S2 uses ResNet-50 as the backbone network. The main network, high-frequency sub-network and low-frequency sub-network extract features of the original image, high-frequency image and low-frequency image respectively; the first convolutional layer of each branch contains 3 bottleneck residual blocks, and the extracted features are down-sampled by maximum pooling to obtain the first layer feature map I Main , I High , I Low .

4. The semi-supervised medical image segmentation method based on two-way feature alignment according to claim 1, characterized in that: The down-sampling feature alignment and fusion in S3 include: The features of the main network and the low-frequency sub-network are subjected to 1×1 convolution dimensionality reduction, and then concatenated and subjected to two 3×3 convolutions to generate the feature deformation field. use The low-frequency sub-network features are spatially deformed and aligned; the deformation features and the main network features are decomposed by wavelet transform respectively, and the high and low frequency components are stacked respectively. Then the low frequency components are convolved by 3×3, and the high frequency components are convolved by 1×1, and then restored to the fusion feature I by inverse wavelet transform. Output .

5. The semi-supervised medical image segmentation method based on two-way feature alignment according to claim 4, characterized in that: The generation of the characteristic deformation field satisfies: Among them I Main_down and I Low_down These are the characteristics of the main network and low-frequency sub-network after dimensionality reduction.

6. The semi-supervised medical image segmentation method based on two-way feature alignment according to claim 1, characterized in that: The up-sampling feature alignment and fusion in S4 include: The decoder features of the main network and high-frequency sub-network are upsampled and 1×1 convolution is used for dimensionality reduction. After splicing, two 3×3 convolutions are performed to generate the feature deformation field. use The high-frequency sub-network features are spatially deformed and aligned, and the deformed features are concatenated with the main network features and fused through a 3×3 convolution.

7. The semi-supervised medical image segmentation method based on two-way feature alignment according to claim 6, characterized in that: The generation of the characteristic deformation field satisfies: Among them I Main_Dr and I High_Dr are the main network and high-frequency sub-network features after dimensionality reduction.

8. The semi-supervised medical image segmentation method based on two-way feature alignment according to claim 1, characterized in that: The total loss Ltotal in S5 is the weighted sum of the supervised loss Lsup and the unsupervised loss Lunsup: THE total =L sup +λL unsup in λ max is the preset maximum value, epoch is the current training round number, epoch max is the total number of rounds.

9. The semi-supervised medical image segmentation method based on two-way feature alignment according to claim 8, characterized in that: The supervised loss Lsup is the sum of the three-branch network prediction result and the Dice loss of the label: in are the prediction results of the high-frequency sub-network, the main network, and the low-frequency sub-network, respectively. i For label.

10. The semi-supervised medical image segmentation method based on two-way feature alignment according to claim 8, characterized in that: The unsupervised loss Lunsup is the sum of the Dice losses of the three-branch prediction results that are pseudo-labels of each other: in and Represent the bidirectional pseudo-label losses of high-frequency-main network and low-frequency-main network respectively.

Images