Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation

Abstract

The invention discloses a Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation, and relates to the technical field of visual reasoning. Comprising the steps of image frequency domain transformation; performing Gaussian frequency domain masking; calculating similarity; a dynamic threshold strategy; a network prediction result; and loss calculation. According to the method, high-quality medical image segmentation can be realized, and a segmented result has clearer edge details.

Description

Technical Field

[0001] The present invention relates to the field of visual reasoning technology, and more particularly to a Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation. Background Art

[0002] Traditional fully supervised learning methods rely on a large amount of labeled data, which is often difficult to achieve in the medical field. To solve this problem, researchers have gradually turned to semi-supervised learning methods, which can effectively use unlabeled data to supplement the lack of labeled data. Semi-supervised learning aims to improve the segmentation accuracy and generalization ability of the model by constructing self-learning mechanisms, generative adversarial networks (GANs) or graphical models.

[0003] Existing semi-supervised medical image segmentation methods mainly include the following categories:

[0004] Generative Adversarial Networks (GANs): These methods use adversarial training between the generator and the discriminator to enable the model to generate more accurate segmentation results. GANs perform well in handling blurred edges and detail completion, but usually require a complex training process.

[0005] Consistency regularization: By applying different perturbations (such as noise, rotation, etc.) to the input image, the consistency of the model output is maintained, thereby improving the robustness of the model. This method can effectively improve the segmentation accuracy with a small amount of labeled data.

[0006] Self-training: The model is first trained on labeled data, and then the model is retrained on the prediction results of unlabeled data. The key to this method is how to choose reliable pseudo labels to avoid introducing wrong information.

[0007] Frequency domain processing: Some studies use frequency domain techniques such as Fourier transform to enhance the model's ability to perceive high-frequency information. These methods extract features in the frequency domain to improve the model's ability to recognize image details.

[0008] Although a variety of semi-supervised learning methods have been proposed and applied to medical image segmentation, there are still many challenges, such as how to effectively use unlabeled data, improve the model's sensitivity to details, and avoid overfitting. Therefore, research on these issues is still ongoing in order to achieve more efficient and accurate medical image segmentation.

[0009] Therefore, it is an urgent problem for those skilled in the art to propose a Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation to solve the difficulties existing in the prior art. Summary of the invention

[0010] In view of this, the present invention provides a Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation, which is used to solve the technical problems existing in the prior art.

[0011] In order to achieve the above object, the present invention provides the following technical solutions:

[0012] A Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation, comprising the following steps:

[0013] Image frequency domain transformation step: Convert the original input image from the spatial domain to the frequency domain through Fourier transform;

[0014] Gaussian frequency domain masking step: After converting the original input image into the frequency domain, it is processed using a Gaussian mask to obtain a masked frequency domain image;

[0015] Similarity calculation step: convert the masked frequency domain image into the spatial domain, and perform similarity calculation with the original input image to obtain a similarity calculation result;

[0016] Dynamic threshold strategy steps: If the similarity calculation result is lower than the threshold image, readjust the mask degree and repeat the Gaussian frequency domain mask and similarity calculation steps until the similarity calculation result of the masked spatial domain image and the original input image meets the standard;

[0017] Network prediction result step: input the original input image and the mask space domain image after dynamic threshold adjustment into the segmentation network to obtain the true result of the original input image and the prediction result corresponding to the mask space domain image;

[0018] Loss calculation step: convert the true result of the original input image and the predicted result corresponding to the mask space domain image into the frequency domain, and detect the high-frequency signal to calculate the high-frequency loss.

[0019] The above method optionally converts the original input image from the spatial domain to the frequency domain through Fourier transform as follows:

[0020] First, for the original input image I of shape (B, C, H, W, D), B represents the batch size, C represents the number of channels, H, W and D represent the height, width and depth of the original input image, a two-dimensional Fourier transform is performed on each layer of the original input image to obtain a frequency domain representation:

[0021]

[0022] Where F is the Fourier transform operator, b is the batch index, c is the channel index, d is the depth index, μ and v are the indices of the horizontal and vertical spatial frequencies in the Fourier spectrum, are the frequency components of the original input image in the frequency domain, m is the row index, and n is the column index.

[0023] In the above method, optionally, after converting the original input image into the frequency domain, the original input image is processed using a Gaussian mask, and the specific content of the masked frequency domain image is:

[0024] Design a Gaussian mask M, the form is as follows:

[0025]

[0026] Among them, (x, y) is the mask pixel point, (x0, y0) is the center point of the frequency domain, set to σ controls the mask strength. This Gaussian mask is centered around a Fourier shift operation:

[0027] M shift (u,v)=F shift {M(x,y)};

[0028] In the frequency domain, the Fourier transformed image I f Multiply it with the Gaussian mask M to get the masked frequency domain image

[0029]

[0030] The above method optionally converts the masked frequency domain image into the spatial domain, and performs similarity calculation with the original input image as follows:

[0031] Reconstruct the mask spatial domain image using the inverse Fourier transform of the mask frequency domain image:

[0032]

[0033] Calculate the similarity with the original input image I:

[0034]

[0035] Among them, <·> is the dot product of the vectors, and ||·|| is the norm of the vectors.

[0036] In the above method, the specific content of the optional dynamic threshold strategy is:

[0037] Using cosine similarity for dynamic threshold strategy, when the similarity is higher than the threshold η, the algorithm will use the processed mask frequency domain image for training to ensure the optimization of the model;

[0038] When the similarity is below the threshold, the algorithm gradually increases the control parameter σ to reduce the strength of the Gaussian mask.

[0039] In the above method, optionally, the segmentation model in the segmentation network is an MCF model.

[0040] In the above method, optionally, the specific content of detecting the high-frequency signal to calculate the high-frequency loss is:

[0041] First, initialize a Laplace kernel to detect the image output I outputs High frequency signal in:

[0042]

[0043] Here, the values ​​of α, β, and γ are 0, -1, and 4. The Laplacian kernel is first created as a tensor and expanded to shape (1, 1, 3, 3). Then, the Laplacian kernel is repeated across the channels to obtain the shape (1, C, 3, 3). The kernel is further expanded to shape (1, C, 1, 3, 3) by adding the depth dimension. Next, a high-pass filter is applied using the padding and stride parameters. The convolution operation is defined as:

[0044] High_pass_filter=F.conv3d(image,kernel,padding,stridE);

[0045] Define a high-frequency loss function to quantify the difference between the high-frequency signals in the real image and the predicted image. The loss function uses the high-pass filter defined previously to extract the true value y true and the predicted value y pred The high-frequency signal is calculated as follows:

[0046] h true =High_pass_filter(y true );

[0047] h pred =High_pass_filter(y pred );

[0048] The high-frequency loss is then calculated based on the L2 norm:

[0049] L hf =||h true -h pred ||2.

[0050] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation, which has the following beneficial effects:

[0051] It can achieve high-quality medical image segmentation and make the segmented results have clearer edge details. The Gaussian mask strategy improves the model's ability to capture local details of the image, and the dynamic threshold adjustment strategy avoids misleading in model learning; the mask and adjustment strategy enables the model to maintain high segmentation performance in different medical imaging modalities (such as CT and MRI), improving the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0053] Figure 1 A flowchart of a Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation provided by the present invention;

[0054] Figure 2 A schematic diagram of the design provided by the present invention;

[0055] Figure 3 This is a segmentation effect diagram provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] See also Figure 1 As shown, the embodiment of the present invention discloses a Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation, comprising the following steps:

[0058] Image frequency domain transformation step: Convert the original input image from the spatial domain to the frequency domain through Fourier transform;

[0059] Specifically, because frequency-domain image enhancement has many advantages, frequency-domain processing can more precisely control the preservation and filtering of information. Figure 2 As shown, the low-frequency part contains important structural information, while the high-frequency part contains details. Frequency domain enhancement can specifically enhance or suppress these components, and some frequency domain operations (such as convolution) are more efficient in frequency domain calculation, which can significantly reduce the calculation time, especially for large-scale data.

[0060] Gaussian frequency domain masking step: After converting the original input image into the frequency domain, it is processed using a Gaussian mask to obtain a masked frequency domain image;

[0061] Specifically, the Gaussian mask is a low-pass filter that can remove high-frequency noise in an image and retain the low-frequency part. Through this mask, some high-frequency information in the frequency domain can be selectively removed, thereby reducing noise and highlighting the main structural information of the image. The high-frequency signal will be suppressed or removed, and what remains is the low-frequency part that retains the general structure and morphology.

[0062] Similarity calculation step: convert the masked frequency domain image into the spatial domain, and perform similarity calculation with the original input image to obtain a similarity calculation result;

[0063] Specifically, ensure that the masking process is not excessive, causing damage to the image.

[0064] Dynamic threshold strategy steps: If the similarity calculation result is lower than the threshold image (i.e., the image is over-masked), readjust the masking degree and repeat the Gaussian frequency domain masking and similarity calculation steps until the similarity calculation result of the masked spatial domain image and the original input image meets the standard (i.e., the image is not over-masked and damaged);

[0065] Network prediction result step: input the original input image and the mask space domain image after dynamic threshold adjustment into the segmentation network to obtain the true result of the original input image and the prediction result corresponding to the mask space domain image;

[0066] Loss calculation step: convert the true result of the original input image and the predicted result corresponding to the mask space domain image into the frequency domain, and detect the high-frequency signal to calculate the high-frequency loss.

[0067] Specifically, the above method can solve the problem of blurred edges of medical images, and can optimize the frequency domain features of the image, enhance the learning effect of the model, thereby improving the overall performance and accuracy.

[0068] Furthermore, the specific content of converting the original input image from the spatial domain to the frequency domain through Fourier transform is:

[0069] First, for the original input image I of shape (B, C, H, W, D), B represents the batch size, C represents the number of channels, H, W and D represent the height, width and depth of the original input image, a two-dimensional Fourier transform is performed on each layer of the original input image to obtain a frequency domain representation:

[0070]

[0071] Where F is the Fourier transform operator, b is the batch index, c is the channel index, d is the depth index, μ and v are the indices of the horizontal and vertical spatial frequencies in the Fourier spectrum, are the frequency components of the original input image in the frequency domain, m is the row index, and n is the column index.

[0072] Specifically, mathematically speaking, the 3D volume of a medical image is defined as: I∈R W×H×L The goal of semi-supervised medical image segmentation is to predict the label map y∈{0,1,...,K-1} for each voxel. W×H×L , represents the position of the background and target in I, and K is the class number.

[0073] Furthermore, after the original input image is converted into the frequency domain, it is processed using a Gaussian mask, and the specific content of the masked frequency domain image is:

[0074] Design a Gaussian mask M, the form is as follows:

[0075]

[0076] Among them, (x, y) is the mask pixel point, (x0, y0) is the center point of the frequency domain, set to σ controls the mask strength. This Gaussian mask is centered around a Fourier shift operation:

[0077] M shift (u, v) = F shift {M(x, y)};

[0078] In the frequency domain, the Fourier transformed image I f Multiply it with the Gaussian mask M to get the masked frequency domain image

[0079]

[0080] Furthermore, the masked frequency domain image is converted to the spatial domain, and the specific contents of similarity calculation with the original input image are as follows:

[0081] Reconstruct the mask spatial domain image using the inverse Fourier transform of the mask frequency domain image:

[0082]

[0083] Calculate the similarity with the original input image I:

[0084]

[0085] Among them, <·> is the dot product of the vectors, and ||·|| is the norm of the vectors.

[0086] Furthermore, the specific content of the dynamic threshold strategy is:

[0087] Using cosine similarity for dynamic threshold strategy, when the similarity is higher than the threshold η, the algorithm will use the processed mask frequency domain image for training to ensure the optimization of the model;

[0088] When the similarity is lower than the threshold, the algorithm controls the parameter σ to reduce the strength of the Gaussian mask.

[0089] Specifically, it aims to ensure that extremely important details in the image will not be destroyed by excessive masking. The core is to use cosine similarity to measure the similarity between different 3D images and dynamically adjust the input image accordingly.

[0090] Specific,algorithm: dynamic adjustment and image update

[0091] 1. Initialize similarity threshold: η←0.8

[0092] 2. Initialize control parameters: σ←0.5

[0093] 3. Set the maximum number of iterations: n←10

[0094] 4. Calculate cosine similarity: cosine_sim←Calculate cosine similarity (I,I masked)

[0095] 5. For iterations from 1 to n:

[0096] 6. If cosine_sim>η, then

[0097] 7. Training with dynamically adjusted input:

[0098] 8.I outputs ←Model(I masked )

[0099] 9. Otherwise:

[0100] 10. Add control parameters: σ←σ+0.1

[0101] 11. Dynamically adjust the input image: I masked ←Gaussian mask (I,σ)

[0102] 12. Recalculate cosine similarity: cosine_sim←calculate cosine similarity (I,I masked )

[0103] 13. End if

[0104] 14. End for

[0105] 15. If cosine_sim<η, then

[0106] 16. Use original images for training:

[0107] 17.I outputs ←Model(I)

[0108] End if.

[0109] Specifically, by calculating the cosine similarity between the original input image and the mask space domain image after Gaussian mask processing, the algorithm can evaluate the detail preservation of the image in real time.

[0110] This adjustment mechanism ensures that more original input image information is gradually retained in the absence of details, and by setting the maximum number of iterations (n), the algorithm recalculates the cosine similarity in each iteration to ensure continuous monitoring and adjustment of image quality during training. If the cosine similarity is still below the threshold after the maximum number of iterations, the algorithm will fall back to using the original input image for training. This mechanism ensures that in extreme cases, the model can still obtain enough information for effective learning. Through the above mechanism, this algorithm can dynamically balance the image detail retention and blur processing during training, improve the model's adaptability to complex scenes, and ultimately improve the accuracy and robustness of the output results.

[0111] To assist the model learning process, the present invention uses high-frequency information to calculate the loss to train the network.

[0112] Furthermore, the segmentation model in the segmentation network is an MCF model.

[0113] Furthermore, the specific contents of detecting high-frequency signals and calculating high-frequency losses are as follows:

[0114] First, initialize a Laplace kernel to detect the image output I outputs High frequency signal in:

[0115]

[0116] Here, the values ​​of α, β, and γ are 0, -1, and 4. The Laplacian kernel is first created as a tensor and expanded to shape (1, 1, 3, 3). Then, the Laplacian kernel is repeated across the channels to obtain the shape (1, C, 3, 3). The kernel is further expanded to shape (1, C, 1, 3, 3) by adding the depth dimension. Next, a high-pass filter is applied using the padding and stride parameters. The convolution operation is defined as:

[0117] High_pass_filter=F.conv3d(image, kernel, padding, stride);

[0118] Define a high-frequency loss function to quantify the difference between the high-frequency signals in the real image and the predicted image. The loss function uses the high-pass filter defined previously to extract the true value y true and the predicted value y pred The high-frequency signal is calculated as follows:

[0119] h true =High_pass_filter(y true );

[0120] h pred =High_pass_filter(y pred );

[0121] The high-frequency loss is then calculated based on the L2 norm:

[0122] L hf =||h true -h pred ||2.

[0123] Specifically, the high-frequency loss function is designed to quantify the difference in high-frequency signals between the real image and the predicted image. By extracting high-frequency signals through the Laplacian kernel, the loss function can focus on the edge and detail information in the image, ensuring that the model can retain important visual features when generating images, and combined with the dynamic mask and cosine similarity strategies, the loss function can be adaptively adjusted on different image features to ensure the best training effect. The high-frequency loss provides clear feedback to the network, prompting it to optimize the generated results to reduce blur and missing details, thereby improving the accuracy of the model segmentation results.

[0124] In a specific embodiment, the present invention uses the widely accepted Dice Score (Dice), Jaccard Score (Jaccard), 95% Hausdorff Distance (95HD) and Average Surface Distance (ASD) to evaluate the performance of our method. For two object regions, Dice and Jaccard mainly calculate the percentage of overlap between them. ASD calculates the average distance of the boundaries between them, and 95HD measures the distance between the nearest points between them. The higher the Dice and Jaccard, the higher the consistency between the model prediction results and the true results, while the opposite is true for ASD and 95HD. When our method is trained on the Atrial Segmentation Challenge (LA) dataset, the training labeling amount is 10%, and the Dice, Jaccard, 95HD, and ASD on the test set are 80.88, 80.13, 6.40, and 2.07 respectively; the training labeling amount is 20%, and the Dice, Jaccard, 95HD, and ASD on the test set are 90.54, 82.84, 5.68, and 1.73 respectively. The visualization results are as follows Figure 3 As shown in the first row (GT is the standard result, the rest are the model results and the results of the present invention). When training on the Pancreas-NIH (Pancrease) dataset, the training labeling amount is 20%, and the Dice, Jaccard, 95HD, and ASD on the test set are 78.64, 65.45, 8.35, and 2.44 respectively. The visualization results are as follows Figure 3 As shown in the second row (GT is the standard result, the rest are the model results and the results of the present invention); when the training labeling amount is 30%, Dice, Jaccard, 95HD, and ASD on the test set are 78.09, 65.05, 8.59, and 1.74, respectively.

[0125] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0126] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation, characterized in that: The following steps are involved: Image frequency domain transformation step: Convert the original input image from the spatial domain to the frequency domain through Fourier transform; Gaussian frequency domain masking step: After converting the original input image into the frequency domain, it is processed using a Gaussian mask to obtain a masked frequency domain image; Similarity calculation step: convert the masked frequency domain image into the spatial domain, and perform similarity calculation with the original input image to obtain a similarity calculation result; Dynamic threshold strategy steps: If the similarity calculation result is lower than the threshold image, readjust the mask degree and repeat the Gaussian frequency domain mask and similarity calculation steps until the similarity calculation result of the masked spatial domain image and the original input image meets the standard; Network prediction result step: input the original input image and the mask space domain image after dynamic threshold adjustment into the segmentation network to obtain the true result of the original input image and the prediction result corresponding to the mask space domain image; Loss calculation step: convert the true result of the original input image and the predicted result corresponding to the mask space domain image into the frequency domain, and detect the high-frequency signal to calculate the high-frequency loss.

2. The Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation according to claim 1, characterized in that: The specific content of converting the original input image from the spatial domain to the frequency domain through Fourier transform is: First, for the original input image I of shape (B, C, H, W, D), B represents the batch size, C represents the number of channels, H, W and D represent the height, width and depth of the original input image, a two-dimensional Fourier transform is performed on each layer of the original input image to obtain a frequency domain representation: Where F is the Fourier transform operator, b is the batch index, c is the channel index, d is the depth index, μ and v are the indices of the horizontal and vertical spatial frequencies in the Fourier spectrum, are the frequency components of the original input image in the frequency domain, m is the row index, and n is the column index.

3. The Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation according to claim 1, characterized in that: After the original input image is converted into the frequency domain, it is processed using a Gaussian mask, and the specific content of the masked frequency domain image is: Design a Gaussian mask M, the form is as follows: Among them, (x, y) is the mask pixel point, (x 0, y0) is the center point of the frequency domain, set σ controls the mask strength. This Gaussian mask is centered around a Fourier shift operation: M shift (u,v)=F shift {M(x,y)}; In the frequency domain, the Fourier transformed image I f Multiply it with the Gaussian mask M to get the masked frequency domain image 4. The Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation according to claim 1, characterized in that: The specific contents of converting the masked frequency domain image into the spatial domain and calculating the similarity with the original input image are as follows: Reconstruct the mask spatial domain image using the inverse Fourier transform of the mask frequency domain image: Calculate the similarity with the original input image I: Among them, <·> is the dot product of the vectors, and ||·|| is the norm of the vectors.

5. The Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation according to claim 1, characterized in that: The specific contents of the dynamic threshold policy are as follows: Using cosine similarity for dynamic threshold strategy, when the similarity is higher than the threshold η, the algorithm will use the processed mask frequency domain image for training to ensure the optimization of the model; When the similarity is lower than the threshold, the algorithm controls the parameter σ to reduce the strength of the Gaussian mask.

6. The Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation according to claim 1, characterized in that: The segmentation model in the segmentation network is the MCF model.

7. The Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation according to claim 1, characterized in that: The specific contents of detecting high-frequency signals and calculating high-frequency losses are as follows: First, initialize a Laplace kernel to detect the image output I outputs High frequency signal in: Here, the values ​​of α, β, and γ are 0, -1, and 4. The Laplacian kernel is first created as a tensor and expanded to shape (1, 1, 3, 3). Then, the Laplacian kernel is repeated across the channels to obtain the shape (1, C, 3, 3). The kernel is further expanded to shape (1, C, 1, 3, 3) by adding the depth dimension. Next, a high-pass filter is applied using the padding and stride parameters. The convolution operation is defined as: High_pass_filter=F.conv3d(image,kernel,padding,stride); Define a high-frequency loss function to quantify the difference between the high-frequency signals in the real image and the predicted image. The loss function uses the high-pass filter defined previously to extract the true value y true and the predicted value y pred The high-frequency signal is calculated as follows: h true =High_pass_filter(y true ); h pred =High_pass_filter(y pred ); The high-frequency loss is then calculated based on the L2 norm: L hf =||h true -H pred ||2。

Images