A Fourier Mask and Adaptive Threshold Adjustment Method for Semi-Supervised Medical Image Segmentation

Abstract

The present invention discloses a Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation, which relates to the field of visual reasoning technology. It includes: image frequency domain transformation; Gaussian frequency domain mask; similarity calculation; dynamic threshold strategy; network prediction result; loss calculation. The present invention can achieve high-quality medical image segmentation and make the segmented result have clearer edge details.

Description

Technical Field

[0001] The present invention relates to the technical field of visual reasoning, and more specifically, to a Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation. Background Art

[0002] Traditional fully supervised learning methods rely on a large amount of labeled data, which is often difficult to achieve in the medical field. To solve this problem, researchers have gradually turned to semi-supervised learning methods, which can effectively utilize unlabeled data to supplement the shortage of labeled data. Semi-supervised learning aims to improve the segmentation accuracy and generalization ability of the model by constructing self-learning mechanisms, generative adversarial networks (GANs), or graphical models, etc.

[0003] Existing semi-supervised medical image segmentation methods mainly include the following categories:

[0004] Generative Adversarial Networks (GANs): Such methods enable the model to generate more accurate segmentation results through the adversarial training of a generator and a discriminator. GANs perform well in dealing with fuzzy edges and detail completion, but usually require a complex training process.

[0005] Consistency Regularization: By applying different perturbations (such as noise, rotation, etc.) to the input image and maintaining the consistency of the model output, the robustness of the model is improved. This method can effectively improve the segmentation accuracy with a small amount of labeled data.

[0006] Self-Training: The model is first trained on labeled data, and then the prediction results of the model on unlabeled data are used for retraining. The key to this method lies in how to select reliable pseudo-labels to avoid introducing incorrect information.

[0007] Frequency Domain Processing: Some studies use frequency domain techniques such as Fourier transform to enhance the model's perception ability of high-frequency information. These methods perform feature extraction in the frequency domain to improve the model's recognition ability of image details.

[0008] Although a variety of semi-supervised learning methods have been proposed and applied to medical image segmentation, many challenges still remain, such as how to effectively utilize unlabeled data, improve the model's sensitivity to details, and avoid overfitting. Therefore, research on these issues is still ongoing in order to achieve more efficient and accurate medical image segmentation.

[0009] Therefore, it is an urgent problem for those skilled in the art to propose a Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation to solve the difficulties existing in the prior art. Summary of the Invention

[0010] In view of this, the present invention provides a Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation to solve the technical problems existing in the prior art.

[0011] To achieve the above object, the present invention provides the following technical solutions:

[0012] A Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation, comprising the following steps:

[0013] Image frequency domain transformation step: converting the original input image from the spatial domain to the frequency domain through Fourier transform;

[0014] Gaussian frequency domain mask step: after converting the original input image to the frequency domain, using a Gaussian mask for processing to obtain a masked frequency domain image;

[0015] Similarity calculation step: converting the masked frequency domain image to the spatial domain and performing similarity calculation with the original input image to obtain a similarity calculation result;

[0016] Dynamic threshold strategy step: for images with similarity calculation results lower than the threshold, readjust the masking degree and repeat the Gaussian frequency domain mask and similarity calculation steps until the similarity calculation result between the masked spatial domain image and the original input image meets the standard;

[0017] Network prediction result step: inputting the original input image and the masked spatial domain image after dynamic threshold adjustment into a segmentation network to obtain the true result of the original input image and the prediction result corresponding to the masked spatial domain image;

[0018] Loss calculation step: converting the true result of the original input image and the prediction result corresponding to the masked spatial domain image to the frequency domain and detecting high-frequency signals for high-frequency loss calculation.

[0019] For the above method, optionally, the specific content of converting the original input image from the spatial domain to the frequency domain through Fourier transform is as follows:

[0020] First, for the original input image I with the shape of (B, C, H, W, D), where B represents the batch size, C represents the number of channels, and H, W, and D represent the height, width, and depth of the original input image, perform two-dimensional Fourier transform on each layer of the original input image to obtain a frequency domain representation:

[0021]

[0022] where F is the Fourier transform operator, b is the batch index, c is the channel index, d is the depth index, and μ and v are the indices of the horizontal and vertical spatial frequencies in the Fourier spectrum, are the frequency components of the original input image in the frequency domain, m is the row index, and n is the column index.

[0023] For the above method, optionally, after converting the original input image to the frequency domain, it is processed using a Gaussian mask. The specific content of obtaining the masked frequency-domain image is as follows:

[0024] Design a Gaussian mask M in the following form:

[0025]

[0026] where (x, y) is the mask pixel point, (x0, y0) is the center point of the frequency domain, set as σ controls the mask intensity. This Gaussian mask is centered on the Fourier shift operation:

[0027] M shift (u, v) = F shift {M(x, y)};

[0028] In the frequency domain, multiply the Fourier-transformed image I f by the Gaussian mask M to obtain the masked frequency-domain image

[0029]

[0030] For the above method, optionally, the specific content of converting the masked frequency-domain image to the spatial domain and calculating the similarity with the original input image is as follows:

[0031] Use the inverse Fourier transform of the masked frequency-domain image to reconstruct the masked spatial-domain image:

[0032]

[0033] Calculate the similarity with the original input image I:

[0034]

[0035] where <·> is the dot product of vectors, and ||·|| is the norm of vectors.

[0036] For the above method, optionally, the specific content of the dynamic threshold strategy is as follows:

[0037] Use the cosine similarity for the dynamic threshold strategy. When the similarity is higher than the threshold η, the algorithm will use the processed masked frequency-domain image for training to ensure the optimization of the model;

[0038] When the similarity is lower than the threshold, the algorithm will gradually increase the control parameter σ to reduce the intensity of the Gaussian mask.

[0039] In the above method, optionally, the segmentation model in the segmentation network is the MCF model.

[0040] In the above method, optionally, the specific content of detecting high-frequency signals for high-frequency loss calculation is as follows:

[0041] First, initialize a Laplacian kernel for detecting high-frequency signals in the image output I outputs :

[0042]

[0043] Among them, the values of α, β, and γ are 0, -1, and 4. First, create the Laplacian kernel as a tensor and expand it to the shape (1, 1, 3, 3); then, repeat the Laplacian kernel on the channels to obtain the shape (1, C, 3, 3); by adding a depth dimension, further expand the kernel to the shape (1, C, 1, 3, 3); then, apply high-pass filtering using the padding and stride parameters, and the convolution operation is defined as:

[0044] High_pass_filter = F.conv3d(image, kernel, padding, stridE);

[0045] Define a high-frequency loss function to quantify the difference in high-frequency signals between the real image and the predicted image. The loss function uses the previously defined high-pass filter to extract the high-frequency signals of the real value y true and the predicted value y pred The high-frequency signals are calculated as follows:

[0046] h true = High_pass_filter(y true );

[0047] h pred = High_pass_filter(y pred );

[0048] Then calculate the high-frequency loss according to the L2 norm:

[0049] L hf = ||h true - h pred ||2.

[0050] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation, and its beneficial effects are:

[0051] High-quality medical image segmentation can be achieved, and the segmented results have clearer edge details. The Gaussian mask strategy improves the model's ability to capture local details of the image, and the dynamic threshold adjustment strategy avoids the misleading in model learning; the mask and adjustment strategies enable the model to maintain high segmentation performance in different medical imaging modalities (such as CT and MRI), improving the model's generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0053] Figure 1 It is a flowchart of a Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation provided by the present invention;

[0054] Figure 2 It is a design schematic diagram provided by the present invention;

[0055] Figure 3 It is a segmentation effect diagram provided by the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0057] See Figure 1 As shown, the embodiments of the present invention disclose a Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation, including the following steps:

[0058] Image frequency domain transformation step: The original input image is transformed from the spatial domain to the frequency domain through Fourier transform;

[0059] Specifically, because there are many advantages in image enhancement at the frequency domain level, frequency domain processing can more precisely control the retention and filtering of information. See Figure 2 As shown, the low-frequency part contains important structural information, while the high-frequency part contains details. Frequency domain enhancement can specifically enhance or suppress these components, and some frequency domain operations (such as convolution) are more efficient in the frequency domain, especially for large-scale data, which can significantly reduce the calculation time.

[0060] Gaussian frequency domain masking step: After converting the original input image into the frequency domain, it is processed using a Gaussian mask to obtain the masked frequency domain image;

[0061] Specifically, the Gaussian mask is a low-pass filter that can remove high-frequency noise in the image and retain the low-frequency part. Through this mask, some high-frequency information in the frequency domain can be selectively removed, thereby reducing noise and highlighting the main structural information of the image. High-frequency signals will be suppressed or removed, and what remains is the low-frequency part that retains the general structure and form.

[0062] Similarity calculation step: Convert the masked frequency domain image into the spatial domain and perform similarity calculation with the original input image to obtain the similarity calculation result;

[0063] Specifically, ensure that the masking process is not excessive and does not cause damage to the image.

[0064] Dynamic threshold strategy step: For images whose similarity calculation results are lower than the threshold (i.e., images that are overly masked), readjust the masking degree and repeat the Gaussian frequency domain masking and similarity calculation steps until the similarity calculation result between the masked spatial domain image and the original input image meets the standard (i.e., it is not overly masked and the image is not damaged);

[0065] Network prediction result step: Input the original input image and the masked spatial domain image after dynamic threshold adjustment into the segmentation network to obtain the true result of the original input image and the prediction result corresponding to the masked spatial domain image;

[0066] Loss calculation step: Convert the true result of the original input image and the prediction result corresponding to the masked spatial domain image into the frequency domain and detect high-frequency signals for high-frequency loss calculation.

[0067] Specifically, the above method can solve the problem of blurred edges in medical images, optimize the frequency domain features of the image, enhance the learning effect of the model, and thus improve the overall performance and accuracy.

[0068] Furthermore, the specific content of converting the original input image from the spatial domain to the frequency domain through Fourier transform is as follows:

[0069] First, for the original input image I with the shape of (B, C, H, W, D), where B represents the batch size, C represents the number of channels, and H, W, and D represent the height, width, and depth of the original input image, perform a two-dimensional Fourier transform on each layer of the original input image to obtain the frequency domain representation:

[0070]

[0071] Among them, F is the Fourier transform operator, b is the batch index, c is the channel index, d is the depth index, μ and v are the indices of the horizontal and vertical spatial frequencies in the Fourier spectrum, are the frequency components of the original input image in the frequency domain, m is the row index, and n is the column index.

[0072] Specifically, mathematically, the 3D volume of the medical image is defined as: I ∈ R W×H×L . The goal of semi-supervised medical image segmentation is to predict the label map y ∈ {0, 1,..., K - 1} for each voxel W×H×L , representing the positions of the background and the target in I, and K is the class number.

[0073] Furthermore, after converting the original input image to the frequency domain, it is processed using a Gaussian mask. The specific content of the masked frequency domain image is as follows:

[0074] Design a Gaussian mask M in the following form:

[0075]

[0076] where (x, y) are the mask pixel points, (x0, y0) is the center point of the frequency domain, set as σ controls the mask intensity, and this Gaussian mask is centered on the Fourier shift operation:

[0077] M shift (u, v) = F shift {M(x, y)};

[0078] In the frequency domain, multiply the Fourier-transformed image I f by the Gaussian mask M to obtain the masked frequency domain image

[0079]

[0080] Furthermore, convert the masked frequency domain image to the spatial domain and perform similarity calculation with the original input image. The specific content is as follows:

[0081] Use the inverse Fourier transform of the masked frequency domain image to reconstruct the masked spatial domain image:

[0082]

[0083] Calculate the similarity with the original input image I:

[0084]

[0085] where <·> is the dot product of vectors, and ||·|| is the norm of vectors.

[0086] Furthermore, the specific content of the dynamic threshold strategy is as follows:

[0087] The dynamic threshold strategy is carried out using cosine similarity. When the similarity is higher than the threshold η, the algorithm will use the processed masked frequency-domain image for training to ensure the optimization of the model.

[0088] When the similarity is lower than the threshold, the algorithm will control the parameter σ to reduce the intensity of the Gaussian mask.

[0089] Specifically, it aims to ensure that extremely important details in the image are not damaged due to excessive masking. The core is to use cosine similarity to measure the similarity between different 3D images and dynamically adjust the input image accordingly.

[0090] Specifically, the algorithm: dynamic adjustment and image update

[0091] 1. Initialize the similarity threshold: η ← 0.8

[0092] 2. Initialize the control parameter: σ ← 0.5

[0093] 3. Set the maximum number of iterations: n ← 10

[0094] 4. Calculate the cosine similarity: cosine_sim ← calculate the cosine similarity(I, I masked)

[0095] 5. For the iteration number iteration from 1 to n:

[0096] 6. If cosine_sim > η, then

[0097] 7. Use the dynamically adjusted input for training:

[0098] 8. I outputs ← model(I masked )

[0099] 9. Otherwise:

[0100] 10. Increase the control parameter: σ ← σ + 0.1

[0101] 11. Dynamically adjust the input image: I masked ← Gaussian mask(I, σ)

[0102] 12. Recalculate the cosine similarity: cosine_sim ← calculate the cosine similarity(I, I masked )

[0103] 13. End if

[0104] 14. End for

[0105] 15. If cosine_sim < η, then

[0106] 16. Use the original image for training:

[0107] 17. I outputs ← Model (I)

[0108] End if.

[0109] Specifically, by calculating the cosine similarity between the original input image and the masked spatial domain image processed by the Gaussian mask, the algorithm can evaluate the detail retention of the image in real time.

[0110] This adjustment mechanism can ensure that more original input image information is gradually retained in the case of missing details. And by setting the maximum number of iterations (n), the algorithm recalculates the cosine similarity in each iteration to ensure continuous monitoring and adjustment of the image quality during the training process. If after the maximum number of iterations, the cosine similarity is still lower than the threshold, the algorithm will fall back to using the original input image for training. This mechanism ensures that in extreme cases, the model can still obtain sufficient information for effective learning. Through the above mechanism, the present algorithm can dynamically balance the detail retention and blurring processing of the image during the training process, improve the adaptability of the model to complex scenarios, and ultimately enhance the accuracy and robustness of the output results.

[0111] To assist the model learning process, the present invention uses high-frequency information to calculate the loss to train the network.

[0112] Furthermore, the segmentation model in the segmentation network is the MCF model.

[0113] Furthermore, the specific content of detecting high-frequency signals for high-frequency loss calculation is:

[0114] First, initialize a Laplacian kernel for detecting high-frequency signals in the image output I outputs :

[0115]

[0116] Among them, the values of α, β, and γ are 0, -1, 4. First, create the Laplacian kernel as a tensor and expand it to the shape (1, 1, 3, 3); then, repeat the Laplacian kernel on the channel to obtain the shape (1, C, 3, 3); by adding a depth dimension, further expand the kernel to the shape (1, C, 1, 3, 3); then, apply high-pass filtering using the padding and stride parameters. The convolution operation is defined as:

[0117] High_pass_filter = F.conv3d(image, kernel, padding, stride);

[0118] Define a high-frequency loss function to quantify the difference in high-frequency signals between the real image and the predicted image. The loss function uses the previously defined high-pass filter to extract the real value y true and the predicted value y pred of the high-frequency signal, and the high-frequency signal is calculated as follows:

[0119] h true = High_pass_filter(y true );

[0120] h pred = High_pass_filter(y pred );

[0121] Then, calculate the high-frequency loss according to the L2 norm:

[0122] L hf = ||h true - h pred ||2.

[0123] Specifically, the design of the high-frequency loss function aims to quantify the difference in high-frequency signals between the real image and the predicted image. By extracting high-frequency signals through the Laplacian kernel, the loss function can focus on the edge and detail information in the image, ensuring that the model can retain important visual features when generating images. Combining the strategies of dynamic masking and cosine similarity enables the loss function to adaptively adjust to different image features, ensuring the best training effect. The high-frequency loss provides clear feedback to the network, prompting it to optimize the generation results to reduce blurring and detail loss, thereby improving the accuracy of the model segmentation results.

[0124] In a specific embodiment, the present invention uses the widely accepted Dice Score (Dice), Jaccard Score (Jaccard), 95% Hausdorff Distance (95HD), and Average Surface Distance (ASD) to evaluate the performance of our method. For two object regions, Dice and Jaccard mainly calculate the percentage of overlap between them. ASD calculates the average distance between their boundaries, and 95HD measures the distance between their closest points. The higher the Dice and Jaccard, the higher the consistency between the model prediction result and the true result, while ASD and 95HD are the opposite. When our method is trained on the Atrial Segmentation Challenge (LA) dataset with a training label amount of 10%, the Dice, Jaccard, 95HD, and ASD on the test set are 80.88, 80.13, 6.40, and 2.07 respectively; when the training label amount is 20%, the Dice, Jaccard, 95HD, and ASD on the test set are 90.54, 82.84, 5.68, and 1.73 respectively. The obtained visualization results are as shown in Figure 3 the first row (GT is the standard result, and the rest are the model result and the result of the present invention). When trained on the Pancreas-NIH (Pancrease) dataset with a training label amount of 20%, the Dice, Jaccard, 95HD, and ASD on the test set are 78.64, 65.45, 8.35, and 2.44 respectively. The obtained visualization results are as shown in Figure 3 the second row (GT is the standard result, and the rest are the model result and the result of the present invention); when the training label amount is 30%, the Dice, Jaccard, 95HD, and ASD on the test set are 78.09, 65.05, 8.59, and 1.74 respectively.

[0125] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other.

[0126] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation, characterized in that It includes the following steps: Image frequency domain transformation step: The original input image is transformed from the spatial domain to the frequency domain through Fourier transform; Gaussian frequency domain mask step: After the original input image is transformed into the frequency domain, it is processed using a Gaussian mask to obtain the masked frequency domain image; Similarity calculation step: The masked frequency domain image is transformed into the spatial domain and similarity calculation is performed with the original input image to obtain the similarity calculation result; Dynamic threshold strategy step: For images with similarity calculation results lower than the threshold, the masking degree is readjusted and the steps of Gaussian frequency domain masking and similarity calculation are repeated until the similarity calculation result between the masked spatial domain image and the original input image meets the standard; Network prediction result step: The original input image and the masked spatial domain image after dynamic threshold adjustment are input into the segmentation network to obtain the true result of the original input image and the prediction result corresponding to the masked spatial domain image; Loss calculation step: The true result of the original input image and the prediction result corresponding to the masked spatial domain image are transformed into the frequency domain, and high-frequency signals are detected for high-frequency loss calculation; The specific content of detecting high-frequency signals for high-frequency loss calculation is: First, initialize a Laplacian kernel for detecting high-frequency signals in the image output I outputs : Among them, the values of α, β, and γ are 0, -1, and 4. First, the Laplacian kernel is created as a tensor and expanded to a shape of (1, 1, 3, 3); then, the Laplacian kernel is repeated on the channels to obtain a shape of (1, C, 3, 3); by adding a depth dimension, the kernel is further expanded to a shape of (1, C, 1, 3, 3); then, high-pass filtering is applied using the padding and stride parameters, and the convolution operation is defined as: High_pass_filter = F.conv3d(image, kernel, padding, stride); Define a high-frequency loss function to quantify the difference in high-frequency signals between the real image and the predicted image. The loss function uses the previously defined high-pass filter to extract the real value y true and the predicted value y pred of the high-frequency signal. The high-frequency signal is calculated as follows: h true = High_pass_filter(y true ); h pred = High_pass_filter(y pred ); Then, the high-frequency loss is calculated according to the L2 norm: L hf = ||h true -h pred ||2。 2. A Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation according to claim 1, characterized in that, The specific content of transforming the original input image from the spatial domain to the frequency domain through Fourier transform is: First, for the original input image I with a shape of (B, C, H, W, D), where B represents the batch size, C represents the number of channels, and H, W, and D represent the height, width, and depth of the original input image, a two-dimensional Fourier transform is performed on each layer of the original input image to obtain the frequency domain representation: where F is the Fourier transform operator, b is the batch index, c is the channel index, d is the depth index, μ and v are the indices of the horizontal and vertical spatial frequencies in the Fourier spectrum, are the frequency components of the original input image in the frequency domain, m is the row index, and n is the column index.

3. A Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation according to claim 1, characterized in that, After the original input image is transformed into the frequency domain, the specific content of processing it using a Gaussian mask to obtain the masked frequency domain image is: Design a Gaussian mask M in the following form: where (x, y) is the mask pixel point, and (x0, y0) is the center point in the frequency domain, which is set as σ controls the mask intensity, and this Gaussian mask is centered on the Fourier shift operation: M shift (u, v) = F shift {M(x, y)}; In the frequency domain, multiply the Fourier-transformed image I f by the Gaussian mask M to obtain the masked frequency-domain image 4. A Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation according to claim 1, characterized in that, The specific content of transforming the masked frequency domain image into the spatial domain and performing similarity calculation with the original input image is: Use the inverse Fourier transform of the masked frequency domain image to reconstruct the masked spatial domain image: Calculate the similarity with the original input image I: Among them, <·> is the dot product of vectors, and ||·|| is the norm of vectors.

5. A Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation according to claim 1, characterized in that, The specific content of the dynamic threshold strategy is: Use cosine similarity for the dynamic threshold strategy. When the similarity is higher than the threshold η, the algorithm will use the processed masked frequency domain image for training to ensure the optimization of the model; When the similarity is lower than the threshold, the algorithm will control the parameter σ to reduce the intensity of the Gaussian mask.

6. A Fourier mask and adaptive threshold adjustment method for semi-supervised medical image segmentation according to claim 1, characterized in that, The segmentation model in the segmentation network is the MCF model.

Images