Medical image segmentation method based on combination of wavelet transform and SAM of bridging mode

Abstract

The invention discloses a medical image segmentation method combining wavelet transform and SAM based on a bridging mode, and relates to the field of medical image segmentation. The method comprises the following steps: data preprocessing: performing data enhancement on an input image, and standardizing the input image; feature extraction: extracting preliminary features of the preprocessed image by using a convolutional neural network, and performing multi-scale feature decomposition through wavelet transform to obtain a low-frequency feature map and a high-frequency feature map; multi-scale feature fusion: fusing the low-frequency feature map and the high-frequency feature map through a bridging mode, weighting the fused features through an adaptive attention mechanism, and recovering the weighted features through inverse wavelet transform; carrying out weighted fusion on the feature maps subjected to different processing; and feature decoding and prediction. According to the method, local detail information and global context information are effectively combined, and the method is particularly suitable for a feature extraction task in a medical image under a boundary fuzzy and low-contrast scene.

Description

Technical Field

[0001] The invention belongs to the technical field of medical image segmentation, and in particular relates to a medical image segmentation method based on the combination of wavelet transform and SAM in a bridge mode. Background Art

[0002] Medical image segmentation technology plays an important role in modern medical diagnosis and treatment. It provides doctors with accurate anatomical and pathological information through computer-aided identification and segmentation of organs, tissues and lesions in medical images, supporting clinical decision-making, surgical planning and treatment evaluation. As a type of image widely used in clinical diagnosis, ultrasound images have many unique characteristics. The low contrast and high noise problems of ultrasound images often make it difficult to distinguish the target object from the background, especially when the boundaries between soft tissue and lesion tissue are blurred. In addition, the quality of ultrasound imaging is highly dependent on the operator's skills and equipment parameters, and the acquisition of images is often affected by the angle and position of the probe. These factors make ultrasound image segmentation more complicated than other imaging technologies, and also put higher requirements on the segmentation accuracy of the model. At the same time, since ultrasound images can be acquired in real time, efficient and accurate automatic segmentation technology is crucial to improving the speed and accuracy of clinical diagnosis.

[0003] In recent years, with the rapid development of deep learning in the field of computer vision, convolutional neural networks (CNN) have become the mainstream method for medical image segmentation. However, CNN has certain limitations in processing global information, boundary details and multi-scale features in images, especially in images with complex backgrounds or blurred boundaries, the performance of CNN is often affected. In order to solve these problems, Segment Anything Model (SAM), as a model based on large-scale pre-training, has demonstrated powerful capabilities in natural image segmentation tasks. SAM is trained on a large-scale dataset through the Vision Transformer (ViT) architecture, which can effectively process the global information in the image and provide more accurate segmentation results. However, due to the particularity of medical images, especially ultrasound images in terms of contrast, noise and boundary fuzziness, when SAM is directly applied to medical image segmentation tasks, the effect is often not as expected. The existing technology specifically has the following problems:

[0004] (1) Current medical image segmentation algorithms, especially those based on convolutional neural networks (CNNs), have significant deficiencies in processing multi-scale features and boundary details. This is because the receptive field of CNNs is usually fixed and cannot effectively adapt to the multi-scale features in the image. Due to the low contrast, high noise, and blurred boundaries of ultrasound images, traditional CNN models have difficulty effectively distinguishing the target object from the background, especially in the transition area between diseased tissue and healthy tissue, resulting in decreased segmentation accuracy. These problems have not been effectively solved in traditional CNNs, especially when processing tasks such as ultrasound images with low contrast and complex backgrounds. The characteristics of different scales and detailed areas are still not fully considered, which affects the segmentation accuracy.

[0005] (2) The SAM model (Segment Anything Model) has achieved good results in natural image segmentation tasks, but directly applying it to medical images, especially ultrasound images, often faces great challenges. SAM's pre-training datasets are mainly derived from natural images, and lack relevant data on medical images, resulting in its inability to fully understand the details and features of the image when processing medical images. In particular, the edge blur, low contrast and noise problems in ultrasound images cannot be effectively compensated by SAM's global information modeling capabilities, resulting in the model being unable to accurately segment the lesion area.

[0006] (3) In the task of medical image segmentation, the design of loss function is crucial to the training and accuracy of the model. Currently, most medical image segmentation models mainly rely on Dice coefficient loss and binary cross entropy (BCE) loss. Dice coefficient loss can effectively deal with the problem of class imbalance, but it focuses more on global overlap and has weak processing of boundary details, especially in images with blurred boundaries or low contrast, and cannot provide sufficient signals to optimize edges. BCE loss is suitable for pixel-level classification, but it treats all pixel regions equally and fails to effectively focus on boundary regions, resulting in insufficient segmentation accuracy. In ultrasound images, the boundaries of target objects are usually blurred and have low contrast. Traditional loss functions fail to provide sufficient training signals for these boundary regions, resulting in low segmentation accuracy of the model in the boundary part. In addition, existing loss functions usually do not weight boundary information and cannot effectively improve the segmentation effect of detail regions, thereby affecting the overall segmentation accuracy.

[0007] To this end, the present invention begins to explore the combination method of CNN and SAM, and uses a hybrid model that uses CNN for local feature extraction and SAM for global information modeling, aiming to improve the model's processing capabilities in boundary details and multi-scale features. This combination can not only make up for the shortcomings of CNN in global modeling, but also improve segmentation accuracy through SAM's powerful context understanding ability, thereby providing more accurate and stable segmentation results in high-noise, low-contrast medical images such as ultrasound images.

[0008] In addition, as a technology with advantages in time-frequency analysis, wavelet transform (WT) can effectively process the multi-scale features of images. Unlike the traditional Fourier transform, wavelet transform provides local and global information of images in both time and frequency domains, which is particularly suitable for processing non-stationary signals with multi-scale features. In the task of medical image segmentation, combining convolutional neural network (CNN) and wavelet transform can significantly enhance the model's ability to extract information of different scales. CNN performs well in extracting local features, but has certain limitations in processing multi-scale features and capturing image boundary details, especially in ultrasound images with low contrast, high noise and blurred boundaries. Wavelet transform can decompose images into sub-images of multiple scales and frequencies, thereby effectively capturing high-frequency details in the image, such as edges, textures and tiny lesions, and thus helping the model to better distinguish target objects from backgrounds in complex backgrounds. In addition, the multi-scale decomposition of wavelet transform can also improve the robustness of image processing and reduce the interference of noise on segmentation results. Therefore, combining CNN and wavelet transform can not only improve the segmentation accuracy of the model in ultrasound images, but also enhance its sensitivity to image boundaries and details, especially when dealing with the boundaries of soft tissue and diseased tissue, which can significantly improve the segmentation effect.

[0009] Therefore, the present invention proposes a medical image segmentation method based on the combination of wavelet transform and SAM based on the bridge mode. Summary of the invention

[0010] The purpose of the present invention is to provide a medical image segmentation method based on the combination of wavelet transform and SAM based on the bridge mode, so as to solve the problem that the existing methods fail to fully consider the characteristics of different scales and detail areas, which affects the segmentation accuracy, and the existing loss function cannot effectively improve the segmentation effect of the detail area.

[0011] To achieve the above purpose, the technical solution adopted by the present invention is:

[0012] The first aspect of the present invention proposes a medical image segmentation method based on a combination of wavelet transform and SAM based on a bridge mode, which at least comprises the following steps:

[0013] S1. Data preprocessing: perform data enhancement on the input image and standardize the input image;

[0014] S2, feature extraction: extract preliminary features from the preprocessed image using a convolutional neural network, and perform multi-scale feature decomposition through wavelet transform to obtain low-frequency feature maps and high-frequency feature maps;

[0015] S3, multi-scale feature fusion; low-frequency feature maps and high-frequency feature maps are fused through the bridge mode, the fused features are weighted through the adaptive attention mechanism, and the weighted features are restored through the inverse wavelet transform; the feature maps that have been processed differently are weighted and fused;

[0016] S4, feature decoding and prediction; decode the image features through the self-attention mechanism, and use the deformable attention mechanism to adjust the image features; the decoded features are input to the prediction head, and finally the predicted target box coordinates and target category are output.

[0017] Preferably, the S1 is as follows:

[0018] First, the images are normalized and the image pixel values ​​are uniformly scaled to a standard range; second, the images are data enhanced, which includes random cropping, translation, rotation, and horizontal flipping operations; finally, all images are resized to a uniform size.

[0019] Furthermore, the input image is first normalized. The normalization operation scales each pixel value to the range of [0,1] to eliminate the brightness difference between different images and reduce the deviation during model training. Secondly, data augmentation methods (such as random cropping, rotation, flipping, translation, etc.) are used to improve the robustness of the model and enable it to maintain good performance in the face of different image deformations and noise. Afterwards, the image is resized. In order to avoid image distortion, the original scaling method is adopted, and the blank area of ​​the image is filled with black. Finally, all images are uniformly adjusted to a size of 256×256, and the pixel values ​​are further normalized to ensure the standardized processing of the input image.

[0020] Preferably, S2 is specifically as follows:

[0021] Assuming the input feature is x, the input feature is preliminarily processed through the convolution layer (SingleConv) to obtain the preliminary low-level feature x1; this process is expressed as:

[0022] x1=SingleConv(x1)

[0023] The input feature x1 is decomposed into multiple scales through wavelet transform to extract low-frequency and high-frequency information, which is expressed as:

[0024] x2,x3=DWT(x1)

[0025] Among them, x2 represents the characteristics of the low-frequency part, capturing the overall structure of the image; x3 represents the characteristics of the high-frequency part, capturing the details and edge information of the image; DWT is wavelet transform.

[0026] Furthermore, the wavelet transform performs multi-scale decomposition, as follows:

[0027] The wavelet transform decomposes the input feature map through a filter bank, which is expressed as:

[0028]

[0029] in, and are low-pass filter and high-pass filter respectively, m and n are the coordinates of the image.

[0030] Preferably, S3 is specifically as follows:

[0031] Features x2 and x3 are fused through the bridge mode to retain information at different scales, expressed as:

[0032] x bridge =Concat(x2,x3)

[0033] Among them, Concat represents the bridge layer, x bridge It represents the fusion of the low-frequency features and the high-frequency features.

[0034] For feature x bridge Global information modeling is performed through Transformer, which is expressed as:

[0035] x4=Transformer(x bridge )

[0036] The feature x4 after Transformer modeling is restored to the spatial dimension through inverse wavelet transform to obtain the restored feature x5, which is expressed as:

[0037] x5=IWT(x4)

[0038] Where IWT stands for inverse wavelet transform;

[0039] After wavelet transformation, Transformer modeling, and inverse wavelet transformation, the feature x5 is weightedly fused with the initial input feature x through residual connection to obtain the final output feature map x out , expressed as:

[0040] x oUt =x+0.5×x5.

[0041] Furthermore, the Transformer performs global information modeling, as follows:

[0042] Transformer models the long-range dependencies in the image through the self-attention mechanism to obtain global context information. The calculation process of the self-attention mechanism is as follows:

[0043]

[0044] Among them, Q is the query, K is the key, V is the value, and d is the feature dimension; the self-attention mechanism enhances the modeling of long-range dependencies in the image through these weighted coefficients.

[0045] Furthermore, the inverse wavelet transform is restored to the spatial dimension as follows:

[0046] The process of inverse wavelet transform is to reconstruct the low-frequency and high-frequency features back to the original features, which can be expressed as:

[0047]

[0048] Among them, δ L and δ H It is the inverse filter of low-pass and high-pass, which is used to restore the frequency domain information to the spatial domain.

[0049] The second aspect of the present invention proposes a medical image segmentation system based on a combination of wavelet transform and SAM based on a bridge mode applied to the above method, using a medical image segmentation model, the medical image segmentation model comprising:

[0050] BridgeWave Transformer hybrid encoder, including wavelet transform module, bridge layer, Transformer encoder, inverse wavelet transform module, weighted fusion module; Transformer encoder is based on the ViT architecture in SAM and has an adaptive attention mechanism; it is used for feature extraction and multi-scale feature fusion;

[0051] The decoder includes a multi-head self-attention module, a deformable attention module, and a prediction head; it is used for feature decoding and prediction.

[0052] Preferably, during the training phase, the Dice loss and the boundary-aware BCE loss in the image segmentation task are calculated respectively, and the total loss of the model is obtained by combining the two loss functions, and the gradient of the loss is calculated by the back-propagation algorithm to update the model network parameters;

[0053] For the composite loss function that combines Dice loss with boundary-aware BCE loss, the composite loss function formula is:

[0054] Total Loss=β1·Dice Loss+β2·Boundary_Aware BCE Loss

[0055] Among them, β1 and β2 are hyperparameters used to balance the weights of the two parts of the loss; Dice Loss is the Dice loss, and Boundary_Aware BCE Loss is the boundary-aware BCE loss.

[0056] Furthermore, the Dice loss is specifically:

[0057]

[0058] Among them, y pred is the predicted binary image, y true is the real label image; the Dice coefficient is used to measure the overlap between the prediction and the real label, and the goal of Dice Loss is to minimize the difference between the segmented area and the real area.

[0059] Furthermore, the boundary perception BCE loss is specifically:

[0060]

[0061] Among them, w boundary (y true ) is a weight function that is dynamically adjusted based on boundary information.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] (1) The method of the present invention proposes a feature extraction method that combines wavelet transform and adaptive attention mechanism (SAM). First, the input image is subjected to preliminary feature extraction through a convolutional layer (SingleConv) to extract low-level edge and texture information. Next, the features are decomposed into low-frequency and high-frequency parts through discrete wavelet transform (DWT). The low-frequency part retains the overall structure of the image, while the high-frequency part captures the details and edge information of the image. After multi-scale decomposition, the low-frequency and high-frequency features are spliced ​​and fused through a bridge mode (Bridge Connection) to ensure the effective retention of multi-scale information and the transmission of details. Finally, the fused feature map is input into a Transformer module based on SAM, and the global information in the image is modeled through a self-attention mechanism, further improving the expressiveness of the features. This module effectively combines local detail information and global context information, and is particularly suitable for feature extraction tasks in medical images with blurred boundaries and low-contrast scenes.

[0064] (2) In order to solve the problems of blurred boundaries and loss of small target details in medical image segmentation, the present invention designs a composite loss function that combines the Dice coefficient and the boundary-aware BCE loss. The boundary weight coefficient is introduced on the basis of the standard binary cross entropy (BCE) loss, so that the model can focus on the prediction accuracy of the boundary area. The Dice loss is used to measure the overlap between the predicted segmentation area and the true label area, and is suitable for processing class imbalance and small target tasks. The boundary-aware BCE loss dynamically adjusts the boundary area weight to strengthen the model's training signal for the boundary area, thereby improving the boundary accuracy of the segmentation result. Finally, the composite loss function combines the two weightedly. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a structural block diagram of the medical image segmentation system based on the combination of wavelet transform and SAM in the bridge mode of the present invention;

[0066] Figure 2 It is a structural block diagram of the wavelet transform module in the BridgeWave Transformer hybrid encoder of the present invention;

[0067] Figure 3 It is a structural block diagram of the Transformer encoder in the BridgeWave Transformer hybrid encoder of the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] Embodiment 1:

[0070] The present invention is a medical image segmentation method based on the combination of wavelet transform of bridge mode and Transformer of SAM to solve the problems of insufficient boundary accuracy and detail restoration ability in medical image segmentation in the prior art, especially when processing medical images with blurred boundaries and low contrast.

[0071] In order to achieve the above-mentioned purpose, the present invention provides the following technical scheme, a medical image segmentation method based on the combination of wavelet transform and SAM in bridge mode, and the whole method can be divided into four stages: data preprocessing, feature extraction, multi-scale feature fusion, feature decoding and prediction. In the data preprocessing stage, the input image is normalized, randomly transformed, scaled and other operations are performed to enhance the robustness of the model and standardize the input data. In the feature extraction stage, the preprocessed image is first input into the backbone network, the preliminary features are extracted by convolutional neural network, and multi-scale feature decomposition is performed by wavelet transform to obtain feature information of different frequency bands. In the multi-scale feature fusion stage, the low-frequency and high-frequency features from the wavelet transform are combined, and the features are weighted by adaptive attention mechanism (SAM) to improve image detail recovery and boundary accuracy. In the feature decoding and prediction stage, the fused and weighted feature map will be input into the decoder, and the original size of the image will be restored by upsampling layer by layer, and finally an accurate segmentation result will be output. The whole process can effectively improve the boundary accuracy and detail recovery ability of medical image segmentation, especially when processing low-contrast and blurred images, it has a strong advantage, such as Figure 1 The details are as follows:

[0072] Step 1: Data preprocessing;

[0073] In order to improve the model's utilization of input medical image information, the image needs to be preprocessed before entering the network for feature extraction.

[0074] First, the image is normalized to uniformly scale the image pixel values ​​to a standard range (for example, 0 to 1 or -1 to 1), which can accelerate the convergence process of the model. Next, in order to increase the robustness of the model, the training data will be randomly transformed, including random cropping, translation, rotation, horizontal flipping, etc. These data augmentation methods help the network adapt to different conditions, especially when dealing with noise and deformation in the image. Finally, all training images are resized to a uniform size to meet the input requirements of the subsequent network. It is usually chosen to unify the images to a fixed size (such as 256×256). This process helps ensure the uniformity of the input image and provide standardized data for subsequent processing.

[0075] Step 2: Feature extraction;

[0076] The preprocessed image is input to the feature extraction module. The image is decomposed into low-frequency and high-frequency parts through wavelet transform (DWT), capturing the overall structure and detail information of the image respectively. The low-frequency part mainly extracts the global structure, such as the background and object contours, while the high-frequency part focuses on details and edge features. The low-frequency and high-frequency feature maps after wavelet transform will be passed to the convolutional neural network (CNN) to further extract the low-level features of the image (such as edges, textures, etc.) and generate feature maps of multiple scales to represent the information of the image at different levels.

[0077] Step 3: Multi-scale feature fusion;

[0078] The low-frequency features and high-frequency features will be fused through the bridge mode (Bridge Connection). The role of the bridge mode is to splice low-frequency and high-frequency features from different scales to ensure that the semantic information of different scales is effectively preserved. This fusion method can effectively prevent the loss of details and provide rich multi-level features for the subsequent Transformer module. After the low-frequency and high-frequency features are fused through the bridge layer, the resulting feature map will be sent to the next stage of global information modeling.

[0079] The features after bridging fusion are input into the Transformer encoder, which is based on the ViT architecture in the SAM model. Transformer models the long-range dependencies in the image through the self-attention mechanism (Self-Attention) to capture the global context information of the image. This module enables the network to understand the relationship between distant pixels in the image through global information modeling, thereby improving the segmentation effect. The features processed by the Transformer will be restored through the inverse wavelet transform (IWT), and the restored features will be returned to the spatial domain, thereby retaining the details and structural information of the image.

[0080] Specifically, the feature extraction and multi-scale feature fusion in the present invention are realized by the BridgeWave Transformer hybrid encoder. The function of the BridgeWave Transformer hybrid encoder is to extract image details and global information at multiple scales, especially in medical images with blurred boundaries and low contrast, to improve the feature extraction ability of the model. The module structure is as follows Figure 2-3 shown.

[0081] The main working principle of this module is: assuming that the input feature is x, firstly, the input feature is preliminarily processed through the convolution layer (SingleConv) to obtain the preliminary low-level feature x1, where the convolution kernel size is 3×3 and the number of channels is C. The purpose is to extract low-level features in the image, such as edge and texture information. This process is expressed as:

[0082] x1=SingleConv(x1)

[0083] Then, the input feature x1 is decomposed into multiple scales by wavelet transform (DWT) to extract low-frequency and high-frequency information:

[0084] x2,x3=DWT(x1)

[0085] Among them, x2 represents the low-frequency part, which captures the overall structure of the image; x3 represents the high-frequency part, which captures the details and edge information of the image.

[0086] Specifically, DWT decomposes the input feature map through a filter bank:

[0087]

[0088] in, and are low-pass filter and high-pass filter respectively, m and n are the coordinates of the image. Through this decomposition, the low-frequency part retains the rough information of the image, while the high-frequency part captures the details and edge information.

[0089] Next, features x2 and x3 are fused through the bridge connection mode. The bridge layer concatenates these two features to preserve information at different scales:

[0090] x bridge =Concat(x2,x3)

[0091] This splicing method ensures the retention of multi-scale information, avoids detail loss, and provides rich multi-level features for the subsequent Transformer module, which helps the subsequent model to better preserve boundaries and details when processing images.

[0092] Then, feature x bridge It will be sent to the Transformer module for global information modeling. The Transformer module is based on the ViT (Vision Transformer) architecture in SAM (Segment Anything Model).

[0093] x4=Transformer(x bridge )

[0094] Transformer uses the self-attention mechanism to model the long-range dependencies in the image and obtain global context information. The calculation process of the self-attention mechanism is as follows:

[0095]

[0096] Among them, Q is the query, K is the key, V is the value, and d is the feature dimension. The self-attention mechanism enhances the modeling of long-range dependencies in the image through these weighted coefficients.

[0097] Next, the feature x4 after Transformer modeling will be restored to the spatial dimension through inverse wavelet transform (IWT) to obtain the restored feature x5:

[0098] x5=IWT(x4)

[0099] The process of IWT is to reconstruct low-frequency and high-frequency features back to the original features:

[0100]

[0101] Among them, δ L and δ H It is an inverse filter of low pass and high pass, which is used to restore the frequency domain information to the spatial domain. The inverse wavelet transform restores the features from the frequency domain of the wavelet transform to the spatial domain, thereby preserving the detail information and restoring the structure of the image.

[0102] Finally, the feature x5 after wavelet transform, Transformer modeling, and inverse wavelet transform is weighted fused with the initial input feature x through residual connection to obtain the final output feature map x out :

[0103] x out =x+0.5×x5

[0104] Through this weighted fusion, the model can better combine the initial input features with the output after deep feature modeling, improve the image segmentation and restoration performance, especially in images with blurred boundaries and lost details. Residual connections help avoid the gradient vanishing problem during training and improve the stability of the model.

[0105] Step 4: Feature decoding and prediction;

[0106] This process is implemented using a decoder, which uses a multi-head self-attention module and a deformable attention module to further process the features output by the encoder.

[0107] First, the decoder decodes the image features through the self-attention mechanism to capture long-range dependencies and enhance the understanding of the global information of the image. Then, the deformable attention mechanism is used to further adjust the features and optimize the modeling of details and complex structures. The decoded feature sequence is input into the prediction head, and the predicted target box coordinates and target category are finally output. This process can accurately generate target segmentation maps and classify different targets, improving segmentation accuracy, especially when processing images with blurred details or unclear boundaries, showing strong robustness.

[0108] The above method is implemented through a medical image segmentation model, which includes a BridgeWave Transformer hybrid encoder, decoder, and data preprocessing module. The above is the full process of the model inference stage. When the model is in the training stage, it is necessary to calculate the Dice loss and boundary-aware BCE loss in the image segmentation task separately, and combine the two loss functions to obtain the total loss of the model. The gradient of the loss is calculated through the back-propagation algorithm, and the network parameters are gradually adjusted to optimize the segmentation results.

[0109] The present invention proposes a composite loss function that combines the Dice coefficient and the boundary-aware binary cross entropy loss function (Boundary-Aware BCE Loss), aiming to improve the accuracy in image segmentation tasks, especially when processing images with blurred boundaries and lost details. The combination of these two loss functions can improve the overall segmentation effect and enhance the prediction ability of details and boundary areas. The calculated loss is used to update the network parameters through the back propagation algorithm to improve the segmentation accuracy of the model. Therefore, in the training mode, the Dice coefficient loss and the boundary-aware BCE loss are used to optimize the model.

[0110] The loss function consists of two parts: Dice coefficient loss (Dice Loss) is used to measure the similarity between the predicted image and the true label, which is particularly suitable for processing unbalanced data sets and small target objects; Boundary-Aware BCE Loss (Boundary-Aware BCE Loss) strengthens the focus on the boundary between the target object and the background on the basis of the standard binary cross entropy loss function (BCE), thereby improving the accuracy of the segmentation result in the boundary area. Specifically, by adding the weight of the boundary area, the model can give a higher training signal when segmenting the boundary area, thereby enhancing the prediction accuracy of the boundary.

[0111] The composite loss function combining the Dice coefficient and the boundary-aware BCE coefficient is as follows:

[0112] 1) The Dice coefficient is used to measure the overlap between the prediction and the true label. Its calculation formula is:

[0113]

[0114] Among them, A and B are the predicted and true binary areas respectively. The larger the Dice coefficient, the higher the overlap between the predicted area and the true area. The Dice loss is:

[0115]

[0116] Among them, y pred is the predicted binary image, y true is the real label image. The goal of Dice Loss is to minimize the difference between the segmented region and the true region to improve the overall accuracy of segmentation.

[0117] 2) Boundary-aware BCE loss introduces weights into the standard binary cross entropy loss function to highlight the boundary areas in the image, thereby improving the model's segmentation accuracy for boundary areas. The standard BCE loss is:

[0118]

[0119] The boundary-aware BCE loss modifies this standard formula by adding greater weights to the boundary regions and is defined as:

[0120]

[0121] Among them, y true and pred They are the true label and prediction result of the image respectively;

[0122] w boundary (y ttue ) is a weight function that is dynamically adjusted based on boundary information. Its purpose is to enhance the segmentation accuracy of boundary areas during training, ensure that the model can more accurately predict boundary areas in images, and enhance the training signal of boundary areas. Generally, the weight of boundary areas is larger, which encourages the model to make more accurate predictions in boundary areas.

[0123] 3) Combined loss function

[0124] The final loss function is the weighted sum of the Dice coefficient loss and the boundary-aware BCE loss. The formula for the composite loss function is:

[0125] Total Loss=β1·Dice Loss+β2·Boundary_Aware BCE Loss

[0126] Among them, β1 and β2 are hyperparameters used to balance the weights of the two losses. Usually, the selection of β1 and β2 depends on the requirements of boundary accuracy and overall segmentation accuracy in the specific task.

[0127] Based on the above invention content, the following technical effect evaluation is proposed:

[0128] Performance evaluation,The present invention was experimented and evaluated on the TN3K and BUSI datasets. The TN3K dataset is an ultrasound image dataset for thyroid nodule segmentation, which contains 3493 images and segmentation annotations, which are divided into training sets and test sets. BUSI is a classification and segmentation dataset containing breast ultrasound images, which consists of 780 images, which are divided into three categories: normal, benign and malignant. The training set, test set, and validation set are divided in a ratio of 7:2:1.

[0129] The present invention is compared with various optimization and improvement algorithms of the benchmark model SAM.

[0130] Table 1 Comparative test results on BUSI and TN3K datasets

[0131]

[0132]

[0133] The results show that the proposed method achieves the best performance on both datasets, with a Dice coefficient of 84.50 and a Hausdorff distance (HD) of 28.11 on the TN3K dataset, and a Dice coefficient of 88.27 and a HD of 21.57 on the BUSI dataset. Compared with the benchmark model SAM and various optimization algorithms, the proposed method shows significant advantages in both segmentation accuracy (Dice coefficient) and boundary consistency (HD).

[0134] The above description is only used to help understand the method of the present invention and its core essence, but the protection scope of the present invention is not limited thereto. For those skilled in the art in the art, equivalent replacement or change according to the technical solution and inventive concept of the present invention within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A medical image segmentation method based on wavelet transform combined with SAM based on bridging mode, characterized in that: At least the following steps are included: S1. Data preprocessing: perform data enhancement on the input image and standardize the input image; S2, feature extraction; The preprocessed image is used to extract preliminary features using a convolutional neural network, and multi-scale feature decomposition is performed through wavelet transform to obtain low-frequency feature maps and high-frequency feature maps; S3, multi-scale feature fusion; The low-frequency feature map and the high-frequency feature map are fused through the bridge mode, the fused features are weighted through the adaptive attention mechanism, and the weighted features are restored through the inverse wavelet transform; the feature maps that have been processed differently are weighted and fused; S4, feature decoding and prediction: decode image features through self-attention mechanism, and adjust image features using deformable attention mechanism; The decoded features are input into the prediction head, and the predicted target box coordinates and target category are finally output.

2. The medical image segmentation method based on wavelet transform and SAM combined with bridging mode according to claim 1 is characterized in that: The S2 is specifically as follows: Assuming the input feature is x, the input feature is preliminarily processed through the convolution layer to obtain the preliminary low-level feature x1; this process is expressed as: x1=SingleConv(x1) The input feature x1 is decomposed into multiple scales through wavelet transform to extract low-frequency and high-frequency information, which is expressed as: x2,x3=DWT(x1) Among them, x2 represents the characteristics of the low-frequency part, capturing the overall structure of the image; x3 represents the characteristics of the high-frequency part, capturing the details and edge information of the image; DWT is wavelet transform.

3. The medical image segmentation method based on wavelet transform and SAM combined with bridging mode according to claim 2 is characterized in that: The wavelet transform performs multi-scale decomposition, as follows: The wavelet transform decomposes the input feature map through a filter bank, which is expressed as: in, and are low-pass filter and high-pass filter respectively, m and n are the coordinates of the image.

4. The medical image segmentation method based on wavelet transform and SAM combined with bridging mode according to claim 3 is characterized in that: The S3 is as follows: Features x2 and x3 are fused through the bridge mode to retain information at different scales, expressed as: x bridge =Concat(x2,x3) Among them, Concat represents the bridge layer, x bridge It represents the fusion of the low-frequency features and the high-frequency features. For feature x bridge Global information modeling is performed through Transformer, which is expressed as: x4=Transformer(x bridge ) The feature x4 after Transformer modeling is restored to the spatial dimension through inverse wavelet transform to obtain the restored feature x5, which is expressed as: x5=IWT(x4) Where IWT stands for inverse wavelet transform; After wavelet transformation, Transformer modeling, and inverse wavelet transformation, the feature x5 is weightedly fused with the initial input feature x through residual connection to obtain the final output feature map x out , expressed as: x out =x+0.5×x5。 5. The medical image segmentation method based on wavelet transform and SAM combined with bridging mode according to claim 4 is characterized in that: The Transformer performs global information modeling as follows: Transformer models the long-range dependencies in the image through the self-attention mechanism to obtain global context information. The calculation process of the self-attention mechanism is as follows: Among them, Q is the query, K is the key, V is the value, and d is the feature dimension; the self-attention mechanism enhances the modeling of long-range dependencies in the image through these weighted coefficients.

6. The medical image segmentation method based on wavelet transform and SAM combined with bridging mode according to claim 4 is characterized in that: The inverse wavelet transform is restored to the spatial dimension as follows: The process of inverse wavelet transform is to reconstruct the low-frequency and high-frequency features back to the original features, which can be expressed as: Among them, δ L and δ H It is the inverse filter of low-pass and high-pass, which is used to restore the frequency domain information to the spatial domain.

7. A medical image segmentation system based on the combination of wavelet transform and SAM based on bridging mode applied to any of the methods described in claims 1-6, characterized in that: A medical image segmentation model is used, wherein the medical image segmentation model comprises: BridgeWave Transformer hybrid encoder, including wavelet transform module, bridge layer, Transformer encoder, inverse wavelet transform module, weighted fusion module; Transformer encoder is based on the ViT architecture in SAM and has an adaptive attention mechanism; it is used for feature extraction and multi-scale feature fusion; The decoder includes a multi-head self-attention module, a deformable attention module, and a prediction head; it is used for feature decoding and prediction.

8. The medical image segmentation system based on wavelet transform and SAM combined with bridging mode according to claim 7 is characterized in that: In the training phase, the Dice loss and the boundary-aware BCE loss in the image segmentation task are calculated respectively, and the total loss of the model is obtained by combining the two loss functions. The gradient of the loss is calculated through the back-propagation algorithm to update the model network parameters. For the composite loss function that combines Dice loss with boundary-aware BCE loss, the composite loss function formula is: Total Loss=β1·Dice Loss+β2·Boundary_Aware BCE Loss Among them, β1 and β2 are hyperparameters used to balance the weights of the two parts of the loss; Dice Loss is the Dice loss, and Boundary_Aware BCE Loss is the boundary-aware BCE loss.

9. The medical image segmentation system based on wavelet transform and SAM combined with bridging mode according to claim 8, characterized in that: The Dice loss is specifically: Among them, y pred is the predicted binary image, y true is the real label image; the Dice coefficient is used to measure the overlap between the prediction and the real label, and the goal of Dice Loss is to minimize the difference between the segmented area and the real area.

10. The medical image segmentation system based on wavelet transform and SAM combined with bridge mode according to claim 9, characterized in that: The boundary perception BCE loss is specifically: Among them, w boundary (y true ) is a weight function that is dynamically adjusted based on boundary information.

Images