Ultrasonic image segmentation method based on edge guidance

Abstract

The invention relates to the technical field of medical image processing, in particular to an ultrasonic image segmentation method based on edge guidance, and the method comprises the steps: inputting an ultrasonic image into an edge extraction branch and an image encoder; outputting an edge mask image corresponding to the image through the edge extraction branch; performing noise suppression and topological correction on the edge mask image to generate a closed edge constrained by an anatomical structure; generating prompt box information based on the closed edge, and inputting the prompt box information to a prompt encoder; fusing the image prompt features output by the prompt encoder, the image features extracted by the image encoder and the edge mask information; the fusion features are input into a decoding module, a final segmentation result is obtained, the synergistic effect of edge information and visual prompt is fully utilized, the perception ability of the model for the anatomical structure in the ultrasonic image is improved, and the accuracy and robustness of segmentation are remarkably improved. Therefore, the problems of fuzzy boundary, inaccurate prompt, weak structure identification capability and the like in related technologies are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, and particularly relates to an edge-guided ultrasound image segmentation method. BACKGROUND

[0002] Medical image segmentation is a core technology for identifying and highlighting specific organs, tissues and lesion regions in medical images. As a key component of computer-aided diagnosis systems, it plays an important role in disease research and early detection. In recent years, with the development of deep learning, the accuracy and efficiency of medical image segmentation have been significantly improved. However, existing deep learning-based methods usually rely on a large number of annotated datasets for training, while in the medical field, the acquisition of high-quality labeled data is often costly and limited in availability, which severely limits the generalization ability and promotion value of the model.

[0003] The SAM (Segment Anything Model) model has been trained on more than one billion natural images and their mask data, showing excellent zero-shot generalization ability, providing a new way to solve the data scarcity problem in medical image segmentation. However, when directly applied to medical images, its segmentation accuracy and robustness are obviously insufficient. This is because its training data only covers natural scenes, lacking learning of medical image structural features, making it difficult to cope with challenges such as small targets, complex boundaries and weak contrast in medical images.

[0004] Although the model can be fully fine-tuned on medical datasets to improve performance, such methods not only require a large amount of computing resources, but also easily lead to overfitting of the model to specific datasets, affecting its generalization ability. Therefore, various lightweight adaptation strategies have been proposed, such as adjusting or expanding the parameters of part of the model to reduce training costs while improving adaptability. These strategies mainly include: optimizing the mask decoder to better integrate medical image features; using low-rank parameter adjustment methods to reduce the number of parameters to be updated; or inserting small modules into the visual Transformer structure to introduce task-related information. However, these methods generally use fixed-size non-overlapping image patches in the input processing stage, limiting the model's ability to capture local details, especially for small size lesions, complex shape structures or low contrast regions.

[0005] To further compensate for the above limitations, there are schemes to reconstruct the image encoder of the model, introduce additional convolution branches and cross-branch attention mechanisms, enhance the perception of local features while retaining the original strong generalization ability, and control the overall computational overhead. However, this type of method still relies on random sampling strategies in prompt information generation, lacks accurate positioning of target regions, and is prone to interfere with the final segmentation results. In addition, it does not fully utilize edge information to explicitly guide the segmentation process, making it difficult to further improve segmentation accuracy and boundary quality. SUMMARY

[0006] The present application provides an edge-guided ultrasound image segmentation method based on edge guidance mechanism and prompt information fusion, combining edge feature extraction and segmentation of arbitrary model with powerful image understanding ability. Through structure optimization and information fusion strategy, the recognition ability of anatomical structure and boundary positioning accuracy in ultrasound image are improved, thereby solving the problems of blurred boundary, inaccurate prompt and insufficient structure perception in related art.

[0007] The first aspect of the present application provides an edge-guided ultrasound image segmentation method, comprising the following steps: inputting the ultrasound image into an edge extraction branch and an image encoder respectively, generating an edge mask image corresponding to the ultrasound image through the edge extraction branch, extracting edge features, and extracting image features through the image encoder, wherein in the edge extraction branch, a gated convolution module is used to perform attention weighting processing on the edge features to generate edge attention, and the edge mask image is output; the edge mask image is subjected to noise suppression and topological structure correction to obtain a closed edge contour, a prompt box is generated based on the closed edge contour, and the prompt box is input into a prompt encoder, and the image prompt features are output by the prompt encoder; the image prompt features, the image features and the edge mask image are fused, and the fused features are input into an image decoder, and the image decoder outputs a final segmentation result.

[0008] Optionally, the gated convolution module performs attention weighting processing on the edge features to generate edge attention, comprising: splicing the intermediate features of the edge branch and the texture backbone branch; generating a pixel-level attention map through 1x1 convolution and an activation function; performing element-wise multiplication on the edge features and the attention map, and outputting the edge attention through channel convolution and residual connection after fusion.

[0009] Optionally, the edge extraction branch needs to be trained, and the edge mask obtained by gradient calculation of the semantic segmentation label is used as a supervision signal during the training process, and a binary cross-entropy loss is used for optimization.

[0010] Optionally, the prompt box is generated based on the minimum bounding rectangle of the closed edge contour, and is used to represent the spatial range of the target region.

[0011] Optionally, the fusion of image hint features, image features and edge mask images adopts a multi-scale attention mechanism and channel weighted operation to enhance structure perception ability.

[0012] This application improves the SAM model and obtains SAMUS (Segment Anything Model for Ultrasound, an arbitrary segmentation model in medical ultrasound images). Through the edge information guidance mechanism, the original ultrasound image is simultaneously input into the edge extraction branch and the SAMUS trunk image encoder. The branch outputs an edge mask image, which effectively supplements the expression ability of the SAMUS trunk encoder in weak boundary areas. The edge mask image output by the edge extraction branch is subjected to noise suppression and topological structure correction to obtain a closed edge contour; based on the contour, a minimum circumscribed rectangle is generated as prompt box information to represent the spatial range of the target area, thereby providing accurate prompt input; the prompt box information is input into the prompt encoder of SAMUS to generate an image prompt Features are then fused with the image prompt features, the image features extracted by the image encoder, and the edge mask image. This fusion process uses a multi-scale attention mechanism and a channel weighting strategy to ensure that the fused features have both local accuracy and global context understanding capabilities at the spatial and semantic levels. The fused features are input into the decoding module to generate the final segmented image, achieving accurate segmentation of organs or lesion areas, enhanced boundary recognition capabilities, more structurally constrained prompt box generation, stronger context understanding and adaptability, and lower resource consumption. While improving the accuracy and robustness of medical image segmentation, it reduces the training and deployment costs of the model and has high practical application value. This solves the problems of blurred boundaries, inaccurate prompts, and weak structural recognition capabilities in related technologies.

[0013] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0015] Figure 1 A flowchart of an edge-guided ultrasound image segmentation method according to an embodiment of the present application;

[0016] Figure 2 A schematic diagram of an ultrasound image segmentation method based on an edge-aware SAM model according to an embodiment of the present application;

[0017] Figure 3 A schematic diagram of an edge extraction network structure provided according to an embodiment of the present application; DETAILED DESCRIPTION

[0018] Embodiments of the present application are described below in detail, examples of which are shown in the accompanying drawings, in which the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.

[0019] An edge-guided ultrasound image segmentation method of an embodiment of the present application is described below with reference to the accompanying drawings. In view of the problems of insufficient local detail extraction capability, inaccurate prompt information positioning, and insufficient edge information utilization of the related art mentioned in the background art, the present application provides an edge-guided ultrasound image segmentation method, in which the ultrasound image is input in parallel to an edge extraction network and a segmentation model, the edge extraction network is used to extract an edge mask image of the ultrasound image, the segmentation model includes an image encoder for image feature extraction, a prompt encoder for processing a prompt box, and a decoding module for segmentation prediction based on multi-source fusion features. The image features of the ultrasound image are extracted by the image encoder, and then the edge mask image is subjected to noise suppression and topological correction to generate an anatomical constraint closed contour. Then, the prompt box is calculated according to the anatomical constraint closed contour, and the prompt box is input into the prompt encoder. The prompt encoder outputs image prompt encoding, and the edge mask image and the image features are fused based on the image prompt. The fusion result is input into the image decoder, and the image decoder outputs the segmentation result, which greatly improves the accuracy and reliability of the ultrasound image segmentation, and realizes high-precision medical image segmentation without human intervention. Thus, the problems of fuzzy boundaries, inaccurate prompts, and insufficient structure perception in the related art are solved.

[0020] Specifically, Figure 1 A flowchart of an edge-guided ultrasound image segmentation method provided by an embodiment of the present application is shown in the figure, which is constructed based on an improved SAMUS model.

[0021] As Figure 1 shown, the edge-guided ultrasound image segmentation method includes the following steps:

[0022] In step S101, the ultrasound image is input into an edge extraction branch and an image encoder, respectively, an edge mask image corresponding to the ultrasound image is generated by the edge extraction branch, and edge features are extracted. In the edge extraction branch, a gating convolution module is used to perform attention weighting processing on the edge features, and an edge mask image is output.

[0023] The edge extraction branch includes a residual network (for example, a ResNet-50 (Residual Network-50) network) pre-trained on a medical image dataset and an edge-attention preservation (EAP) module composed of residual blocks and gated convolution blocks (GCBs) stacked alternately, which is dedicated to efficiently extracting edge information in the ultrasound image, enhancing the model's ability to perceive edge information in the image, and thus preserving important contour information in the image. The gated convolution block is a module that combines traditional convolution operations and attention mechanisms to guide the model to pay more attention to specific regions or features. The edge features are processed by the gated convolution block to output an edge mask map. The edge mask map is a map representing the probability or intensity of the presence of edges in different positions in the image.

[0024] Specifically, the edge flow in the edge-attention preservation module uses the multi-layer features extracted by the texture branch encoder of the backbone network as guidance to process and refine the corresponding boundary and shape information layer by layer. This process relies on multiple gated convolution blocks to explicitly model and filter the edge features.

[0025] It can be understood that the embodiments of the present application simultaneously input the ultrasound image into the edge extraction branch and the image encoder. The edge extraction branch is responsible for generating the corresponding edge mask map. In the edge extraction branch, the edge-attention preservation module is used to enhance the model's ability to perceive edge information in the image. The texture features extracted by the backbone network are used to guide the extraction of edge features. The gated convolution module is used to perform attention weighting processing on the extracted edge features, thereby enhancing the representation ability of the key edge region, and finally outputting a refined edge mask map.

[0026] In the embodiments of the present application, the gated convolution module performs attention weighting processing on the edge features to generate edge attention, including: concatenating the intermediate features of the edge branch and the texture backbone branch; generating a pixel-level attention map through 1x1 convolution and an activation function; performing element-wise multiplication on the edge features and the attention map, and then outputting the edge attention through channel convolution and residual connection after fusion.

[0027] The edge branch is a network branch specially used for extracting edge information of an image; the texture backbone branch is a backbone network based on a residual network, and is used for extracting texture and high-level semantic information of the image; the intermediate feature splicing is to splice feature maps of the two branches at the same scale in a channel dimension, to fuse edge and texture information, so that the model considers edge structure and semantic content at the same time when generating an attention map; and the activation function is a Sigmoid activation function, which limits the output in a range of [0, 1], to represent importance of each pixel position.

[0028] It can be understood that in each GCB module, the edge flow feature is spliced with the corresponding layer feature from the main texture flow, and then a 1x1 convolution and a Sigmoid activation function are used to generate a pixel-level attention map, which weights the edge feature, retains the boundary-related region and suppresses the irrelevant background, and further enhances the edge response through channel convolution and residual connection, to obtain edge attention, which strengthens the attention to key edge features and improves the performance of the model.

[0029] In the embodiment of the present application, the edge extraction branch needs to be trained, and in the training process, an edge mask calculated by using a semantic segmentation label gradient is used as a supervision signal, and a binary cross entropy loss is used for optimization.

[0030] In the training process, the edge map output by the edge extraction network is supervised and learned with an edge label map calculated by using a semantic segmentation label gradient, and the optimization target is a standard binary cross entropy loss. The binary cross entropy loss is a loss function commonly used in binary classification problems, and is used to measure the difference between the model prediction result and the true label. This supervision mechanism helps the network to accurately locate the boundary region in the image and improves the structure perception ability in the segmentation stage.

[0031] It can be understood that the edge extraction branch of the embodiment of the present application is trained in a supervised learning manner, with a real edge map extracted from a semantic segmentation label as a target, and a binary cross entropy loss function is used to optimize network parameters, so that the edge extraction module can more accurately identify the boundary information in the image, thereby enhancing the understanding and expression ability of the entire segmentation model for the structure of the image.

[0032] In step S102, noise suppression and topological structure correction are performed on the edge mask map to obtain a closed edge contour, a prompt box is generated based on the closed edge contour, and the prompt box is input into the prompt encoder, and the prompt encoder outputs image prompt features.

[0033] The noise suppression refers to processing the noise in the edge mask map to reduce unwanted interference signals, which can be caused by speckle noise or other artifacts in the ultrasound imaging process. The topology correction refers to filtering abnormal contours according to a preset anatomical constraint rule library to ensure that the generated closed contour conforms to medical knowledge and expected morphology, and can accurately reflect the actual edge position in the ultrasound image to form a complete and reasonable boundary. The generation of the prompt box will be described in detail below and will not be repeated here.

[0034] It can be understood that, after obtaining the edge mask map and image features of the ultrasound image, the edge mask map needs to be subjected to noise suppression and topology correction in the embodiments of the present application to reduce the noise components in the mask map, improve the clarity and accuracy of the edge information, and adjust the contour according to the preset anatomical constraint rule library to ensure that it conforms to the anatomical characteristics in medicine. Finally, an anatomically constrained closed contour is generated, and a prompt box is generated based on this and fed into a prompt encoder to be converted into image prompt features that are helpful for image segmentation tasks.

[0035] It should be noted that, in the embodiments of the present application, the anatomical constraints include at least one of a thyroid edge maximum curvature, a liver blood vessel branch angle, and a fetal heart chamber area ratio threshold monitoring.

[0036] The thyroid edge maximum curvature refers to setting an upper limit value for the maximum bending degree of the edge of the thyroid when performing image segmentation on the thyroid, for example, a maximum curvature ≤ 0.25 mm-1, to ensure that the segmented thyroid shape conforms to the actual anatomical structure. The liver blood vessel branch angle refers to the angle between the branches of the blood vessels when segmenting the liver and its internal blood vessel structure, which should fall within a reasonable range, for example, an angle ∈ [30°, 150°], to ensure the authenticity and accuracy of the segmentation result. The fetal heart chamber area ratio threshold monitoring refers to the area ratio between different chambers of the fetal heart needing to meet a certain threshold range during segmentation, which can ensure that the segmented heart structure conforms to the physiological standard.

[0037] It can be understood that the anatomical constraints of the embodiments of the present application include at least one of a thyroid edge maximum curvature, a liver blood vessel branch angle, and a fetal heart chamber area ratio threshold monitoring, that is, these medical anatomical rules are applied to adjust and verify the contour during the segmentation process to ensure that the generated segmentation result conforms to the conditions that the thyroid edge should not exceed a certain curvature, the angle between the branches of the blood vessels in the liver should be within a specified range, and the area ratio of each chamber of the fetal heart needs to conform to the physiological standard, thereby improving the accuracy and reliability of the ultrasound image segmentation.

[0038] In the embodiments of the present application, the prompt box is generated based on the minimum circumscribed rectangle of the closed edge contour to represent the spatial range of the target region.

[0039] Among them, the minimum bounding rectangle refers to the smallest rectangle that can completely surround a certain shape or a set of points, which can be axis-aligned (parallel to the coordinate axis) or rotated, used to concisely describe the position and size of the target region.

[0040] Specifically, in order to extract the bounding box from the edge mask which can be used for segmentation guidance, based on the spatial distribution of all foreground regions in the edge post-processing result, the pixel point coordinate boundary statistics method is used to generate the bounding box, and the flow includes the following steps:

[0041] Firstly, a clear edge image is obtained by bilateral filtering (a nonlinear filtering method that not only considers the spatial distance between pixels, but also considers the intensity difference, which can smooth the image while preserving the edges) and smoothing enhancement operation (smoothing enhancement operation refers to a series of techniques aimed at improving image quality, including but not limited to noise reduction, contrast adjustment, etc.), then adaptive binarization (a technique for converting a grayscale image to a black and white image, automatically selecting a threshold value based on the brightness of the area around each pixel) and morphological operation (a shape-based image processing method consisting of a sequence of two basic operations, erosion and dilation, first erosion, then dilation, mainly used to remove small objects, noise and smooth object boundaries in the image, while trying to keep the main structural features of the object unchanged) are used to remove noise and obtain a structurally complete edge mask image; Then, the coordinate set of all white pixels in the edge mask image is extracted, and the minimum bounding rectangle of the region is calculated based on the set, that is, the four boundary coordinates [x1, y1, x2, y2] of the bounding box are determined directly according to the maximum / minimum x and y values of all foreground pixels; Unlike the traditional principal component analysis or minimum rotating rectangle method, the horizontal axis-aligned box without rotation angle is used in this application, which has the advantages of simple calculation, high efficiency and stability, and can avoid the influence of small target rotation calculation error on the premise of maintaining boundary accuracy. The generated bounding box will be further normalized, that is, its coordinates will be mapped to the relative coordinate range of [0, 1] to facilitate the use of the bounding box in different size images for guidance operation.

[0042] It can be understood that the bounding box of the embodiments of the present application is based on the significant pixel region in the edge mask, and a compact and effective bounding box is obtained by directly counting its boundary position, which quickly and accurately determines the spatial position of the target region without relying on principal component analysis or rotation geometry, thereby improving the image segmentation effect and efficiency under the guidance of the bounding box.

[0043] It should be noted that in the embodiments of the present application, in order to improve the quality of the edge mask and suppress noise interference in the image, an edge post-processing strategy based on image spatial characteristics is adopted, and the specific processing flow is as follows:

[0044] Firstly, bilateral filtering is applied to the initially obtained edge mask image to achieve smooth noise reduction, and the bilateral filter effectively suppresses high-frequency noise while retaining the edge, thereby maintaining the clear boundary of the anatomical structure; secondly, a Laplacian operator is further introduced for edge enhancement, and the recognizability of the boundary structure is improved by enhancing the edge response area; then, the enhanced image is subjected to Otsu adaptive binarization (an algorithm for automatically selecting a threshold for image binarization, the core idea of which is to find a threshold to maximize the inter-class variance between the foreground and background, and the greater the inter-class variance, the higher the contrast between the foreground and background, and the better the segmentation effect), to separate the edge significant area from the background and obtain a clear binary boundary image; in order to remove pseudo-edges and small noise regions, morphological opening operation is used to process the binary image to further purify the boundary structure; finally, all external contours in the binary image are extracted using a contour detection method, and the minimum circumscribed bounding box of the target region is obtained by counting the boundary coordinates of all foreground pixel points, thereby realizing accurate positioning of the target boundary in the ultrasound image.

[0045] In step S103, the image prompt feature, the image feature and the edge mask image are fused, and the fused feature is input into an image decoder. The image decoder outputs a final segmentation result.

[0046] It can be understood that after obtaining the fusion result through the above steps, the fusion result is input into the image decoder, the image decoder infers the target region using the fusion result, and outputs the segmentation result, i.e., gives the possibility of each pixel in the image belonging to a specific target region. The method of fusing the image prompt feature, the image feature and the edge mask image is as follows:

[0047] In the embodiment of the present application, the fusion of the image prompt feature, the image feature and the edge mask image adopts a multi-scale attention mechanism and a channel weighting operation to enhance the structure perception ability.

[0048] The multi-scale attention mechanism is a technology that can capture important information at different scales. By applying the attention mechanism at multiple scales, details and global structures in the image can be better captured, and the understanding ability of the model for complex structures can be enhanced. The channel weighting operation refers to assigning different weights to different channels during feature fusion, thereby emphasizing important feature channels and suppressing unimportant channels.

[0049] It can be understood that, in order to enhance the perception ability of the model to the image structure, the embodiments of the present application adopt a multi-scale attention mechanism and a channel weighting operation when fusing the image prompt feature, the image feature and the edge mask image. Specifically, the multi-scale attention mechanism can focus on the key area at different scales, and the channel weighting operation highlights important features by dynamically weighting each channel. This fusion strategy not only improves the attention of the model to the target area, but also enhances the understanding of complex structures.

[0050] Specifically, the embodiments of the present application fuse the edge mask image and the image feature based on the image prompt feature to enhance the perception ability of the segmentation model to the target boundary area. The process includes: injecting the edge mask image into the image feature, and controlling the injection intensity by a gating mechanism, thereby obtaining the fused feature.

[0051] Specifically, first, the edge mask image is obtained by an edge extraction network and is input into the fusion module after image-level processing. In order to avoid too strong or too weak interference of edge information on the overall image feature, a gating mechanism is introduced to control the injection intensity of edge information. The gating mechanism uses a lightweight convolution layer combined with a Sigmoid activation function to calculate a gating weight map α to dynamically adjust the injection proportion of edge features, and the formula is:

[0052] α = σ(Conv([F img ; M edge ])) img edge

[0053] wherein, F img represents the main image feature output by the image encoder, M edge represents the edge mask image feature, [;] represents channel dimension splicing, Conv represents convolution operation, and σ represents Sigmoid activation function.

[0054] The final fusion feature calculation method is:

[0055] F fused = α·M edge + F img

[0056] It can be understood that, by introducing the edge mask image into the image feature and dynamically adjusting the introduction proportion of edge information by the gating mechanism, the embodiments of the present application enhance the attention degree of the model to the target boundary area, and improve the accuracy and continuity of the segmentation result at the edge.

[0057] According to the edge-guided ultrasound image segmentation method provided in the embodiments of the present application, the ultrasound image is input in parallel to an edge extraction network and a segmentation model, the edge mask of the ultrasound image is extracted by the edge extraction network, the segmentation model includes an image encoder, a prompt encoder and an image decoder, the image features of the ultrasound image are extracted by the image encoder, then the edge mask is subjected to noise suppression and topological correction to generate an anatomical constraint closed contour, and then the prompt box is calculated according to the anatomical constraint closed contour, the prompt box is input into the prompt encoder, the prompt encoder outputs the image prompt code, and the edge mask and the image features are fused based on the image prompt; the fusion result is input into the image decoder, and the image decoder outputs the segmentation result, which greatly improves the accuracy and reliability of the ultrasound image segmentation, and realizes high-precision medical image segmentation without manual intervention.

[0058] The edge-guided ultrasound image segmentation method will be further described below through a specific embodiment.

[0059] The edge-guided ultrasound image segmentation method based on the edge-aware SAM model provided in the embodiments introduces an edge extraction module in the existing SAMUS (Segment Anything Model for Ultrasound Segmentation, Segment Anything Model for Ultrasound Segmentation) framework, automatically generates a prompt box input into a prompt encoder, and realizes high-precision medical image segmentation without manual intervention. The specific process is as shown in Figure 2 The edge-guided ultrasound image segmentation method based on the edge-aware SAM model provided in the embodiments introduces an edge extraction module in the existing SAMUS (Segment Anything Model for Ultrasound Segmentation, Segment Anything Model for Ultrasound Segmentation) framework, automatically generates a prompt box input into a prompt encoder, and realizes high-precision medical image segmentation without manual intervention. The specific process is as shown in

[0060] 1. Dual-mode parallel processing:

[0061] The present application proposes a dual-mode parallel processing strategy, which extracts features from the edge stream and the main image stream in two directions. Among them:

[0062] (1) The edge feature extraction network adopts an edge attention preserving module, which is composed of multiple residual blocks and gate convolution modules. The module takes the features output by the main image encoder as input, and focuses on extracting information related to the structure boundary. The gate convolution module in the edge attention preserving module calculates the position attention map according to the features of the main image stream and the edge stream, combines residual connection and channel weighting, and refines the edge representation layer by layer;

[0063] (2) The first image main encoder adopts a frozen parameter CNN-ViT parallel hybrid structure (an architecture design combining the advantages of CNN (Convolutional Neural Networks) and ViT (Vision Transformers), aiming to take advantage of CNN's strong points in processing local information and spatial invariance, and ViT's ability to capture global dependencies, thereby improving the model's performance in various visual tasks), combining local convolution perception and global visual modeling capabilities, effectively obtaining organ context semantic features in medical images.

[0064] 2. Anatomically constrained edge optimization:

[0065] Perform the following operations on the edge mask output by the edge extraction network:

[0066] (1) Speckle noise suppression: use an ultrasound-specific filtering algorithm based on the coherent diffusion equation;

[0067] (2) Topology structure correction: filter abnormal contours according to the preset organ anatomy rule library (such as the maximum curvature of the thyroid gland edge ≤0.25 mm-1);

[0068] (3) In the filtered and structure-constrained edge mask, extract the main connected domain and calculate its minimum bounding rectangle. Output the normalized coordinates of the two vertices of the top-left corner and the bottom-right corner of the rectangle frame as prompt information.

[0069] 3. Prompt encoding adaptation:

[0070] Convert the bounding box vertex coordinates obtained in step 2 into an input format that the prompt encoder (i.e. the SAMUS prompt encoder) can recognize, which includes:

[0071] (1) Boundary box format standardization: automatically set invalid boxes to the full image prompt box [-1, -1, h, w], where h and w are the height and width of the image, respectively, and the rest of the cases maintain the original four-tuple format. The final result is a one-dimensional vector [x1, y1, x2, y2], where (x1, y1) and (x2, y2) are the normalized coordinates of the top-left corner and the bottom-right corner of the prompt box, respectively, which can be directly used as input for the SAMUS prompt encoder.

[0072] 4. Edge enhancement feature fusion:

[0073] Introduce EAGU (Edge Attention Gating Unit) in the first decoder of SAMUS, whose workflow is as follows:

[0074] (1) Receive multi-scale feature maps {F1, F2, F3, F4} from the edge extraction network;

[0075] (2) Calculate the feature weight matrix of each layer by the cross-layer attention mechanism:

[0076] α i =Softmax(Conv(F i ⊕F sam ))

[0077] Where, represents channel splicing;

[0078] (3) Output the fused features.

[0079] Specifically, the present embodiment is applied to thyroid image segmentation, as follows:

[0080] 1. Data preprocessing

[0081] Since the present embodiment mainly focuses on the edge-guided segmentation process, no complex data preprocessing operations are performed, only the following basic steps are completed:

[0082] (1) Size adjustment: uniformly scale the image to a fixed size (e.g., 256x256) to adapt to the model input requirements;

[0083] (2) Data type conversion: convert the image to an 8-bit unsigned integer tensor (uint8) for subsequent inference and bounding box generation.

[0084] 2. Edge extraction network training

[0085] The edge extraction network adopts a structure based on the edge attention preservation module, containing multiple residual blocks and gated convolution modules. The gated convolution module, as a key component, is used to guide the edge flow through the main image stream features, focusing only on boundary-related information, and the network structure is as shown in Figure 3 The specific process is as follows:

[0086] Assuming that the main stream feature at the tthposition of each layer is r t , and the edge stream feature is e t , first, splice the two types of features, and generate an attention map a t through 1x1 convolution and sigmoid activation:

[0087] α t =σ(C([e t ⊕r t ]))

[0088] Then, according to the attention map, weight the edge features, and through residual connection and channel weighting kernel w t get the updated feature

[0089]

[0090] Multi-layer gated convolution module sequentially connects each layer feature E of the main encoder l , l e {1, 2, 3, 4}, constituting a complete edge attention guidance process. The final edge branch outputs an edge probability map S e , as an edge supervision path.

[0091] The edge supervision loss adopts a standard binary cross-entropy loss function, and compares the edge branch output S e with the edge label map G s extracted from the semantic segmentation label map G e :

[0092]

[0093] where w, h are the width and height of the edge map respectively, (x, y) is the pixel coordinate, G e is a binary edge map obtained by taking the gradient of G s .

[0094] • Training data: 200 ultrasound images labeled with thyroid edges, Adam (Adaptive Moment Estimation, learning rate lr = 1e-4, batch size = 8) optimizer.

[0095] 3. Suggestion box generation

[0096] • The following processing steps are performed on the edge mask image to generate a standard forward boundary box [x1, y1, x2, y2] for segmentation suggestions:

[0097] (1) Edge map smoothing and enhancement: Perform bilateral filtering on the input mask image to remove noise while preserving edges; then enhance the image gradient information by Laplacian operator and weighted fusion with the smoothed image to improve the contour saliency.

[0098] (2) Automatic binarization processing: Use the Otsu method for automatic threshold binarization to generate clear foreground regions.

[0099] (3) Small region removal:

[0100] Apply morphological opening operation (3x3 convolution kernel, iterate 2 times) to remove isolated small regions and retain the main connected domain.

[0101] (4) Boundary box extraction:

[0102] The foreground pixel positions are accurately extracted using np.where (a function in the NumPy library in Python that can select elements from an array according to conditions), the minimum circumscribed positive rectangular boundary box is calculated, and is converted into a standard format (such as returning the whole image if there is no valid area).

[0103] 4. Model inference

[0104] The forward boundary box coordinates [x1, y1, x2, y2] generated in step 3 are input into the prompt encoder of the segmentation model, and the entire inference process is as follows Figure 2 :

[0105] 1. Image encoder output feature size: 768x32x32

[0106] 2. The output dimension of the prompt encoder is [batch, 2, 256], where batch is the batch size, 2 is the number of features or channels, and 256 is the sequence length or feature dimension.

[0107] 3. The segmentation probability map output by the image decoder after fusion is binarized to obtain the final result by thresholding at 0.5.

[0108] Through the above steps, the performance obtained by the embodiment is shown in Table 1, where Table 1 is the performance table obtained by the embodiment on a test set including 120 examples.

[0109] Table 1

[0110]

[0111] The embodiment improves the Dice coefficient (used to measure the degree of overlap between the predicted area and the true area, and the closer the value is to 1, the higher the matching degree of the predicted result and the actual annotation) by 1.26%, and reduces the boundary Hausdorff distance (compares the difference between two shapes or contours, evaluates whether the boundaries of the predicted result and the actual annotation are close, and the smaller the numerical value, the closer the two boundaries, and the higher the matching degree) by 4.64%, indicating that the method is more accurate in boundary positioning while ensuring the overall segmentation accuracy.

[0112] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0113] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0114] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing a custom logical function or process step, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0115] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, the steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement the method: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0116] A person skilled in the art may understand that all or part of the steps carried out in the method for implementing the above-mentioned embodiment may be completed by instructing the relevant hardware through a program, and the above-mentioned program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0117] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and are not to be construed as limiting the present application, and that changes, modifications, substitutions and variations can be made by those skilled in the art without departing from the scope of the present application.

Claims

1. An edge-guided ultrasound image segmentation method, characterized in that: The method comprises the following steps: Inputting the ultrasound image into the edge extraction branch and the image encoder respectively, generating an edge mask map corresponding to the ultrasound image through the edge extraction branch, extracting edge features, and extracting image features through the image encoder, wherein, in the edge extraction branch, a gated convolution module is used to perform attention weighting processing on the edge features to generate edge attention, and output an edge mask map; Performing noise suppression and topological structure correction on the edge mask image to obtain a closed edge contour, generating a prompt box based on the closed edge contour, and inputting the prompt box into a prompt encoder, which outputs an image prompt feature; The image prompt features, image features and edge mask image are fused, the fused features are input into an image decoder, and the image decoder outputs a final segmentation result.

2. The edge-guided ultrasound image segmentation method according to claim 1, characterized in that: The gated convolution module performs attention weighting processing on edge features to generate edge attention, including: Splice the edge branch with the intermediate features of the texture trunk branch; Generate pixel-level attention map through 1×1 convolution and activation function; The edge features are element-wise multiplied with the attention map, and after fusion, the edge attention is output through channel convolution and residual connection.

3. The edge-guided ultrasound image segmentation method according to claim 1, wherein: The edge extraction branch needs to be trained. During the training process, the edge mask obtained by calculating the gradient with the semantic segmentation label is used as the supervision signal, and the binary cross entropy loss is used for optimization.

4. The edge-guided ultrasound image segmentation method according to claim 1, wherein: The prompt box is generated based on the minimum circumscribed rectangle of the closed edge contour and is used to represent the spatial range of the target area.

5. The edge-guided ultrasound image segmentation method according to claim 1, wherein: The fusion of the image prompt features, image features and edge mask images adopts a multi-scale attention mechanism and channel weighted operation to enhance the structure perception ability.

Images