Medical image annular region segmentation method and system, medium and equipment

Abstract

The invention relates to the technical field of image processing, and provides a medical image annular region segmentation method and system, a medium and equipment, and the method comprises the steps: obtaining a to-be-segmented medical image, and positive points and negative points marked in the medical image; a plurality of positive points are uniformly distributed in an annular region of the medical image; the negative point is positioned in the center of the closed pattern formed by the positive points; based on the positive points and the negative points, annular prompt features are obtained through an annular prior prompt encoder; after feature coding is carried out on a to-be-segmented medical image, an annular connected region feature is obtained through a region connectivity enhanced image coder; and based on the annular prompt feature and the annular connected region feature, obtaining an annular region segmentation result through a feature cooperation mask decoder. And confusion among different region features is reduced, so that the precision of annular region segmentation is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a method, system, medium and device for segmenting circular regions of medical images. Background Art

[0002] The statements in this part merely provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] Medical image segmentation is crucial for automatically annotating data by extracting regions of interest from medical images or videos. With the progress of deep learning, many medical image segmentation algorithms for various medical imaging modalities have emerged, providing support for downstream tasks such as clinical parameter calculation, 3D (three-dimensional) reconstruction, and auxiliary diagnosis.

[0004] In some medical images, the regions of interest for clinicians may include circular lesions or tissues, such as the vessel wall in intravascular ultrasound (IVUS) images and cardiac magnetic resonance images.

[0005] However, due to the frequent occurrence of blurring and unclear boundaries in medical images, and because of the within-class variability and between-class similarity inherent in them, circular regions are easily confused with adjacent regions, resulting in segmentation errors. Therefore, reducing this confusion and achieving accurate segmentation of circular regions is a major challenge. Summary of the Invention

[0006] In order to solve the technical problems existing in the above background art, the present invention provides a method, system, medium and device for segmenting circular regions of medical images. After annotating positive points and negative points in the medical image, a circular prior hint encoder is used to capture complex circular region hint information using multiple points, thereby reducing the confusion between different region features and improving the accuracy of circular region segmentation.

[0007] To achieve the above object, the present invention adopts the following technical solutions: The first aspect of the present invention provides a method for segmenting circular regions of medical images, which includes: Obtaining a medical image to be segmented and positive points and negative points annotated in the medical image; there are multiple positive points, and they are evenly distributed in the circular region of the medical image; there is one negative point, and it is located at the center of the closed figure formed by the positive points; Based on the positive points and negative points, obtaining circular hint features through a circular prior hint encoder; After feature encoding the medical image to be segmented, obtaining circular connected region features through a region connectivity enhanced image encoder; Based on the annular prompt feature and the annular connected region feature, the annular region segmentation result is obtained through the feature collaborative mask decoder.

[0008] Further, the positive points and negative points are obtained by annotating a prompt box in the shape of a 2×2 grid on the medical image to be segmented, and the midpoints of the four outer sides of the prompt box are positive points, and the center of the prompt box is a negative point.

[0009] Further, the processing steps of the annular prior prompt encoder include: Obtain the coordinates of the positive points and negative points, and through Gaussian matrix feature embedding, obtain the coordinate point sparse embedding and label sparse embedding; After concatenating the coordinate point sparse embedding and the label sparse embedding, learn and update the feature embedding through the annular constraint module to obtain the annular prompt feature.

[0010] Further, the processing steps of the region connectivity enhanced image encoder include: Pass the features of the encoded medical image to be segmented through several Transformer blocks and feature decomposition blocks to obtain the initial features; Pass the initial features in parallel through the morphological processing module and the neck module for feature processing to obtain the morphological image embedding and the original image embedding respectively; Use convolution and multi-head attention mechanism to fuse the morphological image embedding and the original image embedding to obtain the annular connected region feature.

[0011] The second aspect of the present invention provides a medical image annular region segmentation system, which includes: A data acquisition module, which is configured to: acquire the medical image to be segmented and the positive points and negative points annotated in the medical image; there are multiple positive points, and they are evenly distributed in the annular region of the medical image; there is one negative point, and it is located at the center of the closed figure composed of positive points; A prompt encoding module, which is configured to: based on the positive points and negative points, obtain the annular prompt feature through the annular prior prompt encoder; An image encoding module, which is configured to: after feature encoding the medical image to be segmented, obtain the annular connected region feature through the region connectivity enhanced image encoder; A segmentation module, which is configured to: based on the annular prompt feature and the annular connected region feature, obtain the annular region segmentation result through the feature collaborative mask decoder.

[0012] Further, the positive points and negative points are obtained by annotating a prompt box in the shape of a 2×2 grid on the medical image to be segmented, and the midpoints of the four outer sides of the prompt box are positive points, and the center of the prompt box is a negative point.

[0013] Furthermore, the processing steps of the circular prior prompt encoder include: Obtain the coordinates of positive points and negative points, and through Gaussian matrix feature embedding, obtain coordinate point sparse embedding and label sparse embedding; After concatenating the coordinate point sparse embedding and the label sparse embedding, learn and update the feature embedding through the circular constraint module to obtain the circular prompt feature.

[0014] Furthermore, the processing steps of the region connectivity enhanced image encoder include: Pass the features of the encoded medical image to be segmented through several Transformer blocks and feature decomposition blocks to obtain initial features; Pass the initial features in parallel through a morphological processing module and a neck module for feature processing to obtain morphological image embedding and original image embedding respectively; Use convolution and multi-head attention mechanism to fuse the morphological image embedding and the original image embedding to obtain circular connected region features.

[0015] The third aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps in a method for segmenting a circular region of a medical image as described above are implemented.

[0016] The fourth aspect of the present invention provides a computer device, including a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor. When the processor executes the program, the steps in a method for segmenting a circular region of a medical image as described above are implemented.

[0017] Compared with the prior art, the beneficial effects of the present invention are: After marking positive points and negative points in the medical image, the present invention uses a circular prior prompt encoder to capture complex circular region prompt information by using multiple points, thereby reducing the confusion between different region features and improving the accuracy of circular region segmentation.

[0018] In order to simplify the prompting operation, the present invention uses a 2×2 grid-shaped prompt box to mark and obtain multiple prompt points simultaneously.

[0019] In order to improve the expression ability of multi-point discrete prompt features, the present invention introduces a circular constraint module to fit curve-based prompt features and provide more comprehensive prompts for the feature collaborative mask decoder.

[0020] The regional connectivity enhanced image encoder of the present invention uses morphological closing operations to identify the connected regions of image features within the image encoder, and achieves multi-level fusion guided by a multi-head attention mechanism to enhance feature connections across regions. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings forming a part of this specification are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation of the present invention.

[0022] Figure 1 is a flowchart of a method for segmenting a circular region of a medical image according to Embodiment 1 of the present invention; Figure 2 is a structural diagram of a circular prior hint encoder according to Embodiment 1 of the present invention; Figure 3 is a structural diagram of a regional connectivity enhanced image encoder according to Embodiment 1 of the present invention; Figure 4 is a structural diagram of a feature collaboration mask decoder according to Embodiment 1 of the present invention; Figure 5 is a schematic diagram of segmentation results for different types of IVUS images according to Embodiment 1 of the present invention; Figure 6 is a schematic structural diagram of a computer device according to Embodiment 4 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0024] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0025] Embodiment 1 This embodiment provides a method for segmenting a circular region of a medical image.

[0026] The method for segmenting a circular region of a medical image provided in this embodiment can solve the problem of circular region confusion caused by intra-class variability and inter-class similarity in medical images, and improve the segmentation accuracy of circular regions in medical images.

[0027] The method for segmenting a circular region of a medical image provided in this embodiment, as Figure 1 shown, includes the following steps: Step 1: Obtain the medical image to be segmented.

[0028] Among them, the medical image to be segmented is the original intravascular ultrasound (IVUS) image.

[0029] Step 2: For a medical image containing a circular lesion or tissue, the user uses the multi-point one-key annotation method to annotate multiple (5) cue points in the circular region of the medical image. The cue points are 4 positive points relatively evenly distributed in the circular region and 1 negative point (negative point) located in the inner region.

[0030] Among them, the multi-point one-key annotation method reduces the complexity of the cueing process by using a cue box in the shape of a 2×2 grid in the image to simultaneously mark and obtain multiple cue points.

[0031] Preferably, the negative point is located at the center of the closed figure formed by the 4 positive points.

[0032] As an implementation, the cue box is a "field" character box. The positions where the "mouth" intersects with the "+" are recorded as positive points, and the intersection of the horizontal and vertical lines in the "+" is recorded as the negative point.

[0033] Among them, the positive points are the points within the area desired by the user, and the negative points are the points within the area not desired by the user.

[0034] Step 3: Input the multiple cue points mentioned in Step 2 into the Annular Prior Prompt Encoder (APPE) to obtain a 5×256-dimensional annular cue feature.

[0035] The annular prior encoder is mainly composed of a Gaussian matrix feature embedding and an Annular Constraint (AC) module, as Figure 2 shown. The specific steps are as follows: Step 301: Based on the coordinates (P coords = {p1, p2, p3, p4, p5}) and labels (P label = {1, 1, 1, 1, 0}) of the 5 cue points, where 1 represents a positive point and 0 represents a negative point, through Gaussian matrix feature embedding, obtain the coordinate point sparse embedding and the label sparse embedding L emb .

[0036] For example, the calculation formula of the coordinate point sparse embedding is as follows: ; ; Among them, represents the calculated coordinate point sparse embedding, GMpe Represents a Gaussian matrix for position encoding, Norm represents a normalization operation, and @ represents matrix multiplication.

[0037] Step 302: Then, perform feature embedding and label feature embedding L emb Concatenate and learn to update the feature embedding through the AC module to obtain a circular prompt feature F ap . As Figure 2 shown, the AC module includes two linearly connected layers, and an activation function is set between the two linearly connected layers. Its calculation formula is as follows: ; ; where, w 1 and w 2 represent the weights of each linear layer, b 1 and b 2 represent the bias parameters of each linear layer, ReLU is the activation function.

[0038] Step 4: After preprocessing the medical image containing the circular lesion or tissue, input it into the Region Connectivity Enhanced Image Encoder (RCEIE) to obtain a circular connectivity region feature with a dimension of 256×64×64, as Figure 3 shown. The specific steps are as follows: Step 401: Preprocess the medical image containing the circular lesion or tissue, mainly by performing feature encoding (feature embedding); Step 402: Pass the encoded image features through 12 Transformer (a model proposed by Google in 2017) modules and a Factorizer to output initial features F , and obtain the circular connectivity region feature through the Morphology Attention (MA) module. The specific steps are as follows: (1) The initial feature F is processed in parallel through the Morphological Processing (MP) module and the Neck Block to obtain the morphological image embedding F mp and the original image embedding F oAmong them, in the MP module, the morphological image embedding is derived by applying morphological closing operation to the image embedding. F mp .

[0039] Specifically, the morphological processing module applies the morphological closing operation, first eroding the features F , then performing dilation, and both dilation and erosion are processed by the activation function after that. This process eliminates the noise points in the image or the feature embedding matrix, repairs the small gaps, and smooths the boundaries of the region of interest.

[0040] The calculation formula of the morphological processing module is as follows: ; where represents the erosion operation, ⊕ represents the dilation operation, K mp is the elliptical structuring element, defined as follows: ; ; , ; where e xy represents K mp the corresponding element in K mp matrix, and if the condition is satisfied, the value is 1; x and y respectively represent C x , C y the number of rows and columns of the K mp matrix, where x = y; ([[]] A x , A y ) represents the coordinates of the center of the ellipse in

[0041] (2) Use convolution and multi-head attention mechanism to fuse the morphological image embedding F mp and the original image embedding F o to enhance the feature connectivity and remove the single-point noise, specifically including: First, use convolution operation to match F o and F mp in size (so that their size dimensions are the same), and perform layer normalization and GeLU activation on F mp ; Then, the output after GeLU activation obtains Q and K through size and dimension transformation; F o V is obtained through size and dimension transformation; based on Q, K, and V, the multi-head attention mechanism is used to fuse these two image embeddings to obtain the circular connected region feature embedding F ic , and its calculation formula is as follows: F ic = concatenate ( h 1 ,h 2 ,…,h i ) W out , where, i represents the number of heads, h i represents the output of the i th head, W out is the output transformation matrix (the output transformation matrix is a model parameter, initially obtained by loading a pre-trained model or initializing the model, and subsequent parameter gradient updates are achieved through model training backpropagation). The output h i of each head is calculated as h i =Att(QW i q , KW i k , VW i v ), where Q, K, and V represent query, key, and value respectively; W i q , W i k and W i v are the query transformation matrix, key transformation matrix, and value transformation matrix of the Att th head respectively,

[0042] In the multi-head attention mechanism (MHAM), the self-attention mechanism is used to calculate the attention, and its calculation formula is as follows: , where dim represents the size of the key.

[0043] In this embodiment, the softmax (normalized exponential) function is used to normalize the weights.

[0044] In this embodiment, Q is composed of F oobtained, and both K and V are obtained from F mp obtained.

[0045] Step 5: Input the annular prompt feature obtained in Step 3 and the annular connected region feature obtained in Step 4 into the Feature Synergy Mask Decoder (FSMD). As Figure 4 shown, the FSMD contains two Transformer layers and one deconvolution layer for feature decoding to obtain the annular region segmentation result.

[0046] A medical image annular region segmentation method (Medical Image Annular Region Segmentation Network, MIARS-Net) provided by the present invention specifically includes an annular prior prompt encoder (Annular Prior Prompt Encoder, APPE). This encoder uses multiple points to capture complex annular region prompt information, thereby reducing the confusion between different region features. To simplify the prompting operation, a 2×2 grid-shaped prompt box is used to simultaneously mark and obtain multiple prompt points. To improve the expression ability of multi-point discrete prompt features, an annular constraint (AC) module is introduced to fit curve-based prompt features to provide more comprehensive prompts for the Feature Synergy Mask Decoder (FSMD). In addition, it includes a region connectivity enhanced image encoder (Region Connectivity Enhanced Image Encoder, RCEIE). This encoder uses morphological closing operations to identify the connected regions of image features within the image encoder. It achieves multi-level fusion guided by the multi-head attention mechanism to enhance cross-region feature connections. The features generated by the above two encoders can work together in the FSMD to improve the accuracy of annular region segmentation.

[0047] As Figure 5 shown, a medical image annular region segmentation method provided by the present invention can accurately segment various-shaped annular regions in medical images (for example, the vascular wall region in IVUS images), providing reliable results for clinical practice.

[0048] Table 1. Quantitative ablation results of each module in the IVUS internal dataset

[0049] The quantitative ablation results of each module using the IVUS dataset are shown in Table 1. AC: Compared with the baseline model, replacing the box hint with a multi-point hint and adding the AC module (baseline + AC) can improve various evaluation metrics. The reason for the significant improvement is that AC can provide a robust annular hint function. MA: In RCEIE, through the MA module, the values of mIoU (Mean Intersection over Union), DSC (Dice Similarity Coefficient), and HD (Hausdorff Distance) will be optimized; the segmentation results of MIARS-Net are significantly closer to the labels; MA provides the connectivity features of the annular region and the background for the model, avoiding region confusion caused by intra-class variability. Factorizer: Adding Factorizer to the baseline model while keeping the hint method unchanged results in a slight performance improvement; removing RCEIE from the proposed MIARS-Net model leads to a performance decline; however, compared with other ablation models, it shows a small advantage in mIoU and DSC, which is due to the main purpose of the factorizer being to improve the convergence speed and efficiency.

[0050] Example 2 This embodiment provides a medical image annular region segmentation system, which specifically includes: A data acquisition module, which is configured to: acquire the medical image to be segmented and the positive points and negative points annotated in the medical image; there are multiple positive points, and they are evenly distributed in the annular region of the medical image; there is one negative point, and it is located at the center of the closed figure composed of positive points; A hint encoding module, which is configured to: based on the positive points and negative points, obtain annular hint features through an annular prior hint encoder; An image encoding module, which is configured to: after feature encoding the medical image to be segmented, obtain annular connected region features through a region connectivity enhanced image encoder; A segmentation module, which is configured to: based on the annular hint features and the annular connected region features, obtain the annular region segmentation result through a feature collaboration mask decoder.

[0051] Among them, the positive points and negative points are obtained by annotating a hint box in the shape of a 2×2 grid on the medical image to be segmented, and the midpoints of the four outer sides of the hint box are positive points, and the center of the hint box is a negative point.

[0052] Among them, the processing steps of the annular prior hint encoder include: Obtain the coordinates of the positive points and negative points, and through Gaussian matrix feature embedding, obtain coordinate point sparse embedding and label sparse embedding; After cascading the coordinate point sparse embedding and the label sparse embedding, the feature embedding is learned and updated through the circular constraint module to obtain the circular prompt feature.

[0053] Among them, the processing steps of the region connectivity enhanced image encoder include: The features of the encoded medical image to be segmented are passed through several layers of Transformer blocks and feature decomposition blocks to obtain initial features; The initial features are passed through the morphological processing module and the neck module in parallel for feature processing to obtain the morphological image embedding and the original image embedding respectively; Using convolution and multi-head attention mechanism, the morphological image embedding and the original image embedding are fused to obtain the circular connected region features.

[0054] It should be noted here that each module in this embodiment corresponds to each step in Embodiment 1 one by one, and the specific implementation process is the same, so it will not be repeated here.

[0055] Embodiment 3 This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the steps in a method for segmenting a circular region of a medical image as described in Embodiment above.

[0056] Embodiment 4 This embodiment provides a computer device, as Figure 6 shown, including a computer-readable storage medium 1003, a processor 1001, a communication interface 1002, and a computer program stored on the computer-readable storage medium 1003 and executable on the processor 1001. Among them, the processor 1001, the communication interface 1002, and the computer-readable storage medium 1003 can be connected through a bus or other means. Among them, the communication interface 1002 is used to receive and send data, and when the processor 1001 executes the program, it implements the steps in a method for segmenting a circular region of a medical image as described in Embodiment above.

[0057] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for segmenting a circular region of a medical image, characterized in that, Including: Obtain the medical image to be segmented, as well as the positive points and negative points marked in the medical image; There are multiple positive points, which are evenly distributed in the circular area of the medical image; There is one negative point, which is located at the center of the closed figure composed of positive points; Based on the positive points and negative points, through the circular prior prompt encoder, obtain the circular prompt feature; After feature encoding the medical image to be segmented, through the region connectivity enhanced image encoder, obtain the circular connectivity region feature; Based on the circular prompt feature and the circular connectivity region feature, through the feature collaborative mask decoder, obtain the circular region segmentation result.

2. The medical image circular region segmentation method according to claim 1, wherein, The positive points and negative points are obtained by marking a 2×2 grid-shaped prompt box on the medical image to be segmented, and the midpoints of the four outer sides of the prompt box are positive points, and the center of the prompt box is the negative point.

3. The method for segmenting an annular region of a medical image according to claim 1, wherein The processing steps of the circular prior prompt encoder include: Obtain the coordinates of the positive points and negative points, and through Gaussian matrix feature embedding, obtain the coordinate point sparse embedding and the label sparse embedding; After cascading the coordinate point sparse embedding and the label sparse embedding, learn and update the feature embedding through the circular constraint module to obtain the circular prompt feature.

4. The medical image annular region segmentation method according to claim 1, wherein The processing steps of the region connectivity enhanced image encoder include: Pass the features of the encoded medical image to be segmented through several Transformer blocks and feature decomposition blocks to obtain the initial features; Pass the initial features in parallel through the morphological processing module and the neck module for feature processing, and obtain the morphological image embedding and the original image embedding respectively; Use convolution and multi-head attention mechanism to fuse the morphological image embedding and the original image embedding to obtain the circular connectivity region feature.

5. A medical image circular region segmentation system, characterized in that, Including: A data acquisition module, which is configured to: obtain the medical image to be segmented, as well as the positive points and negative points marked in the medical image; there are multiple positive points, which are evenly distributed in the circular area of the medical image; there is one negative point, which is located at the center of the closed figure composed of positive points; A prompt encoding module, which is configured to: based on the positive points and negative points, through the circular prior prompt encoder, obtain the circular prompt feature; An image encoding module, which is configured to: after feature encoding the medical image to be segmented, through the region connectivity enhanced image encoder, obtain the circular connectivity region feature; A segmentation module, which is configured to: based on the circular prompt feature and the circular connectivity region feature, through the feature collaborative mask decoder, obtain the circular region segmentation result.

6. The medical image circular region segmentation system according to claim 5, characterized in that, The positive points and negative points are obtained by marking a 2×2 grid-shaped prompt box on the medical image to be segmented, and the midpoints of the four outer sides of the prompt box are positive points, and the center of the prompt box is the negative point.

7. The medical image annular region segmentation system according to claim 5, characterized in that, The processing steps of the circular prior prompt encoder include: Obtain the coordinates of the positive points and negative points, and through Gaussian matrix feature embedding, obtain the coordinate point sparse embedding and the label sparse embedding; After cascading the coordinate point sparse embedding and the label sparse embedding, learn and update the feature embedding through the circular constraint module to obtain the circular prompt feature.

8. The medical image circular region segmentation system according to claim 5, wherein The processing steps of the region connectivity enhanced image encoder include: The features of the encoded medical image to be segmented are passed through several layers of Transformer blocks and feature decomposition blocks to obtain initial features; The initial features are passed through a morphological processing module and a neck module in parallel for feature processing to obtain a morphological image embedding and an original image embedding respectively; Using convolution and multi-head attention mechanisms, the morphological image embedding and the original image embedding are fused to obtain annular connected region features.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps in a method for segmenting an annular region of a medical image according to any one of claims 1-4.

10. A computer device, comprising a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in a method for segmenting an annular region of a medical image according to any one of claims 1-4.

Images