A medical image annular region segmentation method, system, medium and device

Abstract

The present application relates to the technical field of image processing, and provides a medical image annular region segmentation method, system, medium and equipment, which comprises the following steps: obtaining a medical image to be segmented and positive points and negative points labeled in the medical image; the positive points are multiple and uniformly distributed in the annular region of the medical image; the negative point is one and located at the center of the closed figure composed of the positive points; based on the positive points and the negative point, an annular prior prompt encoder is used to obtain annular prompt features; after feature encoding of the medical image to be segmented, an area connectivity enhanced image encoder is used to obtain annular connected region features; based on the annular prompt features and the annular connected region features, a feature collaborative mask decoder is used to obtain an annular region segmentation result. The confusion between different region features is reduced, so that the accuracy of annular region segmentation is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a medical image annular region segmentation method, system, medium and equipment. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] Medical image segmentation is crucial for automatically annotating data by extracting regions of interest from medical images or videos. With the progress of deep learning, many medical image segmentation algorithms for various medical imaging modalities have emerged, providing support for downstream tasks such as clinical parameter calculation, 3D reconstruction, and auxiliary diagnosis.

[0004] In some medical images, the regions of interest for clinicians may include annular lesions or tissues, such as blood vessel walls in Intravenous Ultrasound (IVUS) images and in cardiac magnetic resonance images.

[0005] However, due to the frequent occurrence of blur and unclear boundaries in medical images, annular regions are easily confused with adjacent regions due to their inherent intra-class variability and inter-class similarity, leading to segmentation errors. Therefore, it is a major challenge to alleviate this confusion and achieve accurate segmentation of annular regions. SUMMARY

[0006] To solve the technical problems existing in the background art, the present application provides a medical image annular region segmentation method, system, medium and equipment. After labeling positive points and negative points in a medical image, an annular prior prompt encoder utilizes multiple points to capture complex annular region prompt information, thereby reducing confusion between different region features and improving the accuracy of annular region segmentation.

[0007] To achieve the above-mentioned purposes, the present application adopts the following technical solutions:

[0008] The first aspect of the present application provides a medical image annular region segmentation method, which comprises:

[0009] Obtaining a medical image to be segmented and labeling positive points and negative points in the medical image; the positive points are multiple and uniformly distributed in the annular region of the medical image; the negative point is one and located at the center of the closed figure composed of the positive points;

[0010] Based on the positive points and the negative points, an annular prompt feature is obtained by an annular prior prompt encoder;

[0011] The feature coding is performed on the medical image to be segmented, and a region connectivity enhanced image encoder is used to obtain ring-shaped connected region features.

[0012] Based on the ring-shaped prompt features and the ring-shaped connected region features, a feature collaborative mask decoder is used to obtain a ring-shaped region segmentation result.

[0013] Further, the positive points and the negative points are obtained by labeling a 2*2 grid-shaped prompt box on the medical image to be segmented, and the midpoints of the four outer edges of the prompt box are the positive points, and the center of the prompt box is the negative point.

[0014] Further, the processing steps of the ring-shaped prior prompt encoder include:

[0015] The coordinates of the positive points and the negative points are obtained, and Gaussian matrix feature embedding is used to obtain coordinate point sparse embedding and label sparse embedding;

[0016] After the coordinate point sparse embedding and the label sparse embedding are concatenated, the feature embedding is updated by a ring-shaped constraint module to obtain the ring-shaped prompt features.

[0017] Further, the processing steps of the region connectivity enhanced image encoder include:

[0018] The features of the encoded medical image to be segmented are obtained through a plurality of layers of Transformer blocks and feature decomposition blocks to obtain initial features;

[0019] The initial features are processed in parallel through a morphological processing module and a neck module to obtain morphological image embedding and original image embedding, respectively;

[0020] The morphological image embedding and the original image embedding are fused using convolution and multi-head attention mechanism to obtain ring-shaped connected region features.

[0021] The second aspect of the present application provides a medical image ring-shaped region segmentation system, which includes:

[0022] A data acquisition module configured to acquire a medical image to be segmented and positive points and a negative point labeled in the medical image; the positive points are multiple and uniformly distributed in the ring-shaped region of the medical image; the negative point is one and located at the center of the closed figure composed of the positive points;

[0023] A prompt encoding module configured to obtain ring-shaped prompt features based on the positive points and the negative points through a ring-shaped prior prompt encoder;

[0024] An image encoding module configured to perform feature coding on the medical image to be segmented, and obtain ring-shaped connected region features through a region connectivity enhanced image encoder.

[0025] The segmentation module is configured to obtain a ring region segmentation result through a feature cooperative mask decoder based on the ring prompt feature and the ring connected region feature.

[0026] Further, the positive points and the negative points are obtained by marking a 2*2 grid-shaped prompt box on the medical image to be segmented, and the midpoints of the four outer edges of the prompt box are all positive points, and the center of the prompt box is a negative point.

[0027] Further, the processing steps of the ring prior prompt encoder include:

[0028] The coordinates of the positive points and the negative points are obtained, and the coordinate point sparse embedding and the label sparse embedding are obtained through Gaussian matrix feature embedding;

[0029] After the coordinate point sparse embedding and the label sparse embedding are concatenated, the feature embedding is learned and updated through a ring constraint module to obtain the ring prompt feature.

[0030] Further, the processing steps of the region connectedness enhanced image encoder include:

[0031] The features of the encoded medical image to be segmented are obtained through a plurality of layers of Transformer blocks and feature decomposition blocks to obtain initial features;

[0032] The initial features are processed in parallel through a morphological processing module and a neck module to obtain morphological image embedding and original image embedding, respectively;

[0033] The morphological image embedding and the original image embedding are fused using convolution and multi-head attention mechanism to obtain the ring connected region feature.

[0034] The third aspect of the present application provides a computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of the medical image ring region segmentation method as described above.

[0035] The fourth aspect of the present application provides a computer device, comprising a computer readable storage medium, a processor and a computer program stored on the computer readable storage medium and executable on the processor, wherein the processor executes the program to implement the steps of the medical image ring region segmentation method as described above.

[0036] Compared with the prior art, the present application has the following advantages:

[0037] The application labels positive points and negative points in a medical image, and then uses multiple points to capture complex ring area prompt information by a ring prior prompt encoder, so as to reduce the confusion between different area features and improve the accuracy of ring area segmentation.

[0038] The application uses a 2*2 grid-shaped prompt box to mark and acquire multiple prompt points at the same time, so as to simplify the prompt operation.

[0039] The application introduces a ring constraint module to fit curve-based prompt features, so as to improve the expression ability of multi-point discrete prompt features and provide a more comprehensive prompt for a feature collaborative mask decoder.

[0040] The region connectivity enhanced image encoder adopts a morphological closing operation to identify the connected regions of image features in the image encoder, and realizes multi-level fusion guided by a multi-head attention mechanism to enhance the feature connection across regions. BRIEF DESCRIPTION OF DRAWINGS

[0041] The drawings constituting a part of the specification of the application are used to provide a further understanding of the application, and the schematic embodiments of the application and the description thereof are used to explain the application, and do not constitute an improper limitation on the application.

[0042] Figure 1 is a flowchart of a medical image ring area segmentation method of embodiment one of the application;

[0043] Figure 2 is a structure diagram of a ring prior prompt encoder of embodiment one of the application;

[0044] Figure 3 is a structure diagram of a region connectivity enhanced image encoder of embodiment one of the application;

[0045] Figure 4 is a structure diagram of a feature collaborative mask decoder of embodiment one of the application;

[0046] Figure 5 is a segmentation result schematic diagram of different types of IVUS images of embodiment one of the application;

[0047] Figure 6 is a structure schematic diagram of a computer device of embodiment four of the application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical scheme and advantages of the embodiments of the application clearer, the technical scheme in the embodiments of the application will be described clearly and completely with reference to the drawings in the embodiments of the application.

[0049] It should be noted that the following detailed description is illustrative and aims to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention pertains.

[0050] Example 1

[0051] This example provides a method for segmenting the annular region of medical images.

[0052] The method for segmenting the annular region of medical images provided in this example can solve the problem of confusion in the annular region caused by intra-class variability and inter-class similarity in medical images, and improve the segmentation accuracy of the annular region of medical images.

[0053] The method for segmenting the annular region of medical images provided in this example, as Figure 1 shown, includes the following steps:

[0054] Step 1: Obtain the medical image to be segmented.

[0055] Among them, the medical image to be segmented is the original intravascular ultrasound (IVUS) image.

[0056] Step 2: For the medical image containing annular lesions or tissues, the user uses the multi-point one-key annotation method to annotate multiple (5) cue points in the annular region of the medical image. The cue points are respectively 4 positive points relatively evenly distributed in the annular region and 1 negative point (negative point) located in the internal region.

[0057] Among them, the multi-point one-key annotation method uses a cue box in the shape of a 2×2 grid in the image to simultaneously mark and obtain multiple cue points, so as to reduce the complexity of the cue process.

[0058] Preferably, the negative point is located at the center of the closed figure formed by the 4 positive points.

[0059] As an implementation manner, the cue box is a "field" character box. The positions where the "mouth" intersects with the "+", are recorded as positive points, and the intersection of the horizontal and vertical lines in the "+" is recorded as the negative point.

[0060] Among them, the positive points are the points within the area desired by the user, and the negative points are the points within the area not desired by the user.

[0061] Step 3: Input the multiple cue points mentioned in Step 2 into the Annular Prior Prompt Encoder (APPE) to obtain a 5×256-dimensional annular cue feature.

[0062] The annular prior encoder mainly consists of a Gaussian matrix feature embedding and an annular constraint (AC) module, as shown in Figure 2 The specific steps are as follows:

[0063] Step 301, based on the coordinates (P coords ={p1, p2, p3, p4, p5}) and labels (P label ={1, 1, 1, 1, 0}) of the five prompt points, 1 representing positive points and 0 representing negative points, the coordinate point sparse embedding and the label sparse embedding L emb are obtained through Gaussian matrix feature embedding.

[0064] For example, the calculation formula of the coordinate point sparse embedding is as follows:

[0065] ;

[0066] ;

[0067] wherein, represents the calculated coordinate point sparse embedding, GM pe represents the Gaussian matrix used for position encoding, Norm represents a normalization operation, and @ represents matrix multiplication.

[0068] Step 302, the feature embedding is concatenated with the label feature embedding L emb to update the feature embedding through the AC module, and the annular prompt feature F ap is obtained. As shown in Figure 2 , the AC module includes two linear layers connected in sequence, and an activation function is arranged between the two linear layers. The calculation formula is as follows:

[0069] ;

[0070] ;

[0071] wherein, w 1 and w 2 represent the weight of each linear layer, b 1 and b 2 represent the bias parameter of each linear layer, ReLU and f is the activation function.

[0072] Step 4: After image preprocessing, the medical image containing annular lesions or tissues is input into the Region Connectivity Enhanced Image Encoder (RCEIE), and a 256x64x64-dimensional annular connectivity region feature is obtained, as shown in Figure 3 The specific steps are as follows:

[0073] Step 401: The medical image containing annular lesions or tissues is preprocessed by image preprocessing, mainly feature encoding (feature embedding);

[0074] Step 402: The encoded image features are output after passing through a 12-layer Transformer module and a Factorizer block F , and the annular connectivity region feature is obtained by a Morphology Attention (MA) module. The specific steps are as follows:

[0075] (1) The initial feature F is processed in parallel by a Morphological Processing (MP) module and a Neck Block to obtain morphological image embedding F mp and original image embedding F o , respectively. In the MP module, the morphological image embedding F mp is derived by applying a morphological closing operation to the image embedding.

[0076] Specifically, the morphological processing module applies a morphological closing operation, first erodes the feature F , then performs dilation, and both the dilation and erosion are processed by an activation function. This process eliminates noise points in the image or feature embedding matrix, repairs small gaps, and smooths the boundaries of the region of interest.

[0077] The calculation formula of the morphological processing module is as follows: ; where represents the erosion operation, ⊕ represents the dilation operation, K mp is an elliptical structural element, defined as follows:

[0078] ;

[0079] ;

[0080] , ;

[0081] wherein, e xy denotes K mp the corresponding element in K mp the number of rows and columns of the matrix, wherein x=y; ( C x , C y ) denotes the coordinates of the center of the ellipse in K mp ; A x , A y ) denotes the length of the x-axis and y-axis of the ellipse.

[0082] (2) using convolution and multi-head attention mechanism to embed the morphological image into F mp and the original image embedding F o are fused to enhance the feature connectivity and remove single-point noise, which specifically includes:

[0083] First, use convolution operation to match the size of F o and F mp (making the size dimension of the two same), and perform layer normalization and GeLU activation on F mp ;

[0084] Then, the output after GeLU activation is obtained Q, K through size and dimension transformation; F o V is obtained through size and dimension transformation; based on Q, K and V, the two image embeddings are fused using multi-head attention mechanism to obtain the annular connected region feature embedding F ic , the calculation formula is as follows: F ic = concatenate ( h 1 ,h 2 ,…,h i ) W out , wherein, i denotes the number of heads, h i denotes the output of the i th head, W outis an output transformation matrix (the output transformation matrix is a model parameter, which is obtained by loading a pre-trained model or initializing a model at the beginning, and parameter gradient updating is realized through model training back propagation subsequently), the output of each head h i The calculation formula is h i =Att(QW i q , KW i k , VW i v ), Q, K and V represent query, key and value respectively; W i q , W i k and W i v are the query transformation matrix, key transformation matrix and value transformation matrix of the i-th head respectively, Att is an attention calculation function.

[0085] In the multi-head attention mechanism (MHAM), the self-attention mechanism is used to calculate attention, and the calculation formula is as follows: , wherein dim represents the size of the key.

[0086] In this embodiment, the softmax function is used to normalize the weight.

[0087] In this embodiment, Q is obtained by F o , and K and V are obtained by F mp .

[0088] Step 5: input the annular prompt feature obtained in step 3 and the annular connected region feature obtained in step 4 into a feature synergy mask decoder (FSMD), as shown in Figure 4 , the FSMD contains two layers of Transformer layers and one layer of deconvolution layers for feature decoding, to obtain an annular region segmentation result.

[0089] The application provides a medical image annular region segmentation method (Medical Image Annular Region Segmentation Network, MIARS-Net), specifically comprising an annular prior prompt encoder (Annular Prior Prompt Encoder, APPE), which uses multiple points to capture complex annular region prompt information, thereby reducing the confusion between different region features; in order to simplify the prompt operation, a 2x2 grid-shaped prompt box is used to mark and obtain multiple prompt points at the same time; in order to improve the expression ability of the multi-point discrete prompt features, an annular constraint (AC) module is introduced to fit the curve-based prompt features, and a feature synergy mask decoder (Feature Synergy Mask Decoder, FSMD) is provided with more comprehensive prompts; in addition, a region connectivity enhanced image encoder (Region Connectivity Enhanced Image Encoder, RCEIE) is included, which uses a morphological closing operation to identify the connected regions of image features in the image encoder, and realizes multi-level fusion guided by a multi-head attention mechanism to enhance the feature connection across regions; the features generated by the above two encoders can work together in the FSMD to improve the accuracy of annular region segmentation.

[0090] As Figure 5 indicated, the medical image annular region segmentation method provided by the application can accurately segment various morphological annular regions (for example, the blood vessel wall region in the IVUS image) in the medical image, and provide reliable results for the clinic.

[0091] Table 1, quantitative ablation results of each module in the IVUS internal data set

[0092]

[0093] Quantitative ablation results of each module using IVUS dataset are shown in Table 1. AC: Compared with the baseline model, replacing the box prompt with the multi-point prompt and adding the AC module (baseline + AC) can improve various evaluation indicators, and the reason for significant improvement is that AC can provide robust annular prompt function. MA: In RCEIE, the values of mIoU (Mean Intersection over Union), DSC (Dice Similarity Coefficient) and HD (Hausdorff Distance) will be optimized through the MA module; the segmentation result of MIARS-Net is obviously closer to the label; MA provides the connectivity features of annular regions and background for the model, avoiding the confusion of regions caused by intra-class variability. Factorizer: Adding Factorizer to the baseline model while keeping the prompt method unchanged leads to a slight improvement in performance; removing RCEIE from the proposed MIARS-Net model will result in a decrease in performance; however, compared with other ablation models, it shows a small advantage in mIoU and DSC, which is due to the main purpose of the factorizer to improve convergence speed and efficiency.

[0094] Embodiment Two

[0095] The embodiment provides a medical image annular region segmentation system, which specifically comprises:

[0096] A data acquisition module configured to acquire a medical image to be segmented and positive points and negative points labeled in the medical image; the positive points are multiple and uniformly distributed in an annular region of the medical image; and the negative point is one and located at the center of a closed figure composed of the positive points.

[0097] A prompt encoding module configured to obtain annular prompt features through an annular prior prompt encoder based on the positive points and the negative point.

[0098] An image encoding module configured to obtain annular connectivity region features through a region connectivity enhancement image encoder after feature encoding of the medical image to be segmented.

[0099] A segmentation module configured to obtain annular region segmentation results through a feature collaborative mask decoder based on the annular prompt features and the annular connectivity region features.

[0100] The positive points and the negative point are obtained by labeling a 2x2 grid-shaped prompt box on the medical image to be segmented, and the midpoints of the four outer edges of the prompt box are the positive points, and the center of the prompt box is the negative point.

[0101] The processing steps of the ring prior hint encoder include:

[0102] Get the coordinates of positive and negative points, and embed them into Gaussian matrix features to get sparse embedding of coordinate points and sparse embedding of labels;

[0103] After cascading the coordinate point sparse embedding and the label sparse embedding, the feature embedding is updated through the ring constraint module to obtain the ring prompt feature.

[0104] The processing steps of the region connectivity enhanced image encoder include:

[0105] The encoded features of the medical image to be segmented are passed through several layers of Transformer blocks and feature decomposition blocks to obtain the initial features;

[0106] The initial features are processed in parallel by a morphological processing module and a neck module to obtain a morphological image embedding and an original image embedding respectively;

[0107] Using convolution and multi-head attention mechanisms, the morphological image embedding and the original image embedding are fused to obtain the ring-connected region features.

[0108] It should be noted here that the various modules in this embodiment correspond one-to-one to the various steps in Example 1, and the specific implementation processes are the same, which will not be repeated here.

[0109] Example 3

[0110] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps in the method for segmenting annular regions in medical images as described in the first embodiment above are implemented.

[0111] Example 4

[0112] This embodiment provides a computer device, such as Figure 6 As shown, the present invention includes a computer-readable storage medium 1003, a processor 1001, a communication interface 1002, and a computer program stored on the computer-readable storage medium 1003 and executable on the processor 1001. The processor 1001, the communication interface 1002, and the computer-readable storage medium 1003 may be connected via a bus or other means. The communication interface 1002 is configured to receive and transmit data, and when the processor 1001 executes the program, the steps of the method for segmenting annular regions in medical images as described in the first embodiment are implemented.

[0113] The above merely provides the preferred embodiments of the present application, and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the principles and technical scope of the present application shall fall into the scope of the present application.

Claims

1. A medical image annular region segmentation method, characterized by, The method comprises the following steps: obtaining a medical image to be segmented and positive points and negative points labeled in the medical image; the positive points are multiple and uniformly distributed in the annular region of the medical image; the negative point is one and located at the center of the closed figure composed of the positive points; based on the positive points and the negative points, an annular prior prompt encoder is used to obtain annular prompt features; after feature encoding of the medical image to be segmented, an area connectivity enhanced image encoder is used to obtain annular connected region features; based on the annular prompt features and the annular connected region features, a feature collaborative mask decoder is used to obtain an annular region segmentation result; the positive points and the negative points are obtained by labeling a 2x2 grid-shaped prompt box on the medical image to be segmented, and the midpoints of the four outer edges of the prompt box are the positive points, and the center of the prompt box is the negative point; the processing steps of the annular prior prompt encoder include obtaining the coordinates of the positive points and the negative points, embedding the coordinates by a Gaussian matrix feature, obtaining coordinate point sparse embedding and label sparse embedding; after the coordinate point sparse embedding and the label sparse embedding are concatenated, the feature embedding is updated by an annular constraint module to obtain the annular prompt features; the annular constraint module comprises two linear layers connected in sequence, and an activation function is arranged between the two linear layers; the processing steps of the area connectivity enhanced image encoder include: using a plurality of layers of Transformer blocks and feature decomposition blocks to obtain initial features from the encoded features of the medical image to be segmented; the initial features are processed by a morphology processing module and a neck module in parallel to obtain morphological image embedding and original image embedding respectively; the morphological image embedding and the original image embedding are fused by using convolution and multi-head attention mechanism to obtain annular connected region features; the feature collaborative mask decoder comprises two layers of Transformer layers and one layer of deconvolution layers for feature decoding to obtain the annular region segmentation result.

2. A medical image annular region segmentation system, characterized by, The method comprises the following steps: a data acquisition module configured to obtain a medical image to be segmented and positive points and negative points labeled in the medical image; the positive points are multiple and uniformly distributed in the annular region of the medical image; the negative point is one and located at the center of the closed figure composed of the positive points; a prompt encoding module configured to obtain annular prompt features by an annular prior prompt encoder based on the positive points and the negative points; an image encoding module configured to obtain annular connected region features by an area connectivity enhanced image encoder after feature encoding of the medical image to be segmented; a segmentation module configured to obtain an annular region segmentation result by a feature collaborative mask decoder based on the annular prompt features and the annular connected region features; the positive points and the negative points are obtained by labeling a 2x2 grid-shaped prompt box on the medical image to be segmented, and the midpoints of the four outer edges of the prompt box are the positive points, and the center of the prompt box is the negative point; The processing steps of the ring-shaped prior prompt encoder include: obtaining the coordinates of positive points and negative points, obtaining coordinate point sparse embedding and label sparse embedding through Gaussian matrix feature embedding; after the coordinate point sparse embedding and the label sparse embedding are concatenated, the feature embedding is learned and updated through a ring-shaped constraint module to obtain the ring-shaped prompt feature; the ring-shaped constraint module includes two linear layers connected in sequence, and an activation function is arranged between the two linear layers; The processing steps of the region connectivity enhanced image encoder include: obtaining initial features by passing the features of the encoded medical image to be segmented through a plurality of layers of Transformer blocks and feature decomposition blocks; performing feature processing on the initial features in parallel through a morphological processing module and a neck module to obtain morphological image embedding and original image embedding, respectively; and using convolution and a multi-head attention mechanism to fuse the morphological image embedding and the original image embedding to obtain ring-shaped connected region features. The feature collaborative mask decoder includes two layers of Transformer layers and one layer of deconvolution layers for feature decoding to obtain ring-shaped region segmentation results.

3. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps in the medical image ring-shaped region segmentation method of claim 1.

4. A computer device comprising a computer readable storage medium, a processor, and a computer program stored on the computer readable storage medium and executable on the processor, characterized in that, The processor executes the program to implement the steps in the medical image ring-shaped region segmentation method of claim 1.

Images