Method and system for efficiently marking and segmenting lesion area in medical image
By applying a deep convolutional neural network architecture based on visual cues and annotation propagation algorithm in medical images, the dependence on a large amount of labeled data and the inefficiency of manual labeling in the prior art is solved, and efficient and low-cost lesion region segmentation is achieved.
Patent Information
- Application Number
- CN202510313333.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-17
- Publication Date
- 2025-06-17
AI Technical Summary
The prior art relies on a large amount of high-quality labeling data in the automated segmentation of lesion areas in medical images, and manual labeling is inefficient, making it difficult to meet the needs of clinical and scientific research.
A deep convolutional neural network architecture based on visual cues is adopted to generate visual cues through a small number of initial annotations, and the information is spread to the entire image and other unlabeled images using the labeling propagation algorithm, and the model performance is optimized through interactive correction.
It significantly reduces the demand for high-quality annotation data, improves the efficiency of annotation and segmentation, reduces the work burden of medical staff, and improves the accuracy and reliability of segmentation results.
Smart Images

Figure CN120163981A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical imaging, and in particular to a method and system for efficiently marking and segmenting lesion areas in medical images. Background Art
[0002] In today's era, deep neural networks are developing rapidly, and the automated segmentation of lesion areas in medical images has made significant progress. However, the existing fully supervised segmentation methods are heavily dependent on a large amount of high-quality labeled data, and the labeling of medical images is not an easy task. It usually requires experienced medical staff to invest a lot of time and energy to label the lesion areas one by one. Therefore, medical image datasets that can achieve high-quality labeling have become extremely scarce, which undoubtedly sets up many obstacles for the application and promotion of deep learning algorithms in the field of medical image segmentation.
[0003] At the same time, in the field of deep learning, visual cueing technology has gradually emerged, which has indeed opened up new development space for the labeling and segmentation of lesion areas in medical images. However, the traditional manual labeling method has many disadvantages. Doctors need to mark or segment each lesion one by one. The whole process is complicated and time-consuming, and it is highly dependent on the professional knowledge of medical workers. Especially when faced with complex images containing multiple lesions, the workload of manual labeling will increase exponentially, and the efficiency is extremely low. In clinical practice and scientific research, there is an urgent need for efficient labeling, and the existing labeling methods are difficult to meet such needs.
[0004] In recent years, Meta AI has proposed the Segment Anything Model (SAM), a general segmentation model based on visual cues. SAM can accept cues in various forms such as points, boxes, and text to achieve segmentation of arbitrary objects in natural images. Kirillov et al. introduced the model in detail in their paper. However, SAM is mainly trained on large-scale natural image datasets, has limited applicability to medical imaging data, and is not based on labeled masks as cues. Medical images have special grayscale distributions and subtle anatomical differences. If SAM is applied directly, it may result in poor segmentation performance. In addition, SAM is not optimized for annotation propagation and interactive learning in medical images, and lacks a mechanism to propagate annotation information within and across images, so it cannot effectively reduce the annotation burden on doctors.
[0005] Even though the University of Toronto proposed MedSAM as a general medical image segmentation base model, which showed certain performance in some tasks. However, MedSAM relies on a large amount of training data to improve the generalization ability of the model. For specific fine-grained lesion areas, especially tiny lesions, a large amount of labeled data may still be required to achieve satisfactory segmentation results.
[0006] Through the retrieval of patent literature, it is found that the invention patent with the publication number CN110993064A discloses a medical image annotation method and device for deep learning. The method includes the following steps: preprocessing the input image diagnosis report, electronic medical record information, and medical images to generate data containing medical images and extracted relevant diagnosis information; pre-annotating the medical images based on deep learning. Among them, the images are segmented based on image semantic segmentation technology to obtain the boundary ranges of each lesion area, pixel-level segmentation annotation of the images is achieved, and the disease types to which the images belong are classified and annotated based on image classification technology combined with image-related diagnosis information; the pre-annotated images and relevant diagnosis information are displayed through an interface to receive interactive instructions for doctors to fine-tune the pre-annotated image results and export the annotation results. This patent does not utilize reference image learning features and annotation propagation algorithms, making it difficult to achieve automatic detection and segmentation.
[0007] In summary, aiming at the problems of the above-mentioned existing technologies, researching a method and system for efficiently annotating and segmenting lesion areas in medical images has become a key task that needs to be solved urgently at present. Summary of the Invention
[0008] Aiming at the defects in the prior art, the purpose of the present invention is to provide a method and system for efficiently annotating and segmenting lesion areas in medical images.
[0009] According to a method for efficiently annotating and segmenting lesion areas in medical images provided by the present invention, it includes the following steps:
[0010] Step S1, collect a medical image dataset containing complex lesion areas, and divide the medical image dataset into a training set and a test set;
[0011] Step S2, select a reference image from the training set, perform initial annotation and generate visual cues to obtain a reference image with visual cues;
[0012] Step S3, based on the reference image with visual cues, construct a deep convolutional neural network architecture, and obtain a trained model through model training;
[0013] Step S4, use the trained model to perform predictive segmentation on the unannotated images in the test set to generate predictive segmentation results;
[0014] Step S5, review and correct the predicted segmentation results, and improve the model performance through interactive optimization.
[0015] Preferably, step S1 includes preprocessing the medical image dataset, and the preprocessing includes the following sub-steps:
[0016] Step S1.1, perform maximum-minimum normalization on the gray values of the images in the medical image dataset, map them to the interval [0, 1], and obtain the normalized medical image dataset;
[0017] Step S1.2, use data augmentation methods to diversify the images in the normalized medical image dataset, and obtain the preprocessed medical image dataset.
[0018] Preferably, in step S1.2, the data augmentation methods include random Gamma transformation, random brightness transformation, random contrast transformation, and random flipping.
[0019] Preferably, in step S1, the preprocessed medical image dataset is divided into a training set and a test set, and is divided by means of 4-fold cross-validation, and the pictures in the training set and the pictures in the test set are mutually exclusive.
[0020] Preferably, step S2 includes the following sub-steps:
[0021] Step S2.1, an experienced doctor makes initial annotations on specific lesion regions and background regions in the reference image;
[0022] Step S2.2, convert the initial annotations into visual cues that the model can process, and obtain the reference image with visual cues.
[0023] Preferably, in step S2.1, the initial annotations are in the form of drawing bounding boxes or outlining contours, and only representative lesion regions and background regions are initially annotated to control the number of annotations;
[0024] Step S2.2 is implemented by generating corresponding region masks, and the visual cues include the position, shape, and size of the lesion region.
[0025] Preferably, step S3 includes the following sub-steps:
[0026] Step S3.1, based on the reference image with visual cues, construct a deep convolutional neural network architecture, including an image encoder, a visual cue encoder, a prediction mask decoder, and a pre-trained deep feature extractor;
[0027] Step S3.2, the image encoder extracts features from the reference image with visual cues, generates an image encoder feature map, and the size of the image encoder feature map is 256x1024;
[0028] Step S3.3: The visual prompt encoder encodes the received N lesion marking maps to generate prompt features, and fuses them with the feature maps of the image encoder through an attention mechanism or feature concatenation to output visual prompt encoder feature maps, with the size of the visual prompt encoder feature maps being Nx1024;
[0029] Step S3.4: Concatenate the trainable Query Tokens with the visual prompt encoder feature maps to generate masked feature maps, with the size of the masked feature maps being 256x1024;
[0030] Step S3.5: In the prediction mask decoder, perform feature extraction and fusion on the masked feature maps and the image encoder feature maps through the Cross - Attention Block and Self - Attention Block to generate prediction feature maps, with the size of the prediction feature maps being 256x1024;
[0031] Step S3.6: Process the prediction feature maps through a transposed convolutional layer to generate refined feature maps, and use the refined feature maps to generate refined segmentation masks, and perform weakly supervised training through the binary cross - entropy loss L bce and the Dice loss L dice to obtain the trained model;
[0032] The loss function is defined as follows:
[0033]
[0034] L total = αL bce + βL dice
[0035] where the weight α of the binary cross - entropy loss and the weight β of the Dice loss are both 0.5.
[0036] Preferably, step S4 includes the following sub - steps:
[0037] Step S4.1: For the refined feature maps, generate lesion masks through a set threshold; and calculate the image depth feature maps using a pre - trained deep feature extractor, and screen similar regions through a cosine similarity matrix to achieve label propagation within and across images;
[0038] Step S4.2: Use the trained model to perform prediction segmentation on the unlabeled images in the test set, and generate prediction segmentation results through screening by the similarity matrix.
[0039] Preferably, step S5 includes the following sub - steps:
[0040] Step S5.1: Present the predicted segmentation result to the doctor, who marks and corrects the omitted or misdetected areas to generate an updated visual cue.
[0041] Step S5.2: Use the updated visual cue as input and re - execute Steps S3 to S4 to generate an updated predicted segmentation result. Through multiple interactions, gradually optimize the model performance.
[0042] The present invention also provides a system for efficiently annotating and segmenting lesion areas in medical images, including:
[0043] Module M1: Collect a medical image dataset containing complex lesion areas and divide the medical image dataset into a training set and a test set.
[0044] Module M2: Select a reference image from the training set, perform initial annotation and generate a visual cue to obtain a reference image with a visual cue.
[0045] Module M3: Based on the reference image with a visual cue, construct a deep convolutional neural network architecture and obtain a trained model through model training.
[0046] Module M4: Use the trained model to perform predictive segmentation on the unannotated images in the test set to generate a predicted segmentation result.
[0047] Module M5: Review and correct the predicted segmentation result and improve the model performance through interactive optimization.
[0048] Compared with the prior art, the present invention has the following beneficial effects:
[0049] 1. The present invention only requires doctors to perform initial annotation on a small number of lesion areas, solves the problem that existing fully - supervised segmentation methods rely heavily on a large amount of high - quality annotated data, significantly reduces the demand for high - quality annotated data, and alleviates the work burden of medical staff.
[0050] 2. The present invention introduces an annotation propagation technology based on visual cues, and uses a deep neural network to propagate a small amount of annotation information to the entire image and other unannotated images, solves the problem that it is time - consuming, laborious and costly to manually annotate lesion areas one by one, and greatly improves the annotation efficiency and segmentation efficiency.
[0051] 3. The present invention introduces an interactive correction and feedback mechanism, enabling doctors to correct and optimize the segmentation result of the model, solves the problem that the initial segmentation result of the model may be inaccurate or have errors, and improves the accuracy and reliability of the segmentation result.
[0052] 4. The present invention constructs an innovative deep learning architecture that combines visual prompting, depth feature extraction, and annotation propagation, solving the problem that existing methods are difficult to fully utilize a small amount of annotated data to achieve high-precision lesion segmentation, and providing an efficient, low-cost, and easy-to-popularize solution for the automatic segmentation of lesion regions in medical images.
[0053] 5. The operation steps of the present invention include: First, a doctor performs initial annotation to generate visual prompting, then combines a deep convolutional neural network for feature extraction and segmentation mask prediction, then optimizes the segmentation result through interactive correction, and propagates the annotation to other unannotated images. This method reduces the dependence on high-quality annotated data, improves the annotation efficiency and segmentation accuracy, and provides an efficient and low-cost solution for medical image lesion segmentation. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] By reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings, other features, objects, and advantages of the present invention will become more apparent:
[0055] Figure 1 It is a flowchart of a method for efficiently annotating and segmenting lesion regions in medical images in an embodiment of the present invention;
[0056] Figure 2 It is a weakly supervised training model and label propagation strategy in an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION
[0057] The present invention will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that those of ordinary skill in the art can make several changes and improvements without departing from the concept of the present invention. These all belong to the protection scope of the present invention.
[0058] The present invention proposes a visual cue-based label propagation and interactive learning method, which is applied to the efficient annotation and segmentation of lesion regions in medical images. The invention constructs a method and system specifically for the efficient annotation and segmentation of lesion regions in medical images. Compared with the prior art, this method reduces the annotation burden on doctors. Doctors only need to annotate a small number of lesion regions (positive samples) and background regions (negative samples) as visual cues (Visual Prompting). Through the label propagation algorithm based on neural network proposed by us, these limited annotation information is spread to the remaining lesion regions in the whole image. At the same time, the system fully considers the professional judgment value of doctors and allows doctors to perform interactive correction and feedback operations on the results generated by the algorithm. Through multiple rounds of iterative optimization, this algorithm can not only improve the accuracy of annotation, but also extend its application scope to other unannotated images in the whole dataset, thus achieving the goal of efficient and low-cost lesion segmentation.
[0059] The core idea of the method of the present invention is to transform the segmentation task of lesion regions into an open-set object detection and segmentation task integrating visual cues. Through the visual cue information set on the reference image, such as lesion region marker masks, the model can learn and extract the unique feature information of the lesion regions, and then detect and segment all lesion regions with similar features in the target image, thus providing a new, highly efficient and accurate solution for the annotation and segmentation of lesion regions in medical images.
[0060] Example 1:
[0061] Figure 1 It is a flowchart of a method for efficiently annotating and segmenting lesion regions in medical images in an embodiment of the present invention.
[0062] As Figure 1 shown, this embodiment provides a method for efficiently annotating and segmenting lesion regions in medical images, including the following steps:
[0063] Step S1, collect a medical image dataset containing complex lesion regions, and divide the medical image dataset into a training set and a test set.
[0064] In this embodiment, the medical image dataset contains multiple tiny lesion regions, such as a fundus image lesion dataset.
[0065] Specifically, step S1 includes preprocessing the medical image dataset, and the preprocessing includes the following sub-steps:
[0066] Step S1.1, perform min-max normalization on the grayscale values of the images in the medical image dataset, map them to the interval [0, 1], and obtain the normalized medical image dataset; this step is used to eliminate the grayscale differences caused by different devices and imaging conditions.
[0067] Step S1.2, use data augmentation methods to diversify the images in the normalized medical image dataset, and obtain the preprocessed medical image dataset.
[0068] Specifically, the data augmentation methods include random Gamma transformation, random brightness transformation, random contrast transformation, and random flipping, etc., which are used to improve the robustness of the model.
[0069] Furthermore, divide the preprocessed medical image dataset into a training set and a test set, and use 4-fold cross-validation for the division, and the pictures in the training set are mutually exclusive with the pictures in the test set. To ensure the objectivity and accuracy of the test results, that is, any picture in the test set has not been used during the training process.
[0070] Step S2, select a reference image from the training set, perform initial annotation and generate visual cues to obtain a reference image with visual cues.
[0071] Specifically, Step S2 includes the following sub-steps:
[0072] Step S2.1, an experienced doctor performs initial annotation on specific lesion regions (positive samples) and background regions (negative samples) in the reference image.
[0073] In this embodiment, the initial annotation is in the form of drawing a bounding box or outlining the contour, and only representative lesion regions and background regions are initially annotated to control the number of annotations and thus reduce the doctor's workload.
[0074] Step S2.2, convert the initial annotation into visual cues that the model can process to obtain a reference image with visual cues. Specifically, it is achieved by generating corresponding region masks. The visual cues include prior information such as the location, shape, and size of the lesion region, which are used to guide the model to focus on specific target regions.
[0075] Step S3, based on the reference image with visual cues, construct a deep convolutional neural network architecture, and obtain a trained model through model training;
[0076] Figure 2 This is the weakly supervised training model and label propagation strategy in the embodiments of the present invention.
[0077] As Figure 2 shown, Step S3 includes the following sub-steps:
[0078] Step S3.1: Based on the reference image with visual cues, construct a deep convolutional neural network architecture, including an image encoder, a visual cue encoder, a prediction mask decoder, and a pre-trained deep feature extractor.
[0079] Specifically, construct an image encoder based on the industry-advanced backbone network Vision Transformer. Then, construct a visual cue encoder based on convolutional layers and linear layers. Next, construct a prediction mask decoder based on Vision Transformer. Finally, construct a pre-trained deep feature extractor based on Vision Transformer. By introducing trainable Query Tokens, candidate regions are generated. Then, through calculating the similarity between the prediction mask generated by the decoder and the user-annotated region, weak supervision learning is performed.
[0080] Step S3.2: The image encoder extracts features from the reference image with visual cues to generate an image encoder feature map, and the size of the image encoder feature map is 256x1024.
[0081] In this embodiment, a pre-trained deep neural network (Vision Transformer) is used to extract multi-scale and high-level visual features of medical images. The extracted features include information such as spatial resolution and number of channels, and are used to characterize the texture, edge, and morphological features of the lesion region.
[0082] Step S3.3: The visual cue encoder encodes the received N lesion marking maps to generate cue features, and fuses them with the image encoder feature map through an attention mechanism or feature concatenation to output a visual cue encoder feature map, and the size of the visual cue encoder feature map is Nx1024.
[0083] Specifically, assume that a doctor provides N lesion markings in the reference image with visual cues. The multiple lesion markings provided by the doctor are separated into N lesion marking maps according to the edges, and these N lesion marking maps are used as the input of the visual cue encoder to generate encoded cue features. Then, an attention mechanism or a feature concatenation-based feature fusion method is used to fuse the cue features with the features of the reference image with visual cues, so that the model can better combine the spatial position information of the lesion region, make full use of the prior knowledge of the anatomical structure of organs and lesions in medical images, and strengthen the attention degree to the lesion region.
[0084] Step S3.4: Concatenate the trainable Query Tokens (with a dimension of (256 - N)x1024) and the visual cue encoder feature map along the first dimension to generate a mask feature map, and the size of the mask feature map is 256x1024.
[0085] Step S3.5, in the prediction mask decoder, the mask feature map and the image encoder feature map are subjected to feature extraction and fusion through the Cross-Attention Block and the Self-Attention Block to generate a prediction feature map with a size of 256x1024.
[0086] Specifically, the first block is the Cross-Attention Block, and the remaining blocks are Self-Attention Blocks.
[0087] Step S3.6, the prediction feature map is processed through a transposed convolution layer to generate a refined feature map, and the refined feature map is used to generate a refined segmentation mask. Through the binary cross-entropy loss L bce and the Dice loss L dice perform weakly supervised training to obtain the trained model;
[0088] Specifically, the prediction feature map is dimension-changed to 256x32x32, and then the first N-dimensional data are respectively processed through three transposed convolution layers with a convolution kernel of 4 and a stride of 2 to generate a refined feature map with a dimension of 256x256x256. Then, through the sigmoid layer, a refined segmentation mask is generated. N represents the number of data samples. Then, the generated refined segmentation masks of the first Nx256x256 are compared and analyzed with the N lesion marking maps provided by the doctor. Among them, each element value p n in the refined segmentation mask is the predicted probability value, and the element value y n at the corresponding position in the lesion marking map is the true label value (taking values of 0 or 1).
[0089] Furthermore, the relevant loss functions are defined as follows:
[0090] Binary cross-entropy loss:
[0091]
[0092] Dice loss:
[0093]
[0094] Total loss:
[0095] L total = αL bce + βL dice
[0096] where the weight α of the binary cross-entropy loss and the weight β of the Dice loss are both 0.5.
[0097] Step S4: Use the trained model to perform predictive segmentation on the unlabeled images in the test set and generate a predictive segmentation result.
[0098] Specifically, step S4 includes the following sub-steps:
[0099] Step S4.1: For the fine feature map, generate a lesion mask through a set threshold; and use a pre-trained deep feature extractor to calculate the image depth feature map, and screen similar regions through a cosine similarity matrix to achieve label propagation within and across images.
[0100] Specifically, for the fine feature map (with a size of 256x256x256), generate a lesion mask through a set threshold, and the pixel values in the mask are 0 or 1. Subsequently, extract the region where the lesion mask is located from the input medical image and input it into the pre-trained deep feature extractor to obtain an image depth feature map with a dimension of 256x1024. By calculating the inner product of the image depth feature map and its transpose, generate a similarity matrix based on cosine similarity. Then, use a set threshold (0.5 in this embodiment) to screen out the feature map numbers in the similarity matrix whose similarity with the first N dimensions is greater than the threshold. Use the selected numbers to screen the predicted lesion mask to achieve label propagation within and across images. Specifically, calculate the deep feature similarity (using the cosine similarity calculation method) between the image regions represented by the Query Tokens and the Tokens marked by the user through the pre-trained deep feature extractor, and combine it with a set threshold (0.5) to complete label propagation within and across the labeled images.
[0101] Step S4.2: Use the trained model to perform predictive segmentation on the unlabeled images in the test set, and generate a predictive segmentation result through screening by the similarity matrix.
[0102] Specifically, the trained model performs predictive segmentation on the unlabeled images in the test set based on the lesion region features learned on the reference images, and generates a predictive segmentation result through screening by the similarity matrix, thereby realizing automatic detection and segmentation of lesion regions of other patients or different images.
[0103] Step S5: Review and correct the predictive segmentation result, and improve the model performance through interactive optimization.
[0104] Specifically, step S5 includes the following sub-steps:
[0105] Step S5.1: Present the predictive segmentation result to the doctor in a visual way, and the doctor marks and corrects the missed or misdetected regions to generate an updated visual prompt.
[0106] Specifically, the doctor reviews the predictive segmentation result to determine whether the following situations exist:
[0107] 1. Lesion regions that actually exist but were not detected by the model (missed detections);
[0108] 2. The model wrongly determines non-lesion regions as lesion regions (false detections).
[0109] If the above situations exist, the doctor annotates and corrects the missed or misdetected regions and generates updated visual cues (positive or negative samples). In this embodiment, the predicted segmentation result will be superimposed on the preprocessed medical image so that the doctor can intuitively and clearly view the annotation and segmentation of the lesion regions predicted by the model.
[0110] Step S5.2: Use the updated visual cue as the input, and re-execute steps S3 to S4 to generate an updated predicted segmentation result. Through multiple interactions, gradually optimize the model performance.
[0111] Specifically, use the new visual cue added in step S5.1 as an additional visual cue. After updating the cue features, re-execute model training and prediction segmentation. Through multiple interactions, gradually optimize the model performance, and finally extract a predicted segmentation result that more conforms to the actual lesion regions.
[0112] During the optimization process, perform a new round of weakly supervised training on the model based on binary cross-entropy loss and Dice loss. The total loss formula is:
[0113] L′ total = αL′ bce + βL′ dice
[0114] where α and β are both 0.5. Through repeated interactions and optimizations, the model can achieve more accurate segmentation and label propagation of lesion regions in medical images.
[0115] Embodiment 2:
[0116] The present invention also provides a system for efficiently annotating and segmenting lesion regions in medical images. The system for efficiently annotating and segmenting lesion regions in medical images can be implemented by executing the process steps of the method for efficiently annotating and segmenting lesion regions in medical images. That is, those skilled in the art can understand the method for efficiently annotating and segmenting lesion regions in medical images as a preferred implementation manner of the system for efficiently annotating and segmenting lesion regions in medical images.
[0117] The system for efficiently annotating and segmenting lesion regions in medical images includes:
[0118] Module M1, which collects a medical image dataset containing complex lesion regions and divides the medical image dataset into a training set and a test set;
[0119] Module M2 selects reference images from the training set, performs initial annotation and generates visual cues to obtain reference images with visual cues;
[0120] Module M3 constructs a deep convolutional neural network architecture based on the reference images with visual cues and obtains a trained model through model training;
[0121] Module M4 uses the trained model to perform predictive segmentation on the unannotated images in the test set and generates predictive segmentation results;
[0122] Module M5 reviews and corrects the predictive segmentation results and improves the model performance through interactive optimization.
[0123] Those skilled in the art know that in addition to implementing the system and its various devices, modules, and units provided by the present invention in the form of pure computer-readable program code, the method steps can be logically programmed to enable the system and its various devices, modules, and units provided by the present invention to be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers to achieve the same functions. Therefore, the system and its various devices, modules, and units provided by the present invention can be regarded as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be regarded as the structure within the hardware component; the devices, modules, and units for implementing various functions can also be regarded as either software modules for implementing the method or the structure within the hardware component.
[0124] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. Without conflict, the embodiments of the present application and the features in the embodiments can be combined arbitrarily with each other.
Claims
1. A method for efficiently labeling and segmenting lesion areas in medical images, characterized in that: The following steps are involved: Step S1, collecting a medical image data set containing a complex lesion area, and dividing the medical image data set into a training set and a test set; Step S2, selecting a reference image from the training set, performing initial annotation and generating visual cues, and obtaining a reference image with visual cues; Step S3, constructing a deep convolutional neural network architecture based on the reference image with visual cues, and obtaining a trained model through model training; Step S4, using the trained model to perform predictive segmentation on the unlabeled images in the test set to generate a predictive segmentation result; Step S5, reviewing and correcting the predicted segmentation results, and improving model performance through interactive optimization.
2. A method for efficiently marking and segmenting lesion areas in medical images according to claim 1, characterized in that: The step S1 includes preprocessing the medical image data set, and the preprocessing includes the following sub-steps: Step S1.1, performing maximum and minimum normalization on the grayscale values of the images in the medical image dataset and mapping them to the interval [0, 1] to obtain a normalized medical image dataset; Step S1.2, using a data enhancement method to perform diversified processing on the images in the normalized medical image dataset to obtain a preprocessed medical image dataset.
3. A method for efficiently marking and segmenting lesion areas in medical images according to claim 2, characterized in that: In step S1.2, the data enhancement method includes random Gamma transformation, random brightness transformation, random contrast transformation and random flipping.
4. A method for efficiently marking and segmenting lesion areas in medical images according to claim 3, characterized in that: In the step S1, the preprocessed medical image data set is divided into a training set and a test set, and the division is performed by a 4-fold cross-validation method, and the images in the training set and the images in the test set are mutually exclusive.
5. A method for efficiently marking and segmenting lesion areas in medical images according to claim 1, characterized in that: The step S2 includes the following sub-steps: Step S2.1, an experienced doctor initially marks a specific lesion area and a background area in the reference image; Step S2.2, converting the initial annotation into visual cues that can be processed by the model to obtain a reference image with visual cues.
6. A method for efficiently marking and segmenting lesion areas in medical images according to claim 5, characterized in that: In step S2.1, the initial annotation is in the form of drawing a bounding box or outlining an outline, and only representative lesion areas and background areas are initially annotated to control the number of annotations; The step S2.2 is implemented by generating a corresponding area mask, and the visual cue includes the position, shape and size of the lesion area.
7. A method for efficiently marking and segmenting lesion areas in medical images according to claim 1, characterized in that: The step S3 includes the following sub-steps: Step S3.1, based on the reference image with visual cues, constructing a deep convolutional neural network architecture, including an image encoder, a visual cue encoder, a prediction mask decoder and a pre-trained deep feature extractor; Step S3.2, the image encoder extracts features from the reference image with visual cues to generate an image encoder feature map, wherein the size of the image encoder feature map is 256x1024; Step S3.3, the visual cue encoder encodes the received N lesion marking images to generate cue features, and fuses them with the image encoder feature map through an attention mechanism or feature splicing to output a visual cue encoder feature map, wherein the size of the visual cue encoder feature map is Nx1024; Step S3.4, concatenating the trainable Query Tokens with the visual cue encoder feature map to generate a mask feature map, wherein the size of the mask feature map is 256x1024; Step S3.5, in the prediction mask decoder, the mask feature map and the image encoder feature map are subjected to feature extraction and fusion through the Cross-Attention Block and the Self-Attention Block to generate a prediction feature map, the size of which is 256x1024; Step S3.6, the predicted feature map is processed through a deconvolution layer to generate a fine feature map, and the fine feature map is used to generate a fine segmentation mask, and the binary cross entropy loss L is used. bce With Dice loss L dice Perform weakly supervised training to obtain the trained model; The loss function is defined as follows: L total =αL bce +βL dice Among them, the weight α of binary cross entropy loss and the weight β of Dice loss are both 0.
5.
8. A method for efficiently marking and segmenting lesion areas in medical images according to claim 7, characterized in that: The step S4 includes the following sub-steps: Step S4.1, for the fine feature map, generate a lesion mask through a set threshold; and use the pre-trained deep feature extractor to calculate the image deep feature map, and use the cosine similarity matrix to screen similar areas to achieve label propagation within and across images; Step S4.2, using the trained model, predictive segmentation is performed on the unlabeled images in the test set, and the predicted segmentation results are generated through similarity matrix screening.
9. A method for efficiently marking and segmenting lesion areas in medical images according to claim 1, characterized in that: The step S5 comprises the following sub-steps: Step S5.1, presenting the predicted segmentation results to the doctor, who then annotates and corrects the missed or misdetected areas and generates updated visual cues; Step S5.2, taking the updated visual cue as input, re-execute steps S3 to S4 to generate updated predicted segmentation results, and gradually optimize the model performance through multiple interactions.
10. A system for efficiently labeling and segmenting lesion areas in medical images, characterized in that: include: Module M1, collecting a medical image dataset containing a complex lesion area, and dividing the medical image dataset into a training set and a test set; Module M2, selecting a reference image from the training set, performing initial annotation and generating visual cues, and obtaining a reference image with visual cues; Module M3, constructing a deep convolutional neural network architecture based on the reference image with visual cues, and obtaining a trained model through model training; Module M4, using the trained model to perform predictive segmentation on the unlabeled images in the test set to generate a predictive segmentation result; Module M5 reviews and corrects the predicted segmentation results and improves model performance through interactive optimization.
Citation Information
Patent Citations
Medical image labeling method and device for deep learning
CN110993064A
Cited By
Medical ultrasound image data set construction method and system and storage medium
CN120766056A
A medical ultrasound image dataset construction method, system, and storage medium
CN120766056B
Rectal tumor image segmentation method and device
CN121329996A
Rectal tumor image segmentation method and device
CN121329996B
