A method and apparatus for segmenting medical images

Abstract

The present disclosure provides a method and apparatus for medical image segmentation. By obtaining multiple rough annotation information in medical images, the present disclosure realizes the preliminary annotation of images and generates corresponding rough annotation data, effectively reducing the annotation cost and improving the efficiency of medical image research at the same time. Based on multiple trained high-precision segmentation models, a multi-expert system is constructed to deeply analyze the rough annotation information in medical images and generate fine annotation information. By voting on each pixel point, the pixel points of interest are determined, and then the regions of interest in medical images are identified. The progressive annotation information processing method not only realizes the high-precision segmentation of medical images, but also significantly improves the accuracy of annotation, avoiding misjudgment that may occur in a single high-precision segmentation model, thereby enhancing the robustness of region segmentation.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and more particularly, to a method, apparatus, medium, and electronic device for segmenting medical images. Background Art

[0002] In the field of medical image processing, accurately segmenting medical images is crucial for diagnosis and treatment planning.

[0003] Medical images usually have high dimensionality and diversity, requiring a large number of accurate pixel-level annotations, which not only incur high time costs but also require a great deal of expert knowledge. Experts must carefully annotate regions of interest, such as tumors, organs, or other pathological structures. This high-precision annotation work is very time-consuming and greatly increases the research time cost.

[0004] Therefore, the present disclosure provides a method for segmenting medical images to solve one of the above technical problems. Summary of the Invention

[0005] The purpose of the present disclosure is to provide a method, apparatus, medium, and electronic device for segmenting medical images, which can solve at least one of the above-mentioned technical problems. The specific solutions are as follows:

[0006] According to a specific embodiment of the present disclosure, in a first aspect, the present disclosure provides a method for segmenting medical images, including:

[0007] Obtain a plurality of rough annotation information on a medical image;

[0008] Apply the medical images including the plurality of rough annotation information to a plurality of trained high-precision segmentation models respectively, and obtain a plurality of fine annotation information of each of the plurality of high-precision segmentation models on the medical image;

[0009] Vote on each pixel point on the medical image based on the plurality of fine annotation information of each of the plurality of high-precision segmentation models on the medical image, and determine a plurality of pixel points of interest on the medical image;

[0010] Determine the region of interest on the medical image based on the plurality of pixel points of interest on the medical image.

[0011] Optionally, the step of voting on each pixel point on the medical image based on the plurality of fine annotation information of each of the plurality of high-precision segmentation models on the medical image to determine a plurality of pixel points of interest on the medical image includes:

[0012] Statistically count the number of annotations of each pixel point on the medical image in all the fine annotation information of the multiple high-precision segmentation models to represent the number of votes, and determine the number of models of the multiple high-precision segmentation models;

[0013] When the ratio of the number of votes of any pixel point on the medical image to the number of models is greater than a preset ratio, or the number of votes of any pixel point is greater than the product value of the number of models and the preset ratio, determine that the any pixel point is an interested pixel point.

[0014] Optionally, the preset ratio includes 0.5 to 0.7.

[0015] Optionally, determining the region of interest on the medical image based on multiple interested pixel points on the medical image includes:

[0016] Determine multiple hollow regions on the medical image based on multiple interested pixel points on the medical image, where the boundary of each hollow region is constructed by multiple interested pixel points;

[0017] Add at least one interested pixel point in each hollow region on the medical image to the queue corresponding to the hollow region;

[0018] Extract interested pixel points from the queue of each hollow region respectively;

[0019] Determine the neighboring pixel points adjacent to the extracted interested pixel points in each hollow region;

[0020] When any neighboring pixel point in each hollow region is a non-interested pixel point, determine that the any neighboring pixel point is an interested pixel point in the corresponding hollow region, add the any neighboring pixel point in the corresponding hollow region to the queue corresponding to the hollow region, and trigger the execution of the step of extracting interested pixel points from the queue of each hollow region respectively until the queues of all hollow regions are empty;

[0021] Determine the region of interest on the medical image based on all the interested pixel points in each hollow region.

[0022] Optionally, adding at least one interested pixel point in each hollow region on the medical image to the queue corresponding to the hollow region includes:

[0023] Add the multiple interested pixel points on the boundary of each hollow region on the medical image to the queue corresponding to the hollow region.

[0024] Optionally, the multiple high-precision segmentation models include: VT-UNet segmentation model, Vnet segmentation model, and UNETR segmentation model.

[0025] Optionally, the obtaining of multiple rough annotation information on the medical image includes:

[0026] Applying the medical image to a pre-trained low-precision segmentation model to obtain multiple rough annotation information on the medical image.

[0027] Optionally, the low-precision segmentation model includes a pre-trained MedSAM segmentation model.

[0028] Optionally, the multi-task loss function of the MedSAM segmentation model includes a cross-entropy loss function and a Dice loss function.

[0029] According to a specific embodiment of the present disclosure, in a second aspect, the present disclosure provides a segmentation device for medical images, including:

[0030] An acquisition unit for acquiring multiple rough annotation information on a medical image;

[0031] An application unit for applying the medical image including the multiple rough annotation information to multiple pre-trained high-precision segmentation models respectively to obtain multiple fine annotation information of each of the multiple high-precision segmentation models on the medical image;

[0032] A pixel determination unit for voting on each pixel point on the medical image based on the multiple fine annotation information of each of the multiple high-precision segmentation models on the medical image to determine multiple interesting pixel points on the medical image;

[0033] A region determination unit for determining an interesting region on the medical image based on the multiple interesting pixel points on the medical image.

[0034] Optionally, the voting on each pixel point on the medical image based on the multiple fine annotation information of each of the multiple high-precision segmentation models to determine multiple interesting pixel points on the medical image includes:

[0035] Counting the number of annotations in all the fine annotation information of the multiple high-precision segmentation models for each pixel point on the medical image as the voting number, and determining the number of the multiple high-precision segmentation models;

[0036] When the ratio of the voting number of any pixel point on the medical image to the number of models is greater than a preset ratio, or the voting number of the any pixel point is greater than the product value of the number of models and the preset ratio, determining the any pixel point as an interesting pixel point.

[0037] Optionally, the preset ratio includes 0.5 to 0.7.

[0038] Optionally, determining the region of interest on the medical image based on multiple pixels of interest on the medical image includes:

[0039] Determining multiple hollow regions on the medical image based on multiple pixels of interest on the medical image, where the boundary of each hollow region is constructed by multiple pixels of interest;

[0040] Adding at least one pixel of interest in each hollow region on the medical image to the queue of the corresponding hollow region;

[0041] Extracting pixels of interest from the queue of each hollow region respectively;

[0042] Determining the neighborhood pixels adjacent to the extracted pixels of interest in each hollow region;

[0043] When any neighborhood pixel in each hollow region is a non-pixel of interest, determining the any neighborhood pixel as a pixel of interest in the corresponding hollow region, adding the any neighborhood pixel in the corresponding hollow region to the queue of the corresponding hollow region, and triggering the execution of the step of extracting pixels of interest from the queue of each hollow region respectively until the queue of each hollow region is empty;

[0044] Determining the region of interest on the medical image based on all pixels of interest in each hollow region.

[0045] Optionally, adding at least one pixel of interest in each hollow region on the medical image to the queue of the corresponding hollow region includes:

[0046] Adding multiple pixels of interest on the boundary of each hollow region on the medical image to the queue of the corresponding hollow region.

[0047] Optionally, the multiple high-precision segmentation models include: VT-UNet segmentation model, Vnet segmentation model, and UNETR segmentation model.

[0048] Optionally, obtaining multiple rough annotation information on the medical image includes:

[0049] Applying the medical image to a trained low-precision segmentation model to obtain multiple rough annotation information on the medical image.

[0050] Optionally, the low-precision segmentation model includes a trained MedSAM segmentation model.

[0051] Optionally, the multi-task loss function of the MedSAM segmentation model includes a cross-entropy loss function and a Dice loss function.

[0052] According to a specific embodiment of the present disclosure, in a third aspect, the present disclosure provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the segmentation method of the medical image described in any one of the above is implemented.

[0053] According to a specific embodiment of the present disclosure, in a fourth aspect, the present disclosure provides an electronic device, including: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the segmentation method of the medical image described in any one of the above.

[0054] The above solution of the embodiment of the present disclosure has at least the following beneficial effects compared with the prior art:

[0055] The present disclosure provides a segmentation method, device, medium and electronic device for medical images. The present disclosure obtains a plurality of rough annotation information on a medical image, realizes rough annotation on the medical image, generates rough annotation information, reduces the cost of annotating rough annotation information, and improves the research efficiency of medical images. Through a plurality of trained high-precision segmentation models, a multi-expert system is formed. The high-precision segmentation models analyze the rough annotation information of the medical image, vote on each pixel point on the medical image through the fine annotation information obtained by the analysis, determine the pixels of interest, and then determine the region of interest on the medical image through the pixels of interest. By means of progressive annotation information, high-precision segmentation of medical images is realized, the accuracy of annotation is improved, the judgment error of a single high-precision segmentation model is avoided, and the robustness of region segmentation is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 Shows a flowchart of a segmentation method of a medical image according to an embodiment of the present disclosure;

[0057] Figure 2 Shows a unit block diagram of a segmentation device of a medical image according to an embodiment of the present disclosure;

[0058] Figure 3 Shows a schematic connection structure diagram of an electronic device provided according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] In order to make the purpose, technical solution and advantages of the present disclosure clearer, the present disclosure will be further described in detail below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present disclosure.

[0060] The terms used in the embodiments of the present disclosure are for the purpose of describing specific embodiments only and are not intended to limit the present disclosure. The singular forms "a", "the", and "said" used in the embodiments of the present disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. "Plural" generally includes at least two.

[0061] It should be understood that the term "and / or" used herein is merely a description of the associated relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the preceding and following associated objects.

[0062] It should be understood that although terms such as first, second, and third may be used in the embodiments of the present disclosure for description, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, without departing from the scope of the embodiments of the present disclosure, the first can also be referred to as the second, and similarly, the second can also be referred to as the first.

[0063] Depending on the context, the words "if" and "when" as used herein can be interpreted as "when" or "while" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if determined" or "if detecting (stated condition or event)" can be interpreted as "when determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)".

[0064] It should also be noted that the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a commodity or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such commodity or device. Without further limitation, the element defined by the statement "including one..." does not exclude the existence of another identical element in the commodity or device including the said element.

[0065] It should be specifically noted that symbols and / or numbers existing in the specification, if not marked in the figure description, are not figure labels.

[0066] The optional embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0067] Embodiment 1

[0068] For the embodiments provided by the present disclosure, that is, embodiments of a method for segmenting medical images.

[0069] The following combinationFigure 1 A detailed description of the embodiments of the present disclosure will be given.

[0070] Step S101: Obtain multiple rough annotation information on the medical image.

[0071] The annotation information of the medical image is used for medical image analysis and diagnosis, helping doctors save time, make more informed decisions, and improve patient treatment outcomes.

[0072] The rough annotation information refers to low-precision annotation information. The rough annotation information serves as the basic information for fine annotation. For example, the rough annotation information can be annotated manually.

[0073] In some specific embodiments, the obtaining of multiple rough annotation information on the medical image includes the following steps:

[0074] Step S101a: Apply the medical image to a trained low-precision segmentation model to obtain multiple rough annotation information on the medical image.

[0075] In this specific embodiment, a small amount of high-precision annotation information is manually annotated on the medical image sample, and the low-precision segmentation model is trained using the medical image sample to achieve the preliminary automatic recognition of the lesion area on the medical image. Multiple rough annotation information on the medical image can be obtained through the trained low-precision segmentation model, thereby realizing the automatic generation of rough annotation information on the medical image, reducing the cost of manually annotating the rough annotation information, and improving the research efficiency of the medical image.

[0076] In some other specific embodiments, the low-precision segmentation model includes the MedSAM segmentation model.

[0077] The MedSAM segmentation model (full English name: Medical Segment Anything Model) is a deep learning-based model that particularly utilizes the powerful functions of Convolutional Neural Networks (abbreviated as CNN). The MedSAM segmentation model integrates multiple existing high-quality pre-trained models, such as the U-Net model and the Faster R-CNN model, which have shown excellent performance in medical image segmentation tasks.

[0078] The MedSAM segmentation model framework is based on modular design, providing the ability to fine-tune the model and train custom models, enabling users to easily add new network layers or loss functions and optimize according to their own needs to adapt to the evolving AI research. The MedSAM segmentation model also supports GPU acceleration, greatly improving the computing efficiency and making it possible to process high-resolution medical images. The MedSAM segmentation model has performed excellently in the recognition of various biological tissues and structures and has a powerful basic segmentation ability.

[0079] In this specific embodiment, the MedSAM segmentation model is trained using medical image samples in a large-scale medical image dataset to obtain a trained MedSAM segmentation model. Then, the trained MedSAM segmentation model is used to perform cell segmentation on medical images.

[0080] First, initialize the MedSAM segmentation model, that is, load the pre-trained MedSAM segmentation model. Then, fine-tune the parameters, and use a relatively low learning rate (such as 0.2 - 0.4) to fine-tune the deep network structure of the model to ensure that the model retains its original useful features and learning ability while adapting to tumor-specific images.

[0081] In some other specific embodiments, the multi-task loss function of the MedSAM segmentation model includes a cross-entropy loss function and a Dice loss function, which are used to optimize the performance of the MedSAM segmentation model in terms of pixel-level classification accuracy and segmentation region coherence.

[0082] For example, the specific formula of the multi-task loss function is:

[0083] L = λL CE +(1 - λ)L Dice ;

[0084] where L represents the multi-task loss function, L CE represents the cross-entropy loss function, L Dice represents the Dice loss function, and λ represents the weight coefficient for balancing the two losses. The choice of λ critically affects the model performance.

[0085] Cross-entropy loss function:

[0086]

[0087] where N represents the number of medical image samples, y i represents the true result of the i-th medical image sample, and p i represents the true result of the i-th medical image sample.

[0088] Dice loss function:

[0089]

[0090] In step S102, the medical images including the multiple rough annotation information are respectively applied to the multiple trained high-precision segmentation models, and multiple fine annotation information of each of the multiple high-precision segmentation models on the medical images is obtained.

[0091] The fine annotation information refers to the annotation information that is more accurate than the rough annotation information.

[0092] In the embodiments of the present disclosure, a progressive segmentation method is adopted. First, the medical images are roughly annotated, and then the multiple rough annotation information is optimized by combining multiple trained advanced high-precision segmentation models, so as to obtain multiple fine annotation information of each high-precision segmentation model, in order to reduce the computational amount, improve the segmentation efficiency, and enhance the accuracy of medical image segmentation.

[0093] In some specific embodiments, the multiple high-precision segmentation models include: VT-UNet segmentation model, Vnet segmentation model, and UNETR segmentation model.

[0094] The VT-UNet segmentation model reformulates volumetric segmentation from a sequence-to-sequence perspective and proposes a UNet-shaped volumetric Transformer for multi-modal medical image segmentation. An encoder block is designed, which contains two consecutive self-attention layers to jointly capture local and global context information. In addition, a decoder block is designed to support self- and cross-attention based on parallel (shifted) windows. This parallelization uses a shared projection of the queries and independently computes cross- and self-attention. To further enhance the function in decoding, a convex combination method of Fourier position encoding is proposed. Combining the proposed design choices greatly limits the model parameters and maintains a low failure rate compared with existing methods. Extensive evaluations are carried out, indicating that the VT-UNet segmentation model achieves the best volumetric segmentation results and also enhances the robustness to data artifacts.

[0095] The Vnet segmentation model proposes a volume-based and FCN-based 3D image segmentation method. It is trained end-to-end on the described MRI volume and learns to predict the segmentation of the entire volume at once. A new objective function is introduced in the Vnet segmentation model and optimized according to the Dice coefficient during training to handle the strong imbalance between foreground and background voxels. To cope with the limited amount of labeled information, random non-linear transformations and histogram matching are used to augment the data. The characteristics of the Vnet segmentation model are as follows: it can better combine the underlying information and high-level information of the data. The underlying information is the low-resolution information obtained after multiple downsamplings of the Vnet segmentation model. It can provide the context semantic information of the segmentation target in the whole image, which can be understood as the features reflecting the relationship between the target and its environment and helps to judge the category of the object. The high-level information is the high-resolution information directly passed from the encoder to the decoder at the same level through the concatenate operation. It can provide more refined features for segmentation, such as gradients.

[0096] The UNETR segmentation model directly uses volume data; it adopts Transformer as the main encoder of the segmentation network and is directly connected to the decoder through skip connections instead of using it as the attention layer in the segmentation network; it does not rely on the backbone CNN model to generate the input sequence but directly uses tokenized patches. The UNETR segmentation model is a Transformer-based 3D medical image segmentation model specially designed to process large and complex 3D medical images. The UNETR segmentation model introduces a new type of efficient paired attention (i.e., EPA) block. The EPA block effectively learns discriminative features in the spatial and channel dimensions through a pair of parallel branches based on spatial and channel attention. It not only reduces the number of parameters and computational costs of the UNETR segmentation model and speeds up the inference speed, but also enhances the information exchange between branches by sharing query and key mapping functions, thus improving the overall performance of the model, significantly reducing the computational complexity, and at the same time improving the segmentation accuracy. The UNETR segmentation model has demonstrated excellent performance on multiple public datasets and is easy to integrate into existing medical image processing pipelines.

[0097] Step S103: Vote on each pixel point on the medical image based on the multiple fine annotation information of each of the multiple high-precision segmentation models on the medical image, and determine multiple pixel points of interest on the medical image.

[0098] In an embodiment of the present disclosure, an expert system is established through multiple high-precision segmentation models, and each pixel point on the medical image is voted to determine whether the corresponding pixel point is a pixel of interest based on different decision results of the corresponding pixel point by the multiple high-precision segmentation models. This is to achieve high precision, improve the accuracy of annotation, avoid misjudgment of a single high-precision segmentation model, and improve the robustness of region segmentation.

[0099] In some specific embodiments, voting on each pixel point on the medical image based on multiple fine annotation information of each of the multiple high-precision segmentation models on the medical image to determine multiple pixels of interest on the medical image includes the following steps:

[0100] Step S103-1: Count the number of annotations in all the fine annotation information of the multiple high-precision segmentation models for each pixel point on the medical image as the voting number, and determine the number of models of the multiple high-precision segmentation models.

[0101] For example, multiple high-precision segmentation models include: the first high-precision segmentation model, the second high-precision segmentation model, and the third high-precision segmentation model; for the pixel point p1 on the medical image, if both the first high-precision segmentation model and the third high-precision segmentation model determine p1 as fine annotation information, and the second high-precision segmentation model determines the pixel point p1 as non-fine annotation information, then the number of annotations (i.e., the voting number) of the pixel point p1 is 2, and the number of models is 3.

[0102] Step S103-2: When the ratio of the voting number of any pixel point on the medical image to the number of models is greater than a preset ratio, or the voting number of the any pixel point is greater than the product value of the number of models and the preset ratio, determine the any pixel point as a pixel of interest.

[0103] For example, continuing the above example, if the preset ratio is 0.6, the voting number of any pixel point p1 on the medical image is 2, and the number of models is 3, then 2 / 3 > 0.6, or 2 > (0.6 × 3), and it is determined that the pixel point p1 is a pixel of interest. Of course, the setting of the preset ratio in this specific embodiment is not limited to this. The higher the preset ratio, the higher the accuracy of the obtained pixels of interest.

[0104] In some other specific embodiments, the preset ratio includes 0.5 to 0.7.

[0105] In this specific embodiment, as long as the ratio obtained from the voting results of multiple high-precision segmentation models for any pixel on the medical image is between 0.5 and 0.7, it can be determined that the pixel is a pixel of interest. This is to be able to retain the details in the segmented area of the medical image and is conducive to timely detection of lesions.

[0106] Step S104: Determine the region of interest on the medical image based on multiple pixels of interest on the medical image.

[0107] The region of interest is the region with research value and diagnostic value for the condition.

[0108] In some specific embodiments, the determining the region of interest on the medical image based on multiple pixels of interest on the medical image includes the following steps:

[0109] Step S104-1: Determine multiple hollow regions on the medical image based on multiple pixels of interest on the medical image.

[0110] Wherein, the boundary of each hollow region is at least constructed by multiple pixels of interest.

[0111] Since the pixels of interest determined after decision-making may produce incoherent regions, in this specific embodiment, multiple hollow regions on the medical image are first determined by using multiple pixels of interest on the medical image. The multiple hollow regions are not connected to each other. For example, the first type of hollow region has only an outer boundary and does not include any pixels of interest inside the outer boundary; the second type of hollow region includes an outer boundary and an inner boundary, includes pixels of interest inside the outer boundary, and the peripheral pixels among the pixels of interest included inside the outer boundary are constructed as the inner boundary of the second type of hollow region.

[0112] Step S104-2: Add at least one pixel of interest in each hollow region on the medical image to the queue corresponding to the hollow region.

[0113] A queue is a linear data structure that follows the principle of first in, first out, that is, the element that enters the queue first is the first to be popped out of the queue.

[0114] In this specific embodiment, a queue is established for each hollow region. One or more pixels of interest are selected from each hollow region and added to the queue corresponding to the hollow region. For example, if pixels of interest p1, p2, and p3 are added to the queue in sequence, when the information of the pixels of interest needs to be obtained from the queue, then the pixel of interest p1 is first popped out of the queue, and then the pixel of interest p2 can be popped out of the queue, and then the pixel of interest p3 can be popped out of the queue. So as to start from the pixels of interest put into the queue and gradually spread to other pixels in the hollow region to find out whether the pixel to be determined is a pixel of interest. By traversing each pixel in the hollow region, the verification work of each pixel is realized.

[0115] In some other specific embodiments, adding at least one pixel of interest in each hole region on the medical image to the queue corresponding to the hole region includes:

[0116] Step S104-2a: adding multiple pixels of interest on the boundary of each hole region to the queue corresponding to the hole region.

[0117] In this specific embodiment, first, multiple pixels of interest on the boundary of each hole region on the medical image are added to the queue corresponding to the hole region. Since it is not possible to determine whether the pixels outside the hole region are pixels of interest, the multiple pixels of interest placed in the queue can only gradually spread towards the center of the hole region to find out whether the pixels to be determined are pixels of interest. Thus, the verification work for the pixels to be determined can be completed in an orderly manner, avoiding repeated verification work and improving the verification efficiency.

[0118] Step S104-3: extracting the pixels of interest from the queue of each hole region respectively.

[0119] In this specific embodiment, the determined and unvisited pixels of interest can be obtained in an orderly manner through the queue of each hole region.

[0120] Step S104-4: determining the neighborhood pixels adjacent to the extracted pixels of interest in each hole region.

[0121] The neighborhood pixels include four-neighborhood pixels or eight-neighborhood pixels. For example, the four-neighborhood pixels of the pixel of interest p1 are the pixels at the four adjacent positions above, below, left, and right of the pixel of interest p1; and the eight-neighborhood pixels also include the pixels at the four adjacent positions of upper left, upper right, lower left, and lower right.

[0122] In this specific embodiment, each pixel in the hole region is gradually verified by gradually spreading to the adjacent pixels step by step.

[0123] Step S104-5: when any neighborhood pixel in each hole region is a non-pixel of interest, determining the any neighborhood pixel as a pixel of interest in the corresponding hole region, adding the any neighborhood pixel in the corresponding hole region to the queue corresponding to the hole region, and triggering the execution of the step of extracting the pixels of interest from the queue of each hole region respectively until the queues of all hole regions are empty.

[0124] In this specific embodiment, when any neighborhood pixel point in each hole area is an interested pixel point, it indicates that this neighborhood pixel point has been visited and there is no need to repeat the verification. When any neighborhood pixel point in each hole area is a non-interested pixel point, it indicates that this neighborhood pixel point has not been visited, then it is determined as an interested pixel point and added to the queue of the corresponding hole area as an unvisited interested pixel point. Then step S104-3 is executed again, and such cyclic operations are performed until all pixel points in each hole area have been visited, that is, until there are no interested pixel points in the queue of each hole area.

[0125] Step S104-6, determine the region of interest on the medical image based on all the interested pixel points in each hole area.

[0126] Since the interested pixel points determined after the prediction of the high-precision segmentation model may produce discontinuous regions. For this reason, in this specific embodiment, each pixel point in the hole area is traversed in a way of gradually spreading from the interested pixel points to other pixel points in the hole area to determine whether each pixel point is an interested pixel point. Thus, the connectivity of the hole area is achieved, and further the region of interest on the medical image can be accurately determined.

[0127] The embodiments of the present disclosure obtain multiple rough annotation information on the medical image, implement rough annotation on the medical image, generate rough annotation information, reduce the cost of annotating rough annotation information, and improve the research efficiency of the medical image. Through multiple trained high-precision segmentation models, a multi-expert system is formed. The high-precision segmentation models analyze the rough annotation information of the medical image, vote on each pixel point on the medical image through the refined annotation information obtained through the analysis to determine the interested pixel points, and then determine the region of interest on the medical image through the interested pixel points. By means of progressive annotation information, high-precision segmentation of the medical image is realized, the accuracy of annotation is improved, the judgment error of a single high-precision segmentation model is avoided, and the robustness of region segmentation is improved.

[0128] The embodiments of the present disclosure adopt an advanced fusion strategy to integrate the outputs of each model to achieve automatic segmentation of the region of interest with higher precision and robustness. Through continuous learning and model fusion, the accuracy and efficiency of the automatic segmentation model are continuously optimized, and a comprehensive performance evaluation is carried out on actual medical images to ensure its application value in actual medical scenarios, especially its contribution in recurrence prediction analysis.

[0129] Embodiment 2

[0130] The present disclosure also provides an apparatus embodiment that follows the above embodiments, which is used to implement the method steps described in the above embodiments. Based on the same interpretation of the meaning of the same name as in the above embodiments, it has the same technical effects as in the above embodiments, and will not be elaborated here.

[0131] As Figure 2 shown, the present disclosure provides a medical image segmentation apparatus 200, including:

[0132] An acquisition unit 201, configured to acquire a plurality of rough annotation information on a medical image;

[0133] An application unit 202, configured to apply the medical images including the plurality of rough annotation information to a plurality of trained high-precision segmentation models respectively, and obtain a plurality of fine annotation information of each of the plurality of high-precision segmentation models on the medical images;

[0134] A pixel determination unit 203, configured to vote on each pixel point on the medical image based on the plurality of fine annotation information of each of the plurality of high-precision segmentation models on the medical image, and determine a plurality of interested pixel points on the medical image;

[0135] A region determination unit 204, configured to determine an interested region on the medical image based on the plurality of interested pixel points on the medical image.

[0136] Optionally, the voting on each pixel point on the medical image based on the plurality of fine annotation information of each of the plurality of high-precision segmentation models to determine a plurality of interested pixel points on the medical image includes:

[0137] Count the number of annotations in all the fine annotation information of the plurality of high-precision segmentation models for each pixel point on the medical image as the voting number, and determine the number of the plurality of high-precision segmentation models;

[0138] When the ratio of the voting number of any pixel point on the medical image to the number of models is greater than a preset ratio, or the voting number of the any pixel point is greater than the product value of the number of models and the preset ratio, determine the any pixel point as an interested pixel point.

[0139] Optionally, the preset ratio includes 0.5 to 0.7.

[0140] Optionally, the determining the interested region on the medical image based on the plurality of interested pixel points on the medical image includes:

[0141] Determine a plurality of hole regions on the medical image based on the plurality of interested pixel points on the medical image, wherein the boundary of each hole region is constructed by a plurality of interested pixel points;

[0142] Add at least one pixel of interest in each hole region on the medical image to the queue corresponding to the hole region;

[0143] Extract the pixels of interest from the queue of each hole region respectively;

[0144] Determine the neighboring pixels adjacent to the extracted pixels of interest in each hole region;

[0145] When any neighboring pixel in each hole region is a non-pixel of interest, determine the any neighboring pixel as the pixel of interest in the corresponding hole region, add the any neighboring pixel in the corresponding hole region to the queue corresponding to the hole region, and trigger the execution of the step of extracting the pixels of interest from the queues of each hole region respectively until the queues of each hole region are all empty;

[0146] Determine the region of interest on the medical image based on all the pixels of interest in each hole region.

[0147] Optionally, the adding at least one pixel of interest in each hole region on the medical image to the queue corresponding to the hole region includes:

[0148] Add multiple pixels of interest on the boundary of each hole region on the medical image to the queue corresponding to the hole region.

[0149] Optionally, the multiple high-precision segmentation models include: VT-UNet segmentation model, Vnet segmentation model, and UNETR segmentation model.

[0150] Optionally, the obtaining multiple rough annotation information on the medical image includes:

[0151] Apply the medical image to a trained low-precision segmentation model to obtain multiple rough annotation information on the medical image.

[0152] Optionally, the low-precision segmentation model includes a trained MedSAM segmentation model.

[0153] Optionally, the multi-task loss function of the MedSAM segmentation model includes a cross-entropy loss function and a Dice loss function.

[0154] Embodiments of the present disclosure obtain multiple rough annotation information on medical images, implement rough annotation on medical images, generate rough annotation information, reduce the cost of annotating rough annotation information, and improve the research efficiency of medical images. Through multiple trained high-precision segmentation models, a multi-expert system is formed. The high-precision segmentation models analyze the rough annotation information of the medical images, vote on each pixel point on the medical images through the fine annotation information obtained through the analysis, determine the pixels of interest, and then determine the region of interest on the medical image through the pixels of interest. By means of progressive annotation information, high-precision segmentation of medical images is achieved, the accuracy of annotation is improved, the judgment errors of a single high-precision segmentation model are avoided, and the robustness of region segmentation is improved.

[0155] Embodiments of the present disclosure adopt an advanced fusion strategy to integrate the outputs of each model, achieving automatic segmentation of regions of interest with higher precision and robustness. Through continuous learning and model fusion, the precision and efficiency of the automatic segmentation model are continuously optimized, and a comprehensive performance evaluation is carried out on actual medical images, ensuring its application value in actual medical scenarios, especially its contribution in recurrence prediction analysis.

[0156] Embodiment 3

[0157] As Figure 3 shown, this embodiment provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method steps as described in the above embodiments.

[0158] Embodiment 4

[0159] Embodiments of the present disclosure provide a non-volatile computer storage medium storing computer-executable instructions that can execute the method steps as described in the above embodiments.

[0160] Embodiment 5

[0161] Next, refer to Figure 3 , which shows a schematic structural diagram of an electronic device suitable for implementing embodiments of the present disclosure. The terminal device in embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3The electronic device shown is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

[0162] As Figure 3 shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 301, which may perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage device 308 into the random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. The input / output (I / O) interface 305 is also connected to the bus 304.

[0163] Generally, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 305 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 3 an electronic device with various devices is shown, it should be understood that it is not required to implement or include all the shown devices. Instead, more or fewer devices may be implemented or included.

[0164] Specifically, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, the embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above functions defined in the methods of the embodiments of the present disclosure are performed.

[0165] It should be noted that the above computer-readable medium in the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0166] The above computer-readable medium can be included in the above electronic device; or it can exist separately without being assembled into the electronic device.

[0167] Computer program code for performing the operations of the present disclosure can be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0168] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0169] The units involved in the embodiments described in the present disclosure can be implemented in software or in hardware. Among them, the name of the unit does not constitute a limitation on the unit itself in some cases.

[0170] Finally, it should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0171] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present disclosure.

Claims

1. A method for segmenting medical images, characterized in that, Including: Obtaining multiple rough annotation information on a medical image; Applying the medical images including the multiple rough annotation information to multiple trained high-precision segmentation models respectively, and obtaining multiple fine annotation information of each of the multiple high-precision segmentation models on the medical image; Voting on each pixel point on the medical image based on the multiple fine annotation information of each of the multiple high-precision segmentation models on the medical image, and determining multiple interesting pixel points on the medical image; Determining the region of interest on the medical image based on the multiple interesting pixel points on the medical image, including: Determining multiple hole regions on the medical image based on the multiple interesting pixel points on the medical image, wherein the boundary of each hole region is constructed by multiple interesting pixel points; Adding at least one interesting pixel point in each hole region on the medical image to the queue of the corresponding hole region; Extracting interesting pixel points from the queue of each hole region respectively; Determining the neighborhood pixel points adjacent to the extracted interesting pixel points in each hole region; When any neighborhood pixel point in each hole region is a non-interesting pixel point, determining the any neighborhood pixel point as an interesting pixel point in the corresponding hole region, adding the any neighborhood pixel point in the corresponding hole region to the queue of the corresponding hole region, and triggering the execution of the step of extracting interesting pixel points from the queue of each hole region respectively until the queue of each hole region is empty; Determining the region of interest on the medical image based on all the interesting pixel points in each hole region.

2. The method according to claim 1, characterized in that The voting on each pixel point on the medical image based on the multiple fine annotation information of each of the multiple high-precision segmentation models on the medical image and determining multiple interesting pixel points on the medical image includes: Counting the annotation quantity in all the fine annotation information of the multiple high-precision segmentation models of each pixel point on the medical image as the voting quantity, and determining the number of the multiple high-precision segmentation models; When the ratio of the voting quantity of any pixel point on the medical image to the number of models is greater than a preset ratio, or the voting quantity of the any pixel point is greater than the product value of the number of models and the preset ratio, determining the any pixel point as an interesting pixel point.

3. The method according to claim 2, characterized in that The preset ratio includes 0.5 to 0.

7.

4. The method according to claim 1, wherein The adding at least one interesting pixel point in each hole region on the medical image to the queue of the corresponding hole region includes: Adding the multiple interesting pixel points on the boundary of each hole region on the medical image to the queue of the corresponding hole region.

5. The method according to claim 1, characterized in that, The multiple high-precision segmentation models include: VT-UNet segmentation model, Vnet segmentation model and UNETR segmentation model.

6. The method according to claim 1, characterized in that The obtaining multiple rough annotation information on a medical image includes: Applying the medical image to a trained low-precision segmentation model, and obtaining multiple rough annotation information on the medical image.

7. The method according to claim 6, wherein The low-precision segmentation model includes the trained MedSAM segmentation model.

8. The method according to claim 7, wherein The multi-task loss function of the MedSAM segmentation model includes a cross-entropy loss function and a Dice loss function.

9. A segmentation device for medical images, characterized in that, Comprising: An acquisition unit configured to acquire a plurality of rough annotation information on a medical image; An application unit configured to respectively apply the medical images including the plurality of rough annotation information to a plurality of trained high-precision segmentation models, and obtain a plurality of fine annotation information of each of the plurality of high-precision segmentation models on the medical image; A pixel determination unit configured to vote on each pixel point on the medical image based on the plurality of fine annotation information of each of the plurality of high-precision segmentation models on the medical image, and determine a plurality of interested pixel points on the medical image; A region determination unit configured to determine an interested region on the medical image based on the plurality of interested pixel points on the medical image, including: determining a plurality of hole regions on the medical image based on the plurality of interested pixel points on the medical image, wherein the boundary of each hole region is constructed by a plurality of interested pixel points; adding at least one interested pixel point in each hole region on the medical image to the queue of the corresponding hole region; respectively extracting interested pixel points from the queue of each hole region; determining neighboring pixel points adjacent to the extracted interested pixel points in each hole region; when any neighboring pixel point in each hole region is a non-interested pixel point, determining the any neighboring pixel point as an interested pixel point in the corresponding hole region, adding the any neighboring pixel point in the corresponding hole region to the queue of the corresponding hole region, and triggering the execution of the step of respectively extracting interested pixel points from the queue of each hole region until the queues of all hole regions are empty; determining the interested region on the medical image based on all the interested pixel points in each hole region.

Images