Medical image segmentation method and device

Abstract

The invention provides a method and a device for medical image segmentation. According to the method and the device, the multiple pieces of rough annotation information in the medical image are acquired to realize preliminary annotation of the image, and the corresponding rough annotation data are generated, so that the annotation cost is effectively reduced, and meanwhile, the medical image research efficiency is improved. Based on a plurality of trained high-precision segmentation models, a multi-expert system is constructed, rough labeling information in a medical image is deeply analyzed, and fine labeling information is generated. Voting is carried out on each pixel point, the interested pixel points are determined, and then the interested area in the medical image is recognized. According to the method, a progressive annotation information processing mode is adopted, high-precision segmentation of the medical image is achieved, the annotation accuracy is remarkably improved, misjudgment possibly caused by a single high-precision segmentation model is avoided, and therefore the robustness of region segmentation is enhanced.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to a medical image segmentation method, device, medium and electronic device. Background Art

[0002] In the field of medical image processing, accurate medical image segmentation is crucial for diagnosis and treatment planning.

[0003] Medical images are usually high-dimensional and diverse, requiring a large amount of accurate pixel-level annotation, which is not only time-consuming but also requires a lot of expert knowledge. Experts must carefully annotate areas of interest, such as tumors, organs, or other pathological structures. This high-precision annotation work is very time-consuming and greatly increases the time cost of research.

[0004] Therefore, the present disclosure provides a medical image segmentation method to solve one of the above technical problems. Summary of the invention

[0005] The purpose of the present disclosure is to provide a medical image segmentation method, device, medium and electronic device, which can solve at least one of the above-mentioned technical problems. The specific solution is as follows:

[0006] According to a specific embodiment of the present disclosure, in a first aspect, the present disclosure provides a medical image segmentation method, comprising:

[0007] Obtain multiple rough annotation information on medical images;

[0008] Applying the medical image including the plurality of coarse annotation information to the plurality of trained high-precision segmentation models respectively to obtain a plurality of fine annotation information of each of the plurality of high-precision segmentation models on the medical image;

[0009] Voting for each pixel on the medical image based on multiple fine annotation information of each of the multiple high-precision segmentation models on the medical image to determine multiple pixels of interest on the medical image;

[0010] A region of interest on the medical image is determined based on a plurality of pixels of interest on the medical image.

[0011] Optionally, voting for each pixel on the medical image based on multiple fine annotation information of each of the multiple high-precision segmentation models on the medical image to determine multiple pixels of interest on the medical image includes:

[0012] Counting the number of annotations of each pixel on the medical image in all the fine annotation information of the multiple high-precision segmentation models to represent the number of votes, and determining the number of models of the multiple high-precision segmentation models;

[0013] When the ratio of the number of votes for any pixel point on the medical image to the number of models is greater than a preset ratio, or the number of votes for any pixel point is greater than the product of the number of models and the preset ratio, the any pixel point is determined to be a pixel of interest.

[0014] Optionally, the preset ratio includes 0.5 to 0.7.

[0015] Optionally, determining the region of interest on the medical image based on a plurality of pixels of interest on the medical image includes:

[0016] Determining a plurality of hole regions on the medical image based on a plurality of pixels of interest on the medical image, wherein a boundary of each hole region is constructed by the plurality of pixels of interest;

[0017] Adding at least one pixel of interest in each hole region on the medical image to a queue corresponding to the hole region;

[0018] Extract the pixels of interest from the queue of each hole area respectively;

[0019] Determine the neighboring pixels adjacent to the extracted pixel of interest in each hole area;

[0020] When any neighboring pixel point in each hole area is a non-pixel point of interest, determine the any neighboring pixel point as a pixel point of interest in the corresponding hole area, add the any neighboring pixel point in the corresponding hole area to the queue of the corresponding hole area, and trigger the step of extracting the pixel points of interest from the queue of each hole area respectively until the queue of each hole area is empty;

[0021] The region of interest on the medical image is determined based on all the pixels of interest in each hole region.

[0022] Optionally, adding at least one pixel of interest in each hole region on the medical image to a queue corresponding to the hole region includes:

[0023] A plurality of pixels of interest at the boundary of each hole region on the medical image are added to a queue corresponding to the hole region.

[0024] Optionally, the multiple high-precision segmentation models include: VT-UNet segmentation model, Vnet segmentation model and UNETR segmentation model.

[0025] Optionally, the obtaining of a plurality of rough annotation information on the medical image includes:

[0026] The medical image is applied to a trained low-precision segmentation model to obtain a plurality of rough annotation information on the medical image.

[0027] Optionally, the low-precision segmentation model includes a trained MedSAM segmentation model.

[0028] Optionally, the multi-task loss function of the MedSAM segmentation model includes a cross entropy loss function and a Dice loss function.

[0029] According to a specific embodiment of the present disclosure, in a second aspect, the present disclosure provides a medical image segmentation device, comprising:

[0030] An acquisition unit, used for acquiring a plurality of rough annotation information on the medical image;

[0031] An application unit, configured to apply the medical image including the plurality of coarse annotation information to the plurality of trained high-precision segmentation models respectively, to obtain a plurality of fine annotation information of each of the plurality of high-precision segmentation models on the medical image;

[0032] A pixel determination unit, configured to vote for each pixel on the medical image based on a plurality of fine annotation information of each of the plurality of high-precision segmentation models on the medical image, so as to determine a plurality of pixels of interest on the medical image;

[0033] A region determining unit is used to determine a region of interest on the medical image based on a plurality of pixels of interest on the medical image.

[0034] Optionally, voting for each pixel on the medical image based on multiple fine annotation information of each of the multiple high-precision segmentation models on the medical image to determine multiple pixels of interest on the medical image includes:

[0035] Counting the number of annotations of each pixel on the medical image in all the fine annotation information of the multiple high-precision segmentation models to represent the number of votes, and determining the number of models of the multiple high-precision segmentation models;

[0036] When the ratio of the number of votes for any pixel point on the medical image to the number of models is greater than a preset ratio, or the number of votes for any pixel point is greater than the product of the number of models and the preset ratio, the any pixel point is determined to be a pixel of interest.

[0037] Optionally, the preset ratio includes 0.5 to 0.7.

[0038] Optionally, determining the region of interest on the medical image based on a plurality of pixels of interest on the medical image includes:

[0039] Determining a plurality of hole regions on the medical image based on a plurality of pixels of interest on the medical image, wherein a boundary of each hole region is constructed by the plurality of pixels of interest;

[0040] Adding at least one pixel of interest in each hole region on the medical image to a queue corresponding to the hole region;

[0041] Extract the pixels of interest from the queue of each hole area respectively;

[0042] Determine the neighboring pixels adjacent to the extracted pixel of interest in each hole area;

[0043] When any neighboring pixel point in each hole area is a non-pixel point of interest, determine the any neighboring pixel point as a pixel point of interest in the corresponding hole area, add the any neighboring pixel point in the corresponding hole area to the queue of the corresponding hole area, and trigger the step of extracting the pixel points of interest from the queue of each hole area respectively until the queue of each hole area is empty;

[0044] The region of interest on the medical image is determined based on all the pixels of interest in each hole region.

[0045] Optionally, adding at least one pixel of interest in each hole region on the medical image to a queue corresponding to the hole region includes:

[0046] A plurality of pixels of interest at the boundary of each hole region on the medical image are added to a queue corresponding to the hole region.

[0047] Optionally, the multiple high-precision segmentation models include: VT-UNet segmentation model, Vnet segmentation model and UNETR segmentation model.

[0048] Optionally, the obtaining of a plurality of rough annotation information on the medical image includes:

[0049] The medical image is applied to a trained low-precision segmentation model to obtain a plurality of rough annotation information on the medical image.

[0050] Optionally, the low-precision segmentation model includes a trained MedSAM segmentation model.

[0051] Optionally, the multi-task loss function of the MedSAM segmentation model includes a cross entropy loss function and a Dice loss function.

[0052] According to a specific embodiment of the present disclosure, in a third aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the medical image segmentation method as described in any one of the above items is implemented.

[0053] According to the specific implementation of the present disclosure, in a fourth aspect, the present disclosure provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the medical image segmentation method as described in any one of the above items.

[0054] Compared with the prior art, the above solution of the embodiment of the present disclosure has at least the following beneficial effects:

[0055] The present disclosure provides a medical image segmentation method, device, medium and electronic device. The present disclosure obtains multiple rough annotation information on the medical image, implements rough annotation on the medical image, generates rough annotation information, reduces the cost of annotating rough annotation information, and improves the research efficiency of medical images. Through multiple trained high-precision segmentation models, a multi-expert system is formed, and the rough annotation information of the medical image is analyzed by the high-precision segmentation model. The fine annotation information obtained by the analysis is voted for each pixel on the medical image to determine the pixel of interest, and then the region of interest on the medical image is determined by the pixel of interest. By progressively annotating information, high-precision segmentation of medical images is achieved, the accuracy of annotation is improved, the judgment errors of a single high-precision segmentation model are avoided, and the robustness of regional segmentation is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A flow chart of a medical image segmentation method according to an embodiment of the present disclosure is shown;

[0057] Figure 2 A unit block diagram of a medical image segmentation device according to an embodiment of the present disclosure is shown;

[0058] Figure 3 A schematic diagram of a connection structure of an electronic device provided according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0060] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments, and are not intended to limit the present disclosure. The singular forms "a", "said" and "the" used in the embodiments of the present disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two.

[0061] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0062] It should be understood that although the terms first, second, third, etc. may be used to describe in the disclosed embodiments, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, without departing from the scope of the disclosed embodiments, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0063] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0064] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the commodity or device including the elements.

[0065] It should be particularly noted that any symbols and / or numbers in the specification that are not marked in the accompanying drawings are not drawing marks.

[0066] The optional embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0067] Example 1

[0068] The embodiment provided in the present disclosure is an embodiment of a medical image segmentation method.

[0069] Combine the following Figure 1 The embodiments of the present disclosure are described in detail.

[0070] Step S101, obtaining a plurality of rough annotation information on a medical image.

[0071] Annotate medical images for analysis and diagnosis, helping doctors save time, make more informed decisions, and improve patient treatment outcomes.

[0072] Rough annotation information refers to low-precision annotation information. Rough annotation information serves as the basic information for fine annotation. For example, rough annotation information can be annotated manually.

[0073] In some specific embodiments, the step of obtaining a plurality of rough annotation information on the medical image comprises the following steps:

[0074] Step S101a: applying the medical image to a trained low-precision segmentation model to obtain a plurality of rough annotation information on the medical image.

[0075] In this specific embodiment, a small amount of high-precision annotation information is manually annotated on the medical image samples, and the low-precision segmentation model is trained using the medical image samples to achieve preliminary automatic recognition of the lesion area on the medical image. The trained low-precision segmentation model can obtain multiple rough annotation information on the medical image, and then the rough annotation information is automatically generated on the medical image, which reduces the cost of manually annotating the rough annotation information and improves the research efficiency of medical images.

[0076] In some other specific embodiments, the low-precision segmentation model includes a MedSAM segmentation model.

[0077] The MedSAM segmentation model (Medical Segment Anything Model) is a deep learning model that specifically leverages the power of convolutional neural networks (CNNs). The MedSAM segmentation model integrates a variety of existing high-quality pre-trained models, such as the U-Net model and the Faster R-CNN model, which have demonstrated excellent performance in medical image segmentation tasks.

[0078] The MedSAM segmentation model framework is based on a modular design and provides the ability to fine-tune the model and train custom models, allowing users to easily add new network layers or loss functions and optimize according to their needs to adapt to the evolving AI research. The MedSAM segmentation model also supports GPU acceleration, which greatly improves computing efficiency and makes it possible to process high-resolution medical images. The MedSAM segmentation model has performed well in the recognition of a variety of biological tissues and structures and has strong basic segmentation capabilities.

[0079] In this specific embodiment, the MedSAM segmentation model is trained using medical image samples in a large-scale medical image data set to obtain a trained MedSAM segmentation model, and then the trained MedSAM segmentation model is used to perform cell segmentation on the medical image.

[0080] First, the MedSAM segmentation model is initialized, that is, the pre-trained MedSAM segmentation model is loaded. Then the parameters are fine-tuned and the deep network structure of the model is fine-tuned using a lower learning rate (such as 0.2-0.4) to ensure that the model retains its original useful features and learning ability while adapting to tumor-specific images.

[0081] In some other specific embodiments, the multi-task loss function of the MedSAM segmentation model includes a cross entropy loss function and a Dice loss function, which are used to optimize the performance of the MedSAM segmentation model in terms of pixel-level classification accuracy and segmentation region coherence.

[0082] For example, the specific formula of the multi-task loss function is:

[0083] L=λL CE +(1-λ)L Dice ;

[0084] Among them, L represents the multi-task loss function, L CE represents the cross entropy loss function, L Dice represents the Dice loss function, λ represents the weight coefficient for balancing the two losses, and the selection of λ critically affects the model performance.

[0085] Cross entropy loss function:

[0086]

[0087] Where N represents the number of medical image samples, y i Describe the true result of the i-th medical image sample, p i Describe the true result of the i-th medical image sample.

[0088] Dice loss function:

[0089]

[0090] Step S102: Apply the medical image including the plurality of coarse annotation information to the plurality of trained high-precision segmentation models respectively to obtain a plurality of fine annotation information of each of the plurality of high-precision segmentation models on the medical image.

[0091] Fine annotation information refers to annotation information that is more accurate than coarse annotation information.

[0092] In the disclosed embodiment, a progressive segmentation method is adopted, in which the medical image is first roughly annotated, and then the multiple rough annotation information is optimized by combining multiple advanced high-precision segmentation models that have been trained to obtain multiple fine annotation information of each high-precision segmentation model, so as to reduce the amount of calculation, improve the segmentation efficiency, and enhance the accuracy of medical image segmentation.

[0093] In some specific embodiments, the multiple high-precision segmentation models include: a VT-UNet segmentation model, a Vnet segmentation model, and a UNETR segmentation model.

[0094] VT-UNet segmentation model, reformulates volume segmentation from a sequence-to-sequence perspective and proposes a UNet-shaped volume transformer for multimodal medical image segmentation. An encoder block is designed with two consecutive self-attention layers to jointly capture local and global contextual information. In addition, a decoder block is designed to support self- and cross-attention based on parallel (shifted) windows. This parallelization uses a shared projection of the query and independently computes cross- and self-attention. To further enhance the functionality in decoding, a convex combination method of Fourier position encoding is proposed. Combined with the proposed design choices, the model parameters are greatly constrained, maintaining a low failure rate compared to existing methods. Extensive evaluation is performed and shows that the VT-UNet segmentation model achieves the best volume segmentation results while also improving robustness to data artifacts.

[0095] Vnet segmentation model, a volume-based, FCN-based 3D image segmentation method is proposed. End-to-end training is performed on the description of MRI volume, and the segmentation of the entire volume is predicted at once. A new objective function is introduced in the Vnet segmentation model, which is optimized according to the Dice coefficient during training to achieve a state that can handle the strong imbalance between foreground voxels and background voxels. In response to the limited amount of annotation information, random nonlinear transformation and histogram matching are used to enhance the data. Features of the Vnet segmentation model: It can better combine the underlying information and high-level information of the data. The underlying information is the low-resolution information obtained by the Vnet segmentation model after multiple downsampling. It can provide contextual semantic information of the segmentation target in the entire image, which can be understood as the characteristics of the relationship between the target and its environment, which is helpful for the category judgment of the object. The high-level information is directly transferred from the encoder to the high-resolution information on the decoder at the same height through the concatenate operation. It can provide more refined features for segmentation, such as gradients.

[0096] The UNETR segmentation model directly utilizes volumetric data; uses Transformer as the main encoder of the segmentation network and directly connects it to the decoder through skip connections instead of using it as an attention layer in the segmentation network; does not rely on the backbone CNN model for generating input sequences but directly utilizes tokenized patches. The UNETR segmentation model is a Transformer-based 3D medical image segmentation model, which is specially designed to process large and complex 3D medical images. The UNETR segmentation model introduces a new type of efficient paired attention (EPA) block. The EPA block effectively learns discriminative features of spatial and channel dimensions through a pair of parallel branches based on spatial and channel attention. It not only reduces the number of parameters and computational cost of the UNETR segmentation model, but also speeds up the inference speed. It also enhances the information exchange between branches by sharing query and key mapping functions, thereby improving the overall performance of the model, significantly reducing the computational complexity, and improving the segmentation accuracy. The UNETR segmentation model has demonstrated excellent performance on multiple public datasets and is easy to integrate into existing medical image processing pipelines.

[0097] Step S103 , voting for each pixel on the medical image based on multiple fine annotation information of each of the multiple high-precision segmentation models on the medical image, to determine multiple pixels of interest on the medical image.

[0098] In the embodiment of the present disclosure, an expert system is established through multiple high-precision segmentation models to vote for each pixel on the medical image, so as to determine whether the corresponding pixel is a pixel of interest through different decision results of multiple high-precision segmentation models on the corresponding pixel. In order to achieve high precision, improve the accuracy of annotation, avoid misjudgment of a single high-precision segmentation model, and improve the robustness of regional segmentation.

[0099] In some specific embodiments, voting for each pixel on the medical image based on multiple fine annotation information of each of the multiple high-precision segmentation models on the medical image to determine multiple pixels of interest on the medical image includes the following steps:

[0100] Step S103 - 1 , counting the number of annotations of each pixel on the medical image in all the fine annotation information of the multiple high-precision segmentation models to represent the number of votes, and determining the number of models of the multiple high-precision segmentation models.

[0101] For example, multiple high-precision segmentation models include: a first high-precision segmentation model, a second high-precision segmentation model and a third high-precision segmentation model; for pixel point p1 on the medical image, if the first high-precision segmentation model and the third high-precision segmentation model both determine that p1 is fine annotation information, and the second high-precision segmentation model determines that the pixel point p1 is non-fine annotation information, then the number of annotations (i.e., the number of votes) of the pixel point p1 is 2, and the number of models is 3.

[0102] Step S103-2, when the ratio of the number of votes of any pixel point on the medical image to the number of models is greater than a preset ratio, or the number of votes of any pixel point is greater than the product of the number of models and the preset ratio, determine that any pixel point is a pixel of interest.

[0103] For example, continuing the above example, if the preset ratio is 0.6, the number of votes for any pixel p1 on the medical image is 2, and the number of models is 3, then 2 / 3>0.6, or 2>(0.6ⅹ3), and the pixel p1 is determined to be a pixel of interest. Of course, the setting of the preset ratio in this specific embodiment is not limited to this. The higher the preset ratio, the higher the accuracy of the obtained pixel of interest.

[0104] In some other specific embodiments, the preset ratio includes 0.5-0.7.

[0105] In this specific embodiment, as long as the ratio of the voting results of multiple high-precision segmentation models for any pixel on the medical image is between 0.5 and 0.7, the pixel can be determined as a pixel of interest, so that the details in the segmented area on the medical image can be retained, which is conducive to timely detection of lesions.

[0106] Step S104: determining a region of interest on the medical image based on a plurality of pixels of interest on the medical image.

[0107] The area of ​​interest is the area that has research value and diagnostic value.

[0108] In some specific embodiments, determining a region of interest on the medical image based on a plurality of pixels of interest on the medical image comprises the following steps:

[0109] Step S104 - 1 , determining a plurality of hole regions on the medical image based on a plurality of pixels of interest on the medical image.

[0110] The boundary of each hole area is constructed by at least a plurality of pixels of interest.

[0111] Since the pixels of interest determined after the decision may produce incoherent areas, for this reason, this specific embodiment first uses multiple pixels of interest on the medical image to determine multiple hole areas on the medical image. The two hole areas are not connected. For example, the first type of hole area has only an outer boundary, and no pixels of interest are included inside the outer boundary; the second type of hole area includes an outer boundary and an inner boundary, and includes pixels of interest inside the outer boundary, and the peripheral pixels of the pixels of interest included inside the outer boundary are constructed as the inner boundary of the second type of hole area.

[0112] Step S104 - 2 , adding at least one pixel of interest in each hole region on the medical image to a queue corresponding to the hole region.

[0113] A queue is a linear data structure that follows the first-in-first-out principle, that is, the element that first enters the queue is the first to be popped out of the queue.

[0114] In this specific embodiment, a queue is established for each hole area. One or more pixels of interest are selected from each hole area and added to the queue of the corresponding hole area. For example, if the pixels of interest p1, p2 and p3 are added to the queue in sequence, when the information of the pixels of interest needs to be obtained from the queue, the pixel of interest p1 is first popped out of the queue, and then the pixel of interest p2 can be popped out of the queue, and then the pixel of interest p3 can be popped out of the queue. So as to start from the pixel of interest put into the queue, gradually spread to other pixels in the hole area to find out whether the pixel to be determined is a pixel of interest. By traversing each pixel in the hole area, the verification of each pixel is achieved.

[0115] In some other specific embodiments, adding at least one pixel of interest in each hole region on the medical image to a queue corresponding to the hole region includes:

[0116] Step S104-2a, adding multiple pixels of interest at the boundary of each hole area to a queue corresponding to the hole area.

[0117] In this specific embodiment, multiple pixels of interest at the boundary of each hole region on the medical image are first added to the queue of the corresponding hole region. Since it is not determined whether the pixels outside the hole region are pixels of interest, the multiple pixels of interest put into the queue can only gradually spread to the center of the hole region to find out whether the pixel to be determined is a pixel of interest. In this way, the verification work of the pixels to be determined can be completed in an orderly manner, avoiding repeated verification work and improving verification efficiency.

[0118] Step S104-3, extracting pixels of interest from the queue of each hole area respectively.

[0119] In this specific embodiment, determined pixels of interest that have not yet been visited can be obtained in an orderly manner through the queue of each hole area.

[0120] Step S104-4, determining neighboring pixels adjacent to the extracted pixel of interest in each hole area.

[0121] The neighboring pixel points include four neighboring pixel points or eight neighboring pixel points. For example, the four neighboring pixel points of the pixel point of interest p1 are the pixel points at the top, bottom, left and right of the pixel point of interest p1; and the eight neighboring pixel points also include the pixel points at the top left, top right, bottom left and bottom right.

[0122] This specific implementation verifies each pixel in the hole area step by step by gradually diffusing to adjacent pixels.

[0123] Step S104-5, when any neighborhood pixel point in each hole area is a pixel point of no interest, determine that any neighborhood pixel point is a pixel point of interest in the corresponding hole area, add any neighborhood pixel point in the corresponding hole area to the queue of the corresponding hole area, trigger the execution of the step of extracting pixels of interest from the queue of each hole area respectively, until the queue of each hole area is empty.

[0124] In this specific embodiment, when any neighboring pixel in each hole area is a pixel of interest, it indicates that the neighboring pixel has been visited and no repeated verification is required. When any neighboring pixel in each hole area is a pixel of non-interest, it indicates that the neighboring pixel has not been visited, and it is determined as a pixel of interest, and added to the queue of the corresponding hole area as a pixel of interest that has not been visited. Then, step S104-3 is executed again, and the operation is repeated until all pixels in each hole area have been visited, that is, there are no more pixels of interest in the queue of each hole area.

[0125] Step S104-6: determining a region of interest on the medical image based on all pixels of interest in each hole region.

[0126] Since the pixels of interest determined after the high-precision segmentation model prediction may produce incoherent areas, in this specific embodiment, each pixel in the hole area is traversed in a way of gradually diffusing from the pixel of interest to other pixels in the hole area to determine whether each pixel is a pixel of interest. This achieves the connectivity of the hole area, and can accurately determine the region of interest on the medical image.

[0127] The disclosed embodiment obtains multiple rough annotation information on the medical image, implements rough annotation on the medical image, generates rough annotation information, reduces the cost of annotating rough annotation information, and improves the research efficiency of medical images. Through multiple trained high-precision segmentation models, a multi-expert system is formed, and the rough annotation information of the medical image is analyzed by the high-precision segmentation model. The fine annotation information obtained by the analysis votes for each pixel on the medical image to determine the pixel of interest, and then the region of interest on the medical image is determined by the pixel of interest. By progressively annotating information, high-precision segmentation of medical images is achieved, the accuracy of annotation is improved, the judgment errors of a single high-precision segmentation model are avoided, and the robustness of regional segmentation is improved.

[0128] The disclosed embodiment uses an advanced fusion strategy to integrate the outputs of each model to achieve more accurate and robust automatic segmentation of the region of interest. Through continuous learning and model fusion, the accuracy and efficiency of the automatic segmentation model are continuously optimized, and a comprehensive performance evaluation is performed on actual medical images to ensure its application value in actual medical scenarios, especially its contribution to recurrence prediction analysis.

[0129] Example 2

[0130] The present disclosure also provides an apparatus embodiment that is consistent with the above-mentioned embodiment, and is used to implement the method steps described in the above-mentioned embodiment. The explanation based on the same name meaning is the same as that of the above-mentioned embodiment, and has the same technical effect as that of the above-mentioned embodiment, and will not be repeated here.

[0131] like Figure 2 As shown, the present disclosure provides a medical image segmentation device 200, comprising:

[0132] An acquisition unit 201 is used to acquire a plurality of rough annotation information on a medical image;

[0133] An application unit 202 is used to apply the medical image including the plurality of coarse annotation information to the plurality of trained high-precision segmentation models respectively, to obtain a plurality of fine annotation information of each of the plurality of high-precision segmentation models on the medical image;

[0134] A pixel determination unit 203, configured to vote for each pixel on the medical image based on a plurality of fine annotation information of each of the plurality of high-precision segmentation models on the medical image, to determine a plurality of pixels of interest on the medical image;

[0135] The region determining unit 204 is configured to determine a region of interest on the medical image based on a plurality of pixels of interest on the medical image.

[0136] Optionally, voting for each pixel on the medical image based on multiple fine annotation information of each of the multiple high-precision segmentation models on the medical image to determine multiple pixels of interest on the medical image includes:

[0137] Counting the number of annotations of each pixel on the medical image in all the fine annotation information of the multiple high-precision segmentation models to represent the number of votes, and determining the number of models of the multiple high-precision segmentation models;

[0138] When the ratio of the number of votes for any pixel point on the medical image to the number of models is greater than a preset ratio, or the number of votes for any pixel point is greater than the product of the number of models and the preset ratio, the any pixel point is determined to be a pixel of interest.

[0139] Optionally, the preset ratio includes 0.5 to 0.7.

[0140] Optionally, determining the region of interest on the medical image based on a plurality of pixels of interest on the medical image includes:

[0141] Determining a plurality of hole regions on the medical image based on a plurality of pixels of interest on the medical image, wherein a boundary of each hole region is constructed by the plurality of pixels of interest;

[0142] Adding at least one pixel of interest in each hole region on the medical image to a queue corresponding to the hole region;

[0143] Extract the pixels of interest from the queue of each hole area respectively;

[0144] Determine the neighboring pixels adjacent to the extracted pixel of interest in each hole area;

[0145] When any neighboring pixel point in each hole area is a non-pixel point of interest, determine the any neighboring pixel point as a pixel point of interest in the corresponding hole area, add the any neighboring pixel point in the corresponding hole area to the queue of the corresponding hole area, and trigger the step of extracting the pixel points of interest from the queue of each hole area respectively until the queue of each hole area is empty;

[0146] The region of interest on the medical image is determined based on all the pixels of interest in each hole region.

[0147] Optionally, adding at least one pixel of interest in each hole region on the medical image to a queue corresponding to the hole region includes:

[0148] A plurality of pixels of interest at the boundary of each hole region on the medical image are added to a queue corresponding to the hole region.

[0149] Optionally, the multiple high-precision segmentation models include: VT-UNet segmentation model, Vnet segmentation model and UNETR segmentation model.

[0150] Optionally, the obtaining of a plurality of rough annotation information on the medical image includes:

[0151] The medical image is applied to a trained low-precision segmentation model to obtain a plurality of rough annotation information on the medical image.

[0152] Optionally, the low-precision segmentation model includes a trained MedSAM segmentation model.

[0153] Optionally, the multi-task loss function of the MedSAM segmentation model includes a cross entropy loss function and a Dice loss function.

[0154] The disclosed embodiment obtains multiple rough annotation information on the medical image, implements rough annotation on the medical image, generates rough annotation information, reduces the cost of annotating rough annotation information, and improves the research efficiency of medical images. Through multiple trained high-precision segmentation models, a multi-expert system is formed, and the rough annotation information of the medical image is analyzed by the high-precision segmentation model. The fine annotation information obtained by the analysis votes for each pixel on the medical image to determine the pixel of interest, and then the region of interest on the medical image is determined by the pixel of interest. By progressively annotating information, high-precision segmentation of medical images is achieved, the accuracy of annotation is improved, the judgment errors of a single high-precision segmentation model are avoided, and the robustness of regional segmentation is improved.

[0155] The disclosed embodiment uses an advanced fusion strategy to integrate the outputs of each model to achieve more accurate and robust automatic segmentation of the region of interest. Through continuous learning and model fusion, the accuracy and efficiency of the automatic segmentation model are continuously optimized, and a comprehensive performance evaluation is performed on actual medical images to ensure its application value in actual medical scenarios, especially its contribution to recurrence prediction analysis.

[0156] Example 3

[0157] like Figure 3 As shown, this embodiment provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method steps described in the above embodiment.

[0158] Example 4

[0159] An embodiment of the present disclosure provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions can execute the method steps described in the above embodiment.

[0160] Example 5

[0161] Reference below Figure 3 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0162] like Figure 3 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In RAM 303, various programs and data required for the operation of the electronic device are also stored. The processing device 301, ROM 302, and RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0163] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 305 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0164] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0165] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0166] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0167] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0168] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0169] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0170] Finally, it should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0171] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A medical image segmentation method, characterized in that: include: Obtain multiple rough annotation information on medical images; Applying the medical image including the plurality of coarse annotation information to the plurality of trained high-precision segmentation models respectively to obtain a plurality of fine annotation information of each of the plurality of high-precision segmentation models on the medical image; Voting for each pixel on the medical image based on multiple fine annotation information of each of the multiple high-precision segmentation models on the medical image to determine multiple pixels of interest on the medical image; A region of interest on the medical image is determined based on a plurality of pixels of interest on the medical image.

2. The method according to claim 1, characterized in that The step of voting for each pixel on the medical image based on the multiple fine annotation information of each of the multiple high-precision segmentation models on the medical image to determine multiple pixels of interest on the medical image includes: Counting the number of annotations of each pixel on the medical image in all the fine annotation information of the multiple high-precision segmentation models to represent the number of votes, and determining the number of models of the multiple high-precision segmentation models; When the ratio of the number of votes for any pixel point on the medical image to the number of models is greater than a preset ratio, or the number of votes for any pixel point is greater than the product of the number of models and the preset ratio, the any pixel point is determined to be a pixel of interest.

3. The method according to claim 2, characterized in that The preset ratio includes 0.5 to 0.

7.

4. The method according to claim 1, characterized in that: The determining the region of interest on the medical image based on a plurality of pixels of interest on the medical image comprises: Determining a plurality of hole regions on the medical image based on a plurality of pixels of interest on the medical image, wherein a boundary of each hole region is constructed by the plurality of pixels of interest; Adding at least one pixel of interest in each hole region on the medical image to a queue corresponding to the hole region; Extract the pixels of interest from the queue of each hole area respectively; Determine the neighboring pixels adjacent to the extracted pixel of interest in each hole area; When any neighboring pixel point in each hole area is a non-pixel point of interest, determine the any neighboring pixel point as a pixel point of interest in the corresponding hole area, add the any neighboring pixel point in the corresponding hole area to the queue of the corresponding hole area, and trigger the step of extracting the pixel points of interest from the queue of each hole area respectively until the queue of each hole area is empty; The region of interest on the medical image is determined based on all the pixels of interest in each hole region.

5. The method according to claim 4, characterized in that The step of adding at least one pixel of interest in each hole region on the medical image to a queue corresponding to the hole region comprises: A plurality of pixels of interest at the boundary of each hole region on the medical image are added to a queue corresponding to the hole region.

6. The method according to claim 1, characterized in that The multiple high-precision segmentation models include: VT-UNet segmentation model, Vnet segmentation model and UNETR segmentation model.

7. The method according to claim 1, characterized in that The obtaining of a plurality of rough annotation information on the medical image comprises: The medical image is applied to a trained low-precision segmentation model to obtain a plurality of rough annotation information on the medical image.

8. The method according to claim 7, characterized in that The low-precision segmentation model includes a trained MedSAM segmentation model.

9. The method according to claim 8, characterized in that The multi-task loss function of the MedSAM segmentation model includes a cross entropy loss function and a Dice loss function.

10. A medical image segmentation device, characterized in that: include: An acquisition unit, used for acquiring a plurality of rough annotation information on the medical image; An application unit, configured to apply the medical image including the plurality of coarse annotation information to the plurality of trained high-precision segmentation models respectively, to obtain a plurality of fine annotation information of each of the plurality of high-precision segmentation models on the medical image; A pixel determination unit, configured to vote for each pixel on the medical image based on a plurality of fine annotation information of each of the plurality of high-precision segmentation models on the medical image, so as to determine a plurality of pixels of interest on the medical image; A region determining unit is used to determine a region of interest on the medical image based on a plurality of pixels of interest on the medical image.

Images