Medical image segmentation method and device, computer device and storage medium
By constructing a large model cluster and a feature interaction model, the problem of insufficient ability to perceive long-range dependencies and details in existing medical image segmentation methods is solved, achieving more efficient medical image segmentation results, especially in 3D medical images.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HEBEI HUIREN MEDICAL EQUIP TECH CO LTD
- Filing Date
- 2024-06-26
- Publication Date
- 2026-06-16
AI Technical Summary
Existing medical image segmentation methods are insufficient in capturing long-range dependencies and detail perception, especially when medical images differ significantly from natural images, leading to decreased model generalization ability and unsatisfactory segmentation results.
A large model cluster is constructed, and the segmentation models learn from each other through feature interaction models, including models such as SAM, UNETR, and U-Net. The segmentation results are optimized by using feature interaction and fusion models, thereby enhancing global modeling and detail perception capabilities.
It significantly improves the performance of medical image segmentation, enhances the model's generalization ability and segmentation accuracy, and performs particularly well in complex data and 3D medical images.
Smart Images
Figure CN120088273B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical image segmentation methods, and in particular to a medical image segmentation method, apparatus, computer equipment, and storage medium. Background Technology
[0002] Medical imaging technology is a crucial tool in the medical field. Through various physical principles and equipment, it acquires information about the internal structure and function of the human body, aiding doctors in diagnosing diseases and guiding treatment. Common medical imaging techniques include CT, MRI, and PET, which are widely used due to their painless, non-invasive nature and the provision of high-quality images. However, extracting useful information from these high-quality images is challenging, heavily reliant on the doctor's clinical experience. Therefore, advanced image analysis tools are essential for processing this image data, helping doctors better understand diseases, develop more scientific treatment plans, and improve medical standards and patients' quality of life.
[0003] In the field of medical image segmentation, various deep learning-based methods have been proposed to extract regions of interest and structures in medical images for treatment detection and disease diagnosis. Since the groundbreaking U-Net network, CNN-based networks have rapidly achieved state-of-the-art results on various 2D and 3D medical image segmentation tasks due to their strong detail perception, adaptability, and ease of training. These networks extract data features through convolutional layers and learn the mapping relationship between the original image and the segmentation result from a large amount of training data. To compensate for the inability of convolutional layers to capture long-range dependencies, Transformer-based methods transform the image segmentation problem into a sequence prediction problem. By introducing the Transformer module, the inability of convolutional layers to capture long-range dependencies is effectively compensated for. Ultimately, the powerful global modeling capability of the Transformer effectively improves the segmentation performance of the model, especially when dealing with long-range dependencies and images of different scales. The SAM model is a fundamental large-scale segmentation model. This model was pre-trained on 11 million natural images with over 1 billion masks. Thanks to the massive training data and general model architecture, SAM has demonstrated amazing zero-shot performance on various natural image segmentation tasks. The SAM model consists of three parts: an image encoder, a cue encoder, and a mask generator. The cue encoder receives various cues such as points, bounding boxes, and text to enhance the model's ability to learn features. Compared to traditional segmentation networks, the SAM model has advantages such as strong generalization, strong robustness, and strong feature representation capabilities.
[0004] Despite the development of numerous algorithms for medical image segmentation, existing methods still have many shortcomings. CNN-based segmentation methods, limited by the structure of convolutional kernels, cannot effectively capture long-range dependencies, which is detrimental to medical image segmentation tasks. While Transformer-based methods can effectively improve this, Transformers suffer from slow convergence and weak detail perception, often requiring large datasets to converge to a satisfactory state—a challenge given the scarcity of medical data. Furthermore, existing segmentation methods are trained on very limited data, preventing the models from learning strong representations, leading to poor performance with complex data and a significant decrease in generalization ability. The SAM model demonstrates impressive performance in natural image segmentation tasks, but due to the significant differences between medical and natural images, directly applying the SAM model to medical image tasks yields unsatisfactory, even poor, results. Moreover, as a two-dimensional model, SAM cannot utilize the three-dimensional information in medical images, which is also a disadvantage for medical image segmentation. Summary of the Invention
[0005] Therefore, it is necessary to address the technical problem of poor medical image segmentation performance in existing technologies by proposing a medical image segmentation method, device, computer equipment, and storage medium.
[0006] In a first aspect, a medical image segmentation method is provided, the method comprising:
[0007] Acquiring medical images;
[0008] Image segmentation is performed based on the medical image and the trained medical image segmentation model to obtain the segmentation result of the medical image. The medical image segmentation model includes various segmentation models, feature interaction models corresponding to each segmentation model, and fusion models.
[0009] Secondly, a medical image segmentation method apparatus is provided, the apparatus comprising:
[0010] The acquisition module is used to acquire medical images;
[0011] The image segmentation module is used to perform image segmentation based on the medical image and the trained medical image segmentation model to obtain the segmentation result of the medical image. The medical image segmentation model includes various segmentation models, feature interaction models corresponding to each segmentation model, and fusion models.
[0012] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described medical image segmentation method.
[0013] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described medical image segmentation method.
[0014] The medical image segmentation method proposed in this invention acquires a medical image, and then performs image segmentation based on the medical image and a trained medical image segmentation model to obtain the segmentation result of the medical image. The medical image segmentation model includes various segmentation models, feature interaction models corresponding to each segmentation model, and a fusion model. The feature interaction model enables the various segmentation models to learn from each other, thereby significantly improving the effect of medical image segmentation. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] in:
[0017] Figure 1 This is an application environment diagram of a medical image segmentation method in one embodiment;
[0018] Figure 2 Here is a flowchart of a medical image segmentation method in one embodiment;
[0019] Figure 3 Here is the model structure of a medical image segmentation model for a medical image segmentation method in one embodiment;
[0020] Figure 4 Here is the model structure of the feature interaction model of a medical image segmentation method in one embodiment;
[0021] Figure 5 This is the model structure of a fusion model for a medical image segmentation method in one embodiment;
[0022] Figure 6 This is a comparative result of a medical image segmentation method in one embodiment;
[0023] Figure 7This is another comparative result of the medical image segmentation method in one embodiment;
[0024] Figure 8 This is a visualization result of a medical image segmentation method in one embodiment;
[0025] Figure 9 This is another visualization result of the medical image segmentation method in one embodiment;
[0026] Figure 10 This is a structural block diagram of a medical image segmentation method apparatus in one embodiment;
[0027] Figure 11 This is a structural block diagram of a computer device in one embodiment;
[0028] Figure 12 This is a structural block diagram of a computer device in another embodiment. Detailed Implementation
[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.
[0030] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0032] The medical image segmentation method provided in this invention can be applied to, for example... Figure 1In the application environment, client 110 communicates with server 120 via a network. Server 120 can use the medical image segmentation method proposed in this embodiment to acquire a medical image, and then perform image segmentation based on the medical image and a trained medical image segmentation model to obtain the segmentation result. The medical image segmentation model includes various segmentation models, corresponding feature interaction models for each segmentation model, and a fusion model. The feature interaction model enables the various segmentation models to learn from each other, thereby significantly improving the medical image segmentation effect. Client 110 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. Server 120 can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.
[0033] Please see Figure 2 As shown, Figure 2 A flowchart illustrating a medical image segmentation method according to an embodiment of the present invention includes the following steps:
[0034] Step S101: Acquire medical images;
[0035] Step S102: Perform image segmentation based on the medical image and the trained medical image segmentation model to obtain the segmentation result of the medical image. The medical image segmentation model includes various segmentation models, feature interaction models corresponding to each segmentation model, and fusion models.
[0036] The segmentation model can be U-Net, nnU-Net, UNETR, Swin-UNETR, etc. The feature interaction model is obtained by training the model based on the attention mechanism and convolution. The feature interaction model is used to learn the interaction of features extracted from medical images by the segmentation model. The fusion model can be obtained by training the model based on convolutional neural networks.
[0037] It should be noted that while models like U-Net, nnU-Net, UNETR, and Swin-UNETR have achieved good results due to their respective advantages, they also have their limitations. For example, their inability to capture long-range dependencies and weak detail perception capabilities hinder their development. In recent years, the powerful generalization and emergence capabilities of large models have gradually driven a shift in the development paradigm of deep learning models. However, the massive computational resources and data required to train large models are difficult to achieve in the field of medical image processing. Although the SAM model has achieved great success in natural image processing, the significant differences between natural and medical images mean that directly applying SAM to medical images does not yield ideal results. However, SAM's powerful generalization capabilities give it the potential for application in medical image segmentation.
[0038] The model structure of the medical image segmentation model is as follows: Figure 3 As shown, by constructing a large model cluster, the advantages of each model can be fully utilized to achieve the optimal segmentation result. The large model cluster refers to the various segmentation models and their corresponding feature interaction models. Furthermore, existing basic large models can be rapidly transferred through continuous interaction with proprietary models trained on medical images within the cluster.
[0039] In one embodiment, the segmentation model includes a first segmentation model, a second segmentation model, and a third segmentation model; the feature interaction model includes a first feature interaction model corresponding to the first segmentation model, a second feature interaction model corresponding to the second segmentation model, and a third feature interaction model corresponding to the third segmentation model; the step of performing image segmentation based on the medical image and the trained medical image segmentation model to obtain the segmentation result of the medical image includes:
[0040] Step S201: Input the medical image into the first segmentation model to obtain the first feature output by the first segmentation model;
[0041] Step S202: Input the medical image into the second segmentation model to obtain the second feature output by the second segmentation model;
[0042] Step S203: Input the medical image into the third segmentation model to obtain the third feature output by the third segmentation model;
[0043] Step S204: Perform image segmentation based on the first feature, the second feature, the third feature, the first feature interaction model, the second feature interaction model, the third feature interaction model, and the fusion model to obtain the segmentation result of the medical image.
[0044] The first segmentation model can be the SAM model, the second segmentation model can be the U-Net model, and the third segmentation model can be the UNETR model. These three segmentation models represent the basic large model with strong generalization ability, the CNN model with strong detail awareness ability, and the Transformer model with strong global modeling ability, respectively.
[0045] As an example, the SAM model: To enhance the utilization of medical 3D information by the original SAM model, the processed medical image is first fed into a 3D adapter to extract volume information. Let m be the feature map and F be the output of the 3D adapter. This process can then be represented as:
[0046]
[0047] in, Here, Norm represents the activation function, Conv3D represents the 3D convolutional layer, and the 3D convolutional layer is used to extract 3D information. Then, the output of the 3D adapter... The image is fed into the SAM image encoder for further feature extraction and to generate the encoded output. , That is it.
[0048]
[0049] UNETR Model: The UNETR model is a medical image segmentation model based on the Transformer architecture. Leveraging the Transformer's powerful global modeling capabilities, the UNETR model can effectively capture long-range dependencies in 3D medical data. Define x as the model input feature, then the model output... Represented as:
[0050]
[0051] U-Net Model: The U-Net model effectively captures detailed information through its convolutional structure, and is mainly used here to supplement detailed features. Let the input feature be x, then the model's output... It can be represented as:
[0052]
[0053] In one embodiment, the step of performing image segmentation based on the first feature, the second feature, the third feature, the first feature interaction model, the second feature interaction model, the third feature interaction model, and the fusion model to obtain the segmentation result of the medical image includes:
[0054] Step S301: Determine the fourth feature based on the first feature, the second feature, the third feature, and the first feature interaction model;
[0055] Step S302: Determine the fifth feature based on the first feature, the second feature, the third feature, and the second feature interaction model;
[0056] Step S303: Based on the first feature, the second feature, the third feature, and the interaction model of the third feature, determine the sixth feature;
[0057] Step S304: Perform image segmentation based on the fourth feature, the fifth feature, the sixth feature, and the fusion model to obtain the segmentation result of the medical image.
[0058] In this embodiment, the feature interaction model enables different features to guide each other, compensating for their own shortcomings through continuous interaction with other models. For example, the SAM model, as a basic large model, has strong generalization capabilities, but because it is trained on natural images and is a two-dimensional model, its direct application to medical images is not very effective. However, the Transformer-based UNETR model can effectively capture long-range dependencies in three-dimensional data, and the CNN-based U-NET network can fully focus on detailed information. Therefore, the SAM model can continuously improve its global modeling and detail perception capabilities through interaction with the UNETR and U-Net models via the first feature interaction model. Simultaneously, under the guidance of the UNETR and U-Net models, the SAM model can more efficiently transfer medical image data. Furthermore, with future improvements in hardware, this network framework can continuously expand the scale of the model cluster.
[0059] In one embodiment, the step of determining the fourth feature based on the first feature, the second feature, the third feature, and the first feature interaction model includes:
[0060] Step S401: Based on the linear transformation of the second feature, obtain the first Key and the first Value;
[0061] Step S402: Based on the linear transformation of the third feature, obtain the first Query;
[0062] Step S403: Input the first Key, first Value, and first Query into the first attention model in the first feature interaction model to obtain the seventh feature;
[0063] Step S404: Input the seventh feature into the convolutional layer in the first feature interaction model to obtain the eighth feature output by the convolutional layer. Based on the linear transformation of the eighth feature, obtain the second Key and the second Value.
[0064] Step S405: Based on the linear transformation of the first feature, obtain the second Query;
[0065] Step S406: Input the second Key, second Value, and second Query into the second attention model in the first feature interaction model to obtain the fourth feature.
[0066] Among them, the first attention model and the second attention model are both models that use the attention mechanism. The first key refers to the key (K) in the attention mechanism, the first value refers to the value (V) in the attention mechanism, and the first query refers to the query (Q) in the attention mechanism.
[0067] As an example, the model structure of the first feature interaction model is as follows: Figure 4 As shown.
[0068] In one embodiment, the step of determining the fifth feature based on the first feature, the second feature, the third feature, and the second feature interaction model includes:
[0069] Step S501: Based on the linear transformation of the first feature, obtain the third Key and the third Value;
[0070] Step S502: Based on the linear transformation of the third feature, obtain the third Query;
[0071] Step S503: Input the third Key, third Value, and third Query into the third attention model in the second feature interaction model to obtain the ninth feature;
[0072] Step S504: Input the ninth feature into the convolutional layer in the second feature interaction model to obtain the tenth feature output by the convolutional layer. Based on the linear transformation of the tenth feature, obtain the fourth key and the fourth value.
[0073] Step S505: Based on the linear transformation of the second feature, obtain the fourth Query;
[0074] Step S506: Input the fourth Key, fourth Value, and fourth Query into the fourth attention model in the second feature interaction model to obtain the fifth feature.
[0075] In one embodiment, the step of determining the fifth feature based on the first feature, the second feature, the third feature, and the second feature interaction model includes:
[0076] Step S601: Based on the linear transformation of the first feature, obtain the fifth Key and the fifth Value;
[0077] Step S602: Based on the linear transformation of the second feature, obtain the fifth query;
[0078] Step S603: Input the fifth Key, fifth Value, and fifth Query into the fifth attention model in the third feature interaction model to obtain the eleventh feature;
[0079] Step S604: Input the eleventh feature into the convolutional layer in the third feature interaction model to obtain the twelfth feature output by the convolutional layer. Based on the linear transformation of the twelfth feature, obtain the sixth key and the sixth value.
[0080] Step S605: Based on the linear transformation of the third feature, obtain the sixth Query;
[0081] Step S606: Input the sixth Key, sixth Value, and sixth Query into the sixth attention model in the third feature interaction model to obtain the sixth feature.
[0082] In one embodiment, the step of performing image segmentation based on the fourth feature, the fifth feature, the sixth feature, and the fusion model to obtain the segmentation result of the medical image includes:
[0083] Step S701: Concatenate the fourth feature, the fifth feature, and the sixth feature to obtain the target feature;
[0084] Step S702: Input the target features into the 3D global average pooling layer in the fusion model to obtain a first vector, and input the first vector into the activation layer in the fusion model to obtain a second vector;
[0085] Step S703: Input the target features into the 3D global max pooling layer in the fusion model to obtain the third vector, and input the third vector into the activation layer in the fusion model to obtain the fourth vector;
[0086] Step S704: Add the second and fourth vectors together to obtain the fifth vector, and then activate the fifth vector using an activation function to obtain the probability matrix;
[0087] Step S705: Multiply the probability matrix with the target features to obtain the segmentation result of the medical image.
[0088] As an example, the model structure of the fusion model is as follows: Figure 5 As shown.
[0089] It is important to emphasize that this invention proposes a novel social training model. Based on this training framework, a large model cluster strategy is proposed. Models within the cluster continuously optimize through interaction, ultimately achieving the optimal segmentation result. This invention demonstrates the feasibility of this social network framework using the SAM basic large model, UNETR, and the 3D U-Net model. Furthermore, existing models based on other basic large models such as GPT, CLIP, and other traditional deep models such as U-Net, Mamba, Swin-UNETR, Swin-UNet, and nnFormer are also applicable to this invention. This invention uses a 3D medical image segmentation task as an example; however, this network framework is also applicable to various other segmentation tasks, computer vision tasks, and other fields such as NLP. This invention uses attention modules and convolutional modules to achieve interaction between models; other methods for fusing different feature interactions are also applicable to this invention. This invention emphasizes building model clusters to enrich feature learning; other methods, such as using parallel networks to fuse the output features of different networks, are also applicable to this invention. For the SAM model, this invention uses FacT parameter fine-tuning technology to efficiently transfer the SAM model to medical image segmentation tasks. There are many methods for parameter fine-tuning, such as additive, selective, and reparameterization methods, all of which are applicable to this invention. This invention uses a feature interaction module to guide mutual optimization of the models and a fusion module to supplement features in order to output the optimal segmentation result.
[0090] This invention offers the following advantages: It constructs a large model cluster through this training framework. Within the cluster, segmentation models undergo a "model storm" through a feature interaction model, learning the optimal, most accurate, and comprehensive feature information. Furthermore, a feature fusion module supplements the features to achieve the best output results. This invention redefines the large-scale artificial intelligence model. Compared to a single, general-purpose large model, this invention places greater emphasis on the interaction of existing artificial intelligence models, building a large model cluster based on existing models, and providing a new approach for the future development of artificial intelligence models. Moreover, with future improvements in hardware, theoretically, the medical image segmentation method proposed in this invention can infinitely expand the model cluster size, breaking through the information cocoon of existing models and encompassing multimodal and multidimensional information. Although this method has been validated in medical image segmentation tasks, it is also applicable to other computer vision tasks and other fields such as NLP.
[0091] As an example, the proposed medical image segmentation method was validated on six public prostate datasets and the Sliver07 liver dataset, and compared with classic traditional segmentation methods. The same VIT_L pre-trained weights were used in both the MA-SAM and medical image segmentation methods. Figure 6 The results show the comparison between the present invention and other methods on six prostate datasets. Figure 7 The results show a comparison between this invention and other methods on the Sliver07 liver dataset. The results demonstrate that the proposed medical image segmentation method significantly improves upon existing methods. Furthermore, the segmentation results of various methods are visualized, such as... Figure 8 , Figure 9 As shown in the visualization results, the medical image segmentation method proposed in this invention is significantly superior to existing methods in both the completeness of segmentation and the perception of boundary details.
[0092] The medical image segmentation method proposed in this embodiment acquires a medical image, and then performs image segmentation based on the medical image and a trained medical image segmentation model to obtain the segmentation result of the medical image. The medical image segmentation model includes various segmentation models, feature interaction models corresponding to each segmentation model, and a fusion model. The feature interaction model enables the various segmentation models to learn from each other, thereby significantly improving the effect of medical image segmentation.
[0093] Please see Figure 10 As shown, in one embodiment, a medical image segmentation method apparatus is provided, the apparatus comprising: an acquisition module 10, used to acquire medical images;
[0094] The image segmentation module 20 is used to perform image segmentation based on the medical image and the trained medical image segmentation model to obtain the segmentation result of the medical image. The medical image segmentation model includes various segmentation models, feature interaction models corresponding to each segmentation model, and fusion models.
[0095] The image segmentation module 20 is used to input the medical image into the first segmentation model to obtain a first feature output by the first segmentation model; input the medical image into the second segmentation model to obtain a second feature output by the second segmentation model; input the medical image into the third segmentation model to obtain a third feature output by the third segmentation model; and perform image segmentation based on the first feature, the second feature, the third feature, the first feature interaction model, the second feature interaction model, the third feature interaction model, and the fusion model to obtain the segmentation result of the medical image.
[0096] The image segmentation module 20 is used to determine a fourth feature based on the first feature, the second feature, the third feature, and the first feature interaction model; determine a fifth feature based on the first feature, the second feature, the third feature, and the second feature interaction model; determine a sixth feature based on the first feature, the second feature, the third feature, and the third feature interaction model; and perform image segmentation based on the fourth feature, the fifth feature, the sixth feature, and the fusion model to obtain the segmentation result of the medical image.
[0097] Image segmentation module 20 is used to obtain a first Key and a first Value based on a linear transformation of the second feature; obtain a first Query based on a linear transformation of the third feature; input the first Key, the first Value, and the first Query into a first attention model in a first feature interaction model to obtain a seventh feature; input the seventh feature into a convolutional layer in the first feature interaction model to obtain an eighth feature output by the convolutional layer; obtain a second Key and a second Value based on a linear transformation of the eighth feature; obtain a second Query based on a linear transformation of the first feature; and input the second Key, the second Value, and the second Query into a second attention model in the first feature interaction model to obtain a fourth feature.
[0098] Image segmentation module 20 is configured to: obtain a third Key and a third Value based on a linear transformation of the first feature; obtain a third Query based on a linear transformation of the third feature; input the third Key, the third Value, and the third Query into a third attention model in a second feature interaction model to obtain a ninth feature; input the ninth feature into a convolutional layer in the second feature interaction model to obtain a tenth feature output by the convolutional layer; obtain a fourth Key and a fourth Value based on a linear transformation of the tenth feature; obtain a fourth Query based on a linear transformation of the second feature; and input the fourth Key, the fourth Value, and the fourth Query into a fourth attention model in the second feature interaction model to obtain a fifth feature.
[0099] The image segmentation module 20 is used to obtain a fifth Key and a fifth Value based on a linear transformation of the first feature; to obtain a fifth Query based on a linear transformation of the second feature; to input the fifth Key, fifth Value, and fifth Query into the fifth attention model in the third feature interaction model to obtain an eleventh feature; to input the eleventh feature into the convolutional layer in the third feature interaction model to obtain a twelfth feature output by the convolutional layer; to obtain a sixth Key and a sixth Value based on a linear transformation of the twelfth feature; to obtain a sixth Query based on a linear transformation of the third feature; and to input the sixth Key, sixth Value, and sixth Query into the sixth attention model in the third feature interaction model to obtain a sixth feature.
[0100] The image segmentation module 20 is used to concatenate the fourth, fifth, and sixth features to obtain a target feature; input the target feature into a 3D global average pooling layer in the fusion model to obtain a first vector, and input the first vector into an activation layer in the fusion model to obtain a second vector; input the target feature into a 3D global maximum pooling layer in the fusion model to obtain a third vector, and input the third vector into an activation layer in the fusion model to obtain a fourth vector; add the second and fourth vectors to obtain a fifth vector, and activate the fifth vector using an activation function to obtain a probability matrix; multiply the probability matrix with the target feature to obtain the segmentation result of the medical image.
[0101] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 11 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When executed by the processor, the computer program implements the functions or steps of a medical image segmentation method on the server side.
[0102] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 12As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements the functions or steps of a medical image segmentation method on the client side.
[0103] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, performs the following steps:
[0104] Acquiring medical images;
[0105] Image segmentation is performed based on the medical image and the trained medical image segmentation model to obtain the segmentation result of the medical image. The medical image segmentation model includes various segmentation models, feature interaction models corresponding to each segmentation model, and fusion models.
[0106] The medical image segmentation method proposed in this embodiment acquires a medical image, and then performs image segmentation based on the medical image and a trained medical image segmentation model to obtain the segmentation result of the medical image. The medical image segmentation model includes various segmentation models, feature interaction models corresponding to each segmentation model, and a fusion model. The feature interaction model enables the various segmentation models to learn from each other, thereby significantly improving the effect of medical image segmentation.
[0107] In one embodiment, a computer-readable storage medium is provided that stores a computer program, which, when executed by a processor, performs the following steps:
[0108] Acquiring medical images;
[0109] Image segmentation is performed based on the medical image and the trained medical image segmentation model to obtain the segmentation result of the medical image. The medical image segmentation model includes various segmentation models, feature interaction models corresponding to each segmentation model, and fusion models.
[0110] The medical image segmentation method proposed in this embodiment acquires a medical image, and then performs image segmentation based on the medical image and a trained medical image segmentation model to obtain the segmentation result of the medical image. The medical image segmentation model includes various segmentation models, feature interaction models corresponding to each segmentation model, and a fusion model. The feature interaction model enables the various segmentation models to learn from each other, thereby significantly improving the effect of medical image segmentation.
[0111] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.
[0112] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0113] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0114] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A medical image segmentation method, characterized in that, The medical image segmentation method includes: Acquiring medical images; Image segmentation is performed based on the medical image and the trained medical image segmentation model to obtain the segmentation result of the medical image. The medical image segmentation model includes various segmentation models, feature interaction models corresponding to each segmentation model, and fusion models. The segmentation model includes a first segmentation model, a second segmentation model, and a third segmentation model. The feature interaction model includes a first feature interaction model corresponding to the first segmentation model, a second feature interaction model corresponding to the second segmentation model, and a third feature interaction model corresponding to the third segmentation model. The step of performing image segmentation based on the medical image and the trained medical image segmentation model to obtain the segmentation result of the medical image includes: The medical image is input into the first segmentation model to obtain the first feature output by the first segmentation model; The medical image is input into the second segmentation model to obtain the second feature output by the second segmentation model; The medical image is input into the third segmentation model to obtain the third feature output by the third segmentation model; Image segmentation is performed based on the first feature, the second feature, the third feature, the first feature interaction model, the second feature interaction model, the third feature interaction model, and the fusion model to obtain the segmentation result of the medical image; The step of performing image segmentation based on the first feature, the second feature, the third feature, the first feature interaction model, the second feature interaction model, the third feature interaction model, and the fusion model to obtain the segmentation result of the medical image includes: Based on the first feature, the second feature, the third feature, and the first feature interaction model, a fourth feature is determined; Based on the first feature, the second feature, the third feature, and the interaction model of the second feature, a fifth feature is determined; Based on the first feature, the second feature, the third feature, and the interaction model of the third feature, a sixth feature is determined; Image segmentation is performed based on the fourth feature, the fifth feature, the sixth feature, and the fusion model to obtain the segmentation result of the medical image.
2. The medical image segmentation method according to claim 1, characterized in that, The step of determining the fourth feature based on the first feature, the second feature, the third feature, and the first feature interaction model includes: Based on a linear transformation of the second feature, the first Key and the first Value are obtained; Based on a linear transformation of the third feature, the first query is obtained; The first Key, first Value, and first Query are input into the first attention model in the first feature interaction model to obtain the seventh feature; The seventh feature is input into the convolutional layer in the first feature interaction model to obtain the eighth feature output by the convolutional layer. Based on the linear transformation of the eighth feature, the second key and the second value are obtained. Based on a linear transformation of the first feature, a second query is obtained; The second Key, second Value, and second Query are input into the second attention model in the first feature interaction model to obtain the fourth feature.
3. The medical image segmentation method according to claim 1, characterized in that, The step of determining the fifth feature based on the first feature, the second feature, the third feature, and the second feature interaction model includes: Based on a linear transformation of the first feature, the third Key and the third Value are obtained; Based on a linear transformation of the third feature, a third query is obtained; The third Key, third Value, and third Query are input into the third attention model in the second feature interaction model to obtain the ninth feature; The ninth feature is input into the convolutional layer in the second feature interaction model to obtain the tenth feature output by the convolutional layer. Based on the linear transformation of the tenth feature, the fourth key and the fourth value are obtained. Based on a linear transformation of the second feature, the fourth query is obtained; The fourth Key, fourth Value, and fourth Query are input into the fourth attention model in the second feature interaction model to obtain the fifth feature.
4. The medical image segmentation method according to claim 1, characterized in that, The step of determining the sixth feature based on the first feature, the second feature, the third feature, and the interaction model of the third feature includes: Based on a linear transformation of the first feature, the fifth Key and the fifth Value are obtained; Based on a linear transformation of the second feature, the fifth query is obtained; The fifth Key, fifth Value, and fifth Query are input into the fifth attention model in the third feature interaction model to obtain the eleventh feature; The eleventh feature is input into the convolutional layer in the third feature interaction model to obtain the twelfth feature output by the convolutional layer. Based on the linear transformation of the twelfth feature, the sixth key and the sixth value are obtained. Based on the linear transformation of the third feature, the sixth query is obtained; The sixth key, sixth value, and sixth query are input into the sixth attention model in the third feature interaction model to obtain the sixth feature.
5. The medical image segmentation method according to claim 1, characterized in that, The steps for performing image segmentation based on the fourth feature, the fifth feature, the sixth feature, and the fusion model to obtain the segmentation result of the medical image include: The fourth feature, the fifth feature, and the sixth feature are concatenated to obtain the target feature; The target features are input into the 3D global average pooling layer in the fusion model to obtain a first vector, and the first vector is input into the activation layer of the fusion model to obtain a second vector; The target features are input into the 3D global max pooling layer in the fusion model to obtain the third vector, and the third vector is input into the activation layer of the fusion model to obtain the fourth vector; The second and fourth vectors are added together to obtain the fifth vector, and the fifth vector is activated by an activation function to obtain the probability matrix. The probability matrix is multiplied by the target features to obtain the segmentation result of the medical image.
6. A medical image segmentation device, characterized in that, The medical image segmentation apparatus for performing the medical image segmentation method according to any one of claims 1 to 5 includes: an acquisition module for acquiring medical images; The image segmentation module is used to perform image segmentation based on the medical image and the trained medical image segmentation model to obtain the segmentation result of the medical image. The medical image segmentation model includes various segmentation models, feature interaction models corresponding to each segmentation model, and fusion models.
7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the medical image segmentation method as described in any one of claims 1 to 5.
8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the medical image segmentation method as described in any one of claims 1 to 5.