Medical image segmentation method and device, electronic equipment and storage medium
By training the target image segmentation model and performing image size adjustment and denoising processing, the accuracy problem of deep learning models when processing noise in medical images is solved, achieving higher accuracy and clarity of medical image segmentation, and for medical diagnosis Provides more accurate information.
Patent Information
- Application Number
- CN202411012179.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-12-03
AI Technical Summary
When existing medical image segmentation methods have obvious image noise in original medical images, deep learning models cannot effectively process it, resulting in low accuracy of image segmentation and inability to provide doctors with accurate medical images.
By obtaining medical segmented image samples marked with medical tissue labels, training the target image segmentation model, and performing image size adjustment and denoising processing on the original medical image, denoising using the dependencies between image pixels to improve the clarity of image segmentation degree and accuracy.
It effectively improves the accuracy and clarity of medical image segmentation, can more accurately identify and segment medical tissues, and provide more valuable information for medical diagnosis.
Smart Images

Figure CN119068186A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the fields of artificial intelligence and medical technology, and in particular to a medical image segmentation method, device, electronic device and storage medium. Background Art
[0002] Medical image segmentation can extract clear lesion boundaries, tissue and organ contours from original medical images, assist doctors in pathological diagnosis and improve the accuracy of diagnosis.
[0003] At present, the medical image segmentation method mainly uses deep learning models for image segmentation. However, in actual application scenarios, when there is obvious image noise in the original medical image, the deep learning model cannot effectively process the image noise, resulting in low accuracy of image segmentation, and thus unable to provide accurate medical images for doctors.
[0004] Therefore, how to improve the accuracy of medical image segmentation has become a technical problem that needs to be solved urgently. Summary of the invention
[0005] The main purpose of the embodiments of the present application is to propose a medical image segmentation method, device, electronic device and storage medium, aiming to improve the accuracy of medical image segmentation.
[0006] To achieve the above object, a first aspect of an embodiment of the present application proposes a medical image segmentation method, the method comprising:
[0007] Obtaining medical segmentation image samples annotated with medical tissue labels;
[0008] Obtaining original medical images;
[0009] Training a preset original image segmentation model based on the medical segmentation image samples to obtain a target image segmentation model;
[0010] Adjusting the image size of the original medical image based on a preset target image size to obtain an initial medical image;
[0011] Performing medical tissue segmentation on the initial medical image by using the target image segmentation model to obtain a segmented medical image;
[0012] The segmented medical image is denoised using the target image segmentation model to obtain a target medical image.
[0013] In some embodiments, the original image segmentation model includes an original image segmentation sub-model and an original image denoising sub-model, and the training of the preset original image segmentation model based on the medical segmentation image sample to obtain the target image segmentation model includes:
[0014] Performing data preprocessing on the medical segmentation image samples to obtain target training data; wherein the target training data includes medical tissue sample images and medical tissue segmentation images;
[0015] Performing medical tissue segmentation on the medical tissue sample image by using the original image segmentation sub-model to obtain initial segmented image data;
[0016] The original image segmentation sub-model and the original image denoising sub-model are jointly trained based on the initial segmented image data and the medical tissue segmentation image to obtain the target image segmentation model.
[0017] In some embodiments, the original image segmentation sub-model and the original image denoising sub-model are jointly trained based on the initial segmented image data and the medical tissue segmented image to obtain the target image segmentation model, including:
[0018] Performing model optimization on the original image segmentation sub-model based on the initial segmentation image data and the medical tissue segmentation image to obtain a target image segmentation sub-model;
[0019] Performing noise processing on the medical tissue segmentation image by using the original image denoising sub-model to obtain target noise image data;
[0020] Optimizing the original image denoising sub-model based on the initial segmented image data and the target noisy image data to obtain a target image denoising sub-model;
[0021] The target image segmentation sub-model and the target image denoising sub-model are fused to obtain the target image segmentation model.
[0022] In some embodiments, the optimizing the original image denoising sub-model based on the initial segmented image data and the target noisy image data to obtain the target image denoising sub-model includes:
[0023] Performing prediction error calculation on the initial segmented image data and the target noise image data to obtain prediction error data;
[0024] The parameters of the original image segmentation sub-model are adjusted based on the prediction error data to obtain the target image denoising sub-model.
[0025] In some embodiments, the optimizing the original image segmentation sub-model based on the initial segmentation image data and the medical tissue segmentation image to obtain the target image segmentation sub-model includes:
[0026] Performing loss calculation on the initial segmented image data and the medical tissue segmented image to obtain segmentation loss data;
[0027] The parameters of the original image segmentation sub-model are adjusted based on the segmentation loss data to obtain the target image segmentation sub-model.
[0028] In some embodiments, performing data preprocessing on the medical segmentation image samples to obtain target training data includes:
[0029] Performing data cleaning on the medical segmentation image samples to obtain first image training data;
[0030] Filter out images whose image size is greater than or equal to the target image size from the first image training data to obtain second image training data;
[0031] The images in the second image training data whose image size is larger than the target image size are cropped to obtain the target training data; wherein the image size of the target training data is the target image size.
[0032] In some embodiments, performing denoising on the segmented medical image using the target image segmentation model to obtain a target medical image includes:
[0033] Generate random Gaussian noise data having the target image size;
[0034] The segmented medical image is iteratively denoised based on the random Gaussian noise data to obtain the target medical image.
[0035] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a medical image segmentation device, the device comprising:
[0036] A sample data acquisition module is used to acquire medical segmentation image samples marked with medical tissue labels;
[0037] A medical image acquisition module, used for acquiring original medical images;
[0038] A model training module, used for training a preset original image segmentation model based on the medical segmentation image samples to obtain a target image segmentation model;
[0039] An image resizing module, configured to resize the original medical image based on a preset target image size to obtain an initial medical image;
[0040] A medical tissue segmentation module, used for performing medical tissue segmentation on the initial medical image by using the target image segmentation model to obtain a segmented medical image;
[0041] The denoising module is used to perform denoising processing on the segmented medical image through the target image segmentation model to obtain a target medical image.
[0042] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.
[0043] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.
[0044] The medical image segmentation method, device, electronic device and storage medium proposed in the present application obtain medical segmentation image samples marked with medical tissue labels; obtain original medical images; train a preset original image segmentation model based on the medical segmentation image samples to obtain a target image segmentation model; adjust the image size of the original medical image based on a preset target image size to obtain an initial medical image; perform medical tissue segmentation on the initial medical image through the target image segmentation model to obtain a segmented medical image; perform denoising on the segmented medical image through the target image segmentation model to obtain a target medical image, thereby being able to effectively utilize the dependency between image pixels to process image noise, achieve denoising, and improve the clarity and accuracy of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 is a flow chart of a medical image segmentation method provided in an embodiment of the present application;
[0046] Figure 2 yes Figure 1 Flow chart of step S103 in FIG.
[0047] Figure 3 yes Figure 2 Flow chart of step S201 in FIG.
[0048] Figure 4 yes Figure 2 Flow chart of step S203 in FIG.
[0049] Figure 5 yes Figure 4 Flow chart of step S401 in FIG.
[0050] Figure 6 yes Figure 4 Flow chart of step S403 in FIG.
[0051] Figure 7yes Figure 1 Flow chart of step S106 in FIG.
[0052] Figure 8 is a schematic diagram of the structure of a medical image segmentation device provided in an embodiment of the present application;
[0053] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION
[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0055] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0057] First, some nouns involved in this application are analyzed:
[0058] Artificial intelligence (AI) is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems.
[0059] Medical image segmentation refers to the process of using computer vision and image processing technology to accurately separate and identify tissue structures, organs or lesion areas in medical images. By analyzing and processing medical images such as CT scans, MRIs and X-rays, various structures or abnormal areas in the image are accurately separated so that doctors can study and diagnose diseases in more detail and develop more effective treatment plans. Medical image segmentation is of great significance for accurate medical diagnosis, treatment monitoring and pathological analysis. It provides important support and data basis for clinical decision-making, assists doctors in pathological diagnosis, and improves the accuracy of diagnosis.
[0060] Medical tissue, in medical images such as CT scans, MRI and X-rays, refers to various biological tissue structures and organs in the human body. These tissues include but are not limited to muscles, bones, blood vessels, organs (such as heart, lungs, liver, etc.), nerve tissues and other body structures. In medical image analysis, accurately segmenting and identifying these tissue structures is critical for diagnosing diseases, formulating treatment plans, and studying the relationship between human body structure and function. Through imaging technologies such as CT, X-rays and MRI, doctors can observe and analyze the morphology, density, location and other characteristics of these tissues in detail, thereby helping to make accurate medical decisions and treatment plans.
[0061] Diffusion model,Diffusion model is a technique for noise reduction in image processing, which reduces noise and enhances the continuity and clarity of the image by comparing and adjusting the grayscale value of each pixel with the grayscale value of its neighboring pixels. This method is widely used in digital image processing and is suitable for removing random noise and smoothing images.
[0062] At present, medical image segmentation methods mainly use deep learning models for image segmentation, such as convolutional neural networks, visual Transformer models, etc. However, in actual application scenarios, when there is obvious image noise in the original medical images, the deep learning models cannot effectively process the image noise, resulting in low image segmentation accuracy, and thus unable to provide doctors with accurate medical images.
[0063] Based on this, the embodiments of the present application provide a medical image segmentation method, device, electronic device and storage medium, aiming to improve the accuracy of medical image segmentation.
[0064] The medical image segmentation method, device, electronic device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the medical image segmentation method in the embodiments of the present application is described.
[0065] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0066] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0067] The medical image segmentation method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The medical image segmentation method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the medical image segmentation method, etc., but is not limited to the above forms.
[0068] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.
[0069] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.
[0070] Figure 1 is an optional flowchart of the medical image segmentation method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S106.
[0071] Step S101, obtaining a medical segmentation image sample annotated with a medical tissue label;
[0072] Step S102, obtaining an original medical image;
[0073] Step S103, training a preset original image segmentation model based on the medical segmentation image samples to obtain a target image segmentation model;
[0074] Step S104, adjusting the image size of the original medical image based on a preset target image size to obtain an initial medical image;
[0075] Step S105, performing medical tissue segmentation on the initial medical image using the target image segmentation model to obtain a segmented medical image;
[0076] Step S106, performing denoising processing on the segmented medical image using the target image segmentation model to obtain a target medical image.
[0077] Steps S101 to S106 shown in the embodiment of the present application are as follows: obtaining a medical segmentation image sample labeled with a medical tissue label; obtaining an original medical image; training a preset original image segmentation model based on the medical segmentation image sample to obtain a target image segmentation model; resizing the original medical image based on a preset target image size to obtain an initial medical image; performing medical tissue segmentation on the initial medical image using the target image segmentation model to obtain a segmented medical image; and denoising the segmented medical image using the target image segmentation model to obtain a target medical image, thereby being able to effectively utilize the dependency between image pixels to process image noise, achieve denoising, and improve the clarity and accuracy of image segmentation.
[0078] In step S101 of some embodiments, the medical segmentation image sample data is sample data pre-labeled with medical tissue labels by technicians, and is used to train the original image segmentation model. The medical segmentation image sample data includes: sample images that have not been segmented and pre-segmented segmented images. Specifically, the segmented images correspond to medical tissue labels.
[0079] In some embodiments, the original image segmentation model includes an original image segmentation sub-model and an original image denoising sub-model.
[0080] Among them, the original image segmentation sub-model can be a visual transformer semantic model, and its model form is a Unet network, including but not limited to an encoder, a decoder, a feature fusion module, a (self) attention module, etc. It can learn, understand and represent complex structures in images from fewer images, and is suitable for medical image segmentation tasks, especially performing well on small sample data sets.
[0081] The original image denoising sub-model may be a diffusion model (DDMP), and its model form is a Unet network.
[0082] In step S102 of some embodiments, the original medical image is data that needs to be segmented, and may specifically be medical images such as CT scans, MRI, and X-rays of patients, but is not limited thereto.
[0083] See also Figure 2 In some embodiments, step S103 may include but is not limited to steps S201 to S203:
[0084] Step S201, performing data preprocessing on the medical segmentation image samples to obtain target training data; wherein the target training data includes medical tissue sample images and medical tissue segmentation images;
[0085] Step S202, performing medical tissue segmentation on the medical tissue sample image by using the original image segmentation sub-model to obtain initial segmented image data;
[0086] Step S203 , jointly training the original image segmentation sub-model and the original image denoising sub-model based on the initial segmented image data and the medical tissue segmentation image to obtain a target image segmentation model.
[0087] In steps S201 to S203 shown in the embodiment of the present application, data preprocessing is performed on medical segmentation image samples to obtain target training data, wherein the target training data includes medical tissue sample images and medical tissue segmentation images; medical tissue segmentation is performed on the medical tissue sample images through the original image segmentation sub-model to obtain initial segmentation image data; based on the initial segmentation image data and the medical tissue segmentation image, the original image segmentation sub-model and the original image denoising sub-model are jointly trained to obtain a target image segmentation model, so that the image segmentation sub-model and the image denoising sub-model can be combined with each other, fully utilize the information in the segmented image samples, optimize the segmentation performance of the model, and simultaneously reduce the impact of image noise on the segmentation results, thereby significantly improving the accuracy and robustness of the image segmentation model, more accurately identifying and segmenting medical tissues, and providing more valuable information for medical diagnosis.
[0088] It should be noted that joint training means first fixing the original image denoising sub-model and training the original image segmentation sub-model. After the original image segmentation sub-model is trained, the original image denoising sub-model is trained based on the initial segmented image data obtained by processing the original image segmentation sub-model as a condition. The training process of the two sub-models is executed alternately until the training is completed.
[0089] See also Figure 3 In some embodiments, step S201 may include but is not limited to steps S301 to S303:
[0090] Step S301, performing data cleaning on medical segmentation image samples to obtain first image training data;
[0091] Step S302, filtering out images whose image size is greater than or equal to the target image size from the first image training data to obtain second image training data;
[0092] Step S303, cropping the images in the second image training data whose image size is larger than the target image size to obtain target training data; wherein the image size of the target training data is the target image size.
[0093] In steps S301 to S303 shown in the embodiment of the present application, first image training data is obtained by performing data cleaning on medical segmentation image samples; then images with an image size greater than or equal to a target image size are screened out from the first image training data to obtain second image training data; and images in the second image training data with an image size greater than the target image size are cropped to obtain target training data; wherein the image size of the target training data is the target image size, which can ensure that the image segmentation model can receive image inputs of consistent size and high quality during the training process, so that the image segmentation model can better learn the feature information in the image, improve the accuracy and efficiency of segmentation, and provide more accurate and reliable support for subsequent medical image analysis and diagnosis.
[0094] It should be noted that when performing data cleaning, size screening and size cropping on the medical segmentation image sample data, the same processing process will be performed on the sample image and the segmentation image simultaneously, so as to ensure the correspondence between the sample image and the segmentation image.
[0095] Based on this, the target training data finally obtained includes medical tissue sample images and medical tissue segmentation images, which are data preprocessed, and the medical tissue sample images and the medical tissue segmentation images are in a corresponding relationship.
[0096] In step S301 of some embodiments, data cleaning may include but is not limited to:
[0097] Image quality assessment, identifying and removing blurry, noisy or distorted images;
[0098] Image deduplication, only one copy of repeated images is retained;
[0099] Image augmentation, processing images through operations such as rotation, flipping, translation, cropping, and scaling, thereby enriching the sample data used for training.
[0100] It should be noted that the target image size is preset to ensure that the image size of each input model is uniform, improve the efficiency of model training and reasoning, and simplify the data processing process. Specifically, the target image size can be set to 224*224 pixels or 256*256 pixels, which needs to be selected according to the actual application scenario, but is not limited to this.
[0101] In step S302 of some embodiments, images with an image size greater than or equal to the target image size are screened out from the first image training data to obtain second image training data, thereby avoiding the situation where the image size is too small and insufficient image information is provided for model training, which helps to improve the accuracy and robustness of the model.
[0102] It is understandable that if in the first image training data, an image whose size is smaller than the target image size will be eliminated.
[0103] In step S303 of some embodiments, images in the second image training data whose image size is equal to the target image size remain unchanged; images in the second image training data whose image size is larger than the target image size need to be cropped according to the target size to finally obtain the target training data; wherein the image size of the target training data is the target image size.
[0104] It should be noted that the main medical tissues will be positioned when the size is cut to avoid cutting off part of the image of the medical tissues.
[0105] After step S303 in some embodiments, the medical image segmentation method further includes performing correspondence detection on the medical tissue sample image and the medical tissue segmentation image in the target training data, thereby ensuring that the medical tissue sample image and the medical tissue segmentation image are completely and accurately matched.
[0106] In step S202 of some embodiments, medical tissue segmentation is performed on the medical tissue sample image through the encoder-decoder structure of the original image segmentation sub-model (visual transformer semantic model) to obtain initial segmented image data; specifically, medical tissue segmentation is an image semantic prediction processing process that can identify the contour, range, etc. of the medical tissue in the medical tissue sample image.
[0107] See also Figure 4 In some embodiments, step S203 may include but is not limited to steps S401 to S404:
[0108] Step S401, optimizing the original image segmentation sub-model based on the initial segmentation image data and the medical tissue segmentation image to obtain a target image segmentation sub-model;
[0109] Step S402, performing noise processing on the medical tissue segmentation image by using the original image denoising sub-model to obtain target noise image data;
[0110] Step S403, optimizing the original image denoising sub-model based on the initial segmented image data and the target noisy image data to obtain the target image denoising sub-model;
[0111] Step S404: fuse the target image segmentation sub-model and the target image denoising sub-model to obtain a target image segmentation model.
[0112] In steps S401 to S404 shown in the embodiment of the present application, the medical tissue sample image is segmented by the original image segmentation sub-model, and the model is optimized based on the initial segmentation result and the real medical tissue segmentation image, so as to obtain a more accurate target image segmentation sub-model. At the same time, the medical tissue segmentation image is denoised by the original image denoising sub-model, and the denoising sub-model is optimized by the initial segmented image data and the target noise image data after denoising, so as to obtain a more effective target image denoising sub-model. The target image segmentation model obtained by fusing these two sub-models can not only maintain high precision when segmenting medical tissues, but also reduce the influence of noise on the segmentation results when processing noisy images, thereby significantly improving the accuracy and reliability of medical image analysis.
[0113] It can be understood that the embodiments illustrated in step S202 and step S401 constitute a training process for the original image denoising sub-model.
[0114] See also Figure 5 In some embodiments, step S401 may also include but is not limited to steps S501 to S502:
[0115] Step S501, performing loss calculation on the initial segmented image data and the medical tissue segmented image to obtain segmentation loss data;
[0116] Step S502: adjusting the parameters of the original image segmentation sub-model based on the segmentation loss data to obtain a target image segmentation sub-model.
[0117] In the steps S501 to S502 shown in the embodiment of the present application, the performance of the model in the medical tissue segmentation task can be quantitatively evaluated by calculating the loss of the initial segmented image data and the medical tissue segmentation image. Based on the calculated segmentation loss data, the parameters of the original image segmentation sub-model are adjusted and optimized in a targeted manner, so that the original image segmentation sub-model can more accurately identify and segment the medical tissue, so that the segmentation result output by the model is closer to the real medical tissue structure, providing doctors with more accurate and reliable auxiliary diagnosis information.
[0118] In step S501 of some embodiments, loss calculation is performed on the initial segmentation image data and the medical tissue segmentation image at pixel granularity to obtain segmentation loss data; wherein the loss calculation method may be a cross entropy loss function, a mean square error loss function (MSE), a Euclidean distance loss function, etc., but is not limited thereto.
[0119] In step S502 of some embodiments, during the training process of the original image segmentation sub-model, the parameters of the original image segmentation sub-model are continuously adjusted with the goal of minimizing the segmentation loss data, thereby preventing the original image segmentation sub-model from overfitting or underfitting and improving the robustness of the original image segmentation sub-model.
[0120] In step S402 of some embodiments, the medical tissue segmentation image is denoised by the original image denoising sub-model (diffusion model) to obtain target noise image data, thereby realizing forward diffusion of the diffusion model and simulating the process of the image changing from clear to completely noisy.
[0121] Specifically, the noise addition process is to continuously add Gaussian noise to the clear medical tissue segmentation image. During the noise addition process, the probability density function is gradually updated to approximate the real data distribution. By continuously optimizing the probability density function, the diffusion model can better model the data distribution, thereby improving the performance and effect of generating the diffusion model.
[0122] In some embodiments, the Gaussian noise may be image noise obtained by randomly generating a Gaussian distribution function, and the size of the Gaussian noise is the target image size.
[0123] It should be noted that the embodiments illustrated in step S402 and step S403 constitute a training process for the original image denoising sub-model;
[0124] See also Figure 6 In some embodiments, step S403 includes but is not limited to steps S601 to S602:
[0125] Step S601, performing prediction error calculation on the initial segmented image data and the target noise image data to obtain prediction error data;
[0126] Step S602: adjusting the parameters of the original image segmentation sub-model based on the prediction error data to obtain a target image denoising sub-model.
[0127] In the steps S601 to S602 shown in the embodiment of the present application, the prediction error calculation is performed on the initial segmented image data and the target noise image data to obtain error data that can reflect the prediction accuracy of the model. Based on these prediction error data, the parameters of the original image segmentation sub-model are adjusted to obtain a sub-model specifically used for image denoising, namely the target image denoising sub-model, which can specifically improve the segmentation performance of the model when processing noisy images, reduce the impact of noise on the segmentation results, improve the accuracy and clarity of the segmentation results, and provide doctors with more accurate and reliable auxiliary diagnosis information.
[0128] It can be understood that the training process of the diffusion model is the forward diffusion of the training model, so that the diffusion model can learn to what extent the image needs to be denoised in order to be close to or the same as the noise level of the initial segmented image data. Therefore, it is necessary to calculate the error between the initial segmented image data and the target noise image data, that is, the measured error data, so as to adjust the parameters of the original image segmentation sub-model so that the original image segmentation sub-model can learn the degree of denoising, and then in the model reasoning process, the degree of image denoising can be known.
[0129] In step S601 of some embodiments, the initial segmented image data is injected as a condition into the (self) attention mechanism of the diffusion model, so that the prediction error calculation can be performed on the initial segmented image data and the target noise image data.
[0130] Specifically, the method for calculating the prediction error by the (self) attention mechanism may be a cross entropy loss function, a mean square error loss function (MSE), a negative log-likelihood loss, a KL divergence, etc., but is not limited thereto.
[0131] In step S602 of some embodiments, during the training process of the original image denoising sub-model (diffusion model), with the goal of minimizing the prediction error data, the parameters of the original image denoising sub-model (diffusion model) are continuously adjusted through weighted sum, gradient descent, back propagation and other methods, so as to prevent the original image denoising sub-model (diffusion model) from overfitting or underfitting and improve the robustness of the original image denoising sub-model (diffusion model).
[0132] In step S404 of some embodiments, the trained target image segmentation sub-model and the target image denoising sub-model are fused, and the sequence and data flow between the target image segmentation sub-model and the target image denoising sub-model are sorted out, so that when the target image segmentation model is used, image segmentation can be performed according to accurate execution logic, thereby improving the accuracy of medical image segmentation and denoising, and enhancing the robustness of the target image segmentation model.
[0133] In some embodiments, step S104 may include but is not limited to the following steps:
[0134] Screening the original medical image, if the image size of the original medical image is smaller than the target image size, feedback of image acquisition error information is given to prompt the technician to re-acquire the image;
[0135] If the image size of the original medical image is larger than the target image size, the original medical image is preliminarily positioned to obtain the pixel position information of the medical tissue;
[0136] The original medical image is cropped based on the medical tissue pixel position information to obtain an initial medical image, wherein the image size of the initial medical image is the target image size;
[0137] If the image size of the original medical image is equal to the target image size, the original medical image is determined as the initial medical image.
[0138] It is understandable that during the training process of the original image segmentation model, the image size of the training samples is unified to the target image size. Therefore, in the actual reasoning process, the size of the original medical image also needs to be adjusted to the target image size.
[0139] Similarly, when the original medical image is cropped, the main medical tissues are also located to avoid cropping part of the image of the medical tissues.
[0140] It can be understood that the target image segmentation model is obtained by fusion of the target image segmentation sub-model and the target image denoising sub-model. Therefore, it can be considered that the target image segmentation model includes the target image segmentation sub-model and the target image denoising sub-model.
[0141] In step S105 of some embodiments, the initial medical image is segmented into medical tissues using the target image segmentation sub-model in the target image segmentation model to obtain a segmented medical image;
[0142] In step S106 of some embodiments, the segmented medical image is denoised using a target image denoising sub-model in the target image segmentation model to obtain a target medical image.
[0143] See also Figure 7 In some embodiments, step S106 may include but is not limited to steps S701 to S702:
[0144] Step S701, generating random Gaussian noise data with a target image size;
[0145] Step S702, iteratively denoising the segmented medical image based on random Gaussian noise data to obtain a target medical image.
[0146] In steps S701 to S702 shown in the embodiment of the present application, by utilizing the reverse diffusion process of the target image denoising sub-model (diffusion model), the segmented medical image is iteratively denoised based on random Gaussian noise data to obtain a target medical image, which can improve the model calculation effect and generate a high-quality target medical image while maintaining image details, thereby providing doctors with more accurate and reliable auxiliary diagnosis information.
[0147] It should be noted that the training process of the original image denoising sub-model is forward diffusion. In the actual application process, the processing process of the target image denoising sub-model is the inverse process of forward diffusion, thereby achieving denoising and restoring a clear image from the noise.
[0148] It should be noted that the denoising process of the segmented medical image is performed iteratively. Therefore, in each iteration, the random Gaussian noise data used is different and randomly generated. Through multiple iterations of denoising, a more stable and accurate target medical image is generated. Specifically, the number of iterative denoising is determined based on the training effect of the target image segmentation model.
[0149] In some embodiments, the target medical image contains information about the accurate boundaries and regions of specific organs or tissues, such as tumors, organs or blood vessels, etc. The target medical image can help doctors observe the patient's anatomical structure more clearly, diagnose lesions, assess the severity of the disease, and develop personalized treatment plans.
[0150] In addition, the medical image segmentation method provided in the embodiments of the present application also contributes to the development of computer-aided diagnosis systems and improves the automatic analysis and understanding capabilities of medical images.
[0151] The medical image segmentation method provided in the embodiment of the present application extracts spatial relationships and contextual information from the initial medical image through a visual transformer semantic model to obtain deep semantic features, extracts abstract features from the dependency relationships in a mask of a large amount of noise through a diffusion model, and fuses the deep semantic features and abstract features to achieve semantic segmentation of the medical image, thereby improving the accuracy of medical image segmentation.
[0152] See also Figure 8 The present application also provides a medical image segmentation device, which can implement the above-mentioned medical image segmentation method, and the device includes:
[0153] The sample data acquisition module 801 is used to acquire medical segmentation image samples marked with medical tissue labels;
[0154] A medical image acquisition module 802 is used to acquire an original medical image;
[0155] The model training module 803 is used to train the preset original image segmentation model based on the medical segmentation image samples to obtain the target image segmentation model;
[0156] An image resizing module 804 is used to resize the original medical image based on a preset target image size to obtain an initial medical image;
[0157] The medical tissue segmentation module 805 is used to perform medical tissue segmentation on the initial medical image by using the target image segmentation model to obtain a segmented medical image;
[0158] The denoising module 806 is used to perform denoising processing on the segmented medical image through the target image segmentation model to obtain a target medical image.
[0159] In the model training module 803 of some embodiments, the model training module 803 further includes:
[0160] A data preprocessing submodule is used to perform data preprocessing on medical segmentation image samples to obtain target training data; wherein the target training data includes medical tissue sample images and medical tissue segmentation images;
[0161] The segmentation prediction submodule is used to perform medical tissue segmentation on the medical tissue sample image through the original image segmentation submodel to obtain initial segmentation image data;
[0162] The joint training submodule is used to jointly train the original image segmentation submodel and the original image denoising submodel based on the initial segmentation image data and the medical tissue segmentation image to obtain a target image segmentation model.
[0163] In the data preprocessing submodule of some embodiments, the data preprocessing submodule further includes:
[0164] A data cleaning unit, used for performing data cleaning on the medical segmentation image samples to obtain first image training data;
[0165] An image size screening unit, used to screen images whose image sizes are greater than or equal to the target image size from the first image training data to obtain second image training data;
[0166] The size cropping unit is used to crop the image in the second image training data whose image size is larger than the target image size to obtain the target training data; wherein the image size of the target training data is the target image size.
[0167] In some embodiments of the joint training submodule, the joint training submodule further includes:
[0168] A first model optimization unit is used to optimize the original image segmentation sub-model based on the initial segmentation image data and the medical tissue segmentation image to obtain a target image segmentation sub-model;
[0169] A noise processing unit, used for performing noise processing on the medical tissue segmentation image through the original image denoising sub-model to obtain target noise image data;
[0170] A second model optimization unit is used to optimize the original image denoising sub-model based on the initial segmented image data and the target noisy image data to obtain the target image denoising sub-model;
[0171] The model fusion unit is used to fuse the target image segmentation sub-model and the target image denoising sub-model to obtain the target image segmentation model.
[0172] In the first model optimization unit of some embodiments, the first model optimization unit further includes:
[0173] A loss calculation subunit, used for performing loss calculation on the initial segmentation image data and the medical tissue segmentation image to obtain segmentation loss data;
[0174] The first parameter adjustment sub-unit is used to adjust the parameters of the original image segmentation sub-model based on the segmentation loss data to obtain the target image segmentation sub-model.
[0175] In the second model optimization unit of some embodiments, the second model optimization unit further includes:
[0176] The error calculation subunit is used to calculate the prediction error of the initial segmented image data and the target noise image data to obtain the prediction error data;
[0177] The second parameter adjustment sub-unit is used to adjust the parameters of the original image segmentation sub-model based on the prediction error data to obtain the target image denoising sub-model.
[0178] In some embodiments of the denoising module 806, the denoising module 806 further includes:
[0179] Gaussian noise generation submodule, used to generate random Gaussian noise data with target image size;
[0180] The iterative denoising submodule is used to iteratively denoise the segmented medical image based on random Gaussian noise data to obtain a target medical image.
[0181] The specific implementation of the medical image segmentation device is basically the same as the specific implementation of the above-mentioned medical image segmentation method, and will not be repeated here.
[0182] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned medical image segmentation method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.
[0183] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:
[0184] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;
[0185] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the medical image segmentation method of the embodiment of this application;
[0186] Input / output interface 903, used to implement information input and output;
[0187] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WI FI, Bluetooth, etc.);
[0188] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);
[0189] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .
[0190] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned medical image segmentation method is implemented.
[0191] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0192] The medical image segmentation method, device, electronic device and storage medium provided in the embodiments of the present application obtain medical segmentation image samples marked with medical tissue labels; obtain an original medical image; train a preset original image segmentation model based on the medical segmentation image samples to obtain a target image segmentation model; resize the original medical image based on a preset target image size to obtain an initial medical image; perform medical tissue segmentation on the initial medical image using the target image segmentation model to obtain a segmented medical image; and denoise the segmented medical image using the target image segmentation model to obtain a target medical image, thereby being able to effectively utilize the dependency between image pixels to process image noise, achieve denoising, and improve the clarity and accuracy of image segmentation.
[0193] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0194] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.
[0195] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0196] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.
[0197] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0198] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0199] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0200] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0201] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0202] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.
[0203] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.
Claims
1. A medical image segmentation method, characterized in that: The method comprises: Obtaining medical segmentation image samples annotated with medical tissue labels; Obtaining original medical images; Training a preset original image segmentation model based on the medical segmentation image samples to obtain a target image segmentation model; Adjusting the image size of the original medical image based on a preset target image size to obtain an initial medical image; Performing medical tissue segmentation on the initial medical image by using the target image segmentation model to obtain a segmented medical image; The segmented medical image is denoised using the target image segmentation model to obtain a target medical image.
2. The method according to claim 1, characterized in that The original image segmentation model includes an original image segmentation sub-model and an original image denoising sub-model. The preset original image segmentation model is trained based on the medical segmentation image sample to obtain a target image segmentation model, including: Performing data preprocessing on the medical segmentation image samples to obtain target training data; wherein the target training data includes medical tissue sample images and medical tissue segmentation images; Performing medical tissue segmentation on the medical tissue sample image by using the original image segmentation sub-model to obtain initial segmented image data; The original image segmentation sub-model and the original image denoising sub-model are jointly trained based on the initial segmented image data and the medical tissue segmentation image to obtain the target image segmentation model.
3. The method according to claim 2, characterized in that The method of jointly training the original image segmentation sub-model and the original image denoising sub-model based on the initial segmented image data and the medical tissue segmented image to obtain the target image segmentation model includes: Performing model optimization on the original image segmentation sub-model based on the initial segmentation image data and the medical tissue segmentation image to obtain a target image segmentation sub-model; Performing noise processing on the medical tissue segmentation image by using the original image denoising sub-model to obtain target noise image data; Optimizing the original image denoising sub-model based on the initial segmented image data and the target noisy image data to obtain a target image denoising sub-model; The target image segmentation sub-model and the target image denoising sub-model are fused to obtain the target image segmentation model.
4. The method according to claim 3, characterized in that The step of optimizing the original image denoising sub-model based on the initial segmented image data and the target noisy image data to obtain the target image denoising sub-model includes: Performing prediction error calculation on the initial segmented image data and the target noise image data to obtain prediction error data; The parameters of the original image segmentation sub-model are adjusted based on the prediction error data to obtain the target image denoising sub-model.
5. The method according to claim 3, characterized in that: The step of optimizing the original image segmentation sub-model based on the initial segmentation image data and the medical tissue segmentation image to obtain a target image segmentation sub-model includes: Performing loss calculation on the initial segmented image data and the medical tissue segmented image to obtain segmentation loss data; The parameters of the original image segmentation sub-model are adjusted based on the segmentation loss data to obtain the target image segmentation sub-model.
6. The method according to claim 2, characterized in that The step of performing data preprocessing on the medical segmentation image samples to obtain target training data includes: Performing data cleaning on the medical segmentation image samples to obtain first image training data; Filter out images whose image size is greater than or equal to the target image size from the first image training data to obtain second image training data; The images in the second image training data whose image size is larger than the target image size are cropped to obtain the target training data; wherein the image size of the target training data is the target image size.
7. The method according to any one of claims 1 to 6, characterized in that: The step of performing denoising on the segmented medical image by using the target image segmentation model to obtain a target medical image comprises: Generate random Gaussian noise data having the target image size; The segmented medical image is iteratively denoised based on the random Gaussian noise data to obtain the target medical image.
8. A medical image segmentation device, characterized in that: The device comprises: A sample data acquisition module is used to acquire medical segmentation image samples marked with medical tissue labels; A medical image acquisition module, used for acquiring original medical images; A model training module, used for training a preset original image segmentation model based on the medical segmentation image samples to obtain a target image segmentation model; An image resizing module, configured to resize the original medical image based on a preset target image size to obtain an initial medical image; A medical tissue segmentation module, used for performing medical tissue segmentation on the initial medical image by using the target image segmentation model to obtain a segmented medical image; The denoising module is used to perform denoising processing on the segmented medical image through the target image segmentation model to obtain a target medical image.
9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the medical image segmentation method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the medical image segmentation method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Medical image feature map denoising method and system based on diffusion model
CN116630200A
Subthalamic nucleus segmentation method and model construction method for magnetic resonance image
CN116758098A
Cited By
Model training method, low-dose PET (positron emission tomography) image segmentation method without CT (computed tomography) dependence and related equipment
CN121170474A
