Organ boundary sketching model training method, electronic equipment, medium and sketching method

Abstract

The invention relates to the technical field of medical image processing, and particularly provides an organ boundary delineation model training method, an electronic device, a medium and a delineation method, an organ boundary delineation model comprises a plurality of reverse denoising networks, and the model training method comprises the following steps: obtaining an organ label noise image based on a multi-modal original image; splicing the original image of each mode with the organ tag noise image and the time step number to obtain a to-be-denoised image corresponding to each mode, and inputting the to-be-denoised image corresponding to each mode into different reverse denoising networks for denoising to obtain a denoised image of each mode; training the reverse de-noising network based on the de-noised images of all modalities to obtain a trained organ boundary sketching model; according to the method, the reverse denoising network is trained in combination with the original images of different modalities, so that the anti-interference capability of the model to noise is effectively improved when organ boundary sketching is performed on the multi-modal medical image, and the accuracy and robustness of the model are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to an organ boundary delineation model training method, electronic equipment, medium and delineation method. Background Art

[0002] In radiotherapy, accurate organ delineation is crucial for developing personalized treatment plans, which helps to accurately irradiate radiation while reducing damage to healthy tissues. Automatic delineation technology for medical images can improve the accuracy and efficiency of delineation, thereby optimizing treatment plans and improving treatment effects and safety. When delineating organ boundaries, different imaging modes have significant differences in imaging principles, resolution, contrast, noise type, grayscale value, and texture characteristics. Different modality images have different imaging effects on different human organs and tissues. For example, CT (Computed Tomography) can clearly display bones and calcified areas, and MR (Magnetic Resonance) has high imaging contrast for soft tissues such as the nervous system and cardiovascular system; enhanced CT has clearer imaging of blood vessels and abdominal organs than plain CT; effectively integrating multiple modality medical image data to form a more comprehensive and accurate understanding of organs and tissues is the key to improving the accuracy of organ delineation.

[0003] In order to solve the problem of unclear organ boundaries under a single modality, multimodal medical images are usually used for registration and fusion to realize ROI (Region of Interest) boundary detection. There are many types of noise and artifacts in images of different modalities, which affect the quality of image fusion. The traditional boundary delineation methods commonly used for ROI (Region of Interest) include edge detection algorithms based on Laplacian operators and threshold segmentation algorithms based on grayscale histograms. These methods have low computational complexity and are easy to implement, but they have some shortcomings, such as sensitivity to noise and high requirements for image quality, especially when the image background is complex or the grayscale difference between the target and the background is small, resulting in poor accuracy of the delineation results obtained by traditional boundary delineation methods for images of different modalities, and poor delineation effects.

[0004] Accordingly, the art needs a new organ boundary delineation model training scheme to solve the above problems. Summary of the invention

[0005] In order to overcome the above defects, the present invention is proposed to solve or at least partially solve the technical problem that the existing organ boundary delineation model is sensitive to noise, resulting in poor accuracy of delineation results obtained after images of different modalities are input into the organ boundary delineation model.

[0006] In a first aspect, a method for training an organ boundary delineation model is provided, wherein the organ boundary delineation model includes a plurality of inverse denoising networks, and the model training method includes:

[0007] Obtain multi-modal original images;

[0008] Obtaining an organ label noise image based on the multimodal original image;

[0009] The original image of each modality is spliced ​​with the organ label noise image and the time step number to obtain the image to be denoised corresponding to each modality, and the image to be denoised corresponding to each modality is input into different inverse denoising networks for denoising to obtain the denoised image of each modality; wherein the initial value of the time step number is determined based on the time step number corresponding to the organ label noise image;

[0010] The inverse denoising network is iteratively trained based on the denoised images of all modalities to obtain a trained organ boundary delineation model; wherein the number of time steps corresponding to each iterative training is obtained by gradually decreasing the number of time steps corresponding to the previous iterative training.

[0011] In a technical solution of the above-mentioned organ boundary delineation model training method, the step of obtaining the organ label noise image based on the multimodal original image includes:

[0012] Performing registration and fusion on the multi-modal original images to obtain registered and fused images;

[0013] Delineating the organ boundaries on the registered and fused images to obtain a true label image;

[0014] An organ label noise image is obtained based on the true label image.

[0015] In a technical solution of the above-mentioned organ boundary delineation model training method, obtaining the organ label noise image based on the real label image is a forward noise addition step, and the forward noise addition step includes:

[0016] Within a preset total time step, gradually add noise to the real label image until the preset total time step is reached and stop adding noise to obtain an organ label noise image; wherein the time step corresponding to obtaining the organ label noise image is the preset total time step;

[0017] The method further comprises:

[0018] The preset total number of time steps is used as the initial value of the number of time steps.

[0019] In a technical solution of the above-mentioned organ boundary delineation model training method, the inverse denoising network is trained based on the denoised images of all modalities to obtain a trained organ boundary delineation model, including:

[0020] Performing weighted processing on the denoised images of all modalities to obtain denoised organ boundary delineation images;

[0021] The inverse denoising network is iteratively trained based on the denoised organ boundary delineation image and the real label image to obtain a trained organ boundary delineation model.

[0022] In a technical solution of the above-mentioned organ boundary delineation model training method, each of the inverse denoising networks includes an encoder, a decoder and an attention enhancement module;

[0023] Inputting the to-be-denoised images corresponding to each modality into different inverse denoising networks for denoising respectively to obtain denoised images of each modality, including:

[0024] Inputting the to-be-denoised image corresponding to each modality into the encoder of the corresponding inverse denoising network respectively to obtain an encoded visual feature image of each modality;

[0025] Obtaining a detailed feature image of each modality greater than a preset frequency threshold based on the original image of each modality;

[0026] Inputting the encoded visual feature image of each modality, the detail feature image of each modality and the organ label noise image into the attention enhancement module of the corresponding inverse denoising network to obtain an attention enhanced feature image of each modality;

[0027] The attention-enhanced feature image of each modality is input into the decoder of the corresponding inverse denoising network to obtain the denoised image of each modality.

[0028] In a technical solution of the above-mentioned organ boundary delineation model training method, iteratively training the inverse denoising network based on the denoised organ boundary delineation image and the real label image includes:

[0029] Calculating a loss function based on the denoised organ boundary delineation image and the true label image;

[0030] Updating parameters of the inverse denoising network based on the optimization algorithm and the loss function;

[0031] The denoised organ boundary outline image is input into the inverse denoising network after parameter update to obtain an updated denoised organ boundary outline image and a loss function, and the inverse denoising network is iteratively trained with the updated denoised organ boundary outline image and the loss function until the inverse denoising network converges to obtain a trained organ boundary outline model; wherein each iterative training uses the denoised organ boundary outline image obtained in the previous training as the input of the inverse denoising network.

[0032] In a technical solution of the above-mentioned organ boundary delineation model training method, before splicing the original image of each modality with the organ label noise image and the time step to obtain the image to be denoised corresponding to each modality, it includes:

[0033] Normalize the original images of each modality;

[0034] The normalized original image of each modality is concatenated with the organ label noise image and the time step to obtain the image to be denoised corresponding to each modality.

[0035] In a second aspect, a method for delineating an organ boundary is provided, comprising:

[0036] Acquire an image of the organ whose boundary is to be delineated;

[0037] Inputting the image of the organ boundary to be delineated into a trained organ boundary delineation model to obtain an organ boundary delineation result;

[0038] Wherein, the organ boundary delineation model is trained based on the organ boundary delineation model training method described in any one of the technical solutions of the above-mentioned organ boundary delineation model training method.

[0039] According to a third aspect, an electronic device is provided, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any one of the technical solutions of the above-mentioned organ boundary delineation model training method is implemented.

[0040] In a fourth aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, wherein the program codes are suitable for being loaded and run by a processor to execute the method described in any one of the technical solutions of the above-mentioned organ boundary delineation model training method.

[0041] The above one or more technical solutions of the present invention have at least one or more of the following beneficial effects: In the technical solution of the organ boundary delineation model training method provided by the present invention, the organ boundary delineation model includes multiple inverse denoising networks, and the model training method includes: obtaining multi-modal original images; obtaining organ label noise images based on the multi-modal original images; splicing the original images of each modality with the organ label noise images and the time steps to obtain the image to be denoised corresponding to each modality, and inputting the image to be denoised corresponding to each modality into different inverse denoising networks for denoising to obtain the denoised image of each modality; training the inverse denoising network based on the denoised images of all modalities to obtain a trained organ boundary delineation model; the present invention trains the inverse denoising network in combination with original images of different modalities, effectively improving the model's anti-interference ability to noise when delineating organ boundaries in multi-modal medical images, and improving the accuracy and robustness of the organ boundary delineation model. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The disclosure of the present invention will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. Among them:

[0043] Figure 1 is a flow chart of main steps of a method for training an organ boundary delineation model according to an embodiment of the present invention;

[0044] Figure 2 is a schematic diagram of training an inverse denoising network according to an embodiment of the present invention;

[0045] Figure 3 It is a schematic flow chart of the main steps of selecting CT modality original images and MR modality original images to train a model according to an embodiment of the present invention;

[0046] Figure 4 is a schematic flow chart of main steps of a method for delineating organ boundaries according to an embodiment of the present invention;

[0047] Figure 5 It is a schematic diagram of the main structure of an electronic device according to an embodiment of the present invention.

[0048] Reference numerals:

[0049] 51: memory; 52: processor. DETAILED DESCRIPTION

[0050] Some embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0051] In the description of the present invention, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, and memories, and may also include software parts, such as program codes, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware, or a combination of the two. Computer-readable storage media include any suitable medium that can store program codes, such as a magnetic disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, and the like. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B", and may include only A, only B, or A and B. The singular terms "one" and "the" may also include plural forms.

[0052] The relevant user personal information that may be involved in the various embodiments of the present invention is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.

[0053] The user personal information processed by the present invention will vary depending on the specific product / service scenario, and shall be based on the specific scenario in which the user uses the product / service, and may involve the user's account information, device information, driving information, vehicle information or other related information. The present invention will treat the user's personal information and its processing with a high degree of diligence.

[0054] The present invention attaches great importance to the security of user personal information and has adopted reasonable and feasible security protection measures that meet industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.

[0055] See attached Figure 1 , Figure 1 FIG. 1 is a flow chart of the main steps of a method for training an organ boundary delineation model according to an embodiment of the present invention. The organ boundary delineation model includes multiple inverse denoising networks, such as Figure 1 As shown, the organ boundary delineation model training method in the embodiment of the present invention mainly includes the following steps S101 to S104.

[0056] Step S101: Acquire a multi-modal original image;

[0057] In this embodiment, the multi-modal original images include original medical images of different modalities acquired by imaging technologies such as CT (Computed Tomography) and MR (Magnetic Resonance).

[0058] In this embodiment, after original medical images of different modalities are acquired, normalization processing is performed on the original medical images of each modality.

[0059] In one embodiment, a grayscale normalization processing method is used for the original medical images of each modality; due to factors such as imaging equipment, scanning parameters, and individual differences, the grayscale value distribution of medical images of different modalities may be different, so a grayscale normalization processing method is used to map the grayscale value of the original medical image of each modality to the same preset grayscale value range, which is conducive to the iterative training of subsequent models. It should be noted that the grayscale normalization processing method used for the original medical images of each modality is only an example of an implementation method and does not constitute a limitation on the implementation methods of the present invention.

[0060] Step S102: obtaining an organ label noise image based on the multimodal original image;

[0061] In this embodiment, obtaining an organ label noise image based on a multimodal original image includes:

[0062] Register and fuse the original multi-modal images to obtain the registered and fused images;

[0063] Delineate the organ boundaries on the registered and fused images to obtain the true label images corresponding to the multi-modalities;

[0064] Obtain an organ label noisy image based on the true label image.

[0065] When processing medical images, multimodal medical image registration and fusion is a key step. Medical images of different modalities acquired from different imaging techniques (such as CT, MRI, etc.) are integrated into a unified image space to provide more comprehensive diagnostic information.

[0066] In one embodiment, professional medical imaging software is used to align images of different modalities into the same spatial coordinate system so that the positions of the same anatomical structures in different images are consistent, thereby obtaining a registered multi-modal image; after registration, a fusion method such as a weighted average method or a principal component analysis method is used to integrate the registered multi-modal image information to obtain a registered and fused image.

[0067] In one implementation, the registered and fused images are input into professional medical imaging software for rough outlining, and calibration, verification and modification are performed based on the rough outlining to obtain a true labeled image.

[0068] Optionally, 3D Slicer software is used as professional medical imaging software for registration and delineation. It should be noted that using 3D Slicer as professional medical imaging software for registration and delineation is only an example of an implementation method and does not constitute a limitation on the implementation methods of the present invention. In actual applications, different professional medical imaging software can be selected for registration and delineation.

[0069] In this embodiment, the method further includes a forward denoising step, and obtaining an organ label noise image based on a real label image is the forward denoising step, and the forward denoising step specifically includes:

[0070] Within a preset total number of time steps, noise is gradually added to the real label image until the preset total number of time steps is reached and the noise addition is stopped to obtain an organ label noise image; wherein the time step corresponding to obtaining the organ label noise image is the preset total number of time steps.

[0071] In one embodiment, a total number of time steps is preset, and noise is gradually added based on the real label image. With each additional time step, a layer of Gaussian noise that conforms to the standard normal distribution is added to the real label image. When the preset total number of time steps is reached, the noise addition is stopped to obtain an organ label noise image.

[0072] The organ label noisy image is:

[0073]

[0074] Among them, x t is the organ label noise image; x0 is the real label image; t is the time step; α i =1-β i , β i is the noise weight, β i As the number of time steps increases, it gradually increases; ∈ t represents the noise vector randomly sampled from the standard normal distribution at t time steps, ∈ t ~N(0,I), I is the unit matrix; when the number of time steps is initially increased, β i The smaller the value, the less noise is added. As the number of time steps increases, β i As the value increases, more and more noise is added.

[0075] The probability density function of Gaussian noise has a normal distribution feature, which is conducive to calculating the impact of noise on data, calculating the loss function of the model, and optimizing model parameters; Gaussian noise is continuous, and the change in value is relatively smooth. The intensity and distribution range of Gaussian noise can be controlled by adjusting the mean, standard deviation and other parameters of the Gaussian distribution; in the process of adding noise, the continuity and smoothness of Gaussian noise are easy to be learned and modeled by the model. In the process of inverse denoising, the model is more likely to predict and remove Gaussian noise, thereby restoring the original data.

[0076] Step S103: splicing the original image of each modality with the organ label noise image and the time step (Time Step) to obtain the image to be denoised corresponding to each modality, and inputting the image to be denoised corresponding to each modality into different inverse denoising networks for denoising to obtain the denoised image of each modality;

[0077] In this embodiment, see the attached Figure 2 , Figure 2 FIG. 1 is a schematic diagram of training a reverse denoising network according to an embodiment of the present invention. Figure 2 As shown, the original image of each modality is used as prior information and spliced ​​with the organ label noise image and the time step number to obtain the spliced ​​feature image corresponding to each modality, and the spliced ​​feature image corresponding to each modality is input into different inverse denoising networks through different paths; wherein, the spliced ​​feature image corresponding to each modality is the image to be denoised corresponding to each modality.

[0078] In one embodiment, the initial value of the time step is determined based on the time step of obtaining the organ label noise image, and the time step of the organ label noise image obtained after the forward noise addition is completed is the preset total time step. Therefore, the method further includes:

[0079] The preset total number of time steps is used as the initial value of the time step number.

[0080] In this embodiment, each inverse denoising network includes an encoder, a decoder, and an attention enhancement module;

[0081] The denoised images corresponding to each modality are input into different inverse denoising networks for denoising respectively, and the denoised images of each modality are obtained, including:

[0082] The denoised images corresponding to each modality are input into the encoder of the corresponding inverse denoising network to obtain the encoded visual feature image of each modality;

[0083] Obtaining a detailed feature image of each modality greater than a preset frequency threshold based on the original image of each modality;

[0084] Input the encoded visual feature image of each modality, the detail feature image of each modality, and the organ label noise image into the attention enhancement module of the corresponding inverse denoising network to obtain the attention enhanced feature image of each modality;

[0085] The attention-enhanced feature image of each modality is input into the decoder of the corresponding inverse denoising network to obtain the denoised image of each modality.

[0086] In one embodiment, the encoder includes a convolution layer and a pooling layer, etc. The convolution layer detects local features at different positions of the image, and the pooling layer downsamples the local features to reduce the data dimension, so that the size of the feature image is gradually reduced and key information is retained; the encoder is used to extract features of the image to be denoised corresponding to each modality, capture the image's edge features, texture features, shape features, noise distribution and other information, and obtain the encoded visual feature image of each modality.

[0087] In one embodiment, the decoder includes a deconvolution layer and an upsampling layer, etc., which upsample and reconstruct the feature map of the encoded visual feature image through the deconvolution layer and the upsampling layer, so that the size of the feature image is gradually restored, and finally an output image with a size close to that of the input image is generated; the decoder is used to remap the low-dimensional feature image back to the spatial dimension of the encoder feature, extract higher-level features, improve the quality and accuracy of image generation, and finally obtain a denoised image of each modality.

[0088] In one embodiment, a skip connection is used between the encoder and the decoder to ensure that the semantic information in the encoding process is supplemented in the process of decoding and restoring the image information; when the skip connection is used, an attention enhancement module is used; the input features of the attention enhancement module are the encoded visual feature image of each modality, the detail feature image of each modality, and the organ label noise image.

[0089] Optionally, an EAE boundary attention enhancement module can be selected during the jump connection, and the attention enhancement feature image of each modality can be obtained through the EAE boundary attention enhancement module. It should be noted that the type of attention enhancement module selected during the jump connection here is only an exemplary description and does not constitute a limitation on the implementation of the present invention. In actual applications, different selections can be made as needed, as long as the attention enhancement feature image of each modality can be obtained.

[0090] Optionally, the original image of each modality is input into one of the Laplacian operator, Prewitt operator or Sobel operator for feature extraction to obtain a high-frequency detail feature image, wherein the high-frequency detail feature image is a detail feature image of each modality greater than a preset frequency threshold, the preset frequency threshold is specifically a significant change area (edge) in the image, and the detail feature image refers to a significant change area (edge) in the image extracted by the operator, which contains edge information in the image and can be used to detect image edges and improve the model's perception of boundaries; high-frequency detail features are combined with encoded visual feature images and organ label noise images to detect image edges and improve the model's perception of boundaries. It should be noted that extracting features from the original image of each modality based on one of the Laplacian operator, Prewitt operator or Sobel operator is only an example of an implementation method, and does not constitute a limitation on the implementation methods of the present invention. In practical applications, different feature extraction methods can be selected to extract high-frequency features from the original image of each modality.

[0091] S104: iteratively training the inverse denoising network based on denoised images of all modalities to obtain a trained organ boundary delineation model;

[0092] In this embodiment, the denoised images of all modalities are weighted to obtain a denoised organ boundary delineation image;

[0093] The inverse denoising network is iteratively trained based on the denoised organ boundary delineation image and the real label image to obtain a trained organ boundary delineation model.

[0094] Optionally, the denoised images of all modalities are averaged to obtain a denoised organ boundary delineation image. It should be noted that obtaining the denoised organ boundary delineation image by averaging is only an exemplary description and does not constitute a limitation on the embodiments of the present invention.

[0095] In this embodiment, iterative training of the inverse denoising network based on the denoised organ boundary delineation image and the real label image includes:

[0096] Calculate the loss function based on the denoised organ boundary outline image and the true label image;

[0097] Update the parameters of the inverse denoising network based on the optimization algorithm and loss function;

[0098] The denoised organ boundary outline image is input into the inverse denoising network with updated parameters to obtain an updated denoised organ boundary outline image and a loss function. The inverse denoising network is iteratively trained with the updated denoised organ boundary outline image and the loss function until the inverse denoising network converges to obtain a trained organ boundary outline model.

[0099] Optionally, a mean square error (MSE) loss function is obtained based on the denoised organ boundary delineation image and the true label image; by calculating the MSE loss function, the difference between the denoised organ boundary delineation image and the true label image is measured, and the smaller the value of the loss function is, the closer the denoised organ boundary delineation image is to the true label image, and the better the performance of the organ boundary delineation model is;

[0100] According to the calculated MSE loss function, back propagation is performed to calculate the gradient of the MSE loss function with respect to the parameters of the reverse denoising network (including the weights of the encoder, decoder and attention enhancement module), and the parameters of the reverse denoising network are updated according to the gradient change using the optimization algorithm; after the parameters are updated, the accuracy of the reverse denoising network can also be evaluated using Frechet InceptionDistance (FID). When the MSE loss function value no longer decreases and the FID index no longer decreases, it indicates that the reverse denoising network tends to be stable and the reverse denoising network converges. The training is stopped to obtain a trained organ boundary delineation model. It should be noted that the selection of the MSE loss function to train the model and the selection of the FID index to evaluate the model accuracy are only examples of implementation methods, and do not constitute a limitation on the implementation methods of the present invention. In practical applications, different selections can be made as needed, as long as the model training and model accuracy evaluation can be achieved.

[0101] In this embodiment, after the parameters are updated, the denoised organ boundary delineation image obtained in the previous training is feature spliced ​​with the original images of different modalities and the gradually decreasing time steps, and used as the input for the next iterative training to train the inverse denoising network;

[0102] During iterative training, the number of time steps takes the total preset time steps as the initial value, and gradually decreases with the progress of iterations. The noise of the image gradually decreases, and it becomes closer and closer to the real label image. The time steps corresponding to each iterative training are obtained by gradually decreasing the time steps corresponding to the previous iterative training. The gradually decreasing time steps during the iterative training of the reverse denoising network are symmetrical to the gradually increasing time steps during the forward denoising.

[0103] In an application scenario according to an embodiment of the present invention, the original medical images obtained by CT imaging technology and the original medical images obtained by MR imaging technology are selected as multi-modal original images for training the organ boundary delineation model, hereinafter referred to as CT modality original images and MR modality original images. It should be noted that the selection of CT modality original images and MR modality original images as multi-modal original images in this embodiment is only an example of application scenario, and is not a limitation of this embodiment. In actual applications of the embodiments of the present invention, the type of actual modality can be selected according to actual conditions.

[0104] See attached Figure 3 , Figure 3 FIG. 1 is a flow chart of the main steps of selecting CT modality original images and MR modality original images to train the model according to an embodiment of the present invention. The organ boundary delineation model includes multiple inverse denoising networks, such as Figure 3 As shown, the organ boundary delineation model training method in the embodiment of the present invention mainly includes the following steps S301 to S305.

[0105] Step S301: acquiring a CT modality original image and an MR modality original image;

[0106] In this embodiment, a CT modality original image b1 and an MR modality original image b2 are acquired, and grayscale normalization processing is performed on the CT modality original image b1 and the MR modality original image b2 to obtain the grayscale normalized CT modality original image b1 and the MR modality original image b2.

[0107] Step S302: obtaining a true label image based on the CT modality original image and the MR modality original image;

[0108] In this embodiment, 3D Slicer software is used to register and fuse the CT modality original image b1 and the MR modality original image b2, and to delineate the organ boundaries, so as to obtain a real label image x0 corresponding to the two modalities.

[0109] Step S303: obtaining an organ label noise image based on the real label image;

[0110] In this embodiment, the method further includes a forward noise adding step;

[0111] Obtaining the organ label noise image based on the real label image is the forward denoising step; the forward denoising step specifically includes:

[0112] Preset the total time step number T, take the real label image as x0 as the benchmark, gradually add noise, and add a layer of Gaussian noise ∈ 0 that conforms to the standard normal distribution on the real label image x0 for each additional time step t, when the preset total time step number T is reached, stop adding noise and get the organ label noise image x t ;

[0113] Organ label noisy image x t for:

[0114]

[0115] Among them, x t is the organ label noise image; x0 is the real label image; t is the time step; α i =1-β i , β i is the noise weight, β i As the number of time steps increases, it gradually increases; ∈ t represents the noise vector randomly sampled from the standard normal distribution at t time steps, ∈ t ~N(0,I), I is the unit matrix; when the number of time steps is initially increased, β i The smaller the value, the less noise is added. As the number of time steps increases, β i As the value increases, more and more noise is added.

[0116] Step S304: splicing the original image of each modality with the organ label noise image and the time step number to obtain the image to be denoised corresponding to each modality, and inputting the image to be denoised corresponding to each modality into different inverse denoising networks for denoising to obtain the denoised image of each modality;

[0117] In this embodiment, Figure 2 As shown in the figure, the CT modality original image b1 and the MR modality original image b2 are respectively compared with the organ label noise image x t , time step t splicing, to obtain the CT modality image x to be denoised b1,t , the MR modality image to be denoised x b2,t , the CT modality image to be denoised x b1,t , the MR modality image to be denoised x b2,t Different paths are used to input different inverse denoising networks for denoising, and the denoised image x of the CT modality is obtained. b1,t-1 and the denoised image x of the MR modality b2,t-1 ; At this time, the time step number t is the preset total time step number T;

[0118] Each inverse denoising network includes an encoder, a decoder, and an EAE boundary attention enhancement module;

[0119] The denoised images corresponding to each modality are input into different inverse denoising networks for denoising respectively, and the denoised images of each modality are obtained, including:

[0120] CT modality image to be denoised x b1,t , the MR modality image to be denoised x b2,t Input the encoder of the corresponding inverse denoising network to obtain the encoded visual feature image x of the CT modality b1,t ' and the encoded visual feature image x of the MR modality b2,t ';

[0121] Based on the Laplacian operator, feature extraction is performed on the CT modality original image b1 and the MR modality original image b2 to obtain a detail feature image b1' of the CT modality greater than a preset frequency threshold and a detail feature image b2' of the MR modality greater than a preset frequency threshold;

[0122] The encoded visual feature image x of the CT modality b1,t ', encoded visual feature image x of MR modality b2,t ', detail feature image b1' of CT modality greater than the preset frequency threshold, detail feature image b2' of MR modality greater than the preset frequency threshold, and organ label noise image x t Input the EAE boundary attention enhancement module of the corresponding inverse denoising network to obtain the attention enhanced feature image of each modality;

[0123] Input the attention-enhanced feature image of each modality into the decoder of the corresponding inverse denoising network to obtain the denoised image x of the CT modality b1,t-1 and the denoised image x of the MR modality b2,t-1 .

[0124] Step S305: iteratively training the inverse denoising network based on the denoised images of all modalities to obtain a trained organ boundary delineation model;

[0125] In this embodiment, the denoised image x of the CT modality is b1,t-1 and the denoised image x of the MR modality b2,t-1 Take the average value to obtain the denoised organ boundary delineation image x t-1 ;

[0126] Organ boundary delineation image based on denoising t-1 Calculate the loss function with the real label image x0, and update the parameters of the inverse denoising network based on the optimization algorithm and the loss function;

[0127] Delineate the denoised organ boundary image x t-1 They are respectively spliced ​​with the CT modality original image b1 and the MR modality original image b2 and the gradually decreasing time step number T-1, and input into the reverse denoising network after the updated parameters to obtain the updated denoised organ boundary delineation image x t-2 and loss function to outline the image x with the updated denoised organ boundaryt-2 The inverse denoising network is iteratively trained with the loss function, and the number of time steps gradually decreases from T until the loss function becomes stable and the inverse denoising network converges to obtain a trained organ boundary delineation model. At this time, the denoised organ boundary delineation image obtained is the true label image x0.

[0128] See attached Figure 4 , Figure 4 FIG. 1 is a flow chart showing the main steps of a method for delineating organ boundaries according to an embodiment of the present invention. Figure 4 As shown, the organ boundary delineation method in the embodiment of the present invention mainly includes the following steps S401 to S402.

[0129] Step S401: Acquire an image of the organ boundary to be delineated;

[0130] Step S402: inputting the image of the organ boundary to be delineated into the trained organ boundary delineation model to obtain the organ boundary delineation result;

[0131] The organ boundary delineation model is trained based on the organ boundary delineation model training method described in the above embodiment.

[0132] In this embodiment, original images of different modalities are obtained, and after the original images of different modalities are spliced ​​with pure noise images, they are input into different inverse denoising networks to obtain denoised images of each modality, and the denoised images of each modality are weighted to obtain organ boundary delineation results.

[0133] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present invention, different steps do not have to be executed in such an order, they can be executed simultaneously (in parallel) or in other orders. These adjusted schemes are equivalent to the technical schemes described in the present invention, and therefore will also fall within the scope of protection of the present invention.

[0134] It is understood by those skilled in the art that the present invention implements all or part of the processes in the method of the above embodiment, and can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

[0135] Another aspect of the present invention provides a computer-readable storage medium.

[0136] In an embodiment of a computer-readable storage medium according to the present invention, the computer-readable storage medium can be configured to store a program for executing the organ boundary delineation model training method of the above-mentioned method embodiment, and the program can be loaded and run by a processor to implement the above-mentioned organ boundary delineation model training method. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The computer-readable storage medium can be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present invention is a non-temporary computer-readable storage medium.

[0137] Another aspect of the present invention provides an electronic device.

[0138] In an embodiment of an electronic device according to the present invention, the electronic device may include at least one processor; and a memory connected to the at least one processor in communication; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any of the above embodiments is implemented. Figure 5 , Figure 5 FIG. 5 exemplarily shows that the memory 51 and the processor 52 are communicatively connected via a bus.

[0139] In some embodiments of the present invention, the electronic device described in the present invention may be, but is not limited to, a mobile phone, a tablet computer, a desktop, a laptop, a handheld computer, a notebook computer, a vehicle-mounted device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), an augmented reality (AR)\virtual reality (VR) device, etc., and the embodiments of the present invention are not limited to this.

[0140] So far, the technical solution of the present invention has been described in conjunction with an embodiment shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A method for training an organ boundary delineation model, characterized in that: The organ boundary delineation model includes a plurality of inverse denoising networks, and the model training method includes: Obtain multi-modal original images; Obtaining an organ label noise image based on the multimodal original image; The original image of each modality is spliced ​​with the organ label noise image and the time step number to obtain the image to be denoised corresponding to each modality, and the image to be denoised corresponding to each modality is input into different inverse denoising networks for denoising to obtain the denoised image of each modality; wherein the initial value of the time step number is determined based on the time step number corresponding to the organ label noise image; The inverse denoising network is iteratively trained based on the denoised images of all modalities to obtain a trained organ boundary delineation model; wherein the number of time steps corresponding to each iterative training is obtained by gradually decreasing the number of time steps corresponding to the previous iterative training.

2. The organ boundary delineation model training method according to claim 1, characterized in that: The step of obtaining an organ label noise image based on the multimodal original image comprises: Performing registration and fusion on the multi-modal original images to obtain registered and fused images; Delineating the organ boundaries on the registered and fused images to obtain a true label image; An organ label noise image is obtained based on the true label image.

3. The organ boundary delineation model training method according to claim 2, characterized in that: Obtaining the organ label noise image based on the real label image is a forward denoising step, and the forward denoising step includes: Within a preset total time step, gradually add noise to the real label image until the preset total time step is reached and stop adding noise to obtain an organ label noise image; wherein the time step corresponding to obtaining the organ label noise image is the preset total time step; The method further comprises: The preset total number of time steps is used as the initial value of the number of time steps.

4. The organ boundary delineation model training method according to claim 2, characterized in that: The inverse denoising network is trained based on the denoised images of all modalities to obtain a trained organ boundary delineation model, including: Performing weighted processing on the denoised images of all modalities to obtain denoised organ boundary delineation images; The inverse denoising network is iteratively trained based on the denoised organ boundary delineation image and the real label image to obtain a trained organ boundary delineation model.

5. The organ boundary delineation model training method according to claim 1, characterized in that: Each of the inverse denoising networks includes an encoder, a decoder and an attention enhancement module; Inputting the to-be-denoised images corresponding to each modality into different inverse denoising networks for denoising respectively to obtain denoised images of each modality, including: Inputting the to-be-denoised image corresponding to each modality into the encoder of the corresponding inverse denoising network respectively to obtain an encoded visual feature image of each modality; Obtaining a detailed feature image of each modality greater than a preset frequency threshold based on the original image of each modality; Inputting the encoded visual feature image of each modality, the detail feature image of each modality and the organ label noise image into the attention enhancement module of the corresponding inverse denoising network to obtain an attention enhanced feature image of each modality; The attention-enhanced feature image of each modality is input into the decoder of the corresponding inverse denoising network to obtain the denoised image of each modality.

6. The organ boundary delineation model training method according to claim 4, characterized in that: Iteratively training the inverse denoising network based on the denoised organ boundary delineation image and the real label image comprises: Calculating a loss function based on the denoised organ boundary delineation image and the true label image; Updating parameters of the inverse denoising network based on the optimization algorithm and the loss function; The denoised organ boundary outline image is input into the inverse denoising network after parameter update to obtain an updated denoised organ boundary outline image and a loss function, and the inverse denoising network is iteratively trained with the updated denoised organ boundary outline image and the loss function until the inverse denoising network converges to obtain a trained organ boundary outline model; wherein each iterative training uses the denoised organ boundary outline image obtained in the previous training as the input of the inverse denoising network.

7. The organ boundary delineation model training method according to any one of claims 1 to 6, characterized in that: Before the original image of each modality is concatenated with the organ label noise image and the time step to obtain the denoised image corresponding to each modality, the following steps are included: Normalize the original images of each modality; The normalized original image of each modality is concatenated with the organ label noise image and the time step to obtain the image to be denoised corresponding to each modality.

8. A method for delineating organ boundaries, characterized in that: include: Acquire an image of the organ whose boundary is to be delineated; Inputting the image of the organ boundary to be delineated into a trained organ boundary delineation model to obtain an organ boundary delineation result; Wherein, the organ boundary delineation model is trained based on the organ boundary delineation model training method described in any one of claims 1-7.

9. An electronic device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; Wherein, a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the organ boundary delineation model training method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the organ boundary delineation model training method according to any one of claims 1 to 7.

Images