Medical image segmentation method and device, equipment and storage medium

Abstract

The invention discloses a medical image segmentation method and device, equipment and a storage medium. The method comprises the steps that in the current iteration denoising process, the current denoising frequency, a medical image to be segmented and a current denoised image are input into an image noise model composed of an image coding network, a bottleneck network and an image decoding network; acquiring a previous image bottleneck feature output by a bottleneck network in the image noise model in a previous iterative denoising process and previous decoding feature data corresponding to an image decoding network; inputting the last decoding feature data into an image coding network, and inputting the last image bottleneck feature into a bottleneck network; and taking the difference image between the current denoised image and the current noise image output by the image decoding network as the next denoised image in the next iterative denoising process, and taking the current denoised image as the target segmentation image until the current denoising frequency meets the denoising frequency condition, thereby improving the segmentation effect of the medical image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a medical image segmentation method, device, equipment and storage medium. Background Art

[0002] Image segmentation, a key task in computer vision, aims to divide an image into multiple semantically distinct regions, where pixels within each region share similar characteristics and pixels within different regions differ significantly. Image segmentation has important applications in numerous fields, including medicine, where the segmentation of medical images provides crucial imaging support for subsequent treatment planning, surgical planning, and efficacy evaluation.

[0003] Currently, many image segmentation methods are based on diffusion models. These models employ a progressive denoising mechanism, performing the current denoising operation based on the previous denoised image, and so on to obtain the final segmented image. However, traditional diffusion models fail to fully exploit the deep information in medical images, resulting in poor segmentation results. Summary of the Invention

[0004] The embodiments of the present invention provide a medical image segmentation method, apparatus, device and storage medium to solve the problem that traditional diffusion models are insufficient in mining deep-level information of medical images and improve the segmentation effect of medical images.

[0005] According to one embodiment of the present invention, a medical image segmentation method is provided, the method comprising:

[0006] In the current iterative denoising process, the current denoising number, the medical image to be segmented, and the current denoised image are input into an image noise model; wherein the image noise model includes an image encoding network, a bottleneck network, and an image decoding network, and the image encoding network and the image decoding network are skipped.

[0007] Obtain the previous image bottleneck feature output by the bottleneck network in the image noise model during the previous iterative denoising process and the previous decoding feature data corresponding to the image decoding network;

[0008] Inputting the current denoising times, the medical image to be segmented, the current denoised image and the last decoded feature data into the image coding network for coding processing, thereby obtaining output current coding feature data and current image coding features;

[0009] Inputting the bottleneck feature of the previous image and the encoding feature of the current image into the bottleneck network for feature processing to obtain the output bottleneck feature of the current image;

[0010] Inputting the current encoding feature data and the current image bottleneck feature into the image decoding network for decoding processing to obtain an output current noisy image;

[0011] Using a difference image between the current denoised image and the current noisy image as a next denoised image in a next iterative denoising process;

[0012] Until the current denoising times meets the denoising times condition, the current denoised image is used as a target segmented image corresponding to the medical image to be segmented.

[0013] According to another embodiment of the present invention, a medical image segmentation device is provided, the device comprising:

[0014] a current denoised image input module, configured to input the current denoising count, the medical image to be segmented, and the current denoised image into an image noise model during the current iterative denoising process; wherein the image noise model comprises an image encoding network, a bottleneck network, and an image decoding network, and the image encoding network and the image decoding network are connected by a jump connection;

[0015] The last decoding feature data acquisition module is used to obtain the last image bottleneck feature output by the bottleneck network in the image noise model in the last iterative denoising process and the last decoding feature data corresponding to the image decoding network;

[0016] a current image coding feature output module, configured to input the current denoising times, the medical image to be segmented, the current denoised image, and the last decoded feature data into the image coding network for encoding processing, and obtain output current coding feature data and current image coding features;

[0017] a current image bottleneck feature output module, configured to input the previous image bottleneck feature and the current image coding feature into the bottleneck network for feature processing, and obtain an output current image bottleneck feature;

[0018] a current noise image output module, configured to input the current encoding feature data and the current image bottleneck feature into the image decoding network for decoding processing, thereby obtaining an output current noise image;

[0019] a next denoised image determination module, configured to use a difference image between the current denoised image and the current noisy image as a next denoised image in a next iterative denoising process;

[0020] The target segmentation image determination module is configured to use the current denoising image as a target segmentation image corresponding to the medical image to be segmented until the current denoising times meet a denoising times condition.

[0021] According to another embodiment of the present invention, an electronic device is provided, the electronic device including:

[0022] at least one processor; and

[0023] a memory communicatively connected to the at least one processor; wherein,

[0024] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the medical image segmentation method according to any embodiment of the present invention.

[0025] According to another embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the medical image segmentation method according to any embodiment of the present invention when executed.

[0026] According to another embodiment of the present invention, a computer program product is provided, comprising a computer program. When the computer program is executed by a processor, the medical image segmentation method according to any embodiment of the present invention is implemented.

[0027] The technical solution of this embodiment uses the bottleneck features of the previous image output by the bottleneck network in the previous iterative denoising process as the input data of the bottleneck network in the current iterative denoising process in each iterative denoising process, and uses the previous decoding feature data corresponding to the image decoding network in the previous iterative denoising process as the input data of the image encoding network in the current iterative denoising process, thereby enhancing the feature interaction between two adjacent iterative denoising processes. As the denoising process is executed, a deeper feature interpretation of the bottleneck features and the decoding features is achieved, solving the problem of insufficient deep information mining of medical images by traditional diffusion models, thereby improving the segmentation effect of medical images.

[0028] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1A flowchart of a medical image segmentation method provided by one embodiment of the present invention;

[0031] Figure 2 A schematic diagram of the architecture of a specific example of an image noise model provided by one embodiment of the present invention;

[0032] Figure 3 A flowchart of another medical image segmentation method provided by one embodiment of the present invention;

[0033] Figure 4 A schematic diagram of the architecture of a specific example of another image noise model provided by one embodiment of the present invention;

[0034] Figure 5 A schematic structural diagram of a medical image segmentation device provided by one embodiment of the present invention;

[0035] Figure 6 The present invention provides a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0037] It should be noted that the terms "first", "second", "third", "fourth", "to be divided", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] Figure 1This is a flowchart of a medical image segmentation method provided by an embodiment of the present invention. This embodiment is applicable to the case of segmenting medical images. The method can be performed by a medical image segmentation device. The medical image segmentation device can be implemented in the form of hardware and / or software. The medical image segmentation device can be configured in a terminal device. Figure 1 As shown, the method includes:

[0039] S110 , in the current iterative denoising process, inputting the current denoising times, the medical image to be segmented, and the current denoised image into an image noise model.

[0040] Specifically, the current denoising number is used to mark the denoising position of the current iterative denoising process in the entire medical image segmentation process. Exemplarily, the current denoising number can correspond one-to-one to the number of denoising times in the image noise model training process, or it can represent the number of iterations of the current iterative denoising process. For example, if the current denoising number corresponds to the number of denoising times, the current denoising number decreases, and accordingly, the previous denoising number corresponding to the current denoising number t is expressed as t+1. If the current denoising number represents the number of iterations, the current denoising number increases, and accordingly, the previous denoising number corresponding to the current denoising number t is expressed as t-1.

[0041] This embodiment and the following embodiments are all described by taking the correspondence between the current number of denoising times and the number of denoising times as an example.

[0042] The medical image to be segmented is a two-dimensional image or a three-dimensional image. Exemplarily, the medical imaging device to which the medical image to be segmented belongs includes, but is not limited to, an ultrasound device, a direct digital radiography (DR) system, a computed tomography (CT) device, a magnetic resonance imaging (MRI) device, or a positron emission computed tomography (PET) device. The segmentation objects corresponding to the medical image to be segmented may be, but are not limited to, blood vessels, a prostate, or a heart.

[0043] Specifically, when the current denoising count is N, the current denoised image is the preset initial noisy image. In subsequent denoising processes, the current denoised image is the denoised image obtained after n denoising operations have been performed on the initial noisy image. Here, 1≤n≤N, where N represents the number of denoising operations.

[0044] Specifically, the image noise model represents a network model pre-trained using a sample image set. In one optional embodiment, the sample image set includes a training set, a validation set, and a test set. The training set includes training medical images and training noise data, the validation set includes validation medical images and validation noise data, and the test set includes test medical images and test noise data. Exemplarily, the image data volume of the training set, validation set, and test set satisfies an 8:1:1 ratio, but this is not limited to this example.

[0045] In an optional embodiment, the training method of the image noise model includes: performing noise processing on the label segmentation image corresponding to the training medical image based on the training noise data to obtain noise image data; determining the predicted noise data based on the training medical image, the noise image data, the number of noise additions and the untrained image noise model; determining the loss function value based on the predicted noise data and the training noise data, and adjusting the model parameters of the untrained image noise model based on the loss function value; until the loss function value converges, determining the trained image noise model based on the image noise model in the current iterative training process.

[0046] In an optional embodiment, the method further includes: performing image enhancement processing on the original medical image to obtain a training medical image. Exemplarily, the image enhancement processing includes but is not limited to grayscale transformation, rotation, translation, elastic transformation, and filtering processing.

[0047] In an optional embodiment, the training medical images are composed of multiple two-dimensional medical images. Accordingly, the image enhancement processing further includes normalization processing and / or resizing processing, wherein the normalization processing ensures that the pixel values of each two-dimensional medical image in the training medical images satisfy the range [0, 1], and the resizing processing ensures that the image sizes of each two-dimensional medical image in the training medical images are the same.

[0048] Specifically, the preset noise data includes at least two preset noise images, each of which can be the same or different, and the number of noise additions is the same as the number of preset noise images included in the preset noise data. For example, the preset noise data is represented as [ε1,ε2,…,ε N ], the noisy image data can be expressed as [p1,p2,…,p N ], where p N =x+ε1+ε2+…+ε N , x is a training medical image.

[0049] Exemplarily, the loss function corresponding to the loss function value can be a square loss function, a logarithmic loss function, an exponential loss function, a mean square error loss function, a logistic regression loss function, a Huber loss function, a cross entropy loss function, and a Kullback-Leibler divergence loss function, etc., but is not limited to the example case.

[0050] In an optional embodiment, determining the trained image noise model according to the image noise model in the current iterative training process includes: using the image noise model in the current iterative training process as the trained image noise model.

[0051] In another optional embodiment, a trained image noise model is determined based on the image noise model in the current iterative training process, including: determining model performance parameters based on the verification medical image, verification noise data and the image noise model in the current iterative training process, and adjusting the hyperparameters of the image noise model in the current iterative training process based on the model performance parameters; until the model performance parameters meet the performance conditions, the image noise model in the current adjustment stage is used as the trained image noise model.

[0052] Exemplarily, model performance parameters include but are not limited to mean square error, similarity index, and peak signal-to-noise ratio.

[0053] In an optional embodiment, the method further includes: determining model performance parameters of the trained image noise model based on the test medical image and the test noise data.

[0054] In this embodiment, the image noise model includes an image encoding network, a bottleneck network, and an image decoding network, and the image encoding network and the image decoding network are skip-connected.

[0055] Specifically, the image coding layer in the image coding network corresponds to the image decoding layer in the image decoding network in a jump connection, and the encoding order of the image coding layer in the image coding network is opposite to the decoding order of the image decoding layer in the image decoding network.

[0056] S120, obtaining the previous image bottleneck feature output by the bottleneck network in the image noise model in the previous iterative denoising process and the previous decoding feature data corresponding to the image decoding network.

[0057] Specifically, when the current denoising times is N, the previous image bottleneck feature and the previous decoding feature data may be preset. Exemplarily, the previous image bottleneck feature and the previous decoding feature data may be 0, but are not limited to the example case.

[0058] Specifically, the last decoding feature data includes the last decoding feature corresponding to each image decoding layer in the image decoding network. In an optional embodiment, the last decoding feature is the hierarchical decoding feature output by the image decoding layer in the last iterative denoising process.

[0059] S130: Input the current denoising times, the medical image to be segmented, the current denoised image and the last decoded feature data into the image coding network for coding processing to obtain output current coding feature data and current image coding features.

[0060] In an optional embodiment, the encoding input data of the first image coding layer in the image coding network includes the current denoising times, the medical image to be segmented, the current denoised image, and the previous decoding feature corresponding to the first image decoding layer in the previous decoding feature data; the encoding input data of other image coding layers except the first image coding layer include the previous decoding feature corresponding to the image decoding layer with the same sequential level as the image coding layer in the previous decoding feature data and the previous level encoding feature output by the previous image coding layer.

[0061] Taking the current iterative denoising process as the tth denoising process as an example, the encoding input data corresponding to the first image coding layer includes the current denoising times t, the medical image to be segmented x, the current denoised image y t And the previous decoding feature z corresponding to the first image decoding layer 1,t+1 , the coded input data corresponding to the i-th image coding layer includes the previous decoding feature z corresponding to the i-th image decoding layer i,t+1 and the hierarchical coding features output by the i-1th image coding layer, where i≠1.

[0062] In this embodiment, when the image coding layer is the last image coding layer, the hierarchical coding feature output by the image coding layer is the current image coding feature.

[0063] Specifically, the image coding network includes a preset number of image coding layers connected in series, and the hierarchical coding features output by the current image coding layer serve as the input to the next image coding layer connected in series. Exemplarily, the preset number may be 4 or 5, and the network architecture of the image coding layer may be a classic convolutional network, a residual convolutional network (ResNet), a depthwise separable convolutional network, or a grouped convolutional network, but is not limited to the example scenario.

[0064] S140: Input the bottleneck feature of the previous image and the coding feature of the current image into the bottleneck network for feature processing to obtain the output bottleneck feature of the current image.

[0065] Exemplarily, the network architecture of the bottleneck network includes but is not limited to a classical convolutional network, a residual convolutional network, a depthwise separable convolutional network, or a grouped convolutional network.

[0066] S150: Input the current encoding feature data and the current image bottleneck feature into the image decoding network for decoding processing to obtain the output current noise image.

[0067] Specifically, the image decoding network includes the same number of image decoding layers connected in series as the number of image encoding layers. The hierarchical decoding features output by the current image decoding layer serve as the input to the next image decoding layer connected in series. Exemplary network architectures for the image decoding layer include, but are not limited to, classical convolutional networks, residual convolutional networks, depthwise separable convolutional networks, or grouped convolutional networks.

[0068] In an optional embodiment, the image decoding layer includes at least one self-attention decoding module and a decoding output module connected in series, and the previous decoding feature is the attention decoding feature output by the last self-attention decoding module in the image decoding layer.

[0069] In an optional embodiment, the self-attention decoding module includes a convolution unit and a self-attention unit connected in series, and the decoding output module is a convolution module.

[0070] Figure 2 This is a schematic diagram of the architecture of a specific example of an image noise model provided by one embodiment of the present invention. Taking the current iterative denoising process as the t-th denoising process as an example, the previous iterative denoising process is the t+1-th denoising process. Figure 2 Where x represents the medical image to be segmented, y t and y t+1 They represent the denoised images after the t-th denoising process and the t+1-th denoising process, respectively. The image noise model includes an image encoding network consisting of three image encoding layers, a bottleneck network, and an image decoding network consisting of three image decoding layers.

[0071] In a specific embodiment, the encoded input data of the image encoding layer in the t-th denoising process includes the attention decoding features output by the last self-attention decoding module in the image decoding layer of the same sequential level in the t+1-th denoising process.

[0072] S160: Using the difference image between the current denoised image and the current noisy image as the next denoised image in the next iterative denoising process.

[0073] Taking the current iterative denoising process as the tth denoising process as an example, the next denoised image y t-1 = Current denoised image y t - Current noise image ε t .

[0074] S170 , until the current denoising times meet the denoising times condition, taking the current denoised image as a target segmented image corresponding to the medical image to be segmented.

[0075] Specifically, if the current denoising count corresponds to the number of denoising cycles, the current denoising count decreases as the iterative denoising process is executed, and the denoising count condition is that the current denoising count is zero. If the current denoising count represents the number of iterations, the current denoising count increases as the iterative denoising process is executed, and the denoising count condition is that the current denoising count is the number of denoising cycles.

[0076] In the medical image segmentation process, the denoising operation is iteratively performed N times. In one specific embodiment, the current denoising count is decremented by one. If the current denoising count is zero, the current denoised image in the current iterative denoising process is used as the target segmented image. If the current denoising count is not zero, the process returns to step S110.

[0077] The technical solution of this embodiment uses the bottleneck features of the previous image output by the bottleneck network in the previous iterative denoising process as the input data of the bottleneck network in the current iterative denoising process in each iterative denoising process, and uses the previous decoding feature data corresponding to the image decoding network in the previous iterative denoising process as the input data of the image encoding network in the current iterative denoising process, thereby enhancing the feature interaction between two adjacent iterative denoising processes. As the denoising process is executed, a deeper feature interpretation of the bottleneck features and the decoding features is achieved, solving the problem of insufficient deep information mining of medical images by traditional diffusion models, thereby improving the segmentation effect of medical images.

[0078] Figure 3 This is a flowchart of another medical image segmentation method provided by an embodiment of the present invention. This embodiment further refines the "image coding layer" in the above embodiment. In this embodiment, the current denoising times, the medical image to be segmented, the current denoised image, and the last decoded feature data are input into the image coding network for coding processing to obtain the output current coding feature data and the current image coding feature, including: for each image coding layer, the coding input data corresponding to the image coding layer is input into the cross-attention module to obtain the output cross-coding feature; through the self-attention coding module, the cross-attention feature is self-attention processed to obtain the output self-attention feature; through the coding output module, the self-attention feature is feature processed to obtain the output hierarchical coding feature. As shown in FIG. Figure 3 As shown, the method includes:

[0079] S210 . In the current iterative denoising process, the current denoising times, the medical image to be segmented, and the current denoised image are input into an image noise model.

[0080] S220 , obtaining the previous image bottleneck feature output by the bottleneck network in the image noise model in the previous iterative denoising process and the previous decoding feature data corresponding to the image decoding network.

[0081] S210-S220 in this embodiment are similar to those in the above embodiment. Figure 1 S110 - S120 shown correspond to the same or similar steps and are not described in detail in this embodiment.

[0082] S230. For each image coding layer in the image coding network, the coding input data corresponding to the image coding layer is input into the cross-attention module to obtain the output cross-coding features.

[0083] Taking the current iterative denoising process as the tth denoising process as an example, the encoding input data corresponding to the first image coding layer includes the current denoising times t, the medical image to be segmented x, the current denoised image y t And the previous decoding feature z corresponding to the first image decoding layer 1,t+1 , the coded input data corresponding to the i-th image coding layer includes the previous decoding feature z corresponding to the i-th image decoding layer i,t+1 and the hierarchical coding features output by the i-1th image coding layer, where i≠1.

[0084] In this embodiment, the image coding layer includes a cross-attention module, a self-attention coding module and a coding output module connected in series.

[0085] In an optional embodiment, the cross-attention module includes a convolution unit, a cross-attention unit and a splicing unit connected in series in sequence; wherein the convolution unit is used to perform convolution processing on the input content in the encoded input data except the previous decoding feature to obtain the output image convolution feature, the cross-attention unit is used to perform cross-attention processing on the previous decoding feature in the encoded input data according to the image convolution feature to obtain the cross-attention feature, and the splicing unit is used to splice the image convolution feature and the cross-attention feature to obtain the cross-coding feature.

[0086] Exemplarily, the convolution unit is a residual convolution unit, but is not limited to the example case.

[0087] Among them, the cross-attention unit is specifically used to determine the first query vector based on the image convolution feature, and determine the second key vector and the second value vector based on the previous decoding feature, and cross-process the first query vector, the second key vector and the second value vector to determine the cross-attention feature.

[0088] The query vector is used to represent the features to be queried, the key vector is used to filter features, and the value vector is used to represent the features actually extracted.

[0089] For example, the first query vector is represented by Q1, the second key vector and the second value vector are represented by K2 and V2 respectively, then the cross attention feature F(Q1, K2, V2) satisfies the following formula:

[0090]

[0091] in, represents the dimension of the second key vector K2.

[0092] Specifically, the cross-attention feature can be used to reflect the result of the previous decoding feature after weight adjustment, thereby helping to improve the clarity of the segmentation boundary in the target segmentation image.

[0093] S240. Through the self-attention encoding module, the cross-attention feature is processed by self-attention to obtain the output self-attention feature.

[0094] In an optional embodiment, the self-attention encoding module includes a convolution unit and a self-attention unit connected in series.

[0095] S250. Through the encoding output module, feature processing is performed on the self-attention feature to obtain the output hierarchical encoding feature.

[0096] In an optional embodiment, the encoding output module is a residual convolution module.

[0097] S260: Input the bottleneck feature of the previous image and the encoding feature of the current image into the bottleneck network for feature processing to obtain the output bottleneck feature of the current image.

[0098] In an optional embodiment, the bottleneck network includes a cross-attention layer and a self-attention layer; wherein the cross-attention layer is used to determine the cross-bottleneck feature based on the bottleneck feature of the previous image and the encoding feature of the current image, and the self-attention layer is used to perform self-attention processing on the cross-bottleneck feature to obtain the output bottleneck feature of the current image.

[0099] In an optional embodiment, the cross-attention layer includes a convolution module, a cross-attention module and a splicing module connected in series, and the self-attention layer includes a convolution module and a self-attention module connected in series.

[0100] Among them, the convolution module is used to perform convolution processing on the current image coding features to obtain convolution bottleneck features, the cross-attention module is used to determine the third query vector based on the convolution bottleneck features, and determine the fourth key vector and the fourth value vector based on the bottleneck features of the previous image, and cross-process the third query vector, the fourth key vector and the fourth value vector to obtain cross-attention features, and the splicing module is used to splice the convolution bottleneck features and the cross-attention features to obtain cross-bottleneck features.

[0101] S270: Input the current encoding feature data and the current image bottleneck feature into the image decoding network for decoding processing to obtain the output current noise image.

[0102] In an optional embodiment, the current encoding feature data includes the current encoding features corresponding to each image encoding layer, and the current encoding features include cross-encoding features and self-attention features. Accordingly, the image decoding layer includes two self-attention decoding modules and a decoding output module connected in series.

[0103] In a specific embodiment, the current encoding feature data and the current image bottleneck feature are input into the image decoding network for decoding processing to obtain the output current noisy image, including: for each image decoding layer in the image decoding network, through the first self-attention module, the input content in the decoding input data except the cross-coding feature is self-attention processed to obtain the output initial attention feature; through the second self-attention module, the initial attention feature and the cross-coding feature in the decoding input data are self-attention processed to obtain the output attention decoding feature; through the decoding output module, the attention decoding feature is feature processed to obtain the output hierarchical decoding feature.

[0104] In this embodiment, the decoding input data corresponding to the first image decoding layer in the image decoding network includes the current image bottleneck feature and the current coding feature corresponding to the first image coding layer in the current coding feature data, and the decoding input data corresponding to other image decoding layers except the first image decoding layer include the previous level decoding feature output by the previous image decoding layer and the current coding feature corresponding to the image coding layer with the same sequential level as the image decoding layer in the current coding feature data.

[0105] Taking the current iterative denoising process as the t-th denoising process as an example, the decoding input data corresponding to the first image decoding layer includes the current image bottleneck feature and the current coding feature corresponding to the first image coding layer. The decoding input data corresponding to the j-th image decoding layer includes the hierarchical decoding feature output by the j+1-th image decoding layer and the current coding feature corresponding to the j-th image coding layer, where j≠1.

[0106] In an optional embodiment, the first self-attention module includes a convolution unit and a self-attention unit connected in series, the convolution unit is used to perform convolution processing on the bottleneck feature of the current image or the decoding feature of the previous level to obtain a first convolution decoding feature, and the self-attention unit is used to perform self-attention processing on the first convolution decoding feature and the self-attention feature to obtain an initial attention feature.

[0107] In an optional embodiment, the second self-attention module includes a convolution unit and a self-attention unit connected in series, the convolution unit is used to perform convolution processing on the initial attention feature to obtain a second convolution decoding feature, and the self-attention unit is used to perform self-attention processing on the second convolution decoding feature and the cross-coding feature to obtain an attention decoding feature.

[0108] S280: Using the difference image between the current denoised image and the current noisy image as the next denoised image in the next iterative denoising process.

[0109] S290 , until the current denoising times meet the denoising times condition, taking the current denoised image as a target segmented image corresponding to the medical image to be segmented.

[0110] S280-S290 in this embodiment are similar to those in the above embodiment. Figure 1 The steps S160 - S170 shown correspond to the same or similar steps and are not described in detail in this embodiment.

[0111] Figure 4 This is a schematic diagram of the architecture of a specific example of another image noise model provided by one embodiment of the present invention. Specifically, the image coding network in the image noise model includes M image coding layers, and the image decoding network includes M image decoding layers. Taking the image noise model in the t+1th denoising process as an example, Figure 4 The green dashed box in the figure represents the image encoding layer, the blue dashed box represents the bottleneck network, and the red dashed box represents the image decoding layer.

[0112] Where c represents the number of two-dimensional medical images contained in the medical image to be segmented x, w represents the width of the two-dimensional medical image, d represents the height of the two-dimensional medical image, and y t and y t+1 They represent the denoised images of the tth denoising process and the t+1th denoising process, noise t and noise t+1 Represent the noise images of the tth denoising process and the t+1th denoising process respectively, where y t =y t+1 -noise t+1 .

[0113] Among them, the cross attention module and the self-attention module in the image encoding layer have corresponding jump connections with the two self-attention modules in the image decoding layer of the same order level, the bottleneck network is connected to the first image decoding layer, the two adjacent image decoding layers in the image decoding network are connected to each other, and the attention decoding feature z output by the second self-attention module in the i-th image decoding layer in the t+1-th denoising process is i,t+1is the input data of the i-th image coding layer in the t-th denoising process.

[0114] The technical solution of this embodiment, by setting a cross-attention module in the image coding layer, realizes the extraction of features that are beneficial to the current iterative denoising process from the decoding features in the previous iterative denoising process, and connects the self-attention coding module in series under the cross-attention module, thereby improving the context perception ability of the coding features, further improving the feature representation ability of the image noise model from the feature coding dimension, and further improving the segmentation effect of medical images.

[0115] The following is an embodiment of a medical image segmentation device provided in an embodiment of the present invention. The device and the medical image segmentation method of the above-mentioned embodiment belong to the same inventive concept. For details not fully described in the embodiment of the medical image segmentation device, reference can be made to the content of the medical image segmentation method in the above-mentioned embodiment.

[0116] Figure 5 FIG1 is a schematic diagram of a medical image segmentation device provided by an embodiment of the present invention. Figure 5 As shown, the device includes: a current denoised image input module 310, a previous decoding feature data acquisition module 320, a current image encoding feature output module 330, a current image bottleneck feature output module 340, a current noise image output module 350, a next denoised image determination module 360 and a target segmentation image determination module 370.

[0117] The current denoised image input module 310 is used to input the current denoising number, the medical image to be segmented, and the current denoised image into the image noise model during the current iterative denoising process; wherein the image noise model includes an image encoding network, a bottleneck network, and an image decoding network, and the image encoding network and the image decoding network are connected by a jump connection;

[0118] The last decoded feature data acquisition module 320 is used to obtain the last image bottleneck feature output by the bottleneck network in the image noise model in the last iterative denoising process and the last decoded feature data corresponding to the image decoding network;

[0119] The current image coding feature output module 330 is used to input the current denoising times, the medical image to be segmented, the current denoised image, and the last decoded feature data into the image coding network for coding processing, and obtain the output current coding feature data and the current image coding feature;

[0120] The current image bottleneck feature output module 340 is used to input the bottleneck feature of the previous image and the encoding feature of the current image into the bottleneck network for feature processing to obtain the output bottleneck feature of the current image;

[0121] The current noise image output module 350 is used to input the current encoding feature data and the current image bottleneck feature into the image decoding network for decoding processing to obtain the output current noise image;

[0122] a next denoised image determination module 360 , configured to use a difference image between the current denoised image and the current noisy image as the next denoised image in the next iterative denoising process;

[0123] The target segmentation image determination module 370 is configured to use the current denoised image as the target segmentation image corresponding to the medical image to be segmented until the current denoising times meet the denoising times condition.

[0124] The technical solution of this embodiment uses the bottleneck features of the previous image output by the bottleneck network in the previous iterative denoising process as the input data of the bottleneck network in the current iterative denoising process in each iterative denoising process, and uses the previous decoding feature data corresponding to the image decoding network in the previous iterative denoising process as the input data of the image encoding network in the current iterative denoising process, thereby enhancing the feature interaction between two adjacent iterative denoising processes. As the denoising process is executed, a deeper feature interpretation of the bottleneck features and the decoding features is achieved, solving the problem of insufficient deep information mining of medical images by traditional diffusion models, thereby improving the segmentation effect of medical images.

[0125] In an optional embodiment, the last decoded feature data includes the last decoded feature corresponding to each image decoding layer in the image decoding network;

[0126] The encoding input data of the first image encoding layer in the image encoding network includes the current denoising times, the medical image to be segmented, the current denoised image, and the last decoded feature corresponding to the first image decoding layer in the last decoded feature data;

[0127] The encoding input data of the image coding layers other than the first image coding layer includes the previous decoding feature corresponding to the image decoding layer of the same sequential level as the image coding layer in the previous decoding feature data and the previous level encoding feature output by the previous image coding layer;

[0128] Among them, when the image coding layer is the last image coding layer, the hierarchical coding features output by the image coding layer are the current image coding features.

[0129] In an optional embodiment, the image decoding layer includes at least one self-attention decoding module and a decoding output module connected in series, and the previous decoding feature is the attention decoding feature output by the last self-attention decoding module in the image decoding layer.

[0130] In an optional embodiment, the image encoding layer includes a cross-attention module, a self-attention encoding module, and an encoding output module connected in series;

[0131] Accordingly, the current image coding feature output module 330 is specifically configured to:

[0132] For each image coding layer, the encoded input data corresponding to the image coding layer is input into the cross attention module to obtain the output cross-coded features;

[0133] Through the self-attention encoding module, the cross-attention features are processed by self-attention to obtain the output self-attention features;

[0134] Through the encoding output module, the self-attention features are processed to obtain the output hierarchical encoding features;

[0135] Among them, the current coding feature data includes the current coding features corresponding to each image coding layer, and the current coding features include cross-coding features and self-attention features.

[0136] In an optional embodiment, the crisscross attention module includes a convolution unit, a crisscross attention unit, and a splicing unit connected in series;

[0137] Among them, the convolution unit is used to perform convolution processing on the input content in the encoded input data except the previous decoding feature to obtain the output image convolution feature; the cross-attention unit is used to perform cross-attention processing on the previous decoding feature in the encoded input data according to the image convolution feature to obtain the cross-attention feature; the splicing unit is used to splice the image convolution feature and the cross-attention feature to obtain the cross-coding feature.

[0138] In an optional embodiment, the current noise image output module 350 is specifically configured to:

[0139] For each image decoding layer in the image decoding network, the first self-attention module performs self-attention processing on the input content of the decoded input data except the cross-encoded features to obtain the output initial attention features;

[0140] Through the second self-attention module, the initial attention features and the cross-encoded features in the decoded input data are self-attention processed to obtain the output attention decoding features;

[0141] Through the decoding output module, the attention decoding features are processed to obtain the output hierarchical decoding features;

[0142] Among them, the decoding input data corresponding to the first image decoding layer in the image decoding network includes the current image bottleneck feature and the current coding feature corresponding to the first image coding layer in the current coding feature data, and the decoding input data corresponding to other image decoding layers except the first image decoding layer include the previous level decoding feature output by the previous image decoding layer and the current coding feature corresponding to the image coding layer with the same sequential level as the image decoding layer in the current coding feature data.

[0143] In an optional embodiment, the bottleneck network includes a cross-attention layer and a self-attention layer;

[0144] Among them, the cross-attention layer is used to determine the cross-bottleneck feature based on the bottleneck feature of the previous image and the encoding feature of the current image, and the self-attention layer is used to perform self-attention processing on the cross-bottleneck feature to obtain the output bottleneck feature of the current image.

[0145] The medical image segmentation device provided in the embodiment of the present invention can execute the medical image segmentation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0146] Figure 6 A schematic diagram of the structure of an electronic device provided for one embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0147] like Figure 6As shown, the electronic device 10 includes at least one processor 11 and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by the at least one processor 11, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12 and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0148] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information or data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0149] The processor 11 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the medical image segmentation method provided in the above embodiments.

[0150] In some embodiments, the medical image segmentation method provided in the above embodiments may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps in the medical image segmentation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the medical image segmentation method in any other appropriate manner (e.g., by means of firmware).

[0151] Various embodiments of the systems and techniques described herein can be implemented in the following systems or combinations thereof: digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0152] Computer programs for implementing the medical image segmentation methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that, when executed by the processor, the computer programs implement the functions / operations specified in the flowcharts and / or block diagrams. The computer programs can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0153] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable storage medium. Examples of machine-readable storage media can include an electrical connection based on at least one line, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0154] To provide interaction with a user, the systems and techniques described herein can be implemented on a terminal device having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user can provide input to the terminal device. Other types of devices can also provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0155] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0156] A computing system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. The client-server relationship arises through computer programs running on the respective computers and establishing a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within a cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and virtual private server (VPS) services.

[0157] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0158] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A medical image segmentation method, characterized in that: include: In the current iterative denoising process, the current denoising number, the medical image to be segmented, and the current denoised image are input into an image noise model; wherein the image noise model includes an image encoding network, a bottleneck network, and an image decoding network, and the image encoding network and the image decoding network are skipped. Obtain the previous image bottleneck feature output by the bottleneck network in the image noise model during the previous iterative denoising process and the previous decoding feature data corresponding to the image decoding network; Inputting the current denoising times, the medical image to be segmented, the current denoised image and the last decoded feature data into the image coding network for coding processing, thereby obtaining output current coding feature data and current image coding features; Inputting the bottleneck feature of the previous image and the encoding feature of the current image into the bottleneck network for feature processing to obtain the output bottleneck feature of the current image; Inputting the current encoding feature data and the current image bottleneck feature into the image decoding network for decoding processing to obtain an output current noisy image; Using a difference image between the current denoised image and the current noisy image as a next denoised image in a next iterative denoising process; Until the current denoising times meets the denoising times condition, the current denoised image is used as a target segmented image corresponding to the medical image to be segmented.

2. The method according to claim 1, characterized in that The last decoding feature data includes the last decoding feature corresponding to each image decoding layer in the image decoding network; The encoding input data of the first image encoding layer in the image encoding network includes the current denoising times, the medical image to be segmented, the current denoised image, and the last decoding feature corresponding to the first image decoding layer in the last decoding feature data; The encoding input data of other image coding layers except the first image coding layer includes the previous decoding feature corresponding to the image decoding layer of the same sequential level as the image coding layer in the previous decoding feature data and the previous level encoding feature output by the previous image coding layer; When the image coding layer is the last image coding layer, the hierarchical coding feature output by the image coding layer is the current image coding feature.

3. The method according to claim 2, characterized in that The image decoding layer includes at least one self-attention decoding module and a decoding output module connected in series, and the previous decoding feature is the attention decoding feature output by the last self-attention decoding module in the image decoding layer.

4. The method according to claim 3, characterized in that The image encoding layer includes a cross attention module, a self-attention encoding module and an encoding output module connected in series; Accordingly, the current denoising times, the medical image to be segmented, the current denoised image and the previous decoded feature data are input into the image coding network for coding processing to obtain output current coding feature data and current image coding features, including: For each image coding layer, inputting the coded input data corresponding to the image coding layer into the cross attention module to obtain the output cross coding features; Performing self-attention processing on the cross-attention features through the self-attention encoding module to obtain output self-attention features; The self-attention feature is processed by the encoding output module to obtain an output hierarchical encoding feature; The current coding feature data includes the current coding features corresponding to each image coding layer, and the current coding features include cross-coding features and self-attention features.

5. The method according to claim 4, characterized in that The cross attention module includes a convolution unit, a cross attention unit and a splicing unit connected in series; Among them, the convolution unit is used to perform convolution processing on the input content in the encoded input data except the previous decoding feature to obtain the output image convolution feature; the cross-attention unit is used to perform cross-attention processing on the previous decoding feature in the encoded input data according to the image convolution feature to obtain the cross-attention feature; the splicing unit is used to splice the image convolution feature and the cross-attention feature to obtain the cross-coding feature.

6. The method according to claim 4, characterized in that The step of inputting the current encoding feature data and the current image bottleneck feature into the image decoding network for decoding processing to obtain an output current noisy image includes: For each image decoding layer in the image decoding network, a first self-attention module is used to perform self-attention processing on the input content of the decoded input data except the cross-encoded features to obtain an output initial attention feature; Performing self-attention processing on the initial attention feature and the cross-coded feature in the decoded input data through a second self-attention module to obtain an output attention decoding feature; The attention decoding feature is processed by the decoding output module to obtain the output hierarchical decoding feature; Among them, the decoding input data corresponding to the first image decoding layer in the image decoding network includes the current image bottleneck feature and the current coding feature corresponding to the first image coding layer in the current coding feature data, and the decoding input data corresponding to other image decoding layers except the first image decoding layer include the previous level decoding feature output by the previous image decoding layer and the current coding feature corresponding to the image coding layer with the same sequential level as the image decoding layer in the current coding feature data.

7. The method according to any one of claims 1 to 6, characterized in that The bottleneck network includes a cross attention layer and a self-attention layer; Among them, the cross-attention layer is used to determine the cross-bottleneck feature based on the bottleneck feature of the previous image and the encoding feature of the current image, and the self-attention layer is used to perform self-attention processing on the cross-bottleneck feature to obtain the output bottleneck feature of the current image.

8. A medical image segmentation device, characterized in that: include: a current denoised image input module, configured to input the current denoising count, the medical image to be segmented, and the current denoised image into an image noise model during the current iterative denoising process; wherein the image noise model comprises an image encoding network, a bottleneck network, and an image decoding network, and the image encoding network and the image decoding network are connected by a jump connection; The last decoding feature data acquisition module is used to obtain the last image bottleneck feature output by the bottleneck network in the image noise model in the last iterative denoising process and the last decoding feature data corresponding to the image decoding network; a current image coding feature output module, configured to input the current denoising times, the medical image to be segmented, the current denoised image, and the last decoded feature data into the image coding network for encoding processing, and obtain output current coding feature data and current image coding features; a current image bottleneck feature output module, configured to input the previous image bottleneck feature and the current image coding feature into the bottleneck network for feature processing, and obtain an output current image bottleneck feature; a current noise image output module, configured to input the current encoding feature data and the current image bottleneck feature into the image decoding network for decoding processing, thereby obtaining an output current noise image; a next denoised image determination module, configured to use a difference image between the current denoised image and the current noisy image as a next denoised image in a next iterative denoising process; The target segmentation image determination module is configured to use the current denoising image as a target segmentation image corresponding to the medical image to be segmented until the current denoising times meet a denoising times condition.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the medical image segmentation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the medical image segmentation method according to any one of claims 1 to 7 when executed.

Images