Intestinal ultrasound image segmentation method, system, device and medium based on diffusion network and deep metric learning
By combining diffusion network and deep metric learning, the problems of spot-like noise and edge blur in intestinal ultrasound images are solved, and more efficient intestinal ultrasound image segmentation is achieved, improving segmentation accuracy and edge perception ability.
Patent Information
- Application Number
- CN202411245346.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-06
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-09-06
AI Technical Summary
The problems of spot-like noise and blurred edge areas in intestinal ultrasound images have led to challenges in intestinal segmentation tasks. It is difficult for the existing technology to effectively capture global semantic information and improve edge segmentation accuracy.
Using a method based on diffusion network and deep metric learning, the U-Net structure is constructed with Mamba denoising module for implicit denoising and segmentation, the network is pre-trained using a self-supervised domain diffusion method, and the ternary loss of deep metric learning is used. Functions enhance attention in edge areas.
Effective processing of spot-like noise and edge areas is achieved, the accuracy and edge perception ability of intestinal ultrasound image segmentation are improved, and the accuracy of segmentation results and the utilization efficiency of computing resources are significantly improved.
Smart Images

Figure CN118762040B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence and computer vision, and specifically relates to a method, system, device and medium for intestinal ultrasound image segmentation based on diffusion network and deep metric learning. Background Art
[0002] Intestinal ultrasound segmentation is an important task in medical image processing. The automatic segmentation of ultrasound images can achieve early detection and diagnosis of intestinal diseases. This helps to intervene and treat early, and improves patients' survival rate and quality of life. Automatic segmentation technology can significantly reduce the time and energy required by doctors in analyzing ultrasound images, allowing them to handle more cases more efficiently. In addition, this technology can provide doctors with more accurate and consistent segmentation results, reduce subjective differences and observation errors, and thus improve the accuracy of diagnosis. However, due to the problems of speckle noise, artifacts, and low contrast in ultrasound images, and the very complex structure of the intestine and its highly variable morphological characteristics, there are many challenges in intestinal segmentation.
[0003] There have been many studies on the above challenges. Fully Convolutional Networks (FCN) is one of the most commonly used networks. The paper (“Iterative multi-domain regularized deep learning for anatomical structure detection and segmentation from ultrasound images”, MICCAI, 2016: 487-495.) proposed an iterative multi-domain regularized FCN, which can accurately segment the target area when there are artifacts and speckle noise in the image. This work used FCN to solve the ultrasound segmentation problem earlier. The paper (“Investigating shift variance of convolutional neural networks in ultrasound image segmentation”, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 2022, 69(5): 1703-1713.) studied the shift variance of convolutional networks and evaluated the performance of BlurPooling technology in ultrasound image segmentation. As the most classic segmentation network, U-Net has been widely used in the field of ultrasound image segmentation. SMU-Net (“SMU-Net: Saliency-guidedmorphology-aware U-Net for breast lesion segmentation in ultrasound image”, IEEE transactions on medical imaging, 2021, 41(2): 476-490.) is a saliency-guided morphology-aware U-Net network that effectively utilizes the implicit knowledge carried by the saliency map and designs a fusion module to fuse shape-aware and position-aware information. In the literature (“Estan: Enhanced small tumor-aware network forbreast ultrasound image segmentation”, Healthcare, 2022, 10(11): 2262.), researchers used row-column convolution kernels to enhance the model’s sensitivity to tissue anatomical structures and designed two encoder branches to better fuse contextual information at different scales.Although methods based on convolutional neural networks have strong feature expression capabilities, they are limited by the locality of convolution operations and it is difficult for such methods to capture the global semantic information of images.
[0004] The Transformer framework has been verified in various tasks due to its powerful generalization ability and global semantic information extraction ability. The HoVer-Tran model (“Hover-trans: Anatomy-aware hover-transformer forroi-free breast cancer diagnosis in ultrasound images”, IEEE Transactions on Medical Imaging, 2023, 42(6): 1696-1706.) uses the Transformer framework to segment the target area of breast cancer. In addition, the fusion of the convolutional neural network framework and the Transformer has become an important research direction. The Transformer allows the model to focus on the global information of the image, making up for the shortcomings of the convolutional neural network in dealing with long-distance dependencies. HCTNet (“A hybrid CNN-transformer network for breast ultrasound image segmentation”, Computers in Biology and Medicine, 2023, 155: 106629.) and DSEU-net (“DSEU-net: A novel deep supervision SEU-net for medical ultrasound image segmentation”, Expert Systems with Applications, 2023, 223: 119939.) use Transformer modules to extract global context information. The former designs a spatial cross-attention module based on the spatial attention mechanism to narrow the gap between the decoder and the encoder, and the latter designs a deeper squeeze-excitation attention module to reduce regional perturbations that are useless for segmentation tasks. The Transformer architecture can model the entire image and capture global semantic relationships well, but it consumes more computing resources due to the large number of matrix calculations.
[0005] It is worth noting that the Mamba architecture was proposed in December 2023. It is a selective state-space model that can not only capture global contextual relationships like Transformer, but also achieve linear expansion with the increase of context length. U-Mamba ("U-mamba: Enhancing long-range dependency for biomedical imagesegmentation", arXiv preprint arXiv:2401.04722, 2024.) was the first to combine the U-Net model and the Mamba architecture for medical image segmentation of multiple modalities. Experimental results show that the model has achieved state-of-the-art performance in medical image segmentation tasks of multiple modalities. Swin-UMamba ("Swin-umamba: Mamba-based unet with imagenet-based pretraining", arXiv preprint arXiv:2402.03302, 2024.) verified that pre-training on ImageNet plays a very important role in medical image segmentation models. The entire model has reduced the number of parameters by nearly half compared to U-Mamba and achieved similar performance. Semi-Mamba-UNet ("Semi-Mamba-UNet:Pixel-Level Contrastive Cross-Supervised Visual Mamba-based UNet for Semi-Supervised Medical Image Segmentation", arXiv preprint arXiv:2402.07245, 2024.) proposes a semi-supervised method for medical image segmentation, and designs a self-supervised pixel-level contrast learning strategy combined with a dual-channel network architecture to make full use of unlabeled data. Summary of the invention
[0006] In view of the deficiencies in the prior art, the present invention provides a method, system, device and medium for intestinal ultrasound image segmentation based on diffusion network and deep metric learning, which overcomes the problem of speckle noise and blurred edge areas in ultrasound images.
[0007] To achieve the above object, the present invention adopts the following technical solutions:
[0008] A method for intestinal ultrasound image segmentation based on diffusion network and deep metric learning, comprising the following steps:
[0009] S1. constructing a segmentation network, wherein the segmentation network includes a backbone network and a multi-layer perceptron connected to the backbone network; the backbone network is used to implicitly denoise the intestinal ultrasound image, and the multi-layer perceptron is used to segment the intestinal ultrasound image;
[0010] S2, pre-training segmentation network based on self-supervised domain diffusion method;
[0011] S3, deep metric learning-assisted training of segmentation network;
[0012] S4. Use the trained segmentation network to perform image segmentation on intestinal ultrasound images.
[0013] To optimize the above technical solutions, the specific measures taken also include:
[0014] Further, in S1, the backbone network is a three-layer U-net structure, including three encoders, three decoders and an upsampling feature fusion module; one encoder and one decoder constitute one layer;
[0015] Each of the encoders includes a convolution module and a Mamba denoising module. The convolution module is used to extract shallow features, reduce the input resolution by half and double the number of channels. The shallow features are then input into the Mamba denoising module. The Mamba denoising module outputs features of a set size and then inputs them into the decoder of the same layer and the convolution module of the encoder of the next layer. The decoder fuses semantic information and saves the features of the current layer. The upsampling feature fusion module upsamples and fuses the features of different scales saved by each layer of the decoder.
[0016] Before the encoder of the first layer processes the input intestinal ultrasound image, it also includes the operation of dividing the intestinal ultrasound image into several image blocks, and then inputting the image blocks into the convolution module.
[0017] Further, the Mamba denoising module includes: a mapping and position encoding module, a non-subsampled shearlet transform denoising submodule and a visual Mamba submodule;
[0018] The mapping and position encoding module is used to generate high-dimensional features. The high-dimensional features pass through two branches. One branch is input into the non-subsampled shearlet transform denoising submodule for further denoising. The denoised features are input into the visual Mamba submodule to further capture the long-range semantic information in the features. Finally, they are added with the high-dimensional features of the other branch. The added features are then used through a convolution to generate features of a set size.
[0019] Furthermore, the visual Mamba submodule specifically performs the following operations:
[0020] The feature after denoising by the non-subsampled shearlet transform denoising submodule is a sequence block of shape (B, M, D) , where B is the batch size, M is the sequence length, and D is the feature dimension. l -1 is the sequence number of the backbone network layer;
[0021] Sequence Block Perform normalization to obtain normalized sequence blocks , for the normalized sequence block Through two different linear transformations, the first feature x and the second feature z are obtained respectively;
[0022] The first feature x is used for forward processing and reverse processing respectively. In the forward processing, the first feature x is firstly processed by a one-dimensional convolution operation and the SiLU activation function is applied to obtain the third feature ; Through two different linear transformations, the fourth feature is generated and the fifth characteristic , through SoftPlus function and third feature Calculate gate value , gate value With bias parameters Multiply to get the sixth characteristic ; Gate value With the fourth characteristic Multiply to get the seventh characteristic , based on the sixth feature , the seventh characteristic The fifth characteristic and the third feature , generating positive features through the state space model , by combining the positive feature z with the second feature z after SiLU activation Adjust to get the output characteristics after forward gating ;
[0023] The reverse processing process is opposite to the forward processing process. After reverse processing, the output features after reverse gating are generated. ;
[0024] Output characteristics after forward gating And the output characteristics after reverse gating After adding, perform linear transformation and then add it to the original sequence block Perform residual connection to generate the final output sequence block , sequence block The convolutional module that feeds into the encoder of the next layer of the backbone network.
[0025] Further, in S2, the pre-training segmentation network based on the self-supervised domain diffusion method includes a forward process for image degradation and a reverse process for image restoration and segmentation;
[0026] The forward process is specifically as follows: obtaining the original intestinal ultrasound image from the training data set composed of intestinal ultrasound images and its corresponding segmentation labels, and then gradually perform degradation operations on the original intestinal ultrasound image to generate multiple degraded images at different times t ;
[0027] The reverse process is specifically: transforming the degraded image The image is input into the segmentation network for image restoration and segmentation to obtain the segmented image. The loss of the reverse process consists of two parts, one of which is the image restoration loss, which is used to train the backbone network so that the restored image is close to the original intestinal ultrasound image. , and the other part is the segmentation loss, which is used to improve the accuracy of the multi-layer perceptron in segmenting the target area, and use the cross entropy and Dice loss to jointly optimize the reverse process.
[0028] Furthermore, the degradation operation is one or more of adding speckle noise, adding Gaussian noise, adding Poisson noise, blurring, downsampling, deformation or masking.
[0029] Further, in S3, the deep metric learning-assisted training segmentation network is specifically:
[0030] In the process of training the segmentation network of intestinal ultrasound images, the intestinal wall edge area of the intestinal ultrasound images is sampled to obtain anchor points, positive samples and negative samples, and the anchor points, positive samples and negative samples are formed into triplets. The sampling process is guided by the intestinal ultrasound images marked with real labels; a constraint relationship is given to the triplets, and the constraint relationship specifically requires that the distance between the anchor point and the positive sample and the distance between the anchor point and the negative sample be greater than a positive constant; and the triplets loss of deep metric learning is calculated;
[0031] The ternary loss function for deep metric learning is as follows:
[0032] ;
[0033] In the formula, L loss represents the triple loss for deep metric learning, Represents the Euclidean distance between the anchor point and the positive sample positive, Represents the Euclidean distance between the anchor point and the negative sample negative. is the boundary, which is used to ensure that the distance between the positive sample and the anchor point is smaller than the distance between the negative sample and the positive sample;
[0034] The ternary loss function using deep metric learning enables the segmentation network to pay more attention to the edge areas of intestinal ultrasound images.
[0035] The present invention also proposes a system for segmenting intestinal ultrasound images based on diffusion network and deep metric learning, comprising:
[0036] A network construction module, used to construct a segmentation network, wherein the segmentation network includes a backbone network and a multi-layer perceptron connected to the backbone network; the backbone network is used to implicitly denoise the intestinal ultrasound image, and the multi-layer perceptron is used to segment the intestinal ultrasound image;
[0037] A pre-training module for pre-training segmentation networks based on a self-supervised domain diffusion method;
[0038] Auxiliary training module, used to assist in training the segmentation network based on deep metric learning;
[0039] The image segmentation module uses the trained segmentation network to perform image segmentation on intestinal ultrasound images.
[0040] The present invention also proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the intestinal ultrasound image segmentation method based on diffusion network and deep metric learning as described above is implemented.
[0041] The present invention also proposes a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the intestinal ultrasound image segmentation method based on diffusion network and deep metric learning as described above.
[0042] The beneficial effects of the present invention are:
[0043] In response to the problem of speckle noise, the present invention overcomes the speckle noise in intestinal ultrasound from two perspectives: model architecture and domain diffusion pre-training. A network framework based on U-Net and fused with Mamba structure is designed. The network has a strong global semantic capture capability and can enhance the model's feature expression capability while saving computing resources. The present invention designs a domain degradation operator that is more suitable for ultrasound images, so that the model is not only sensitive to common Gaussian noise, but also to speckle noise, electronic noise and even motion artifacts. The backbone network is trained by self-supervised pre-training (domain diffusion) so that the backbone network model can learn a large amount of domain noise knowledge and obtain denoising capabilities for intestinal ultrasound images.
[0044] To solve the problem of blurred edge areas, auxiliary training of deep metric learning is used. In the auxiliary training stage, the edge areas are sampled to obtain pixel-level triplets. A ternary loss function is designed to encourage the model to pay enough attention to the edge areas to improve the segmentation accuracy of the model.
[0045] The entire segmentation network model has acquired sufficient edge perception capabilities to achieve the goal of improving the target edge segmentation accuracy. The model can segment ultrasound images while implicitly denoising. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 This is the overall framework diagram of the intestinal ultrasound image segmentation method based on diffusion network and deep metric learning proposed in the present invention.
[0047] Figure 2 Comparison of the segmentation results of manual annotation of intestinal ultrasound images and the image segmentation method proposed in the present invention. DETAILED DESCRIPTION
[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0049] Embodiment 1
[0050] The present invention proposes a method for intestinal ultrasound image segmentation based on diffusion network and deep metric learning. The overall framework of the method is as follows: Figure 1 As shown, the following steps are included:
[0051] S1. Construct a segmentation network, which includes a backbone network and a multi-layer perceptron connected to the backbone network; the backbone network is used to implicitly denoise the intestinal ultrasound image, and the multi-layer perceptron is used to segment the intestinal ultrasound image; the backbone network is a three-layer U-net structure, including three encoders, three decoders and an upsampling feature fusion module; an encoder and a decoder form a layer;
[0052] Each encoder includes a convolution module and a Mamba denoising module. The convolution module is used to extract shallow features, reducing the input resolution by half while doubling the number of channels. The shallow features are then input into the Mamba denoising module. The Mamba denoising module outputs features of a set size and then inputs them into the decoder of the same layer and the convolution module of the encoder of the next layer. The decoder fuses semantic information and saves the features of the current layer. The upsampling feature fusion module upsamples and fuses the features of different scales saved by the decoder of each layer.
[0053] Before the encoder of the first layer processes the input intestinal ultrasound image, it also includes the operation of dividing the intestinal ultrasound image into several image blocks, and then inputs the image blocks into the convolution module. The input is The image is converted into a two-dimensional intestinal ultrasound image of size , where H and W are the height and width of the image, and C is the number of image channels. Flatten the two-dimensional image blocks, J is the number of image blocks, and P is the size of each image block.
[0054] The Mamba denoising module includes: a mapping and position encoding module, a non-subsampled shearlet transform (NSST) denoising submodule and a visual Mamba submodule. The main idea of the non-subsampled shearlet transform (NSST) is to learn a parameterized attention map and apply it to the NSST frequency domain space. The reason for choosing NSST is that compared with the wavelet transform, the NSST transform has the characteristics of multi-scale and anisotropy, while avoiding the pseudo-Gibbs distortion phenomenon of the shearlet transform. First, the input features are transformed into high-frequency subbands and low-frequency subbands, and then the transformed signal and a parameterized attention map are element-multiplied. The map is learnable, and finally the inverse transformation is returned to the time domain, so as to comprehensively remove the interference of speckle noise in all directions and frequency domains. Mamba is actually a state-space model (State-Space Model, SSM). The standard Mamba architecture can only process one-dimensional sequences. In order to process two-dimensional video images, the present invention designs a visual Mamba architecture.
[0055] The mapping and position encoding module is used to generate high-dimensional features. The high-dimensional features pass through two branches. One branch is input into the non-subsampled shearlet transform denoising submodule for further denoising. The denoised features are input into the visual Mamba submodule to further capture the long-range semantic information in the features. Finally, they are added with the high-dimensional features of the other branch. The added features are then used through a convolution to generate features of a set size.
[0056] Taking the encoder of the first layer as an example, the two-dimensional image block is linearly projected into a vector of size D using the mapping and position encoding module and the position embedding encoding is added to obtain T 0 The sequence block looks like this:
[0057] ;
[0058] in is the Jth image block, is a learnable projection matrix, Indicates the category to which the entire image block belongs. represents the position encoding information of the input two-dimensional image block. Then T 0 The sequence block is input to the next layer encoder to obtain the sequence block T 1 .
[0059] The Visual Mamba submodule specifically performs the following operations:
[0060] The feature after denoising by the non-subsampled shearlet transform denoising submodule is a sequence block of shape (B, M, D) , where B is the batch size, M is the sequence length, and D is the feature dimension. l -1 is the sequence number of the backbone network layer;
[0061] Sequence Block Perform normalization to obtain normalized sequence blocks , for the normalized sequence block Through two different linear transformations, the first feature x and the second feature z are obtained respectively;
[0062] The first feature x is used for forward processing and reverse processing respectively. In the forward processing, the first feature x is firstly processed by a one-dimensional convolution operation and the SiLU activation function is applied to obtain the third feature ; Through two different linear transformations, the fourth feature is generated and the fifth characteristic , through SoftPlus function and third feature Calculate gate value , gate value With bias parameters Multiply to get the sixth characteristic ; Gate value With the fourth characteristic Multiply to get the seventh characteristic , based on the sixth feature , the seventh characteristic The fifth characteristic and the third feature , generating positive features through the state space model , by combining the positive feature z with the second feature z after SiLU activation Adjust to get the output characteristics after forward gating ;
[0063] The reverse processing process is opposite to the forward processing process. After reverse processing, the output features after reverse gating are generated. ;
[0064] Output characteristics after forward gating And the output characteristics after reverse gating After adding, perform linear transformation and then add it to the original sequence block Perform residual connection to generate the final output sequence block , sequence block The convolutional module that feeds into the encoder of the next layer of the backbone network.
[0065] The basic structures of the three decoders are similar. Each decoder contains an addition operation, which jumps the encoder-related features with the current layer features to fuse the semantic information. Subsequently, the fused features are input to the convolutional layer for upsampling and input to the activation layer (ReLU activation function) to obtain the final output of the layer and save the features of the current layer.
[0066] The upsampling feature fusion module performs corresponding upsampling operations on the features of different scales saved by the encoder and fuses them. Finally, the fused features are input into a multi-layer perceptron, which contains three hidden layers, performs nonlinear transformation on the input features, and finally outputs the corresponding segmentation results.
[0067] S2. Pre-train the segmentation network based on the self-supervised domain diffusion method so that the parameters therein learn rich domain noise knowledge. The original diffusion model gradually adds Gaussian noise and then gradually cools it down to a clean sample. This denoising method has great limitations. The model is often only sensitive to Gaussian noise, and there are a lot of patchy noises in ultrasound images. Obviously, the above diffusion model cannot achieve the optimal denoising effect. Therefore, the present invention proposes a self-supervised domain diffusion method. The pre-training segmentation network based on the self-supervised domain diffusion method includes a forward process for image degradation and a reverse process for image restoration and segmentation;
[0068] The forward process is as follows: obtain the original intestinal ultrasound image from the training data set composed of intestinal ultrasound images and its corresponding segmentation labels, and then gradually perform degradation operations on the original intestinal ultrasound image to generate multiple degraded images at different times t ; The degradation operation is one or more of adding speckle noise, adding Gaussian noise, adding Poisson noise, blurring, downsampling, deformation or masking. For Gaussian noise, the present invention adopts a 3D generalized zero-mean Gaussian noise model with a covariance matrix to simulate the noise correlation between the R, G and B channels. Unlike Gaussian noise, speckle noise is a multiplicative noise. We use the generated Gaussian noise to randomly sample and then multiply it with the clean sample. Poisson noise is generally due to the photon emission noise caused by the discrete nature of the charge, and it is also widely present in medical images. In order to simulate different levels of Poisson noise, the clean sample is first multiplied by ,in is a real number uniformly sampled between [2, 4], then Poisson noise is added and then divided by At the same time, we also use blurring, downsampling, deformation, masking and other operations on the clean image to get as close as possible to the noise and even artifacts contained in the real ultrasound image. In addition, the present invention designs a random shuffle degradation strategy. In short, the degradation operation on the image is to randomly sample one or several degradations from the degradation operator set, which can further enrich the degradation types and samples.
[0069] The reverse process is as follows:
[0070] Degrade the image The image is input into the segmentation network for image restoration and segmentation to obtain the segmented image. The loss of the reverse process consists of two parts, one of which is the image restoration loss, which is used to train the backbone network so that the restored image is close to the original intestinal ultrasound image. ,The other part is the segmentation loss, which is used to improve the ,accuracy of segmenting the target area by the multi-layer perceptron, and the ,cross entropy and Dice loss are jointly optimized for the ,reverse process.
[0071] Taking the addition of Gaussian noise as an example, the forward process is as follows: Get the original intestinal ultrasound image from the training dataset consisting of intestinal ultrasound images and its corresponding segmentation labels, and then gradually add Gaussian noise to the original intestinal ultrasound image to generate multiple noisy intestinal ultrasound images (degraded images) at different times t. ; As shown below:
[0072] ;
[0073] In the formula, is the scaling factor of the initial intestinal ultrasound image, is the variance coefficient of the noise, is a Gaussian distribution, is standard Gaussian noise.
[0074] The probability distribution of the forward process with Gaussian noise added is as follows:
[0075] ;
[0076] In the formula, is the probability distribution of the forward process, is the original intestinal ultrasound image, is the noisy intestinal ultrasound image at time t, N(·) represents Gaussian distribution, is the scaling factor of the initial intestinal ultrasound image, is the variance coefficient of the noise, I is the identity matrix.
[0077] Correspondingly, the image restoration process in the reverse process is the process of removing Gaussian noise, and the probability distribution of the denoising process is as follows:
[0078] ;
[0079] In the formula, represents the probability distribution of the denoising process, where are the parameters of the backbone network, is the noisy intestinal ultrasound image at time t, is the intestinal ultrasound image with noise removed at time step t-1, N represents Gaussian distribution, represents the mean value predicted by the backbone network, is the variance of the backbone network prediction.
[0080] S3. Deep metric learning-assisted training of segmentation network. Existing deep metric learning is almost all based on image scale. The present invention proposes a pixel-level speed metric learning, specifically:
[0081] In the process of training the segmentation network of intestinal ultrasound images, the intestinal wall edge area of the intestinal ultrasound image is sampled to obtain anchor points, positive samples and negative samples. The anchor points, positive samples and negative samples are combined into triplets. The sampling process is guided by the intestinal ultrasound image with the real label. For example, the selected anchor point can be a pixel point inside the intestinal wall, then the positive sample is selected from other pixel points on the edge of the intestinal wall, and the negative sample is selected from the background pixel points outside the intestinal wall. Give the triplet constraint relationship, which specifically requires that the distance between the anchor point and the positive sample and the distance between the anchor point and the negative sample be greater than a positive constant; calculate the ternary loss of deep metric learning;
[0082] The ternary loss function for deep metric learning is as follows:
[0083] ;
[0084] In the formula, L loss represents the triple loss for deep metric learning, Represents the Euclidean distance between the anchor point and the positive sample positive, Represents the Euclidean distance between the anchor point and the negative sample negative. is the boundary, which is used to ensure that the distance between the positive sample and the anchor point is smaller than the distance between the negative sample and the positive sample;
[0085] The ternary loss function using deep metric learning enables the segmentation network to pay more attention to the edge areas of intestinal ultrasound images.
[0086] S4. Use the trained segmentation network to perform image segmentation on intestinal ultrasound images.
[0087] The results of intestinal ultrasound image segmentation performed by the present invention are compared with those of manual annotation. Figure 2 shown. Figure 2 (a) is a comparison chart of intestinal wall thickness indicators in intestinal images. The left side shows the calculation of intestinal wall thickness by manually selecting points, and the right side shows the automatic detection of intestinal wall thickness through image segmentation. Figure 2 Middle (b) is a comparison of intestinal fat indicators in intestinal images. The left side is a subjective judgment of whether intestinal fat exists, and the right side is automatic detection of intestinal fat through image segmentation. Figure 2 Middle (c) is a comparison chart of intestinal stratification indicators in intestinal images. The left side shows the manually observed intestinal wall stratification status, and the right side shows the intestinal wall stratification automatically obtained through image segmentation.
[0088] Embodiment 2
[0089] The present invention proposes a system for segmenting intestinal ultrasound images based on diffusion network and deep metric learning corresponding to the method of embodiment 1, comprising:
[0090] A network construction module is used to construct a segmentation network, which includes a backbone network and a multi-layer perceptron connected to the backbone network; the backbone network is used to implicitly denoise the intestinal ultrasound image, and the multi-layer perceptron is used to segment the intestinal ultrasound image;
[0091] A pre-training module for pre-training segmentation networks based on a self-supervised domain diffusion method;
[0092] Auxiliary training module, used to assist in training the segmentation network based on deep metric learning;
[0093] The image segmentation module uses the trained segmentation network to perform image segmentation on intestinal ultrasound images.
[0094] The implementation methods of each module and the functions of the modules in the system are completely consistent with the steps of the method in Example 1, so they will not be repeated here.
[0095] Embodiment 3
[0096] The present invention proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the intestinal ultrasound image segmentation method based on diffusion network and deep metric learning as described in Example 1 is implemented.
[0097] Embodiment 4
[0098] The present invention proposes a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the intestinal ultrasound image segmentation method based on diffusion network and deep metric learning as described in Example 1.
[0099] In the embodiments disclosed in the present application, the computer storage medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specific examples of computer storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0100] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0101] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.
Claims
1. A method for intestinal ultrasound image segmentation based on diffusion network and deep metric learning, characterized in that: The following steps are involved: S1. constructing a segmentation network, wherein the segmentation network includes a backbone network and a multi-layer perceptron connected to the backbone network; the backbone network is used to implicitly denoise the intestinal ultrasound image, and the multi-layer perceptron is used to segment the intestinal ultrasound image; S2, a pre-trained segmentation network based on a self-supervised domain diffusion method; in S2, the pre-trained segmentation network based on a self-supervised domain diffusion method includes a forward process for image degradation and a reverse process for image restoration and segmentation; The forward process is specifically as follows: obtaining the original intestinal ultrasound image from the training data set composed of intestinal ultrasound images and its corresponding segmentation labels, and then gradually perform degradation operations on the original intestinal ultrasound image to generate multiple degraded images at different times t ; The reverse process is specifically: transforming the degraded image The image is input into the segmentation network for image restoration and segmentation to obtain the segmented image. The loss of the reverse process consists of two parts, one of which is the image restoration loss, which is used to train the backbone network so that the restored image is close to the original intestinal ultrasound image. , the other part is the segmentation loss, which is used to improve the accuracy of the multi-layer perceptron in segmenting the target area, and uses cross entropy and Dice loss to jointly optimize the reverse process; S3, based on deep metric learning to assist in training the segmentation network; the deep metric learning to assist in training the segmentation network is specifically: In the process of training the segmentation network of intestinal ultrasound images, the intestinal wall edge area of the intestinal ultrasound images is sampled to obtain anchor points, positive samples and negative samples, and the anchor points, positive samples and negative samples are formed into triplets. The sampling process is guided by the intestinal ultrasound images marked with real labels; a constraint relationship is given to the triplets, and the constraint relationship specifically requires that the distance between the anchor point and the positive sample and the distance between the anchor point and the negative sample be greater than a positive constant; and the triplets loss of deep metric learning is calculated; The ternary loss function for deep metric learning is as follows: ; In the formula, L loss represents the triple loss for deep metric learning, Represents the Euclidean distance between the anchor point and the positive sample positive, Represents the Euclidean distance between the anchor point and the negative sample negative, is the boundary, which is used to ensure that the distance between the positive sample and the anchor point is smaller than the distance between the negative sample and the positive sample; The ternary loss function of deep metric learning is used to make the segmentation network pay more attention to the edge areas of intestinal ultrasound images; S4. Use the trained segmentation network to perform image segmentation on intestinal ultrasound images.
2. The intestinal ultrasound image segmentation method based on diffusion network and deep metric learning according to claim 1, characterized in that: In S1, the backbone network is a three-layer U-net structure, including three encoders, three decoders and an upsampling feature fusion module; one encoder and one decoder form one layer; Each of the encoders includes a convolution module and a Mamba denoising module. The convolution module is used to extract shallow features, reduce the input resolution by half and double the number of channels. The shallow features are then input into the Mamba denoising module. The Mamba denoising module outputs features of a set size and then inputs them into the decoder of the same layer and the convolution module of the encoder of the next layer. The decoder fuses semantic information and saves the features of the current layer. The upsampling feature fusion module upsamples and fuses the features of different scales stored in each layer of decoders; Before the encoder of the first layer processes the input intestinal ultrasound image, it also includes the operation of dividing the intestinal ultrasound image into several image blocks, and then inputting the image blocks into the convolution module.
3. The intestinal ultrasound image segmentation method based on diffusion network and deep metric learning according to claim 2, characterized in that: The Mamba denoising module includes: a mapping and position encoding module, a non-subsampled shearlet transform denoising submodule and a visual Mamba submodule; The mapping and position encoding module is used to generate high-dimensional features. The high-dimensional features pass through two branches. One branch is input into the non-subsampled shearlet transform denoising submodule for further denoising. The denoised features are input into the visual Mamba submodule to further capture the long-range semantic information in the features. Finally, they are added with the high-dimensional features of the other branch. The added features are then used through a convolution to generate features of a set size.
4. The intestinal ultrasound image segmentation method based on diffusion network and deep metric learning according to claim 3, characterized in that: The visual Mamba submodule specifically performs the following operations: The feature after denoising by the non-subsampled shearlet transform denoising submodule is a sequence block of shape (B, M, D) , where B is the batch size, M is the sequence length, and D is the feature dimension. l -1 is the sequence number of the backbone network layer; Sequence Block Perform normalization to obtain normalized sequence blocks , for the normalized sequence block Through two different linear transformations, the first feature x and the second feature z are obtained respectively; The first feature representation x is used for forward processing and reverse processing respectively. In the forward processing, the first feature x is firstly processed by a one-dimensional convolution operation and the SiLU activation function is applied to obtain the third feature ; Through two different linear transformations, the fourth feature is generated and the fifth characteristic , through SoftPlus function and third feature Calculate gate value , gate value With bias parameters Multiply to get the sixth characteristic ; Gate value With the fourth characteristic Multiply to get the seventh characteristic , based on the sixth feature , the seventh characteristic The fifth characteristic and the third feature , generating positive features through the state space model , by combining the positive feature z with the second feature z after SiLU activation Adjust to get the output characteristics after forward gating ; The reverse processing process is opposite to the forward processing process. After reverse processing, the output features after reverse gating are generated. ; Output characteristics after forward gating And the output characteristics after reverse gating After adding, perform linear transformation and then add it to the original sequence block Perform residual connection to generate the final output sequence block , sequence block The convolutional module that feeds into the encoder of the next layer of the backbone network.
5. The intestinal ultrasound image segmentation method based on diffusion network and deep metric learning according to claim 1, characterized in that: The degradation operation is one or more of adding speckle noise, adding Gaussian noise, adding Poisson noise, blurring, downsampling, deformation or masking.
6. A system for intestinal ultrasound image segmentation based on diffusion network and deep metric learning, characterized in that: include: A network construction module, used to construct a segmentation network, wherein the segmentation network includes a backbone network and a multi-layer perceptron connected to the backbone network; the backbone network is used to implicitly denoise the intestinal ultrasound image, and the multi-layer perceptron is used to segment the intestinal ultrasound image; A pre-training module, used for pre-training a segmentation network based on a self-supervised domain diffusion method; the pre-training segmentation network based on a self-supervised domain diffusion method includes a forward process for image degradation and a reverse process for image restoration and segmentation; The forward process is specifically as follows: obtaining the original intestinal ultrasound image from the training data set composed of intestinal ultrasound images and its corresponding segmentation labels, and then gradually perform degradation operations on the original intestinal ultrasound image to generate multiple degraded images at different times t ; The reverse process is specifically: transforming the degraded image The image is input into the segmentation network for image restoration and segmentation to obtain the segmented image. The loss of the reverse process consists of two parts, one of which is the image restoration loss, which is used to train the backbone network so that the restored image is close to the original intestinal ultrasound image. , the other part is the segmentation loss, which is used to improve the accuracy of the multi-layer perceptron in segmenting the target area, and uses cross entropy and Dice loss to jointly optimize the reverse process; An auxiliary training module is used to assist in training the segmentation network based on deep metric learning; the auxiliary training of the segmentation network based on deep metric learning is specifically: In the process of training the segmentation network of intestinal ultrasound images, the intestinal wall edge area of the intestinal ultrasound images is sampled to obtain anchor points, positive samples and negative samples, and the anchor points, positive samples and negative samples are formed into triplets. The sampling process is guided by the intestinal ultrasound images marked with real labels; a constraint relationship is given to the triplets, and the constraint relationship specifically requires that the distance between the anchor point and the positive sample and the distance between the anchor point and the negative sample be greater than a positive constant; and the triplets loss of deep metric learning is calculated; The ternary loss function for deep metric learning is as follows: ; In the formula, L loss represents the triple loss for deep metric learning, Represents the Euclidean distance between the anchor point and the positive sample positive, Represents the Euclidean distance between the anchor point and the negative sample negative, is the boundary, which is used to ensure that the distance between the positive sample and the anchor point is smaller than the distance between the negative sample and the positive sample; The ternary loss function of deep metric learning is used to make the segmentation network pay more attention to the edge areas of intestinal ultrasound images; The image segmentation module uses the trained segmentation network to perform image segmentation on intestinal ultrasound images.
7. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the intestinal ultrasound image segmentation method based on diffusion network and deep metric learning as described in any one of claims 1 to 5 is implemented.
8. A computer-readable storage medium storing a computer program, characterized in that: The computer program enables the computer to execute the intestinal ultrasound image segmentation method based on diffusion network and deep metric learning as described in any one of claims 1 to 5.
