An edge condition control-based generative medical image segmentation method, system, terminal and storage medium

Abstract

The application relates to the technical field of medical image segmentation, and discloses a generative medical image segmentation method, system, terminal and storage medium based on edge condition control, which comprises the following steps: performing edge extraction on a two-dimensional input image to obtain an edge graph, inputting the two-dimensional input image into a pre-trained VAE encoder to perform coding and obtaining an image hidden vector; performing element-by-element fusion on the two-dimensional input image and the edge graph to obtain an edge synthesis graph, inputting the edge synthesis graph into an edge control module to output an edge control feature; inputting the image hidden vector into a hidden space mapping module to perform prediction and obtaining a mask hidden vector, inputting the mask hidden vector and the edge control feature into a hidden vector gating module to perform optimization and obtaining an optimized edge control feature; and jointly inputting the mask hidden vector and the optimized edge control feature into a pre-trained VAE decoder to perform decoding processing and outputting a predicted segmentation mask. The application improves the segmentation precision of the generative medical image segmentation.

Description

Technical Field

[0001] This invention relates to the field of medical image segmentation technology, and in particular to a generative medical image segmentation method, system, terminal, and computer-readable storage medium based on edge condition control. Background Technology

[0002] Lightweight medical image segmentation models aim to achieve automatic extraction of structural regions with lower computational overhead, representing a key technological direction for deploying medical image analysis algorithms in resource-constrained scenarios. Current mainstream segmentation paradigms can be divided into two categories: one is convolutional networks represented by UNet and its variants, and the other is the Transformer architecture represented by SwinUNet. Both employ a strategy of training from scratch, resulting in a large number of model parameters. Furthermore, their network structure design is not specifically optimized for the inherent imaging characteristics of medical images, such as blurred tissue boundaries and uneven contrast. Directly reducing the number of parameters often leads to a significant decrease in segmentation accuracy.

[0003] In recent years, generative models have demonstrated powerful representation capabilities in visual tasks, providing a new solution for lightweight segmentation. For example, freezing a pre-trained generative model as the backbone network and training only a lightweight mapping network has achieved good segmentation results with a minimal number of parameters. However, these methods mainly rely on the global semantic priors of the generative model and lack explicit modeling of local boundary information. This results in a lack of accurate boundary localization capabilities in segmentation results at tissue junctions or low-contrast regions, making it difficult to meet the application requirements of high-precision structural analysis.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The main objective of this invention is to provide a generative medical image segmentation method, system, terminal, and computer-readable storage medium based on edge condition control. This invention aims to solve the problem that existing medical image segmentation technologies cannot simultaneously achieve lightweight model deployment and fine segmentation of target boundaries. Traditional models trained from scratch have a large number of parameters, and their accuracy drops sharply after lightweighting. While generative lightweight methods have fewer parameters, they lack explicit modeling of edge information, resulting in insufficient boundary localization accuracy.

[0006] To achieve the above objectives, the present invention provides a generative medical image segmentation method based on edge condition control, the generative medical image segmentation method based on edge condition control comprising the following steps: A two-dimensional input image is determined, and edge generation processing is performed on the two-dimensional input image to obtain an edge map. The two-dimensional input image is then input into a pre-trained VAE encoder of a generative medical image segmentation network for encoding processing to obtain image latent vectors. The two-dimensional input image and the edge map are fused element by element to obtain an edge composite image. The edge composite image is then input into the edge control module of the edge control network to output edge control features. The image latent vector is input into the latent space mapping module of the generative medical image segmentation network for prediction to obtain the mask latent vector. The mask latent vector and the edge control features are then input into the latent vector gating module of the edge control network for optimization to obtain the optimized edge control features. The mask latent vector and the optimized edge control features are input together into the pre-trained VAE decoder of the generative medical image segmentation network for decoding, and the predicted segmentation mask is output as the image segmentation result.

[0007] Optionally, in the generative medical image segmentation method based on edge condition control, the pre-trained VAE encoder includes three downsampling blocks; The process involves determining a two-dimensional input image, performing edge generation processing on the two-dimensional input image to obtain an edge map, and then inputting the two-dimensional input image into a pre-trained VAE encoder of a generative medical image segmentation network for encoding processing to obtain image latent vectors. Specifically, this includes: Determine the two-dimensional input image The two-dimensional input image is processed using an edge detection algorithm. Perform edge extraction to generate the corresponding edge map. ; The two-dimensional input image The three downsampling blocks input to the pre-trained VAE encoder are used for image size reduction. After processing by the third downsampling block, the latent vector of the output image is processed. .

[0008] Optionally, in the generative medical image segmentation method based on edge condition control, the edge control network includes an edge control module and a latent vector gating module; the edge control module includes one convolutional block and three downsampling blocks; the latent vector gating module includes three gating modules. The step of fusing the two-dimensional input image and the edge map element-wise to obtain a synthesized edge map, and then inputting the synthesized edge map into the edge control module of the edge control network to output edge control features, specifically includes: For the edge map With the two-dimensional input image By adding and blending elements one by one, an edge composite image is obtained. The edge composite image The convolutional block input to the edge control module is used for preliminary feature extraction to obtain preliminary features; The preliminary features are input into the first downsampling block for downsampling processing to obtain the first downsampling feature. The first downsampling feature is then input into the second downsampling block for downsampling processing to obtain the second downsampling feature. The second downsampling feature is then input into the third downsampling block for downsampling processing to obtain the edge control feature. .

[0009] Optionally, in the generative medical image segmentation method based on edge condition control, the preliminary features are input into a first downsampling block for downsampling processing to obtain a first downsampling feature; the first downsampling feature is input into a second downsampling block for downsampling processing to obtain a second downsampling feature; and the second downsampling feature is input into a third downsampling block for downsampling processing to obtain an edge control feature. Specifically, it includes: The initial features are processed through first-level group convolution, normalization, and non-linear activation, simultaneously completing channel number transformation and spatial downsampling: ; ; in, Indicates preliminary characteristics, Conv For the first-level 3×3 group convolution, the number of groups is... , This indicates the number of input channels for the convolution. This indicates the number of output channels of the convolution. Indicates the feature normalization operator, For the LeakyReLU activation function, Conv For the second-level 3×3 group convolution, the number of groups... , This represents the output of the first-level group convolution. This represents the output of the second-level group convolution; Add a residual branch: ; in, This is a third-level 3×3 group convolution. This represents the output of the residual branch convolution; The main path output and the short-circuit branch output are added element by element to complete the residual feature fusion: ; The fused features After nonlinear activation, the final output of the third downsampling block is obtained: ; in, This indicates edge control features.

[0010] Optionally, in the generative medical image segmentation method based on edge condition control, the latent space mapping module is composed of a two-dimensional convolutional layer, an activation function, a normalization layer and a self-attention layer stacked sequentially. The process involves inputting the image latent vector into the latent space mapping module of the generative medical image segmentation network for prediction to obtain a mask latent vector. The mask latent vector and the edge control features are then input into the first, second, and third gating modules of the latent vector gating module for optimization, outputting a first downsampled feature, a second downsampled feature, and optimized edge control features, respectively. Finally, the mask latent vector and edge control features are input into the latent vector gating module of the edge control network for optimization to obtain optimized edge control features. Specifically, this includes: The image latent vector The input is fed into the latent space mapping module, and then sequentially passes through a two-dimensional convolutional layer, an activation function, a normalization layer, and a self-attention layer for prediction processing to obtain the masked latent vector. ; The mask hidden vector and the edge control features The first gating module of the latent vector gating module outputs a first downsampled feature, which is then input into the second gating module to output a second downsampled feature. This second downsampled feature is then input into the third gating module for optimization, resulting in the optimized edge control feature. .

[0011] Optionally, in the generative medical image segmentation method based on edge condition control, the step of using the mask latent vector... and the edge control features The first gating module of the latent vector gating module outputs a first downsampled feature, which is then input into the second gating module to output a second downsampled feature. This second downsampled feature is then input into the third gating module for optimization, resulting in the optimized edge control feature. Specifically, it includes: The edge control features and the mask hidden vector Upsampling is performed to obtain multi-scale guided features that correspond one-to-one with the spatial dimensions of edge features at each level. An attention gating module based on group convolution is then used to perform adaptive modulation operations on the edge features. ; ; ; ; in, Indicates the first Layer edge control features, and Represents 2D grouped convolution. The intermediate features representing the marginal branches, Indicates the first Mask latent vectors sampled on the layer, Represents the intermediate features of the mask branch. Represents 2D convolution. and This represents two activation functions. Represents the attention gating weight map. Indicates the first Optimized edge control features of the layer; The optimized edge control features are obtained by combining the optimized edge control features of all layers. .

[0012] Optionally, in the generative medical image segmentation method based on edge condition control, the pre-trained VAE decoder includes an intermediate layer, three upsampling blocks, and an output layer. The process involves inputting the mask latent vector and the optimized edge control features into a pre-trained VAE decoder of the generative medical image segmentation network for decoding. The pre-trained VAE decoder performs a nonlinear transformation and feature recalibration on the mask latent vector. During upsampling, the optimized edge control features from the corresponding layer of the latent vector gating module are incorporated and added element-wise to restore the image size and perform mapping processing. A predicted segmentation mask is then output as the image segmentation result. Specifically, this includes: The mask hidden vector and the optimized edge control features The inputs are combined and fed into the pre-trained VAE decoder; The intermediate layer provides the mask hidden vector After performing nonlinear transformation and feature recalibration, the three upsampling blocks incorporate optimized edge control features from the corresponding layer of the latent vector gating module during upsampling. The image size is restored by element-wise addition. The output layer then maps the output of the third upsampling block to produce a predicted segmentation mask. and the predicted segmentation mask As a result of image segmentation.

[0013] Furthermore, to achieve the above objectives, the present invention also provides a generative medical image segmentation system based on edge condition control. This system is used to implement the generative medical image segmentation method based on edge condition control. The generative medical image segmentation system based on edge condition control includes: The image latent vector generation module is used to determine a two-dimensional input image, perform edge generation processing on the two-dimensional input image to obtain an edge map, and input the two-dimensional input image into a pre-trained VAE encoder of a generative medical image segmentation network for encoding processing to obtain image latent vectors; An edge control feature generation module is used to perform element-wise fusion processing on the two-dimensional input image and the edge map to obtain an edge composite map, and input the edge composite map into the edge control module of the edge control network to output edge control features; The edge control feature optimization module is used to input the image latent vector into the latent space mapping module of the generative medical image segmentation network for prediction to obtain the mask latent vector. The mask latent vector and the edge control features are then input into the latent vector gating module of the edge control network for optimization processing to obtain the optimized edge control features. The image segmentation result prediction module is used to input the mask latent vector and the optimized edge control features into the pre-trained VAE decoder of the generative medical image segmentation network for decoding, output a predicted segmentation mask, and use the predicted segmentation mask as the image segmentation result.

[0014] Furthermore, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a generative medical image segmentation program based on edge condition control stored in the memory and executable on the processor, wherein when the generative medical image segmentation program based on edge condition control is executed by the processor, it implements the steps of the generative medical image segmentation method based on edge condition control as described above.

[0015] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a generative medical image segmentation program based on edge condition control, which, when executed by a processor, implements the steps of the generative medical image segmentation method based on edge condition control as described above.

[0016] In this invention, a two-dimensional input image is determined, and edge generation processing is performed on the two-dimensional input image to obtain an edge map. The two-dimensional input image is then input into a pre-trained VAE encoder of a generative medical image segmentation network for encoding processing to obtain an image latent vector. The two-dimensional input image and the edge map are then fused element-wise to obtain an edge composite map. The edge composite map is then input into the edge control module of an edge control network to output edge control features. The image latent vector is then input into the latent space mapping module of the generative medical image segmentation network for prediction to obtain a mask latent vector. The mask latent vector and the edge control features are then input into the latent vector gating module of the edge control network for optimization processing to obtain optimized edge control features. Finally, the mask latent vector and the optimized edge control features are input into the pre-trained VAE decoder of the generative medical image segmentation network for decoding processing to output a predicted segmentation mask, which is then used as the image segmentation result. This invention aims to improve the segmentation accuracy of generative medical image segmentation by explicitly introducing edge prior information. At the same time, it designs a dedicated edge information filtering mechanism to distinguish between effective target edges and noisy edges, thereby suppressing segmentation interference caused by invalid edges. Furthermore, it strictly constrains the scale of parameters while improving segmentation performance, ensuring that the model has lightweight characteristics. Attached Figure Description

[0017] Figure 1 This is a flowchart of a preferred embodiment of the generative medical image segmentation method based on edge condition control of the present invention; Figure 2 This is a schematic diagram of the edge-controlled generative medical image segmentation structure in a preferred embodiment of the generative medical image segmentation method based on edge condition control of the present invention. Figure 3 This is a schematic diagram of the edge control network and pre-trained VAE decoder structure in a preferred embodiment of the generative medical image segmentation method based on edge condition control of the present invention. Figure 4 This is a visual comparison diagram of a preferred embodiment of the generative medical image segmentation method based on edge condition control of the present invention; Figure 5 This is a structural diagram of a preferred embodiment of the generative medical image segmentation system based on edge condition control of the present invention; Figure 6 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0019] The preferred embodiment of the generative medical image segmentation method based on edge condition control described in this invention, such as... Figure 1 and Figure 2 As shown, the generative medical image segmentation method based on edge condition control includes the following steps: Step S10: Determine the two-dimensional input image, perform edge generation processing on the two-dimensional input image to obtain an edge map, and input the two-dimensional input image into the pre-trained VAE encoder of the generative medical image segmentation network for encoding processing to obtain the image latent vector.

[0020] Specifically, such as Figure 2 As shown, the generative medical image segmentation network includes a pre-trained VAE encoder, a latent space mapping module, and a pre-trained VAE decoder; the edge control network includes an edge control module and a latent vector gating module.

[0021] The pre-trained VAE encoder includes three downsampling blocks; it determines the two-dimensional input image. The two-dimensional input image is processed using an edge detection algorithm. Perform edge extraction to generate the corresponding edge map. The two-dimensional input image Input to the pre-trained VAE encoder ( The image size is reduced by three downsampling blocks (the three downsampling blocks process the input image sequentially in a manner of "gradually compressing spatial size and gradually expanding channel dimension". For example, the first downsampling block performs low-level feature extraction, the second downsampling block performs mid-level semantic extraction, and the third downsampling block performs high-level semantic compression. These three downsampling blocks not only complete spatial compression, but also realize the encoding transformation from "concrete pixels" to "abstract semantics" by increasing the number of channels layer by layer. The final output latent vector is the compact expression of the original image in the latent space). After processing by the third downsampling block, the output image latent vector is... That is, each downsampling layer reduces the image size, and the output of the last downsampling layer is the image latent vector. .

[0022] Step S20: Perform element-wise fusion processing on the two-dimensional input image and the edge map to obtain an edge composite image. Input the edge composite image into the edge control module of the edge control network and output the edge control features.

[0023] Specifically, such as Figure 3 As shown, the edge control module includes one convolutional block and three downsampling blocks; for the edge map With the two-dimensional input image By adding and blending elements one by one, an edge composite image is obtained. The edge composite image Input to the edge control module ( The Edge Control Module performs preliminary feature extraction on convolutional blocks to obtain preliminary features. These preliminary features are then input into a first downsampling block for downsampling processing to obtain first downsampling features. The first downsampling features are then input into a second downsampling block for downsampling processing to obtain second downsampling features. The second downsampling features are then input into a third downsampling block for downsampling processing to obtain edge control features. .

[0024] Due to morphological variations and low image quality, medical images often result in inaccurate edge segmentation. Existing generative medical image segmentation techniques have not specifically addressed these edge accuracy deficiencies, making it difficult to meet the application requirements of high-precision medical image segmentation. Therefore, to address the shortcomings of existing technologies, this invention proposes an edge control module that introduces image edge information as an additional conditional control signal into the segmentation network. This guides the model to strengthen its feature perception of edge regions during feature decoding, thereby optimizing the edge quality of the segmentation results.

[0025] like Figure 3 As shown, the edge composite image is... Input to edge control module ( The convolutional blocks are used for initial feature extraction, followed by multi-level downsampling. All convolutional operations in the downsampling blocks are implemented using a group convolutional structure.

[0026] A downsampling block consists of two parts. One part processes the initial features through first-level group convolution, normalization, and non-linear activation, simultaneously completing channel number transformation and spatial downsampling. ; ; in, Indicates preliminary characteristics, Conv For the first-level 3×3 group convolution, the number of groups is... That is, the smaller value in the input / output channels. This indicates the number of input channels for the convolution. This indicates the number of output channels of the convolution. Indicates the feature normalization operator, For the LeakyReLU activation function, Conv For the second-level 3×3 group convolution, the number of groups... , This represents the output of the first-level group convolution. This represents the output of the second-level group convolution.

[0027] Additionally, add a residual branch: ; in, This is a third-level 3×3 group convolution. This represents the output of the residual branch convolution.

[0028] The main path output and the short-circuit branch output are added element by element to complete the residual feature fusion: ; The fused features After nonlinear activation, the final output of the third downsampling block is obtained: ; in, This indicates edge control features.

[0029] This downsampling effectively controls the number of module parameters and computational complexity while ensuring the ability to extract edge features in dimensionality reduction, thus meeting the lightweight requirements of generative medical image segmentation models. After multi-level downsampling, edge control features (or multi-scale edge feature maps) are obtained. The dimensions of each level of the edge control features correspond one-to-one with the dimensions of the corresponding level features in the upsampling part of the pre-trained VAE decoder of the generative medical image segmentation network, achieving alignment between edge control features and decoder features. The aligned edge features at each level are injected into the feature map of the corresponding level of the pre-trained VAE decoder in an element-wise addition manner. This injection method can complete the fusion of edge control signals and decoded features without introducing additional learnable parameters, which can improve the model's ability to perceive edges and thus enhance the segmentation effect.

[0030] After injecting edge control information into the pre-trained VAE decoder, the model utilizes edge prior information to constrain the segmentation network without significantly increasing the model's computational overhead. This accurately improves the problems of blurred edge segmentation and contour distortion, effectively enhancing the segmentation accuracy of existing generative medical image segmentation methods.

[0031] Step S30: Input the image latent vector into the latent space mapping module of the generative medical image segmentation network for prediction to obtain the mask latent vector. Input the mask latent vector and the edge control features into the latent vector gating module of the edge control network for optimization to obtain the optimized edge control features.

[0032] Specifically, image latent vectors Latent vectors belonging to the image space are used to reconstruct images in VAE (Variational Autoencoder). However, the segmentation task of this invention requires the generation of a segmentation mask rather than the reconstruction of an image. Therefore, it is necessary to spatially map the image latent vectors into mask latent vectors so that the decoder can decode the segmentation result based on the mapped latent vectors.

[0033] The latent space mapping module consists of a stacked 2D convolutional layer, activation function, normalization layer, and self-attention layer. Thanks to the small spatial size of the latent vectors, introducing self-attention only slightly increases the computational cost. Furthermore, the latent space mapping module employs a skip connection structure to alleviate the gradient vanishing problem and prevent the loss of semantically relevant features. The latent vectors of the image... Input to the latent space mapping module ( The hidden vector is obtained by sequentially passing it through a two-dimensional convolutional layer, an activation function, a normalization layer, and a self-attention layer for prediction processing. .

[0034] The mask hidden vector and the edge control features The inputs are combined into the latent vector gating module ( The first gate module of the Latent Vector Gating Module outputs a first downsampled feature. This first downsampled feature is then input into the second gate module of the Latent Vector Gating Module, which outputs a second downsampled feature. This second downsampled feature is then input into the third gate module of the Latent Vector Gating Module for optimization processing, resulting in optimized edge control features. .

[0035] Since the aforementioned edge map is pre-generated by the edge detection algorithm, it inevitably contains false edges and background noise. If the edge control features are directly injected into the decoder, it is easy to cause noise pollution to the decoding features, resulting in distortion of the edge control information and weakening the effect of edge control on improving segmentation accuracy.

[0036] To address the aforementioned issues, this invention proposes a latent vector gating module, positioned before the fusion of edge control signals and decoded features, to generate higher-quality edge control signals. This module introduces latent vectors as guiding priors, obtained by the encoder of the segmentation network through deep feature extraction of the input image. These latent vectors contain high-level prior information such as the global tissue distribution and semantic category of the input image, effectively distinguishing real target tissue boundaries from background noise. An adaptive weighted filtering of edge features is performed through a gating modulation mechanism, suppressing noise interference while preserving effective edge information and improving the reliability of the edge control signal.

[0037] The specific implementation process is as follows: First, the edge control features are... and the mask hidden vector Upsampling is performed to obtain multi-scale guided features that correspond one-to-one with the spatial dimensions of edge features at each level. To balance modulation functionality with model lightweighting requirements, this invention employs an attention gating module based on group convolution to perform adaptive modulation operations on edge features. ; ; ; ; in, Indicates the first Layer edge control features, and This represents a 2D grouped convolution, corresponding to the number of groups. That is, the smaller value in the input / output channels. The intermediate features representing the marginal branches, Indicates the first Mask latent vectors sampled on the layer, Represents 2D grouped convolution. Represents the intermediate features of the mask branch. Represents 2D convolution. and This represents two activation functions. Represents the attention gating weight map. Indicates the first Optimized edge control features of the layer.

[0038] The optimized edge control features are obtained by combining the optimized edge control features of all layers. .

[0039] After attention-gated adaptive modulation, the edge feature weights corresponding to the true target boundary are actively enhanced, while the feature weights corresponding to noise and pseudo-edges are suppressed, ultimately resulting in a set of semantically enhanced, noise-controlled, high-quality multi-scale edge features (i.e., optimized edge control features). These optimized edge control features are then injected into the corresponding layers of the pre-trained VAE decoder, which can accurately guide the model to strengthen feature learning of the target boundary region while avoiding noise interference, further improving the segmentation results.

[0040] Step S40: Input the mask latent vector and the optimized edge control features into the pre-trained VAE decoder of the generative medical image segmentation network for decoding, output the predicted segmentation mask, and use the predicted segmentation mask as the image segmentation result.

[0041] Specifically, such as Figure 3As shown, the pre-trained VAE decoder ( It includes an intermediate layer, three upsampling blocks, and an output layer; the mask hidden vector is... and the optimized edge control features The common input is fed into the pre-trained VAE decoder; the intermediate layer inputs the mask latent vector. Nonlinear transformations and feature recalibration (such as fully connected or convolutional transformations) are performed to map latent vectors to feature representations suitable for decoding (without restoring size); during upsampling, the three upsampling blocks incorporate optimized edge control features from the corresponding layers of the latent vector gating module. Element-wise addition is performed, with each upsampling block containing interpolation or transpose convolutions to progressively enlarge the spatial size of the current feature map and restore the image size. The output layer (typically composed of 1×1 convolutions (or ordinary convolutions) combined with Sigmoid / Softmax activation functions to map the high-dimensional feature map to the required number of classification channels) maps the output of the third upsampling block and outputs a predicted segmentation mask. and the predicted segmentation mask As a result of image segmentation.

[0042] This invention further incorporates the optimized edge control features output by the latent vector gating module after the upsampling process of the pre-trained VAE decoder. Specifically, the upsampling output of each layer and the optimized edge control features of the corresponding layer are added element by element to achieve the integration of edge control information. This parameter-free design ensures the lightweight nature of the model.

[0043] The edge-controlled generative medical image segmentation algorithm of this invention is shown below:

[0044] The core innovation of this invention lies in proposing a lightweight edge control mechanism with gated filtering for generative medical image segmentation scenarios. For the first time, it introduces explicit edge condition control into the generative segmentation architecture. Image edge information extracted by edge detection algorithms is downsampled at multiple levels to generate multi-scale features, which are then injected into the decoder layer by layer, specifically addressing the shortcomings of insufficient edge accuracy in medical image segmentation. Simultaneously, the latent vector output by the encoder is introduced as a high-level semantic prior. An attention-gating module based on group convolution is used to adaptively weight and modulate the edge features, suppressing noise and pseudo-edge interference in the pre-generated edge map from a semantic level, ensuring the accuracy and effectiveness of the edge control signal. Furthermore, the entire edge processing chain uses group convolution to achieve feature transformation, and fuses with the decoded features using a zero-parameter, element-wise addition method. This achieves edge constraints while strictly controlling the number of model parameters and computational overhead, thus meeting the requirements for lightweight design.

[0045] This invention validates the model performance on a breast ultrasound dataset containing 647 breast ultrasound images. The experiment involved randomly selecting 80% of the samples as the training set, with the remaining samples forming the test set. This invention selects the Destination Similarity Coefficient (DSC), Intersection over Union (IoU), and the number of model parameters (params), all commonly used in medical segmentation, as quantitative evaluation metrics to comprehensively measure segmentation accuracy and lightweight design.

[0046] The comparative experimental results of various methods are shown in Table 1. Traditional UNet and SwinUNet have as many as 14.0M and 27.2M parameters, respectively, and their segmentation accuracy has significant shortcomings. The baseline generative segmentation model GMS compresses the number of parameters to 1.5M, improving DSC and IoU to 81.43 and 72.58, respectively. Compared to GMS, this invention only slightly increases the number of parameters by 0.1M to 1.6M, improving DSC by 1.47 and IoU by 1.68, achieving a significant increase in segmentation accuracy. The experimental results fully demonstrate that this invention can achieve a significant improvement in segmentation performance with extremely low resource increment, balancing the advantages of lightweight design and high accuracy.

[0047] Table 1: Comparison of Quantitative Indicators of Different Models

[0048] The comparison results of the visualization effects are as follows: Figure 4 As shown, Figure 4 In the image, (a) represents the input image. Figure 4 (b) in the text represents the segmentation label. Figure 4 (c) in the text represents the GMS result. Figure 4 (d) in the figure represents the result of the present invention; compared with the GMS baseline method, the present invention can more accurately depict the fine contours of the target region, effectively correct the edge blurring, local undersegmentation and contour distortion problems existing in the baseline, and the segmentation results are more consistent with the real labels, further verifying the promoting effect of edge optimization and gating guidance mechanism on fine segmentation.

[0049] To verify the effectiveness of each core module proposed in this invention, ablation experiments were conducted based on the BUSI dataset. The experimental results are shown in Table 2. Using GMS as the baseline model, its original DSC and IoU were 81.43 and 72.58, respectively. Introducing only the edge control module slightly improved the model's segmentation performance, with DSC increasing to 81.68 and IoU to 72.77, demonstrating that explicit edge information input can effectively compensate for the weak edge modeling ability of generative models and has a positive optimization effect on edge segmentation accuracy. Further adding a latent vector gating module significantly improved the model's performance, with DSC and IoU reaching optimal values ​​of 82.90 and 74.26, respectively. Experimental results show that the edge control module and the latent vector gating module have good complementary gain effects. The collaborative work of the two modules allows the model to focus on edge features and filter noise information, significantly improving overall segmentation accuracy while maintaining the model's lightweight characteristics.

[0050] Table 2: Ablation Experiment Results

[0051] Furthermore, such as Figure 5 As shown, based on the above-described generative medical image segmentation method based on edge condition control, this invention also provides a generative medical image segmentation system based on edge condition control, wherein the generative medical image segmentation system based on edge condition control includes: The image latent vector generation module 51 is used to determine a two-dimensional input image, perform edge generation processing on the two-dimensional input image to obtain an edge map, and input the two-dimensional input image into a pre-trained VAE encoder of a generative medical image segmentation network for encoding processing to obtain an image latent vector. The edge control feature generation module 52 is used to perform element-wise fusion processing on the two-dimensional input image and the edge map to obtain an edge composite map, input the edge composite map into the edge control module of the edge control network, and output edge control features; The edge control feature optimization module 53 is used to input the image latent vector into the latent space mapping module of the generative medical image segmentation network for prediction to obtain the mask latent vector, and input the mask latent vector and the edge control features together into the latent vector gating module of the edge control network for optimization processing to obtain the optimized edge control features. The image segmentation result prediction module 54 is used to input the mask latent vector and the optimized edge control features into the pre-trained VAE decoder of the generative medical image segmentation network for decoding, output the predicted segmentation mask, and use the predicted segmentation mask as the image segmentation result.

[0052] Furthermore, such as Figure 6 As shown, based on the above-mentioned generative medical image segmentation method and system based on edge condition control, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 6 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0053] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a generative medical image segmentation program 40 based on edge condition control, which can be executed by the processor 10 to implement the generative medical image segmentation method based on edge condition control in this application.

[0054] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the generative medical image segmentation method based on edge condition control.

[0055] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The terminal's processor 10, memory 20, and display 30 communicate with each other via a system bus.

[0056] In one embodiment, when the processor 10 executes the edge-condition-controlled generative medical image segmentation program 40 in the memory 20, the following steps are performed: A two-dimensional input image is determined, and edge generation processing is performed on the two-dimensional input image to obtain an edge map. The two-dimensional input image is then input into a pre-trained VAE encoder of a generative medical image segmentation network for encoding processing to obtain image latent vectors. The two-dimensional input image and the edge map are fused element by element to obtain an edge composite image. The edge composite image is then input into the edge control module of the edge control network to output edge control features. The image latent vector is input into the latent space mapping module of the generative medical image segmentation network for prediction to obtain the mask latent vector. The mask latent vector and the edge control features are then input into the latent vector gating module of the edge control network for optimization to obtain the optimized edge control features. The mask latent vector and the optimized edge control features are input together into the pre-trained VAE decoder of the generative medical image segmentation network for decoding, and the predicted segmentation mask is output as the image segmentation result.

[0057] The pre-trained VAE encoder includes three downsampling blocks; The process involves determining a two-dimensional input image, performing edge generation processing on the two-dimensional input image to obtain an edge map, and then inputting the two-dimensional input image into a pre-trained VAE encoder of a generative medical image segmentation network for encoding processing to obtain image latent vectors. Specifically, this includes: Determine the two-dimensional input image The two-dimensional input image is processed using an edge detection algorithm. Perform edge extraction to generate the corresponding edge map. ; The two-dimensional input image The three downsampling blocks input to the pre-trained VAE encoder are used for image size reduction. After processing by the third downsampling block, the latent vector of the output image is processed. .

[0058] The edge control network includes an edge control module and a latent vector gating module; the edge control module includes one convolutional block and three downsampling blocks; the latent vector gating module includes three gating modules. The step of fusing the two-dimensional input image and the edge map element-wise to obtain a synthesized edge map, and then inputting the synthesized edge map into the edge control module of the edge control network to output edge control features, specifically includes: For the edge map With the two-dimensional input image By adding and blending elements one by one, an edge composite image is obtained. The edge composite image The convolutional block input to the edge control module is used for preliminary feature extraction to obtain preliminary features; The preliminary features are input into the first downsampling block for downsampling processing to obtain the first downsampling feature. The first downsampling feature is then input into the second downsampling block for downsampling processing to obtain the second downsampling feature. The second downsampling feature is then input into the third downsampling block for downsampling processing to obtain the edge control feature. .

[0059] Specifically, the preliminary features are input into a first downsampling block for downsampling processing to obtain a first downsampling feature; the first downsampling feature is input into a second downsampling block for downsampling processing to obtain a second downsampling feature; and the second downsampling feature is input into a third downsampling block for downsampling processing to obtain an edge control feature. Specifically, it includes: The initial features are processed through first-level group convolution, normalization, and non-linear activation, simultaneously completing channel number transformation and spatial downsampling: ; ; in, Indicates preliminary characteristics, Conv For the first-level 3×3 group convolution, the number of groups is... , This indicates the number of input channels for the convolution. This indicates the number of output channels of the convolution. Indicates the feature normalization operator, For the LeakyReLU activation function, Conv For the second-level 3×3 group convolution, the number of groups... , This represents the output of the first-level group convolution. This represents the output of the second-level group convolution; Add a residual branch: ; in, This is a third-level 3×3 group convolution. This represents the output of the residual branch convolution; The main path output and the short-circuit branch output are added element by element to complete the residual feature fusion: ; The fused features After nonlinear activation, the final output of the third downsampling block is obtained: ; in, This indicates edge control features.

[0060] The latent space mapping module is composed of a two-dimensional convolutional layer, an activation function, a normalization layer and a self-attention layer stacked in sequence. The process involves inputting the image latent vector into the latent space mapping module of the generative medical image segmentation network for prediction to obtain a mask latent vector. The mask latent vector and the edge control features are then input together into the latent vector gating module of the edge control network for optimization to obtain optimized edge control features. Specifically, this includes: The image latent vector The input is fed into the latent space mapping module, and then sequentially passes through a two-dimensional convolutional layer, an activation function, a normalization layer, and a self-attention layer for prediction processing to obtain the masked latent vector. ; The mask hidden vector and the edge control features The first gating module of the latent vector gating module outputs a first downsampled feature, which is then input into the second gating module to output a second downsampled feature. This second downsampled feature is then input into the third gating module for optimization, resulting in the optimized edge control feature. .

[0061] Wherein, the mask hidden vector and the edge control features The first gating module of the latent vector gating module outputs a first downsampled feature, which is then input into the second gating module to output a second downsampled feature. This second downsampled feature is then input into the third gating module for optimization, resulting in the optimized edge control feature. Specifically, it includes: The edge control features and the mask hidden vector Upsampling is performed to obtain multi-scale guided features that correspond one-to-one with the spatial dimensions of edge features at each level. An attention gating module based on group convolution is then used to perform adaptive modulation operations on the edge features. ; ; ; ; in, Indicates the first Layer edge control features, and Represents 2D grouped convolution. The intermediate features representing the marginal branches, Indicates the first Mask latent vectors sampled on the layer, Represents the intermediate features of the mask branch. Represents 2D convolution. and This represents two activation functions. Represents the attention gating weight map. Indicates the first Optimized edge control features of the layer; The optimized edge control features are obtained by combining the optimized edge control features of all layers. .

[0062] The pre-trained VAE decoder includes an intermediate layer, three upsampling blocks, and an output layer. The process of inputting the mask latent vector and the optimized edge control features into the pre-trained VAE decoder of the generative medical image segmentation network for decoding, outputting a predicted segmentation mask, and using the predicted segmentation mask as the image segmentation result specifically includes: The mask hidden vector and the optimized edge control features The inputs are combined and fed into the pre-trained VAE decoder; The intermediate layer provides the mask hidden vector After performing nonlinear transformation and feature recalibration, the three upsampling blocks incorporate optimized edge control features from the corresponding layer of the latent vector gating module during upsampling. The image size is restored by element-wise addition. The output layer then maps the output of the third upsampling block to produce a predicted segmentation mask. and the predicted segmentation mask As a result of image segmentation.

[0063] The present invention also provides a computer-readable storage medium storing a generative medical image segmentation program based on edge condition control, wherein the generative medical image segmentation program based on edge condition control, when executed by a processor, implements the steps of the generative medical image segmentation method based on edge condition control as described above.

[0064] In summary, this invention provides a generative medical image segmentation method, system, terminal, and computer-readable storage medium based on edge condition control. The method includes: determining a two-dimensional input image; performing edge generation processing on the two-dimensional input image to obtain an edge map; inputting the two-dimensional input image into a pre-trained VAE encoder of a generative medical image segmentation network for encoding processing to obtain an image latent vector; performing element-wise fusion processing on the two-dimensional input image and the edge map to obtain an edge composite map; inputting the edge composite map into an edge control module of an edge control network to output edge control features; inputting the image latent vector into a latent space mapping module of the generative medical image segmentation network for prediction to obtain a mask latent vector; inputting the mask latent vector and the edge control features together into a latent vector gating module of the edge control network for optimization processing to obtain optimized edge control features; inputting the mask latent vector and the optimized edge control features together into a pre-trained VAE decoder of the generative medical image segmentation network for decoding processing to output a predicted segmentation mask, and using the predicted segmentation mask as the image segmentation result. This invention aims to improve the segmentation accuracy of generative medical image segmentation by explicitly introducing edge prior information. At the same time, it designs a dedicated edge information filtering mechanism to distinguish between effective target edges and noisy edges, thereby suppressing segmentation interference caused by invalid edges. Furthermore, it strictly constrains the scale of parameters while improving segmentation performance, ensuring that the model has lightweight characteristics.

[0065] It should be noted that, in this invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0066] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0067] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A generative medical image segmentation method based on edge condition control, characterized in that, The generative medical image segmentation method based on edge condition control is implemented based on a generative medical image segmentation network and an edge control network. The generative medical image segmentation network includes a pre-trained VAE encoder, a latent space mapping module, and a pre-trained VAE decoder. The edge control network includes an edge control module and a latent vector gating module. The generative medical image segmentation method based on edge condition control includes: A two-dimensional input image is determined, and edge generation processing is performed on the two-dimensional input image to obtain an edge map. The two-dimensional input image is then input into a pre-trained VAE encoder of a generative medical image segmentation network for encoding processing to obtain image latent vectors. The two-dimensional input image and the edge map are fused element by element to obtain an edge composite image. The edge composite image is then input into the edge control module of the edge control network to output edge control features. The latent vector of the image is input into the latent space mapping module of the generative medical image segmentation network for prediction to obtain the mask latent vector. The mask latent vector and the edge control features are input into the first gate module, the second gate module and the third gate module of the latent vector gate module for optimization processing, and the first downsampled feature, the second downsampled feature and the optimized edge control feature are output respectively. The mask latent vector and the optimized edge control features are jointly input into the pre-trained VAE decoder of the generative medical image segmentation network for decoding. The pre-trained VAE decoder performs nonlinear transformation and feature recalibration on the mask latent vector. During upsampling, the optimized edge control features from the corresponding layer of the latent vector gating module are added element-wise to restore the image size and perform mapping processing. The predicted segmentation mask is then output and used as the image segmentation result. The latent space mapping module is composed of a two-dimensional convolutional layer, an activation function, a normalization layer and a self-attention layer stacked in sequence; The process involves inputting the image latent vector into the latent space mapping module of the generative medical image segmentation network for prediction to obtain a mask latent vector. The mask latent vector and the edge control features are then input into the first, second, and third gating modules of the latent vector gating module for optimization, outputting a first downsampled feature, a second downsampled feature, and optimized edge control features, respectively. Specifically, this includes: The image latent vector The input is fed into the latent space mapping module, and then sequentially passes through a two-dimensional convolutional layer, an activation function, a normalization layer, and a self-attention layer for prediction processing to obtain the masked latent vector. ; The mask hidden vector and the edge control features The first gating module of the latent vector gating module outputs a first downsampled feature, which is then input into the second gating module to output a second downsampled feature. This second downsampled feature is then input into the third gating module for optimization, resulting in the optimized edge control feature. ; The hidden vector of the mask and the edge control features The first gating module of the latent vector gating module outputs a first downsampled feature, which is then input into the second gating module to output a second downsampled feature. This second downsampled feature is then input into the third gating module for optimization, resulting in the optimized edge control feature. Specifically, it includes: The edge control features and the mask hidden vector Upsampling is performed to obtain multi-scale guided features that correspond one-to-one with the spatial dimensions of edge features at each level. An attention gating module based on group convolution is then used to perform adaptive modulation operations on the edge features. ； ； ； ； in, Indicates the first Layer edge control features, and Represents 2D grouped convolution. The intermediate features representing the marginal branches, Indicates the first Mask latent vectors sampled on the layer, Represents the intermediate features of the mask branch. Represents 2D convolution. and This represents two activation functions. Represents the attention gating weight map. Indicates the first Optimized edge control features of the layer; The optimized edge control features are obtained by combining the optimized edge control features of all layers. ; The pre-trained VAE decoder includes an intermediate layer, three upsampling blocks, and an output layer; The process involves inputting the mask latent vector and the optimized edge control features into a pre-trained VAE decoder of the generative medical image segmentation network for decoding. The pre-trained VAE decoder performs a non-linear transformation and feature recalibration on the mask latent vector. During upsampling, the optimized edge control features from the corresponding layer of the latent vector gating module are incorporated and added element-wise to restore the image size and perform mapping processing. A predicted segmentation mask is then output, and this predicted segmentation mask is used as the image segmentation result. Specifically, this includes: The mask hidden vector and the optimized edge control features The inputs are combined and fed into the pre-trained VAE decoder; The intermediate layer provides the mask hidden vector After performing nonlinear transformation and feature recalibration, the three upsampling blocks incorporate optimized edge control features from the corresponding layer of the latent vector gating module during upsampling. The image size is restored by element-wise addition. The output layer then maps the output of the third upsampling block to produce a predicted segmentation mask. and the predicted segmentation mask As a result of image segmentation.

2. The generative medical image segmentation method based on edge condition control according to claim 1, characterized in that, The pre-trained VAE encoder includes three downsampling blocks; The process involves determining a two-dimensional input image, performing edge generation processing on the two-dimensional input image to obtain an edge map, and then inputting the two-dimensional input image into a pre-trained VAE encoder of a generative medical image segmentation network for encoding processing to obtain image latent vectors. Specifically, this includes: Determine the two-dimensional input image The two-dimensional input image is processed using an edge detection algorithm. Perform edge extraction to generate the corresponding edge map. ; The two-dimensional input image The three downsampling blocks input to the pre-trained VAE encoder are used for image size reduction. After processing by the third downsampling block, the latent vector of the output image is processed. .

3. The generative medical image segmentation method based on edge condition control according to claim 2, characterized in that, The edge control network includes an edge control module and a latent vector gating module; the edge control module includes one convolutional block and three downsampling blocks; the latent vector gating module includes three gating modules. The step of fusing the two-dimensional input image and the edge map element-wise to obtain a synthesized edge map, and then inputting the synthesized edge map into the edge control module of the edge control network to output edge control features, specifically includes: For the edge map With the two-dimensional input image By adding and blending elements one by one, an edge composite image is obtained. The edge composite image The convolutional block input to the edge control module is used for preliminary feature extraction to obtain preliminary features; The preliminary features are input into the first downsampling block for downsampling processing to obtain the first downsampling feature. The first downsampling feature is then input into the second downsampling block for downsampling processing to obtain the second downsampling feature. The second downsampling feature is then input into the third downsampling block for downsampling processing to obtain the edge control feature. .

4. The generative medical image segmentation method based on edge condition control according to claim 3, characterized in that, The preliminary features are input into the first downsampling block for downsampling processing to obtain the first downsampling feature. The first downsampling feature is then input into the second downsampling block for downsampling processing to obtain the second downsampling feature. The second downsampling feature is then input into the third downsampling block for downsampling processing to obtain the edge control feature. Specifically, it includes: The initial features are processed through first-level group convolution, normalization, and non-linear activation, simultaneously completing channel number transformation and spatial downsampling: ； ； in, Indicates preliminary characteristics, For the first-level 3×3 group convolution, the number of groups is... , This indicates the number of input channels for the convolution. This indicates the number of output channels of the convolution. Indicates the feature normalization operator, The LeakyReLU activation function is used. For the second-level 3×3 group convolution, the number of groups... , This represents the output of the first-level group convolution. This represents the output of the second-level group convolution; Add a residual branch: ； in, This is a third-level 3×3 group convolution. This represents the output of the residual branch convolution; The main path output and the short-circuit branch output are added element by element to complete the residual feature fusion: ； The fused features After nonlinear activation, the final output of the third downsampling block is obtained: ； in, This indicates edge control features.

5. A generative medical image segmentation system based on edge condition control, characterized in that, The edge-condition-controlled generative medical image segmentation system is used to implement the edge-condition-controlled generative medical image segmentation method according to any one of claims 1-4, wherein the edge-condition-controlled generative medical image segmentation system comprises: The image latent vector generation module is used to determine a two-dimensional input image, perform edge generation processing on the two-dimensional input image to obtain an edge map, and input the two-dimensional input image into a pre-trained VAE encoder of a generative medical image segmentation network for encoding processing to obtain image latent vectors; An edge control feature generation module is used to perform element-wise fusion processing on the two-dimensional input image and the edge map to obtain an edge composite map, and input the edge composite map into the edge control module of the edge control network to output edge control features; The edge control feature optimization module is used to input the image latent vector into the latent space mapping module of the generative medical image segmentation network for prediction to obtain the mask latent vector. The mask latent vector and the edge control features are then input into the latent vector gating module of the edge control network for optimization processing to obtain the optimized edge control features. The image segmentation result prediction module is used to input the mask latent vector and the optimized edge control features into the pre-trained VAE decoder of the generative medical image segmentation network for decoding, output a predicted segmentation mask, and use the predicted segmentation mask as the image segmentation result.

6. A terminal, characterized in that, The terminal includes: a memory, a processor, and a generative medical image segmentation program based on edge condition control stored in the memory and executable on the processor. When the generative medical image segmentation program based on edge condition control is executed by the processor, it implements the steps of the generative medical image segmentation method based on edge condition control as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a generative medical image segmentation program based on edge condition control, which, when executed by a processor, implements the steps of the generative medical image segmentation method based on edge condition control as described in any one of claims 1-4.

Images