A Curve Structure Segmentation Method and System Based on Edge-Skeleton Dual Constraints

Abstract

This invention relates to the field of artificial intelligence technology, specifically to a method and system for segmenting curved structures based on edge-skeleton dual constraints. The method includes: acquiring a curved target image; preprocessing the curved target image to obtain an edge map, a skeleton map, and text prompts; inputting the edge map, skeleton map, and text prompts into an edge-skeleton dual constraint generator for training to obtain a trained edge-skeleton dual constraint generator; using the trained edge-skeleton dual constraint generator to generate a synthetic image-mask pair; fusing the synthetic image-mask pair with a real dataset to obtain a training dataset; training a curved structure segmentation model using the training dataset to obtain a trained curved structure segmentation model; and inputting the curved target image to be segmented into the trained curved structure segmentation model to obtain the segmentation result. This invention can improve the performance of curved structure segmentation.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method and system for segmenting curve structures based on edge-skeleton dual constraints. Background Technology

[0002] Segmentation tasks involving curvilinear structures, such as blood vessels, concrete cracks, and road networks, differ significantly from the segmentation of common objects due to their unique morphological features. Furthermore, this task is further constrained by data scarcity, annotation difficulties, and the specialized knowledge required across application domains.

[0003] Traditional segmentation methods often suffer from fragmentation or oversegmentation when dealing with such structures due to insufficient modeling capabilities for slender topologies and complex intersection regions. While deep learning methods improve performance through end-to-end learning, in small sample scenarios, the models are susceptible to data distribution biases and struggle to accurately capture the global continuity and local details of curved structures. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a method and system for segmenting curved structures based on edge-skeleton dual constraints, aiming to improve the segmentation performance of curved structures.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] On one hand, embodiments of the present invention provide a curve structure segmentation method based on edge-skeleton dual constraints, the method comprising the following steps:

[0007] S100, acquire the curved target image, preprocess the curved target image to obtain the edge map, skeleton map and text prompt;

[0008] S200, Construct an edge-skeleton dual-constraint generator based on a stable diffusion model and a control network. Input the edge map, skeleton map and text prompt into the edge-skeleton dual-constraint generator for training to obtain a trained edge-skeleton dual-constraint generator.

[0009] S300: Use the trained edge-skeleton dual-constraint generator to generate synthetic image-mask pairs, and fuse the synthetic image-mask pairs with the real dataset to obtain the training dataset;

[0010] S400, Use the training dataset to train the curved structure segmentation model to obtain the trained curved structure segmentation model;

[0011] S500 inputs the curved target image to be segmented into the trained curved structure segmentation model to obtain the segmentation result.

[0012] Optionally, in S100, the preprocessing of the curved target image to obtain an edge map, a skeleton map, and a text prompt includes:

[0013] S110, unify the resolution of the curved target image;

[0014] S120: The curved target image after uniform resolution is smoothed with a Gaussian kernel to obtain a smoothed image; the magnitude and direction of the smoothed image are calculated by Sobel gradient, and non-maximum suppression is performed on the magnitude and direction to obtain an edge map with a width of one pixel.

[0015] S130, using the mask of the curved target image as input, perform Euclidean distance transformation to obtain the skeleton map;

[0016] S140, Build the semantic prompt building submodule to generate text prompts.

[0017] Optionally, in S130, the step of performing Euclidean distance transformation using the mask of the curved target image as input to obtain a skeleton map includes:

[0018] The skeleton map is obtained by calculating the shortest distance from each pixel to the nearest mask boundary using the Euclidean distance transformation formula.

[0019] The Euclidean distance transformation formula is as follows:

[0020] ;in, For mask boundaries, Let p be any pixel position in the image grid, and let p be any boundary pixel position within the boundary. Let x be the Euclidean distance from pixel x to the boundary ∂M.

[0021] Optionally, in S140, the semantic prompt construction submodule generates text prompts, including:

[0022] S141, use ChatGPT to develop a stable cue word template that covers the morphological characteristics, density distribution, directional trend and imaging conditions of curved targets;

[0023] S142, the preprocessed curved target image is input into the Qwen2.5-VL model, which generates a structured description text for the current curved target image based on the prompt word template; the structured description includes shape, density, orientation and imaging conditions;

[0024] S143, The generated structured descriptions are manually sampled and solidified to form standardized text prompts.

[0025] Optionally, in S200, the construction of an edge-skeleton dual-constraint generator based on a stable diffusion model and a control network, wherein the edge map, skeleton map, and text prompt are input into the edge-skeleton dual-constraint generator for training to obtain a trained edge-skeleton dual-constraint generator, includes:

[0026] S210, Construct an edge-skeleton dual-constraint generator based on a stable diffusion model and a control network. The stable diffusion model includes a variational autoencoder, a multimodal Transformer denoiser, and multiple text encoders.

[0027] S220, the variational autoencoder is used to compress the real curved target image to obtain the compressed image in the latent space; multiple text encoders are used to encode the text prompts into corresponding text vectors, and then the vectors are concatenated to obtain a comprehensive text vector; the diffusion step is mapped to a time vector through time step embedding.

[0028] S230, the edge map and skeleton map are stacked in two channels to construct structural conditions, and then input into the control network to output structural constraint features;

[0029] S240, During the training phase, noise samples are added to the compressed image to obtain a noisy image. The noisy image, the integrated text vector, the time vector, and the structural constraint features are input to the multimodal Transformer denoiser, and the predicted noise is output.

[0030] S250 constructs a loss function by comparing the difference between predicted noise and noise samples, optimizes model parameters using backpropagation, and iterates training until the loss function converges to obtain a trained edge-skeleton dual-constraint generator.

[0031] Optionally, the loss function is:

[0032] ;

[0033] in, For loss function, This represents the average error across all possible cases, where t represents the time step. Indicates at time step Potential noise, It is a text embedding obtained from a pre-trained text encoder. Indicates a noise sample. This refers to the Transformer denoising function. These are the model parameters for the Transformer denoiser.

[0034] Optionally, in S300, the process of generating synthetic image-mask pairs using a trained edge-skeleton dual-constraint generator, and fusing the synthetic image-mask pairs with the real dataset to obtain a training dataset, includes:

[0035] S310 takes manually annotated masks and text prompts as control conditions and inputs them into a trained edge-skeleton dual-constraint generator to generate multiple synthetic image-mask pairs.

[0036] S320, the synthesized image-mask pair is mixed with the real dataset to obtain the training dataset.

[0037] On the other hand, embodiments of the present invention provide a curve structure segmentation system based on edge-skeleton dual constraints, comprising:

[0038] At least one processor;

[0039] At least one memory for storing at least one program;

[0040] When the at least one program is executed by the at least one processor, the at least one processor performs the method described above.

[0041] On the other hand, embodiments of the present invention provide a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to perform the above-described method.

[0042] The beneficial effects of this invention are as follows: This invention discloses a curve structure segmentation method and system based on edge-skeleton dual constraints. This invention innovatively proposes an edge-skeleton dual constraint mechanism, using the edge map to constrain the integrity of the curve contour and the skeleton map to guide the continuity of the central axis. Combined with textual prompts to supplement high-level semantic information, it achieves coordinated control of multimodal features. During generator training, the variational autoencoder compresses the image into the latent space, the control network deeply fuses edge and skeleton features, and the multimodal Transformer denoiser integrates textual semantics and structural constraints, dynamically optimizing model parameters through noise prediction loss. This ensures that the generated synthetic data maintains the morphological characteristics of the real curve structure while possessing rich scene diversity. After fusing the synthetic data with the real dataset, it is used to train the segmentation model, effectively alleviating the data scarcity problem and improving the model's robustness to complex backgrounds, morphological variations, and noise interference. Ultimately, it achieves full-process optimization from pixel-level localization to fine contour segmentation of curve structures. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating a curve structure segmentation method based on edge-skeleton dual constraints according to an embodiment of the present invention.

[0045] Figure 2 This is an example diagram of the preprocessing structure for curved data in an embodiment of the present invention;

[0046] Figure 3 This is a structural example diagram of the training edge-skeleton dual-constraint generator in an embodiment of the present invention;

[0047] Figure 4 This is a structural example diagram of the training curve-shaped structure segmentation model in an embodiment of the present invention;

[0048] Figure 5 This is an example diagram comparing segmentation results on different datasets in an embodiment of the present invention. Detailed Implementation

[0049] The following will provide a clear and complete description of the concept, specific structure, and technical effects of the present invention in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, solution, and effects of the present invention. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0051] It is understood that the terms “first,” “second,” etc., used in this invention may be used herein to describe various concepts, but unless specifically stated otherwise, these concepts are not limited by these terms. These terms are used only to distinguish one concept from another. For example, first information may also be referred to as second information without departing from the scope of embodiments of the invention, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to determination” as used herein may be interpreted as “when…” or “when…” or “in response to determination.”

[0052] The terms “at least one,” “multiple,” “each,” “any,” etc., used in this invention, “at least one” includes one, two, or more than two; “multiple” includes two or more than two; “each” refers to each of the corresponding multiple; and “any” refers to any one of the multiple.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0054] Curvilinear structure segmentation (CSS) is a specialized and crucial task. Accurate CSS segmentation helps doctors identify vascular lesions in medical images, supports the extraction of road networks from aerial images, and is used to detect concrete cracks to assess structural damage. Typical characteristics of such structures are their elongated, tortuous shape and relatively small proportion of the overall image. These factors make CSS segmentation significantly different from conventional object segmentation, thus increasing the challenge for deep learning models in this task.

[0055] Based on these analyses, this invention addresses the technical problems in related technologies by proposing an edge-skeleton dual-constraint generator based on a diffusion model. This generator can synthesize a large number of diverse and high-quality image-mask pairs to improve the segmentation performance of curved structures. The edge-skeleton dual-constraint generator proposed in this invention has three major innovations:

[0056] 1) Controllable generation pipeline: Combining a stable diffusion model with a control network, generation is performed based on text prompts and object layout.

[0057] 2) Edge-skeleton dual constraint mechanism: Forces the maintenance of structural consistency and continuity of curve shape during the generation process;

[0058] 3) Precise control condition priority strategy: ensure that layout information can be accurately preserved.

[0059] The curved structure samples generated by the edge-skeleton dual-constraint generator are not only structurally accurate and semantically consistent, but also visually highly realistic, resulting in a significant improvement in segmentation performance.

[0060] refer to Figure 1 ,like Figure 1 The figure shows a curve structure segmentation method based on edge-skeleton dual constraints provided by an embodiment of the present invention. The method includes the following steps:

[0061] S100, acquire the curved target image, preprocess the curved target image to obtain the edge map, skeleton map and text prompt;

[0062] S200, Construct an edge-skeleton dual-constraint generator based on a stable diffusion model and a control network. Input the edge map, skeleton map and text prompt into the edge-skeleton dual-constraint generator for training to obtain a trained edge-skeleton dual-constraint generator.

[0063] S300: Use the trained edge-skeleton dual-constraint generator to generate synthetic image-mask pairs, and fuse the synthetic image-mask pairs with the real dataset to obtain the training dataset;

[0064] S400, Use the training dataset to train the curved structure segmentation model to obtain the trained curved structure segmentation model;

[0065] S500 inputs the curved target image to be segmented into the trained curved structure segmentation model to obtain the segmentation result.

[0066] It should be noted that the edge-skeleton dual-constraint generator proposed in this invention is a diffusion-driven generator oriented towards curved structure segmentation. It combines a stable diffusion model with a control network, enabling controllable generation of image-mask pairs under the dual conditions of text prompts and object layout.

[0067] To ensure structural consistency, the edge-skeleton dual-constraint generator introduces a novel dual-constraint design that preserves detailed edge information and enhances the connectivity of tubular structures, thereby effectively capturing the unique shape of curved targets.

[0068] This invention designs a priority strategy to ensure the accurate preservation of layout information during the generation process, thereby generating a large amount of diverse and high-quality synthetic data and significantly improving the performance of downstream curved structure segmentation models.

[0069] Specifically, the stable diffusion model is a text-to-image framework based on a latent diffusion architecture. This architecture integrates a variational autoencoder (VAE), a multi-modal diffusion transformer (MM-DiT), and multiple text encoders. The VAE maps the image to a latent space for denoising, while the Transformer iteratively removes noise under textual condition C. Its training objective is:

[0070] ;

[0071] in, For loss function, This represents the average error across all possible cases, where t represents the time step. Indicates at time step Potential noise, It is a text embedding obtained from a pre-trained text encoder. Indicates a noise sample. This refers to the Transformer denoising function. These are the model parameters for the Transformer denoiser.

[0072] During the sampling process, latent variables Starting with initial Gaussian random noise, through Under conditions and time The process involves progressive denoising. Finally, the denoised latent variables are converted into an image by a variational autoencoder decoder.

[0073] To enhance structural controllability, this invention employs a control network that injects spatial conditions (such as edges and masks) into a diffuse backbone network. This design enables the generator to produce diverse outputs while maintaining layout consistency and generating structure-aware results.

[0074] The edge-skeleton dual-constraint generator architecture of this invention consists of three parts: 1) collection and preprocessing of high-quality curved structure data; 2) construction and training of the edge-skeleton dual-constraint generator based on a diffusion model; and 3) data synthesis for training the curved structure segmentation model. First, this invention utilizes state-of-the-art large models to describe the semantic information of images and extracts skeleton and edge information from masks using traditional image processing algorithms. Second, this invention combines a stable diffusion model with a control network, where textual information is used for semantic alignment, while edge-skeleton information ensures the structural integrity of the generated results. Finally, this invention provides a large number of accurate cue words and edge-skeleton maps for the edge-skeleton dual-constraint generator to generate high-quality samples, which are then fused with real datasets for training the curved structure segmentation model.

[0075] In some embodiments, S100, the preprocessing of the curved target image to obtain an edge map, a skeleton map, and a text prompt includes:

[0076] S110, unify the resolution of the curved target image;

[0077] S120: The curved target image after uniform resolution is smoothed with a Gaussian kernel to obtain a smoothed image; the magnitude and direction of the smoothed image are calculated by Sobel gradient, and non-maximum suppression is performed on the magnitude and direction to obtain an edge map with a width of one pixel.

[0078] S130, using the mask of the curved target image as input, perform Euclidean distance transformation to obtain the skeleton map;

[0079] S140, Build the semantic prompt building submodule to generate text prompts.

[0080] In this embodiment, standardized text prompts can accurately describe the semantic attributes of curved targets, such as type, morphological features, and texture information, providing clear high-level semantic guidance for the generator. This ensures that the synthesized image accurately matches the semantic category of the target while satisfying structural constraints. In practical applications, the standardized construction of text prompts can combine domain knowledge and statistical data features. For example, for medical vascular images, it can include key descriptive terms such as "artery," "branching structure," and "high-contrast edges," ensuring semantic consistency and relevance of the generated data in different scenarios.

[0081] Specifically, the structure diagram of the preprocessing module is as follows: Figure 2 As shown, the target image is curved. (Images from concrete cracks, retinal blood vessels, and road scenes) are used as input, and processed at a uniform resolution (512). After step 512), three sub-processes will proceed:

[0082] Edge detection employs the Canny algorithm process, using a Gaussian kernel. Smoothing to obtain a smoothed image , ,in To smooth the scale, the magnitude is then calculated using the Sobel gradient. With direction Non-maximum suppression NMS ( , After that, an edge map with a width of one pixel is obtained. .

[0083] skeleton with mask Using the input as input, perform Euclidean distance transformation to obtain the skeleton graph;

[0084] The semantic prompting construction submodule first uses Chat GPT to define a stable template, then Qwen2.5-VL generates a structured description (such as morphology, density, orientation, and imaging conditions) for each image. This description is then manually sampled and solidified to obtain text prompts. .

[0085] The meanings of each parameter in the dataset in the physical world: In concrete crack images, width / opening (pixels), length (pixels), direction / azimuth (degrees), branching degree and density (strips / square meter), boundary sharpness and contrast (related to illumination and humidity) reflect the degree of stress and deterioration; In blood vessel images, diameter / radius (pixels), curvature / bending angle (degrees), perfusion / density, illumination uniformity and optic disc position correspond to the actual blood flow and anatomical structure; In road images, lane lines (pixels), road network density, direction relative to driving direction, asphalt roughness and water / shading conditions, road width scale and camera height / pitch determine the imaging scale and visible geometry.

[0086] In some embodiments, S130, the step of performing Euclidean distance transformation using the mask of the curved target image as input to obtain a skeleton map includes:

[0087] The skeleton map is obtained by calculating the shortest distance D(x) from each pixel to the nearest mask boundary using the Euclidean distance transformation formula; the Euclidean distance transformation formula is:

[0088] ;in, For mask boundaries, Let p be any pixel position in the image grid, and let p be any boundary pixel position within the boundary. Let x be the Euclidean distance from pixel x to the boundary ∂M.

[0089] In this embodiment, the spatial relationship between each pixel and the boundary within the mask area can be accurately quantified through Euclidean distance transformation. The calculation results not only provide a reliable basis for skeleton extraction, but also allow the central region of the curved structure to be filtered out by setting a distance threshold, ensuring that the generated skeleton map retains the topological shape of the original structure and has single-pixel level precision.

[0090] In some embodiments, in S140, the semantic prompt construction submodule generates text prompts, including:

[0091] S141, use ChatGPT to develop a stable cue word template that covers the morphological characteristics, density distribution, directional trend and imaging conditions of curved targets;

[0092] S142, the preprocessed curved target image is input into the Qwen2.5-VL model, which generates a structured description text for the current curved target image based on the prompt word template; the structured description includes shape, density, orientation and imaging conditions;

[0093] S143, The generated structured descriptions are manually sampled and solidified to form standardized text prompts.

[0094] Specifically, ChatGPT is used to develop a stable prompt template that covers key elements such as the morphological features of curved targets (e.g., curvature, number of branches), density distribution, directional trend, and imaging conditions (e.g., illumination intensity, contrast). Next, the preprocessed image of the curved target is input into the Qwen2.5-VL model, which generates structured descriptive text for the current image based on the template. Subsequently, the generated descriptive text is manually sampled to verify its accuracy and completeness. Descriptions that do not meet the requirements are corrected and solidified, ultimately forming a standardized text prompt P, ensuring that the text information accurately guides the semantic alignment of subsequent generation models.

[0095] In some embodiments, S200, the construction of an edge-skeleton dual-constraint generator based on a stable diffusion model and a control network, wherein the edge map, skeleton map, and text prompt are input into the edge-skeleton dual-constraint generator for training to obtain a trained edge-skeleton dual-constraint generator, includes:

[0096] S210, Construct an edge-skeleton dual-constraint generator based on a stable diffusion model and a control network. The stable diffusion model includes a variational autoencoder, a multimodal Transformer denoiser, and multiple text encoders.

[0097] S220, the variational autoencoder is used to compress the real curved target image to obtain the compressed image in the latent space; multiple text encoders are used to encode the text prompts into corresponding text vectors, and then the vectors are concatenated to obtain a comprehensive text vector; the diffusion step is mapped to a time vector through time step embedding.

[0098] S230, the edge map and skeleton map are stacked in two channels to construct structural conditions, and then input into the control network to output structural constraint features;

[0099] S240, During the training phase, noise samples are added to the compressed image to obtain a noisy image. The noisy image, the integrated text vector, the time vector, and the structural constraint features are input to the multimodal Transformer denoiser, and the predicted noise is output.

[0100] S250 constructs a loss function by comparing the difference between predicted noise and noise samples, optimizes model parameters using backpropagation, and iterates training until the loss function converges to obtain a trained edge-skeleton dual-constraint generator.

[0101] In this embodiment, the training process, through the synergistic effect of edge-skeleton dual structural constraints and text semantic guidance, ensures that the generator can learn the morphological features and semantic attributes of the curved structure, laying the foundation for the generation of high-quality synthetic data in the future.

[0102] In some embodiments, the loss function is:

[0103] ;

[0104] in, For loss function, This represents calculating the average, where t represents the time step. Indicates at time step Potential noise, It is a text embedding obtained from a pre-trained text encoder. Indicates a noise sample. This refers to the Transformer denoising function. These are the model parameters for the Transformer denoiser.

[0105] In this embodiment, by minimizing the L2 norm difference between predicted noise and real noise samples, the Transformer denoiser can accurately learn the noise distribution pattern, thereby effectively removing noise interference in the latent space during the generation process. This ensures that the synthesized image retains edge-skeleton structure constraints and textual semantic information while possessing high clarity and realism.

[0106] The structure diagram of the training edge-skeleton dual-constraint generator is as follows: Figure 3 As shown, it consists of CLIP-L, Google T5, and CLIPG, a three-channel text encoder that processes text prompts. Compile into a comprehensive text vector A variational autoencoder is used to extract the true curved target image. Pressed into the latent space to obtain , This represents a compressed image. The diffusion step is embedded through time step embedding. Mapped to time vector The structural conditions are received by the control network, with the input being edge-skeleton constraints. Dual-channel stacking is represented as Images will be compressed during training. Adding noise to an image The noise is then predicted by the backbone of the stable diffusion model, and the generated image is finally obtained.

[0107] The structure diagram of the synthesis training module is as follows: Figure 4 As shown, this invention feeds the constructed prompts and control conditions into an edge-skeleton dual-constraint generator to obtain a generated image. This invention inputs both the original real image and the generated image into the segmentation and detection model, resulting in a detection result superior to the original model trained solely on real images.

[0108] The effectiveness of the pipeline generation in this invention relies on acquiring data with fine filamentary patterns. However, general-purpose image-text corpora often cause diffusion models to favor coarse textures and weak boundaries, which hinders the segmentation of curved structures. To address this, this invention constructs a high-quality segmentation dataset covering three categories: cracks, blood vessels, and roads. Each image is accompanied by two types of structured annotations: Canny edges to preserve contour details and Zhang-Suen skeletons to maintain topological structure. Furthermore, this invention designs a specialized cue word system: first, ChatGPT-5 develops stable templates around key morphological features, which are then manually reviewed to ensure accuracy; subsequently, Qwen2.5-VL generates concise descriptions based on these templates, thereby reducing cue drift and ensuring consistency across batches.

[0109] To effectively synthesize high-quality curved structure data, this invention designs a specialized training strategy for an edge-skeleton dual-constraint generator. This strategy combines a powerful pre-trained diffusion backbone with a domain-adaptive control module. Specifically, this invention is based on a stable diffusion model and introduces a control network to implement explicit conditional constraints on the semantic mask, thereby guiding the generated results to better conform to the slender and tortuous shape of the curved structure. The control network, as a trainable adapter, injects domain priors into the frozen stable diffusion model backbone network without weakening its generalization ability.

[0110] During training, both the image and the object layout are encoded into latent representations using a variational autoencoder. Gaussian noise is added at predefined time steps, and the noisy latent representations are denoised together with the time-step encodings and text embeddings. The control network receives the same object layout at each time step and generates modulated feature maps that are injected into the backbone network, thereby ensuring accurate alignment between the generated image and the input layout.

[0111] To better capture the geometric and topological features of curved structures, the edge-skeleton dual-constraint generator introduces an edge-skeleton dual-constraint mechanism. The edge map enhances local boundary accuracy, while the skeleton maintains global connectivity and continuity. These two mechanisms complement each other, thereby improving the structural fidelity of the generated samples and effectively mitigating common problems such as broken or overly smoothed curved structures.

[0112] In some embodiments, in S300, the process of generating synthetic image-mask pairs using a trained edge-skeleton dual-constraint generator, and fusing the synthetic image-mask pairs with the real dataset to obtain a training dataset includes:

[0113] S310 takes manually annotated masks and text prompts as control conditions and inputs them into a trained edge-skeleton dual-constraint generator to generate multiple synthetic image-mask pairs.

[0114] S320, the synthesized image-mask pair is mixed with the real dataset to obtain the training dataset.

[0115] Specifically, after the edge-skeleton dual-constraint generator is trained, this invention utilizes it to generate a large number of high-quality image-mask pairs for curved structure segmentation. To ensure reliable supervision signals, this invention uses manually labeled masks and text prompts as control conditions, avoiding the use of automatically generated descriptions or noisy layouts. This design ensures both the accuracy of the labels and maintains sufficient diversity in the synthetic data.

[0116] In downstream segmentation tasks, this invention directly combines synthetic datasets with real samples to train the discriminative model. This direct hybrid training strategy is effective in improving performance because it enriches the diversity of the data without adding extra complexity.

[0117] To verify the improvement effect of the present invention, the present invention was evaluated through the following experiments:

[0118] This invention builds a generation pipeline based on SD-v3-medium and combines it with a control network, using the stitched Canny edges and skeleton mask as conditions. In the downstream segmentation task, this invention performs benchmark tests on SegFormer and DeepLabV3+ under default training settings. Evaluation metrics include edge alignment metrics (…). and ), as well as the standard curved structure segmentation indices mIoU and mDice.

[0119] Ablation analysis: During the generation phase, this invention uses long cue words to control the background style and appearance, but their impact on the geometric consistency of the curved structure is limited. Comparison with short cue words (such as "crack", "road", "blood") shows that the segmentation performance difference is small (Table 1), a result consistent with the FID evaluation. This indicates that the generation of curved structures is less dependent on text cue words, and the benefits of complex cue design are limited.

[0120] Table 1: The impact of different prompt words on the performance of the segmentation model;

[0121] type Long prompt words Short prompt words Blood vessel 117.81 125.28 crack 102.52 116.65 the way 113.24 119.03

[0122] To further verify the importance of structural conditions, this invention compared the effects of different combinations of control charts. As shown in Table 2, the method of this invention significantly outperforms a single control chart or other combinations. The results show that the complementarity of edge and skeleton information effectively enhances the geometric alignment of curved structures, thereby providing higher quality generated data for downstream segmentation tasks.

[0123] Table 2: Comparison of skeleton and contour alignment performance under different control conditions;

[0124]

[0125] To evaluate the value of the generated data for downstream tasks, this invention conducted segmentation experiments on blood vessel, crack, and road datasets using DeepLabV3+ and SegFormer, respectively, and compared the performance with and without the generated data. As shown in Table 3, the segmentation performance steadily improved across different tasks and models after introducing the generated data, indicating that the generated data effectively enhanced the model's performance and generalization ability.

[0126] like Figure 5 As shown, after adding the generated data, the segmentation model is able to capture subtle structural details better, especially in areas where the original model easily misses or blurs the filamentary structure.

[0127] Table 3: Segmentation results on different datasets;

[0128]

[0129] Figure 5 This is a qualitative comparison diagram of the segmentation results. From left to right: (a) the original image, (b) the corresponding mask, (c) the segmentation result on the original dataset, (d) the segmentation result on the dataset combined with the generated samples, (e) the segmentation result on the generated image, and (f) the generated image itself.

[0130] Conclusion: The edge-skeleton dual-constraint generator proposed in this invention is a diffusion framework for segmenting curved structures. By combining Stable Diffusion with a control network, an edge-skeleton dual-constraint mechanism, and a control condition priority strategy, the edge-skeleton dual-constraint generator can generate structurally accurate and diverse synthetic data. Experimental results show that this type of data can significantly improve downstream segmentation performance, providing a scalable solution for expensive manual annotation.

[0131] In summary, this invention proposes a novel framework for improving the segmentation of curved structures using a dual-constraint edge-skeleton generator that leverages high-quality synthetic data. It incorporates innovations in the generation pipeline, control mechanism, and data generation strategy: First, it combines a Stable Diffusion Denoising MM-DiT (SD3) model with a Control Net to fully utilize its robust prior knowledge. Second, it introduces a dual-constraint edge-skeleton mechanism to enhance structure guidance, thereby improving the edge details and overall connectivity of generated tubular targets. Finally, it employs a precise control condition priority strategy to generate a large amount of high-quality data, significantly enhancing the performance of downstream semantic segmentation models.

[0132] Experiments show that the introduction of synthetic data can not only significantly improve the performance of CNN-based semantic segmentation models, but also enhance the performance of the Transformer architecture, thus verifying the effectiveness of the proposed method.

[0133] This invention also provides a curve structure segmentation system based on edge-skeleton dual constraints, comprising:

[0134] At least one processor;

[0135] At least one memory for storing at least one program;

[0136] When the at least one program is executed by the at least one processor, the at least one processor performs the method described above.

[0137] The content of the above method embodiments is applicable to this embodiment. The specific functions implemented in this embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments. Therefore, they will not be repeated here.

[0138] This invention also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0139] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0140] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0141] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0142] This invention also provides a computer program product, including a computer program or computer instructions, which are stored in a memory. A processor of a computer device reads the computer program or computer instructions from the memory and executes the computer program or computer instructions, causing the computer device to perform the above-described method.

[0143] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0144] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0145] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0146] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

Claims

1. A curve structure segmentation method based on edge-skeleton double constraints, characterized in that, The method comprises the following steps: S100, obtaining a curve-shaped target image, preprocessing the curve-shaped target image to obtain an edge map, a skeleton map and a text prompt; S200, constructing an edge-skeleton double-constraint generator based on a stable diffusion model and a control network, inputting the edge map, the skeleton map and the text prompt into the edge-skeleton double-constraint generator for training to obtain a trained edge-skeleton double-constraint generator; S300, generating a synthetic image-mask pair by using the trained edge-skeleton double-constraint generator, fusing the synthetic image-mask pair with a real data set to obtain a training data set; S400, training a curve-shaped structure segmentation model by using the training data set to obtain a trained curve-shaped structure segmentation model; S500, inputting a curve-shaped target image to be segmented into the trained curve-shaped structure segmentation model to obtain a segmentation result; S200 specifically comprises: S210, constructing an edge-skeleton double-constraint generator based on a stable diffusion model and a control network, wherein the stable diffusion model comprises a variational autoencoder, a multi-modal Transformer denoiser and a plurality of text encoders; S220, compressing a real curve-shaped target image by using the variational autoencoder to obtain a compressed image in a latent space; encoding a text prompt into a corresponding text vector by using a plurality of text encoders, and obtaining a comprehensive text vector by vector splicing; and mapping a diffusion step into a time vector by time step embedding; S230, stacking the edge map and the skeleton map in a double-channel to construct a structure condition, and inputting the structure condition into the control network to output a structure constraint feature; S240, in a training stage, adding a noise sample to the compressed image to obtain a noisy image, inputting the noisy image, the comprehensive text vector, the time vector and the structure constraint feature into the multi-modal Transformer denoiser to output a predicted noise; S250, constructing a loss function by comparing the difference between the predicted noise and the noise sample, optimizing model parameters by using back propagation, and iteratively training until the loss function converges to obtain a trained edge-skeleton double-constraint generator.

2. The method of claim 1, wherein, In S100, the preprocessing of the curve-shaped target image to obtain the edge map, the skeleton map and the text prompt comprises: S110, unifying the resolution of the curve-shaped target image; S120, smoothing the curve-shaped target image with a unified resolution by using a Gaussian kernel to obtain a smoothed image, calculating the amplitude and direction of the smoothed image by using a Sobel gradient, performing non-maximum suppression on the amplitude and direction to obtain an edge map with a width of one pixel; S130, performing Euclidean distance transformation by taking the mask of the curve-shaped target image as input to obtain a skeleton map; S140, constructing a semantic prompt construction submodule to generate a text prompt.

3. The method of claim 2, wherein, In S130, the Euclidean distance transformation by taking the mask of the curve-shaped target image as input to obtain a skeleton map comprises: calculating the shortest distance from each pixel to the nearest mask boundary according to a Euclidean distance transformation formula to obtain a skeleton map; the Euclidean distance transformation formula is: ; where, is the mask boundary, is any pixel position in the image grid, p is any boundary pixel position in the boundary, is the Euclidean distance of pixel x to the boundary ∂M.

4. The method of claim 2, wherein, In S140, the construction semantic prompt construction submodule generates a text prompt, including: S141, using ChatGPT to formulate a stable prompt word template, which covers the morphological characteristics, density distribution, trend and imaging conditions of the curve-shaped target; S142, input the preprocessed curve-shaped target image into the Qwen2.5-VL model, and generate a structured description text for the current curve-shaped target image according to the prompt word template by the Qwen2.5-VL model; the structured description includes morphology, density, trend and imaging conditions; S143, manually inspect and solidify the generated structured description to form a standardized text prompt.

5. The method of claim 1, wherein, The loss function is: ； wherein, is a loss function, denotes averaging over time steps, denotes the latent noise at time step , is a text embedding obtained from a pre-trained text encoder, denotes a noisy sample, denotes a Transformer denoiser, are model parameters of the Transformer denoiser.

6. The method of claim 1, wherein, In S300, the trained edge-skeleton double constraint generator is used to generate a synthetic image-mask pair, and the synthetic image-mask pair is fused with a real data set to obtain a training data set, including: S310, input the manually annotated mask and the text prompt as control conditions into the trained edge-skeleton double constraint generator to generate a plurality of synthetic image-mask pairs; S320, mix the synthetic image-mask pair with the real data set to obtain a training data set.

7. A curve structure segmentation system based on edge-skeleton dual constraints, characterized in that, It includes: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method of any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program is executed by the processor to implement the method of any one of claims 1 to 6.

Images