Curve structure segmentation method and system based on edge-skeleton double constraints

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a curve structure segmentation method and system based on edge-skeleton double constraint, and the method comprises the steps: obtaining a curve-shaped target image, carrying out the preprocessing of the curve-shaped target image, and obtaining an edge image, a skeleton image and a text prompt; inputting the edge graph, the skeleton graph and the text prompt into an edge-skeleton double-constraint generator for training to obtain a trained edge-skeleton double-constraint generator; generating a composite image-mask pair by using the trained edge-skeleton double-constraint generator, and fusing the composite image-mask pair with a real data set to obtain a training data set; training the curve-shaped structure segmentation model by using the training data set to obtain a trained curve-shaped structure segmentation model; inputting a to-be-segmented curve-shaped target image into the trained curve-shaped structure segmentation model to obtain a segmentation result; the segmentation performance of the curved structure can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a curve structure segmentation method and system based on edge-skeleton double constraint. BACKGROUND

[0002] The segmentation task of curvilinear structure, such as blood vessels, concrete cracks and road networks, is significantly different from the segmentation of common objects, which is rooted in its unique morphological characteristics. In addition, this task is further constrained by data scarcity, labeling difficulty, and the requirement of professional knowledge across application domains.

[0003] Traditional segmentation methods often fail to model the elongated topology and complex intersection regions when dealing with such structures, resulting in segmentation results with cracks or over-segmentation. Although deep learning methods have improved performance through end-to-end learning, in small sample scenarios, the model is easily affected by data distribution bias, making it difficult to accurately capture the global continuity and local detail features of curvilinear structures. SUMMARY

[0004] To solve the above problems, the present application provides a curve structure segmentation method and system based on edge-skeleton double constraint, aiming to improve the performance of curvilinear structure segmentation.

[0005] To achieve the above purpose, the present application provides the following technical solutions:

[0006] On the one hand, the present application provides a curve structure segmentation method based on edge-skeleton double constraint, which comprises the following steps:

[0007] S100, obtaining a curvilinear target image, pre-processing the curvilinear target image to obtain an edge map, a skeleton map and a text prompt;

[0008] S200, constructing an edge-skeleton double constraint generator based on a stable diffusion model and a control network, inputting the edge map, the skeleton map and the text prompt into the edge-skeleton double constraint generator for training, and obtaining a trained edge-skeleton double constraint generator;

[0009] S300, generating a synthetic image-mask pair using the trained edge-skeleton double constraint generator, fusing the synthetic image-mask pair with a real data set to obtain a training data set;

[0010] S400, training a curvilinear structure segmentation model using the training data set to obtain a trained curvilinear structure segmentation model;

[0011] S500, input the curve-shaped target image to be segmented into the trained curve-shaped structure segmentation model to obtain a segmentation result.

[0012] Optionally, in S100, the pre-processing of the curve-shaped target image to obtain an edge map, a skeleton map and a text prompt comprises:

[0013] S110, uniform resolution of the curve-shaped target image;

[0014] S120, smoothing the curve-shaped target image after uniform resolution by a Gaussian kernel to obtain a smoothed image; calculating the amplitude and direction of the smoothed image by Sobel gradient, and performing non-maximum suppression on the amplitude and direction to obtain a one-pixel-wide edge map;

[0015] S130, inputting the mask of the curve-shaped target image to perform Euclidean distance transformation to obtain a skeleton map;

[0016] S140, constructing a semantic prompt construction sub-module to generate a text prompt.

[0017] Optionally, in S130, the input of the mask of the curve-shaped target image to perform Euclidean distance transformation to obtain a skeleton map comprises:

[0018] According to the Euclidean distance transformation formula, the shortest distance of each pixel to the nearest mask boundary is calculated to obtain a skeleton map;

[0019] The Euclidean distance transformation formula is:

[0020] ; wherein, is the mask boundary, is any one pixel position in the image grid, p is any one boundary pixel position in the boundary, is the Euclidean distance of pixel x to the boundary ∂M.

[0021] Optionally, in S140, the construction of the semantic prompt construction sub-module to generate a text prompt comprises:

[0022] S141, using ChatGPT to formulate a stable prompt word template, which covers the morphological characteristics, density distribution, trend and imaging conditions of the curve-shaped target;

[0023] S142, inputting the pre-processed curve-shaped target image into the Qwen2.5-VL model, and generating a structured description text for the current curve-shaped target image according to the prompt word template by the Qwen2.5-VL model; the structured description includes morphology, density, trend and imaging conditions;

[0024] S143, The generated structured descriptions are manually sampled and solidified to form standardized text prompts.

[0025] Optionally, in S200, the construction of an edge-skeleton dual-constraint generator based on a stable diffusion model and a control network, wherein the edge map, skeleton map, and text prompt are input into the edge-skeleton dual-constraint generator for training to obtain a trained edge-skeleton dual-constraint generator, includes:

[0026] S210, Construct an edge-skeleton dual-constraint generator based on a stable diffusion model and a control network. The stable diffusion model includes a variational autoencoder, a multimodal Transformer denoiser, and multiple text encoders.

[0027] S220, the variational autoencoder is used to compress the real curved target image to obtain the compressed image in the latent space; multiple text encoders are used to encode the text prompts into corresponding text vectors, and then the vectors are concatenated to obtain a comprehensive text vector; the diffusion step is mapped to a time vector through time step embedding.

[0028] S230, the edge map and skeleton map are stacked in two channels to construct structural conditions, and then input into the control network to output structural constraint features;

[0029] S240, During the training phase, noise samples are added to the compressed image to obtain a noisy image. The noisy image, the integrated text vector, the time vector, and the structural constraint features are input to the multimodal Transformer denoiser, and the predicted noise is output.

[0030] S250 constructs a loss function by comparing the difference between predicted noise and noise samples, optimizes model parameters using backpropagation, and iterates training until the loss function converges to obtain a trained edge-skeleton dual-constraint generator.

[0031] Optionally, the loss function is:

[0032] ;

[0033] in, For loss function, This represents the average error across all possible cases, where t represents the time step. Indicates at time step Potential noise, It is a text embedding obtained from a pre-trained text encoder. Indicates a noise sample. This refers to the Transformer denoising function. These are the model parameters for the Transformer denoiser.

[0034] Optionally, in S300, the trained edge-skeleton dual constraint generator is used to generate a synthetic image-mask pair, and the synthetic image-mask pair is fused with a real data set to obtain a training data set, including:

[0035] In S310, the manually annotated mask and the text prompt are input as control conditions into the trained edge-skeleton dual constraint generator to generate a plurality of synthetic image-mask pairs.

[0036] In S320, the synthetic image-mask pairs are mixed with a real data set to obtain a training data set.

[0037] In another aspect, an embodiment of the present application provides a curve structure segmentation system based on edge-skeleton dual constraint, including:

[0038] at least one processor;

[0039] at least one memory for storing at least one program;

[0040] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0041] In another aspect, an embodiment of the present application provides a computer readable storage medium, which stores a processor executable program, and the processor executable program is used to execute the above method when executed by a processor.

[0042] The present application has the beneficial effects that: the present application discloses a curve structure segmentation method and system based on edge-skeleton dual constraint, and the present application innovatively proposes an edge-skeleton dual constraint mechanism, which constrains the curve contour integrity through an edge graph, guides the center axis continuity through a skeleton graph, combines text prompts to supplement high-level semantic information, and realizes the collaborative regulation and control of multi-modal features. In the generator training process, the variational autoencoder compresses the image to the latent space, the control network deeply fuses the edge and skeleton features, and the multi-modal Transformer denoiser integrates the text semantics and structure constraints, dynamically optimizes the model parameters through noise prediction loss, so that the generated synthetic data not only maintains the morphological characteristics of the real curve structure, but also has rich scene diversity. After the synthetic data is fused with the real data set, it is used to train the segmentation model, which can effectively alleviate the data scarcity problem, improve the robustness of the model to complex background, morphological variation and noise interference, and finally realize the whole process optimization of the curve structure from pixel-level positioning to fine contour segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below only represent some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0044] Figure 1 is a flow diagram of a curve structure segmentation method based on edge-skeleton double constraint according to an embodiment of the present application;

[0045] Figure 2 is a pre-processing structure example diagram of curve structure data according to an embodiment of the present application;

[0046] Figure 3 is a structure example diagram of training edge-skeleton double constraint generator according to an embodiment of the present application;

[0047] Figure 4 is a structure example diagram of training curve structure segmentation model according to an embodiment of the present application;

[0048] Figure 5 is a segmentation result comparison example diagram on different data sets according to an embodiment of the present application. DETAILED DESCRIPTION

[0049] The disclosed concept, specific structure and generated technical effects will be described clearly and completely in the following with embodiments and drawings, so as to fully understand the purpose, scheme and effects of the present application. It should be noted that the embodiments and features in the embodiments in the present application can be combined with each other without conflict.

[0050] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application will be further described in detail in the following with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application. When the following description relates to the drawings, the same numerals in different drawings represent the same or similar elements unless otherwise indicated. The implementation described in the following exemplary embodiments does not represent all the implementations consistent with the embodiments of the present application, but is only an example of devices and methods consistent with some aspects of the embodiments of the present application as described in the appended claims.

[0051] It is to be understood that the terms "first", "second", and the like used herein can be used to describe various concepts, but unless specifically stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of embodiments of the present application, first information can also be referred to as second information, and similarly, second information can also be referred to as first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon determining" or "in response to determining".

[0052] The terms "at least one", "multiple", "each", "any", and the like used herein include one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one skilled in the art to which the present application belongs. The terms used herein are only for the purpose of describing embodiments of the present application and are not intended to limit the present application.

[0054] Curvilinear structure segmentation (CSS) is a professional and critical task. Accurate curvilinear structure segmentation can help doctors identify vascular lesions in medical images, support the extraction of road networks from aerial images, and be used to detect concrete cracks to evaluate structural damage. The typical characteristics of such structures are elongated, tortuous morphology, and relatively small proportion in the whole image. The above factors make curvilinear structure segmentation significantly different from conventional object segmentation, thereby increasing the challenge of deep learning models on this task.

[0055] Based on these analyses, the present application addresses the technical problems in the related art. The present application proposes an edge-skeleton dual constraint generator based on a diffusion model, which can synthesize a large number of diversified and high-quality image-mask pairs to improve the performance of curvilinear structure segmentation. The edge-skeleton dual constraint generator proposed by the present application has three major innovations:

[0056] 1) Controllable generation pipeline: combining a stable diffusion model with a control network to generate under the condition of text prompts and object layout;

[0057] 2) Edge-skeleton dual constraint mechanism: forcing to maintain the consistency and continuity of the curvilinear morphology in the generation process;

[0058] 3) Precise control condition priority strategy: ensuring that the layout information can be accurately preserved.

[0059] The curve structure sample generated by the edge-skeleton double constraint generator is not only accurate in structure, consistent in semantics, but also highly realistic in vision, thereby bringing significant segmentation performance improvement.

[0060] Reference Figure 1 As Figure 1 The application provides a curve structure segmentation method based on edge-skeleton double constraint, and the method comprises the following steps:

[0061] S100, a curve target image is acquired, and the curve target image is preprocessed to obtain an edge graph, a skeleton graph and a text prompt;

[0062] S200, an edge-skeleton double constraint generator based on a stable diffusion model and a control network is constructed, the edge graph, the skeleton graph and the text prompt are input into the edge-skeleton double constraint generator for training, and a trained edge-skeleton double constraint generator is obtained;

[0063] S300, a synthetic image-mask pair is generated by using the trained edge-skeleton double constraint generator, the synthetic image-mask pair is fused with a real data set, and a training data set is obtained;

[0064] S400, a curve structure segmentation model is trained by using the training data set, and a trained curve structure segmentation model is obtained;

[0065] S500, a curve target image to be segmented is input into the trained curve structure segmentation model, and a segmentation result is obtained.

[0066] It should be noted that the edge-skeleton double constraint generator provided by the application is a diffusion generation generator for curve structure segmentation. It combines a stable diffusion model with a control network, and can generate a controllable image-mask pair under the double conditions of a text prompt and object layout.

[0067] To ensure structural consistency, the edge-skeleton double constraint generator introduces a new double constraint design, retains detailed edge information and enhances the connectivity of tubular structures, thereby effectively capturing the unique morphology of the curve target.

[0068] The application designs a priority strategy to ensure accurate retention of layout information during the generation process, thereby being capable of generating a large amount of diversified and high-quality synthetic data and significantly improving the performance of a downstream curve structure segmentation model.

[0069] In particular, the stable diffusion model is a text-to-image framework based on a latent diffusion architecture that integrates a Variational Auto-Encoder (VAE), a Multi-Modal Diffusion Transformer (MM-DiT), and multiple text encoders. The Variational Auto-Encoder maps images to a latent space for denoising, while the Transformer iteratively removes noise under a text condition C. Its training objective is:

[0070] ;

[0071] where, is a loss function, denotes the average error over all possible cases, t denotes the time step, denotes the latent noise at time step , is a text embedding obtained by a pre-trained text encoder, denotes a noisy sample, denotes a Transformer denoiser, is a model parameter of the Transformer denoiser.

[0072] In the sampling process, the latent variable starts from an initial Gaussian random noise and is gradually denoised by under the condition and time . Finally, the denoised latent variable is converted into an image by the Variational Auto-Encoder decoder.

[0073] To enhance structural controllability, the present application adopts a control network that injects spatial conditions (such as edges, masks) into the diffusion backbone network. This design enables the generator to maintain the consistency of the layout and generate structure-aware results while producing diverse outputs.

[0074] The edge-skeleton double constraint generator of the present application is composed of three parts: 1) collection and preprocessing of high-quality curve-shaped structure data; 2) construction and training of the edge-skeleton double constraint generator based on the diffusion model; and 3) data synthesis for training the curve-shaped structure segmentation model. First, the present application uses the most advanced large model to describe the semantic information of the image, and extracts the skeleton and edge information from the mask through traditional image processing algorithms. Second, the present application combines a stable diffusion model with a control network, in which the text information is used for semantic alignment, and the edge-skeleton information is used to ensure the structural integrity of the generated results. Finally, the present application provides a large number of accurate prompt words and edge-skeleton graphs for the edge-skeleton double constraint generator to generate high-quality samples, and fuses them with the real data set for training the curve-shaped structure segmentation model.

[0075] In some embodiments, in S100, the curve-shaped target image is preprocessed to obtain an edge graph, a skeleton graph, and a text prompt, including:

[0076] In S110, the curve-shaped target image is unified in resolution.

[0077] In S120, the curve-shaped target image after unified resolution is smoothed by a Gaussian kernel to obtain a smoothed image; the amplitude and direction of the smoothed image are calculated by Sobel gradient, and non-maximum suppression is performed on the amplitude and direction to obtain an edge graph with one pixel width.

[0078] In S130, the mask of the curve-shaped target image is input to perform Euclidean distance transformation to obtain a skeleton graph.

[0079] In S140, a semantic prompt construction submodule is constructed to generate a text prompt.

[0080] In the present embodiment, the standardized text prompt can accurately describe the semantic attributes of the curve-shaped target, such as type, morphological characteristics, and texture information, etc., providing clear high-level semantic guidance for the generator, so that the synthesized image meets the structural constraints while accurately matching the semantic category of the target. In practical applications, the standardized construction of the text prompt can be combined with domain knowledge and data statistical characteristics, for example, for medical blood vessel images, it can include key description words such as "arterial blood vessel", "branching structure", and "high-contrast edge", etc., to ensure the semantic consistency and pertinence of the generated data in different scenarios.

[0081] Specifically, the structure diagram of the preprocessing module is as shown in Figure 2 The curve-shaped target image (from concrete cracks, fundus blood vessels, and road scenes) is input, and after unified resolution (512 512), it enters three sub-processes:

[0082] Edge detection adopts Canny algorithm process, and a smoothed image is obtained by using a Gaussian kernel . , , wherein is a smoothing scale, and a magnitude is calculated by a Sobel gradient and a direction . After non-maximum suppression (NMS) , , a one-pixel-wide edge map is obtained .

[0083] The skeleton takes a mask as input, performs Euclidean distance transformation, and obtains a skeleton map.

[0084] The semantic prompt construction submodule first uses Chat GPT to formulate a stable template, and then uses Qwen2.5-VL to generate a structured description (such as morphology, density, strike, and imaging conditions) for each image. Artificial sampling and solidification are performed to obtain a text prompt .

[0085] The meanings of various parameters in the data set in the physical world: in the concrete crack image, the width / opening (pixels), length (pixels), strike / azimuth angle (degrees), branching degree and density (strips per square meter), boundary definition and contrast (related to illumination and humidity) reflect the stress and degradation degree; in the blood vessel image, the caliber / radius (pixels), curvature, bifurcation angle (degrees), perfusion / density, illumination uniformity and optic disc position correspond to the real blood flow and anatomical structure; in the road image, the lane line (pixels), road network density, strike relative to the driving direction, asphalt roughness and water content / shadow condition, road width scale and camera height / tilt determine the imaging scale and visible geometry.

[0086] In some embodiments, in S130, the mask of the curve-shaped target image is input, and a Euclidean distance transformation is performed to obtain a skeleton map, including:

[0087] According to the Euclidean distance transformation formula, the shortest distance D(x) of each pixel to the nearest mask boundary is calculated to obtain a skeleton map; the Euclidean distance transformation formula is:

[0088] ; wherein is a mask boundary, is any one pixel position in the image grid, p is any one boundary pixel position in the boundary, is the Euclidean distance of pixel x to the boundary ∂M.

[0089] In this embodiment, the spatial relationship between each pixel in the mask region and the boundary can be accurately quantified by the Euclidean distance transformation. The calculation result not only provides a reliable basis for skeleton extraction, but also can filter out the center region of the curve-shaped structure by setting a distance threshold, ensuring that the generated skeleton map not only retains the topological morphology of the original structure, but also has a single-pixel-level precision.

[0090] In some embodiments, in S140, the semantic prompt construction submodule generates a text prompt, including:

[0091] In S141, a stable prompt template is formulated using ChatGPT, which covers the morphological features, density distribution, trend, and imaging conditions of the curve-shaped target.

[0092] In S142, the preprocessed curve-shaped target image is input into the Qwen2.5-VL model, and the Qwen2.5-VL model generates a structured description text for the current curve-shaped target image based on the prompt template; the structured description includes morphology, density, trend, and imaging conditions.

[0093] In S143, the generated structured description is manually sampled and solidified to form a standardized text prompt.

[0094] Specifically, a stable prompt template is formulated using ChatGPT, which covers the morphological features (such as bending degree, branch number), density distribution, trend, and imaging conditions (such as light intensity, contrast) of the curve-shaped target; then, the preprocessed curve-shaped target image is input into the Qwen2.5-VL model, and the model generates a structured description text for the current image based on the above template; subsequently, the generated description text is manually sampled to verify its accuracy and completeness, and the description that does not meet the requirements is modified and solidified, finally forming a standardized text prompt P, ensuring that the text information can accurately guide the semantic alignment of the subsequent generation model.

[0095] In some embodiments, in S200, the edge-skeleton dual constraint generator based on the stable diffusion model and the control network is constructed, and the edge map, the skeleton map, and the text prompt are input into the edge-skeleton dual constraint generator for training to obtain a trained edge-skeleton dual constraint generator, including:

[0096] In S210, an edge-skeleton dual constraint generator based on a stable diffusion model and a control network is constructed, and the stable diffusion model includes a variational autoencoder, a multi-modal Transformer denoiser, and multiple text encoders.

[0097] S220, using the variational autoencoder to compress a real curve-shaped target image to obtain a compressed image in a latent space; using a plurality of text encoders to respectively encode text prompts into corresponding text vectors, and then obtaining a comprehensive text vector by vector splicing; mapping diffusion steps into time vectors by time step embedding;

[0098] S230, stacking the edge map and the skeleton map in a double channel to construct a structure condition, and inputting the structure condition into a control network to output a structure constraint feature;

[0099] S240, in the training stage, adding a noise sample to the compressed image to obtain a noisy image, inputting the noisy image, the comprehensive text vector, the time vector, and the structure constraint feature into a multi-modal Transformer denoiser to output a predicted noise;

[0100] S250, constructing a loss function by comparing the difference between the predicted noise and the noise sample, optimizing model parameters by back propagation, and iteratively training until the loss function converges to obtain a trained edge-skeleton double constraint generator.

[0101] The training process in this embodiment ensures that the generator can learn the morphological features and semantic attributes of the curve-shaped structure through the synergistic effect of edge-skeleton double structure constraints and text semantic guidance, laying a foundation for the generation of high-quality synthetic data in the future.

[0102] In some embodiments, the loss function is:

[0103] ;

[0104] wherein, is the loss function, denotes averaging, and t denotes a time step, denotes the latent noise at the time step , is a text embedding obtained by a pre-trained text encoder, denotes a noise sample, denotes a Transformer denoiser, is a model parameter of the Transformer denoiser.

[0105] In this embodiment, by minimizing the L2 norm difference between the predicted noise and the real noise sample, the Transformer denoiser can accurately learn the noise distribution rule, thereby effectively removing the noise interference in the latent space during the generation process, ensuring that the synthetic image has high clarity and authenticity while preserving the edge-skeleton structure constraint and text semantic information.

[0106] The structure diagram of the training edge-skeleton dual-constraint generator is as follows: Figure 3 As shown, it consists of CLIP-L, Google T5, and CLIPG, a three-channel text encoder that processes text prompts. Compile into a comprehensive text vector A variational autoencoder is used to extract the true curved target image. Pressed into the latent space to obtain , This represents a compressed image. The diffusion step is embedded through time step embedding. Mapped to time vector The structural conditions are received by the control network, with the input being edge-skeleton constraints. Dual-channel stacking is represented as Images will be compressed during training. Adding noise to an image The noise is then predicted by the backbone of the stable diffusion model, and the generated image is finally obtained.

[0107] The structure diagram of the synthesis training module is as follows: Figure 4 As shown, this invention feeds the constructed prompts and control conditions into an edge-skeleton dual-constraint generator to obtain a generated image. This invention inputs both the original real image and the generated image into the segmentation and detection model, resulting in a detection result superior to the original model trained solely on real images.

[0108] The effectiveness of the pipeline generation in this invention relies on acquiring data with fine filamentary patterns. However, general-purpose image-text corpora often cause diffusion models to favor coarse textures and weak boundaries, which hinders the segmentation of curved structures. To address this, this invention constructs a high-quality segmentation dataset covering three categories: cracks, blood vessels, and roads. Each image is accompanied by two types of structured annotations: Canny edges to preserve contour details and Zhang-Suen skeletons to maintain topological structure. Furthermore, this invention designs a specialized cue word system: first, ChatGPT-5 develops stable templates around key morphological features, which are then manually reviewed to ensure accuracy; subsequently, Qwen2.5-VL generates concise descriptions based on these templates, thereby reducing cue drift and ensuring consistency across batches.

[0109] To effectively synthesize high-quality curved structure data, this invention designs a specialized training strategy for an edge-skeleton dual-constraint generator. This strategy combines a powerful pre-trained diffusion backbone with a domain-adaptive control module. Specifically, this invention is based on a stable diffusion model and introduces a control network to implement explicit conditional constraints on the semantic mask, thereby guiding the generated results to better conform to the slender and tortuous shape of the curved structure. The control network, as a trainable adapter, injects domain priors into the frozen stable diffusion model backbone network without weakening its generalization ability.

[0110] During training, both images and object layouts are encoded into latent representations by variational autoencoders. Gaussian noise is added at predefined time steps, and the noisy latent representations are denoised together with time step encodings and text embeddings. The control network receives the same object layout at each time step and generates modulated feature maps that are injected into the backbone network, ensuring accurate alignment between generated images and input layouts.

[0111] To better capture the geometric and topological features of curved structures, the edge-skeleton dual constraint generator introduces an edge-skeleton dual constraint mechanism. The edge map is used to enhance local boundary accuracy, while the skeleton is used to maintain global connectivity and continuity. The two complement each other, thereby improving the structural fidelity of generated samples and effectively alleviating common problems such as broken or overly smoothed curved structures.

[0112] In some embodiments, in S300, the trained edge-skeleton dual constraint generator is used to generate a synthetic image-mask pair, and the synthetic image-mask pair is combined with a real dataset to obtain a training dataset, including:

[0113] S310, the manually annotated mask and the text prompt are input as control conditions into the trained edge-skeleton dual constraint generator to generate a plurality of synthetic image-mask pairs;

[0114] S320, the synthetic image-mask pairs are mixed with the real dataset to obtain a training dataset.

[0115] Specifically, after the edge-skeleton dual constraint generator is trained, the present application generates a large number of high-quality image-mask pairs for curved structure segmentation. To ensure reliable supervision signals, the present application uses manually annotated masks and text prompts as control conditions, avoiding the use of automatically generated descriptions or noisy layouts. Such a design can ensure the accuracy of the labels and maintain sufficient diversity in the synthetic data.

[0116] In downstream segmentation tasks, the present application combines the synthetic dataset with real samples directly for training a discriminative model. This direct hybrid training strategy is effective in improving performance, as it enriches the diversity of data without increasing additional complexity.

[0117] To verify the improvement effect of the present application, the present application compares and evaluates through the following experiments:

[0118] The application constructs a generation pipeline based on the SD-v3-medium and combines a control network to splice the Canny edge and skeleton mask as conditions. In the downstream segmentation task, the application benchmarks SegFormer and DeepLabV3+ under the default training setting. The evaluation indicators include edge alignment metrics (F1, F2, and F-measure) and ), and the standard curve structure segmentation indicators mIoU and mDice.

[0119] Ablation analysis: In the generation stage, the application uses long prompt words to control the background style and appearance, but its influence on the geometric consistency of the curve structure is limited. Compared with short prompt words such as "crack", "road", and "blood", the segmentation performance difference is small (Table 1), which is consistent with the FID evaluation. This shows that the generation of the curve structure is less dependent on the text prompt, and the benefit of complex prompt design is limited.

[0120] Table 1: Influence of different prompt words on the performance of the segmentation model;

[0121] Type Long cue Short cue Vessel 117.81 125.28 Fracture 102.52 116.65 Road 113.24 119.03

[0122] To further verify the importance of the structure condition, the application compares the effects of different control map combinations. As shown in Table 2, the method of the application is significantly better than a single control map or other combinations. The results show that the complementarity of edge and skeleton information effectively enhances the geometric alignment of the curve structure, thereby providing higher quality generated data for downstream segmentation tasks.

[0123] Table 2: Comparison of different control conditions in terms of skeleton and contour alignment performance;

[0124]

[0125] To evaluate the value of the generated data for downstream tasks, the application uses DeepLabV3+ and SegFormer to perform segmentation experiments on the blood vessel, crack, and road data sets, respectively, and compares the performance with and without generated data. As shown in Table 3, after introducing the generated data, the segmentation performance is stably improved on different tasks and models, indicating that the generated data effectively enhances the model performance and generalization ability.

[0126] As shown in Figure 5 , after adding the generated data, the segmentation model can better capture subtle structural details, especially in areas where the original model is prone to miss or blur filament structures.

[0127] Table 3: Segmentation results on different data sets;

[0128]

[0129] Figure 5 are qualitative comparison illustrations of segmentation results. From left to right, in order: (a) original image, (b) corresponding mask, (c) segmentation result on the original dataset, (d) segmentation result on the dataset combined with the generated samples, (e) segmentation result on the generated image, and (f) the generated image itself.

[0130] Conclusion: The edge-skeleton dual constraint generator proposed in the application is a diffusion framework for curve-like structure segmentation. By combining Stable Diffusion and control network, edge-skeleton dual constraint mechanism and control condition priority strategy, the edge-skeleton dual constraint generator can generate synthetic data with accurate structure and diversity. Experimental results show that such data can significantly improve the performance of downstream segmentation, providing an expandable solution for expensive manual annotation.

[0131] In summary, the edge-skeleton dual constraint generator proposed in the application uses high-quality synthetic data to improve curve-like structure segmentation. It integrates innovations in generation pipeline, control mechanism and data generation strategy: first, the application combines Stable Diffusion Denoising MM-DiT (SD3) with Control Net to fully utilize its strong prior knowledge. Second, the application introduces an edge-skeleton dual constraint mechanism to strengthen structure guidance and improve edge details and overall connectivity of generated tubular targets. Finally, the application uses an accurate control condition priority strategy to generate a large amount of high-quality data, thereby significantly improving the performance of downstream semantic segmentation models.

[0132] Experiments show that the introduction of synthetic data not only significantly improves the performance of CNN-based semantic segmentation models, but also enhances the performance of Transformer architecture, verifying the effectiveness of the proposed method.

[0133] The embodiment of the application also provides a curve structure segmentation system based on edge-skeleton dual constraint, which comprises:

[0134] at least one processor;

[0135] at least one memory for storing at least one program;

[0136] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned method.

[0137] The contents in the method embodiments are applicable to this embodiment, the function implemented in this embodiment is the same as that in the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments, which will not be repeated here.

[0138] The embodiment of the present application also provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the above method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, an on-board computer, etc.

[0139] It can be understood that the contents in the above method embodiments are applicable to the present device embodiment, the function implemented in the present device embodiment is the same as that in the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0140] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above method.

[0141] It can be understood that the contents in the above method embodiments are applicable to the present storage medium embodiment, the function implemented in the present storage medium embodiment is the same as that in the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0142] The embodiment of the present application also provides a computer program product, which comprises a computer program or computer instructions, the computer program or computer instructions are stored in a memory, a processor of a computer device reads the computer program or computer instructions from the memory, and the processor executes the computer program or computer instructions, so that the computer device executes the above method.

[0143] It can be understood that the contents in the above method embodiments are applicable to the present program product embodiment, the function implemented in the present program product embodiment is the same as that in the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0144] The memory is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0145] As will be appreciated by one of ordinary skill in the art, all or some of the steps, systems, and techniques disclosed herein can be embodied in software, firmware, hardware, or any suitable combination thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a micro-processing unit, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on computer readable media, which can comprise computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those of ordinary skill in the art, the term computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media.

[0146] The device embodiments described above are merely illustrative and the units described as separate units can or can not be physically separate, i.e. can be located in one place or distributed over a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purposes of the embodiments.

Claims

1. A curve structure segmentation method based on edge-skeleton dual constraints, characterized in that, The method includes the following steps: S100, acquire the curved target image, preprocess the curved target image to obtain the edge map, skeleton map and text prompt; S200, Construct an edge-skeleton dual-constraint generator based on a stable diffusion model and a control network. Input the edge map, skeleton map and text prompt into the edge-skeleton dual-constraint generator for training to obtain a trained edge-skeleton dual-constraint generator. S300: Use the trained edge-skeleton dual-constraint generator to generate synthetic image-mask pairs, and fuse the synthetic image-mask pairs with the real dataset to obtain the training dataset; S400, Use the training dataset to train the curved structure segmentation model to obtain the trained curved structure segmentation model; S500: Input the curved target image to be segmented into the trained curved structure segmentation model to obtain the segmentation result; S200 specifically includes: S210, Construct an edge-skeleton dual-constraint generator based on a stable diffusion model and a control network. The stable diffusion model includes a variational autoencoder, a multimodal Transformer denoiser, and multiple text encoders. S220, the variational autoencoder is used to compress the real curved target image to obtain the compressed image in the latent space; multiple text encoders are used to encode the text prompts into corresponding text vectors, and then the vectors are concatenated to obtain a comprehensive text vector; the diffusion step is mapped to a time vector through time step embedding. S230, the edge map and skeleton map are stacked in two channels to construct structural conditions, and then input into the control network to output structural constraint features; S240, During the training phase, noise samples are added to the compressed image to obtain a noisy image. The noisy image, the integrated text vector, the time vector, and the structural constraint features are input to the multimodal Transformer denoiser, and the predicted noise is output. S250 constructs a loss function by comparing the difference between predicted noise and noise samples, optimizes model parameters using backpropagation, and iterates training until the loss function converges to obtain a trained edge-skeleton dual-constraint generator.

2. The method according to claim 1, characterized in that, In step S100, the preprocessing of the curved target image to obtain an edge map, a skeleton map, and a text prompt includes: S110, unify the resolution of the curved target image; S120: The curved target image after uniform resolution is smoothed with a Gaussian kernel to obtain a smoothed image; the magnitude and direction of the smoothed image are calculated by Sobel gradient, and non-maximum suppression is performed on the magnitude and direction to obtain an edge map with a width of one pixel. S130, using the mask of the curved target image as input, perform Euclidean distance transformation to obtain the skeleton map; S140, Build the semantic prompt building submodule to generate text prompts.

3. The method according to claim 2, characterized in that, In S130, the step of taking the mask of the curved target image as input and performing Euclidean distance transformation to obtain a skeleton map includes: The skeleton map is obtained by calculating the shortest distance from each pixel to the nearest mask boundary using the Euclidean distance transformation formula. The Euclidean distance transformation formula is as follows: ;in, For mask boundaries, Let p be any pixel position in the image grid, and let p be any boundary pixel position within the boundary. Let x be the Euclidean distance from pixel x to the boundary ∂M.

4. The method according to claim 2, characterized in that, In S140, the semantic prompt construction submodule generates text prompts, including: S141, use ChatGPT to develop a stable cue word template that covers the morphological characteristics, density distribution, directional trend and imaging conditions of curved targets; S142, the preprocessed curved target image is input into the Qwen2.5-VL model, which generates a structured description text for the current curved target image based on the prompt word template; the structured description includes shape, density, orientation and imaging conditions; S143, The generated structured descriptions are manually sampled and solidified to form standardized text prompts.

5. The method according to claim 1, characterized in that, The loss function is: ； in, For loss function, This represents calculating the average, where t represents the time step. Indicates at time step Potential noise, It is a text embedding obtained from a pre-trained text encoder. Indicates a noise sample. This refers to the Transformer denoising function. These are the model parameters for the Transformer denoiser.

6. The method according to claim 1, characterized in that, In S300, the synthesized image-mask pair is generated using a trained edge-skeleton dual-constraint generator, and the synthesized image-mask pair is fused with the real dataset to obtain the training dataset, including: S310 takes manually annotated masks and text prompts as control conditions and inputs them into a trained edge-skeleton dual-constraint generator to generate multiple synthetic image-mask pairs. S320, the synthesized image-mask pair is mixed with the real dataset to obtain the training dataset.

7. A curve structure segmentation system based on edge-skeleton dual constraints, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor performs the method as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images