Flower embroidery generation method based on semantic perception and field driving

Abstract

The application discloses a flower embroidery generation method based on semantic perception and field driving, mainly solves the three bottlenecks of texture flow stiffness, physical texture loss and production path redundancy of the existing automatic embroidery plate making technology, and the problems of low efficiency of manual plate making, neglect of the essence of embroidery technology by commercial software, and incapability of the existing method to adapt to complex natural flower structure. The application builds a two-stage generation framework, which is composed of a semantic perception module and a discrete mapping generation module. In the perception stage, a multi-task Attention U-Net and a region perception direction field prediction network are used to output a semantic segmentation graph, a leaf vein skeleton graph and a region perception direction field, and a mapping function from a physical color space to a needle trace density is established; in the generation stage, a precise embroidery thread color matching module is designed, an adaptive hybrid needle method is used based on semantic regions, and a global TSP path planning strategy is introduced to realize embroidery path optimization.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of computer vision, digital intelligent manufacturing and embroidery digital pattern making. Specifically, it relates to a method for generating floral embroidery based on semantic perception and field-driven principles. This method is compatible with industrial-grade computer embroidery machines and enables fully automated generation of vector embroidery paths from natural flower images to those that can be directly processed on the machine. Background Technology

[0002] Embroidery is a traditional Chinese craft that combines profound historical and cultural heritage with a large-scale modern industry. Modern computer embroidery technology has achieved high-precision automated processing at the hardware level. However, in stark contrast, the digital pattern-making process for high-quality embroidery still faces serious technical bottlenecks. Flowers, as the most common and representative subject matter in embroidery art, with their complex layered structure, delicate natural textures, and rich light and shadow layers, place extremely high aesthetic and technical demands on digital pattern-making, making it a core challenge that current technology struggles to overcome.

[0003] The current mainstream production model in the embroidery pattern making industry still relies heavily on manual labor: professional pattern makers need to divide the image area piece by piece, manually draw the stitch guide lines, and plan the stitch density and processing sequence. The pattern making cycle for a single complex floral pattern can take several hours or even several days. This is not only extremely inefficient and costly in terms of labor, but also the quality of the finished product depends entirely on the pattern maker's personal experience and artistic skills. It is simply unable to support the growing demand for large-scale personalized customization in the modern e-commerce and apparel industries.

[0004] To address the inefficiency of manual pattern making, mainstream commercial embroidery pattern making software has introduced automatic digitization functions. However, its core design logic has fundamental flaws: these software programs generally reduce embroidery art creation to a simple two-dimensional image processing, using only color threshold segmentation to divide color blocks and then filling them with fixed parallel stitches. They completely ignore the physical essence of embroidery—the physical anisotropy of the thread, the directional continuity of continuous stitching, and the occlusion sequence of layered processes—all of which determine the final light and shadow texture and physical workability of the embroidery. This results in three major bottlenecks in the automatically generated floral embroidery patterns: first, the texture flow is rigid, completely violating the natural growth patterns of flowers and lacking artistic expression; second, the physical texture is lacking, failing to create three-dimensional light and shadow through stitch density, resulting in a flat and dull finished product; and third, the production path is severely redundant, with frequent thread-cutting and skipping commands triggered during cross-regional processing, significantly reducing machine processing efficiency and leaving a large number of thread ends on the base fabric, completely destroying the cleanliness and artistic value of both sides of the embroidery.

[0005] To address the technical shortcomings of commercial software, the academic community has undertaken relevant explorations, mainly focusing on two major technical directions: The first is deep learning-based embroidery style transfer methods. These methods use convolutional neural networks and generative adversarial networks to render the embroidery style of images, generating two-dimensional images that visually approximate the embroidery effect. However, these methods can only output "pixel-level" rendered images and cannot generate "path-level" vector embroidery files containing precise needle entry and exit coordinates, processing timing, and machine instructions. An insurmountable gap remains between digital preview and physical processing. The second is computational graphics-based embroidery streamline generation methods. These methods formalize embroidery design into a streamline optimization problem under the constraints of direction and density fields, generating vector paths that can be machine-processed. However, they heavily rely on manual interactive annotation—users need to manually divide polygonal regions and draw directional guide lines one by one. The semi-automatic mode still cannot escape the dependence on manual intervention. Furthermore, when faced with the complex overlapping structures of multiple petals and leaves in floral patterns, problems such as streamline convergence failure, white areas appearing during region splicing, and abruptly broken stitches easily occur, making it unsuitable for the complex topological structures of natural flowers.

[0006] Furthermore, existing computer vision technologies for flower images, such as semantic segmentation and orientation field prediction, are not deeply integrated with the physical processes of embroidery. They fail to transform visual features into stitch paths that conform to machine processing constraints and traditional embroidery specifications. For path planning in cross-regional processing, existing technologies also lack prior knowledge of physical occlusion in layered embroidery work, failing to address the industry's pain point of redundant thread trimming at its root. In summary, there is currently no complete technical solution capable of fully automated generation of high-quality, processable embroidery paths from natural flower images, which has become a core technological barrier restricting the digital transformation of the embroidery industry. Summary of the Invention

[0007] To address the problems existing in current automatic embroidery pattern making technology, such as rigid texture flow, lack of physical texture, redundant production paths, low efficiency of manual pattern making, commercial software ignoring the physical nature of embroidery processes, and the difficulty of adapting existing methods to complex natural flower structures, this invention designs and proposes a flower embroidery generation method based on semantic perception and field-driven approach, so as to achieve fully automatic generation from natural flower images to vector embroidery path files that can be directly processed by machine.

[0008] The technical solution adopted in this invention is: a method for generating floral embroidery based on semantic perception and field-driven principles, comprising the following steps:

[0009] Step (1): Input a natural flower image, construct a global continuous density field mapping function based on the physical color space, and generate a continuous density field to guide the density arrangement of surface stitches;

[0010] Step (2) uses a multi-task Attention U-Net network to extract multi-scale features from the input image. After extracting high-dimensional features through a shared encoder, the fine-grained semantic segmentation map and leaf vein skeleton map of the flower are output synchronously through dual-branch decoding.

[0011] Step (3): Based on the region mask of the semantic segmentation map, the overlapping regions in the image are physically isolated. The local continuous orientation field of each isolated semantic sub-region is independently predicted by the region-aware orientation field prediction network and then fused to obtain the global continuous orientation field.

[0012] Step (4): Construct a precise matching module for embroidery thread colors, perform color clustering and hierarchical determination with semantic segmentation regions as boundaries, converge image colors to the physical embroidery thread color numbers available on the embroidery machine, and establish a bottom-up same-color processing sequence.

[0013] Step (5): Build an adaptive needle generation engine. Based on the category of each semantic region, combine the density field and orientation field of the corresponding region, and use a needle strategy that is adapted to the semantics of the region to generate the discrete needle drop sequence of the corresponding region.

[0014] Step (6): Based on the physical occlusion prior of embroidery processing, the mask union of subsequent processing layers is defined as an absolutely safe connection area. The cross-region connection planning of the same color processing sequence is transformed into a traveling salesman problem with safe mask constraints. The optimal global processing path of the same color layer is output through the TSP solver.

[0015] Step (7) is to perform adaptive constraint optimization of the single needle step length based on the local curvature of the path, and finally generate a vector embroidery path file that can be directly read and executed by the embroidery machine.

[0016] Furthermore, in step (1), given an input natural flower image Global reference canvas background color For any pixel in the image x and y are spatial coordinate parameters; first calculate the normalized global color depth weights:

[0017]

[0018] Among them, molecules The denominator is the Euclidean distance between the local pixel color and the canvas background color. This is the theoretical maximum contrast distance in the color space. To prevent division by zero by extremely small constants, a global continuous density field mapping function is constructed based on the aforementioned color depth weights:

[0019]

[0020] in, For Gaussian smoothing operators, This is a Min-Max normalization operation; This refers to the maximum feasible surface density threshold allowed by the embroidery machine hardware.

[0021] Furthermore, in step (2), the specific implementation of the multi-task Attention U-Net network is as follows:

[0022] The multi-task Attention U-Net network uses ResNet34 as the encoder backbone. It extracts global color and morphological high-dimensional generalization features of the input natural flower image through a shared deep encoder. The decoding end is set with two parallel task branches, namely the semantic segmentation branch and the leaf vein skeleton extraction branch. After decoding, two types of results are output simultaneously.

[0023] The semantic segmentation branch outputs fine-grained semantic segmentation masks containing four types of regions: petals, leaves, flower stems, and stamens, which serve as the basis for subsequent adaptive needle generation of region routing control.

[0024] The extracted leaf vein skeleton outputs a continuous leaf vein topology skeleton map inside the leaf, providing a geometric coordinate reference for the planning of needle direction along the veins and structural expansion compensation in the leaf region.

[0025] The multi-task Attention U-Net network uses a joint loss function for end-to-end training and optimization. The joint loss function expression is as follows: ,in, To balance the hyperparameters of the gradients for each task, For semantic segmentation loss, a combination of cross-entropy loss and Dice loss is used. For the loss of leaf vein skeleton, a weighted binary cross-entropy loss method is adopted.

[0026] Furthermore, in step (3), the global continuous direction field fusion expression is:

[0027]

[0028] in, For the first Mask of semantic subregions For pixel-by-pixel Hadamard product, This is a region-aware orientation field prediction network built on the VGG-16 backbone. This represents the total number of semantic subregions.

[0029] The region-aware orientation field prediction network is optimized using a joint loss function, which combines L1 distance loss and cosine similarity loss, and is expressed as follows:

[0030]

[0031] in, Foreground mask area, and Positions The true direction vector and the predicted vector at that location, The weights of the angle constraint terms, This represents the total number of pixels in the foreground mask region.

[0032] Furthermore, in step (4), the specific implementation method of the embroidery thread color precise matching module is as follows:

[0033] (4a) Embroidery thread color library preprocessing: The full standard physical embroidery thread color library is reduced in dimensionality through perceptual clustering and converged into a fixed number of core embroidery thread subsets, which serve as the benchmark library for subsequent physical color number matching;

[0034] (4b) Regional color clustering and hierarchical pattern determination: Taking each independent semantic sub-region obtained by semantic segmentation as the boundary, K-Means clustering is performed on the pixels in each sub-region in the HSV color space, with the number of clusters k=2, to obtain two candidate cluster center colors C1 and C2; the color difference between the two candidate cluster center colors in the CIELAB color space CIEDE2000 is calculated. And set a stratification threshold. Execute the corresponding processing mode based on the color difference results:

[0035] when When the color is determined to be a single-layer solid color coverage mode, the color mathematical expectation of all pixels in the sub-region is calculated as the unique primary color, and a single-layer constant high-density stitch fill is performed on the semantic region.

[0036] when When the condition is determined to be a dual-layer light and shadow overlay mode, the cluster center color with a larger pixel proportion is defined as the underlying primary color. The cluster center color with a smaller proportion is defined as the hierarchical color. These correspond to the base layer area and the surface light and shadow area, respectively.

[0037] (4c) Global feature color locking: Extract the pixels of all leaf vein mask-covered areas in the input image, calculate their mathematical expectation in the CIELAB color space, obtain a globally unified leaf vein base color, lock the processing color number of the leaf vein area, and avoid color breaks in continuous leaf veins.

[0038] (4d) Global color convergence under machine constraints: based on the maximum number of available needles on the target embroidery machine. To constrain the process, calculate the pairwise color difference between all independent colors in the entire image, and first merge similar colors whose color difference is less than a set threshold; if the number of colors after merging is still greater than a certain threshold, then... Then, using color difference as the distance metric, restricted agglomerative hierarchical clustering is employed to continuously merge color clusters with the smallest color difference until the total number of colors in the entire image equals the maximum number of needles available for the machine. ;

[0039] (4e) Entity color matching and processing sequence establishment: The converged full image color is projected onto the core embroidery thread subset, and the unique corresponding entity embroidery thread color is obtained through nearest neighbor search matching; the discrete regions bound with the same entity embroidery thread color are merged, and the same color processing sequence is established from bottom to top, following the physical coverage time sequence of embroidery processing.

[0040] Furthermore, the specific implementation method of step (5) is as follows:

[0041] The adaptive needle generation engine uses semantic segmentation mask as the region boundary, continuous direction field of the corresponding region as the line reference, and continuous density field as the basis for needle density modulation. Different needle strategies are adopted for four semantic regions: petals, leaves, stamens and stems, to generate discrete needle drop sequence for the corresponding regions.

[0042] For the petal semantic region, a density-aware ray-filling method based on virtual poles is adopted: First, the main direction statistics of the orientation field of the petal region are calculated, and a far-end virtual pole is constructed by offsetting along the opposite direction of the main direction. Ray beams are emitted from the virtual poles to the boundary of the petal region. For the background layer, global ray filling with a fixed angular spacing is performed. For the foreground layer, the ray angular spacing is dynamically modulated with the local density field. The modulation formula is as follows: ,in, Foreground ray angular spacing, and These are the maximum and minimum permissible ray angular spacing of the machine. This represents the density field value at the corresponding position; at the petal outline boundary, a physical jitter function is introduced to simulate the alternating transition process of long and short needles, and the jitter formula is: ,in, The needle point is the original geometric boundary. This is the actual point where the needle lands after the shaking. Let be the unit direction vector of the current ray. To control the maximum physical disturbance distance of the alternating depths of long and short lengths. Let be a uniformly distributed random variable;

[0043] For the semantic region of the leaf, a cooperative continuous needlework method based on topological decoupling is adopted: using the leaf vein skeleton as the dividing line, the complete leaf is divided into multiple sub-regions with independent texture directions. A directional morphological dilation operation is performed on each sub-region towards the central leaf vein. The dilation formula is as follows: ,in The segmented leaf sub-regions This is the processing area after expansion. It is a linear structural element along the normal direction of the leaf veins. To pre-calculate the expansion compensation amount, the background layer generates parallel scan lines with a fixed spacing along the main direction of the region. The scanning direction of the foreground layer is set at a fixed angle relative to the background layer. The spacing of the foreground scan lines is dynamically modulated by the density field, and the modulation formula is: ,in, Foreground scan line spacing, and These represent the minimum and maximum allowable needle spacing of the machine, respectively. The modulation coefficient is used. After the leaf surface area is laid out, the single-line continuous generation of leaf veins is started. A double offset traversal algorithm based on depth-first search is used to transform the leaf vein skeleton line segments into an undirected graph based on distance constraints. A depth-first search traversal is performed on each independent leaf vein tree.

[0044] For the semantic region of flower stamen, a two-layer scanning needle method based on orthogonal offset is adopted: the background layer generates parallel scan lines with a fixed spacing along the main direction of the region, and the foreground layer is set with a 90° orthogonal offset relative to the background layer. The two paths are superimposed to form a cross support structure; the spacing between scan lines and the spacing between single needle drop points are dynamically modulated by the density field. The needle spacing in the dark area approaches the minimum needle spacing of the machine, and the needle spacing in the highlight area tends to the maximum needle spacing of the machine.

[0045] For the semantic region of the flower stem, an adaptive zigzag stitch method based on the central axis diagram model is adopted: the morphological central axis skeleton of the flower stem region is extracted, and the distance transformation field is calculated to obtain the local dynamic width distributed along the skeleton; the bottom layer generates zigzag stitches that dynamically match the local width along the skeleton normal direction; the surface stitches introduce a dynamic deflection angle relative to the bottom layer normal vector; and the step spacing of the surface stitches is dynamically modulated by the density field.

[0046] Furthermore, the specific implementation method of step (6) is as follows:

[0047] (6a) Establish the physical coverage sequence of embroidery processing: following the layering process of first applying a global base, then overlaying light and shadow, and finally top-pressing the structure, the semantic region of the entire image and the decoupled double-layer structure are planned as a bottom-up processing sequence, expressed as:

[0048]

[0049] Among them, subscript Represents the underlying base layer of the corresponding semantic region, subscript A surface lighting and shadow layer representing the corresponding semantic region;

[0050] (6b) Define the absolutely safe connection area: For the current layer of the same color to be processed, define the union of the masks of all fabric areas that have not yet been embroidered and will be completely covered by the dense layer in the subsequent processing sequence as the absolutely safe connection area. ;

[0051] (6c) Constructing a TSP problem model with security mask constraints: Define the set of discrete sub-regions to be processed in the current same-color layer as... The entry point of each sub-region With the needle exit point The access node is defined as the TSP, the cross-regional movement of the embroidery needle is defined as the Traveling Salesman Path, and the physical transfer cost of the cross-regional needle skipping is defined as the path cost. The path cost function expression is:

[0052]

[0053] in, These are two discrete sub-regions within the same color layer that need to be processed. sub-region The needle exit point, sub-region The needle insertion point, This refers to a cross-regional connection path between two points. This represents the machine's wire-cutting command;

[0054] (6d) Optimal path solution: The above-mentioned constrained TSP problem is solved by a bidirectional greedy algorithm. The node sequence with the minimum connectivity cost is found within the absolutely safe connection zone, and a global continuous processing path of the same color layer is generated. The machine cut-off command is inserted only when two points cannot be connected within the safety mask.

[0055] Furthermore, the specific implementation method of step (7) is as follows:

[0056] (7a) Single-needle step length adaptive constraint optimization: For the global processing path of the same color layer output in step (6), an adaptive constraint function based on the local path curvature is introduced to dynamically modulate the physical spacing of each needle landing point on the path. The expression of the adaptive constraint function is:

[0057]

[0058] In the formula, L max With L min These are the maximum and minimum safe stitch lengths allowed by the hardware of the target embroidery machine. The current routing path in coordinates Local geometric curvature at that point This is the curvature penalty coefficient;

[0059] (7b) Machine instruction adaptation and file generation: The optimized needle drop sequence is adapted to machine instructions. According to the standard instruction specifications of the target embroidery machine, the needle drop sequence, corresponding embroidery thread color number, thread cutting instruction, and empty jump instruction are converted into a vector embroidery path file that can be directly read and executed by the embroidery machine, thus completing the fully automatic generation from natural flower image to embroidery file that can be processed by machine.

[0060] Furthermore, the training of the multi-task Attention U-Net network and the region-aware orientation field prediction network is completed using the multimodal floral embroidery pairing dataset EFD. The specific construction method of the dataset is as follows:

[0061] Data Acquisition and Preprocessing: Several sets of original images of natural flowers and corresponding embroidery patterns were collected. All embroidery patterns were tested on an industrial-grade embroidery machine. Standard embroidery thread was used and the color number of the actual embroidery thread used in the processing was recorded. After image registration, geometric alignment and cropping enhancement processing, standardized image pairing data with uniform resolution was obtained.

[0062] Construction of a multimodal ground truth annotation system: For each pair of data, three types of ground truth annotations are completed. The first is semantic and skeleton annotation, which generates four types of high-precision semantic segmentation masks for petals, leaves, flower stems, and stamens, as well as the center line of the topological skeleton of leaf veins. The second is orientation field annotation, which extracts the local texture orientation statistical features of real embroidery products, combines them with manually corrected orientation guide lines in key areas, and fuses them to generate smooth and continuous orientation field ground truths. The third is supporting annotation, which simultaneously generates density field ground truths, color classification ground truths, and vector path files that can be directly used for machining.

[0063] Dataset partitioning: A certain proportion of the labeled dataset is divided into training, validation, and test sets to provide a standardized multi-task learning benchmark for the training and validation of multi-task networks.

[0064] Furthermore, the multi-task Attention U-Net network and the region-aware orientation field prediction network are jointly trained end-to-end, and the global total loss function used for training is expressed as follows:

[0065]

[0066] in, To balance the hyperparameters of the gradients for each task, The value is greater than and To enhance the network's ability to extract fine leaf vein skeleton structures; For semantic segmentation branch loss, Extracting branch loss from the leaf vein skeleton. The joint loss is for the orientation field; the specific implementation of the end-to-end joint training method is as follows:

[0067] Pre-training initialization: The multi-task Attention U-Net network and the region perception orientation field prediction network are pre-trained independently. The encoder backbones of the two networks are initialized using ImageNet pre-trained weights. During the pre-training stage, the bottom weights of the encoder are frozen, and only the high-level decoding layer and the task output head are fine-tuned to complete the weight initialization of the two sub-networks.

[0068] Joint fine-tuning optimization: Unfreeze all trainable weights of both networks, and apply the global total loss function as described above. To optimize the objective, end-to-end joint training is performed. During training, a gradient pruning mechanism is used to avoid gradient explosion, and a learning rate cosine annealing strategy is used to accelerate model convergence. Finally, a joint training model with consistent global features for the three types of outputs—semantic segmentation, leaf vein extraction, and orientation field prediction—is obtained.

[0069] Compared with the prior art, the beneficial effects of the present invention are:

[0070] This invention unifies and couples multi-task semantic perception, regional isolation direction field prediction, precise matching of embroidery thread colors, adaptive hybrid stitch generation, and TSP global path optimization with security mask constraints. It can directly output machine-producible vector embroidery path files in complex natural flower scenes. Compared with existing methods that can only generate pixel-level embroidery effect images, this invention achieves a complete closed loop from natural images to processing paths. Compared with existing solutions that rely on manual field annotation or ignore physical process constraints, the direction field generated by this invention is smoother and more continuous, the stitch texture is more in line with the natural growth law of plants, and it can effectively reduce cross-region thread cutting and empty jump redundancy, improve machine processing efficiency and the cleanliness of both sides of the embroidery. Attached Figure Description

[0071] Figure 1 This is a flowchart illustrating the overall technical process of the present invention.

[0072] Figure 2 This is a diagram of the framework for generating floral embroidery based on semantic perception and field-driven principles of the present invention.

[0073] Figure 3 This is a schematic diagram of the semantic perception module structure of the present invention.

[0074] Figure 4 This is a schematic diagram of the discrete mapping generation module and global path optimization of the present invention. Detailed Implementation

[0075] To better understand the technical solution of the present invention, the specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0076] like Figure 1 As shown in the figure, the overall implementation process of the flower embroidery generation method based on semantic awareness and field-driven approach provided in this embodiment of the invention is as follows:

[0077] Step (1): Input a natural flower image, construct a global continuous density field mapping function based on the physical color space, and generate a continuous density field to guide the density arrangement of surface stitches;

[0078] Step (2) uses a multi-task Attention U-Net network to extract multi-scale features from the input image. After extracting high-dimensional features through a shared encoder, the fine-grained semantic segmentation map and leaf vein skeleton map of the flower are output synchronously through dual-branch decoding.

[0079] Step (3): Based on the region mask of the semantic segmentation map, the overlapping regions in the image are physically isolated. The local continuous orientation field of each isolated semantic sub-region is independently predicted by the region-aware orientation field prediction network and then fused to obtain the global continuous orientation field.

[0080] Step (4): Construct a precise matching module for embroidery thread colors, perform color clustering and hierarchical determination with semantic segmentation regions as boundaries, converge image colors to the physical embroidery thread color numbers available on the embroidery machine, and establish a bottom-up same-color processing sequence.

[0081] Step (5): Build an adaptive needle generation engine. Based on the category of each semantic region, combine the density field and orientation field of the corresponding region, and use a needle strategy that is adapted to the semantics of the region to generate the discrete needle drop sequence of the corresponding region.

[0082] Step (6): Based on the physical occlusion prior of embroidery processing, the mask union of subsequent processing layers is defined as an absolutely safe connection area. The cross-region connection planning of the same color processing sequence is transformed into a traveling salesman problem with safe mask constraints. The optimal global processing path of the same color layer is output through the TSP solver.

[0083] Step (7) is to perform adaptive constraint optimization of the single needle step length based on the local curvature of the path, and finally generate a vector embroidery path file that can be directly read and executed by the embroidery machine.

[0084] The specific implementation process of each step is as follows:

[0085] In step (1), a natural flower image is input, and a global continuous density field mapping function is constructed based on the physical color space to generate a continuous density field for guiding the density arrangement of surface stitches. Given an input natural flower image... Global reference canvas background color For any pixel in the image x and y are the spatial coordinate parameters in the two-dimensional plane of the digital image, used to uniquely locate a single pixel. They are the core spatial reference for the entire embroidery generation framework, from digital image to physical stitch calculation. First, calculate the normalized global color depth weights:

[0086]

[0087] Among them, molecules The denominator is the Euclidean distance between the local pixel color and the canvas background color. This is the theoretical maximum contrast distance in the color space. To prevent extremely small constants from being divided by zero, this embodiment takes... Based on the aforementioned color depth weights, a global continuous density field mapping function is constructed:

[0088]

[0089] in, The Gaussian smoothing operator is used; in this embodiment, a Gaussian kernel with a kernel size of 5×5 and a standard deviation of 1.5 is employed. This is a Min-Max normalization operation; To define the maximum feasible surface density threshold allowed by the embroidery machine hardware, this embodiment sets the following for the BF-900 computerized embroidery machine: =0.8mm / needle. Final output continuous density field Used to dynamically modulate the spacing between surface stitches in each semantic region.

[0090] In step (2), a multi-task Attention U-Net network is used to extract multi-scale features from the input image. After extracting high-dimensional features through a shared encoder, the fine-grained semantic segmentation map and leaf vein skeleton map of the flower are output simultaneously through dual-branch decoding. The specific implementation is as follows:

[0091] (2a) The multi-task Attention U-Net network uses ResNet34 as the encoder backbone and extracts global color and morphological high-dimensional generalization features of the input natural flower image through a shared deep encoder. The deep encoders used in this scheme are all existing publicly available mature network structures, including ResNet34, VGG-16, etc., which are common image feature extraction techniques in the field of computer vision. The innovation of this invention does not lie in the encoder itself, but in the complete technical solution of using existing encoders in a multi-task perception framework, combining embroidery techniques to achieve joint decoupling of semantic / direction field / density field, and adapting to adaptive stitching and TSP path optimization.

[0092] The decoding end has two parallel task branches: a semantic segmentation branch and a leaf vein skeleton extraction branch. The semantic segmentation branch adopts a 4-level symmetric upsampling architecture, which fuses ResNet34 multi-scale features through skip connections and introduces attention gates to suppress background noise. Finally, it outputs fine-grained semantic segmentation global masks for four categories: petals, leaves, flower stems, and stamens through 1×1 convolution + Softmax, providing accurate region boundaries for subsequent hierarchical decoupling and needle generation. The leaf vein skeleton extraction branch adopts a symmetric upsampling architecture, which focuses on configuring detail enhancement convolutional units to strengthen the fine line features of leaf veins and avoid skeleton breakage. It fuses multi-scale edge and semantic information through skip connections, and finally outputs a continuous leaf vein topology skeleton map through 1×1 convolution + Sigmoid, providing a geometric coordinate reference for planning the needle direction along the veins and compensating for structural expansion in the leaf region.

[0093] (2b) The multi-task Attention U-Net network uses a joint loss function for end-to-end training and optimization. The expression for the joint loss function is: ,in, , To balance the hyperparameters of the gradients for each task, For semantic segmentation loss, a combination of cross-entropy loss and Dice loss is used. For the loss of leaf vein skeleton, a weighted binary cross-entropy loss method is adopted.

[0094] (2c) For semantic segmentation branch loss, a combination of cross-entropy loss and Dice loss is used, expressed as follows:

[0095]

[0096] The cross-entropy loss is:

[0097]

[0098] Dice's loss is:

[0099]

[0100] in, Total number of pixels; This represents the total number of semantic categories (such as petals, leaves, stamens, background, etc.). For specific category indexes; and Pixels In category The actual labels and network prediction probabilities on the network. For balance coefficient, To prevent division by zero smoothing terms, the foreground mask region of the flower subject output by the semantic segmentation branch is set to... The introduction of Dice loss effectively improves the network's fitting accuracy for the boundaries of minimal regions.

[0101] (2d) To extract branching loss for the leaf vein skeleton, a weighted binary cross-entropy loss is used, expressed as:

[0102]

[0103] in, and Pixels The true skeleton labels and predicted probabilities, The penalty weight is for positive samples.

[0104] In this embodiment, the multi-task Attention U-Net network is trained using the Adam optimizer, with an initial learning rate set to 10. -4 The batch size is 8, and the model is iterated for 200 rounds on a single V100 (24GB) GPU. Gradient pruning is used during training to avoid gradient explosion, and cosine annealing is used to accelerate model convergence.

[0105] In step (3), overlapping regions in the image are physically isolated based on the region mask of the semantic segmentation map. A region-aware orientation field prediction network independently predicts the local continuous orientation field for each isolated semantic sub-region, and then fuses them to obtain the global continuous orientation field. The specific implementation is as follows:

[0106] (3a) Specifically, based on the region mask of the semantic segmentation map, the overlapping regions of the image are physically isolated, and the orientation field is predicted for each isolated semantic sub-region. Finally, the global continuous orientation field is fused. The expression for the global continuous orientation field fusion is as follows:

[0107]

[0108] in, For the first Mask of semantic subregions For pixel-by-pixel Hadamard product, This is a region-aware orientation field prediction network built on the VGG-16 backbone. This represents the total number of semantic subregions.

[0109] (3b) The region-aware orientation field prediction network takes the isolated region image features as input and uses the VGG-16 model pre-trained on ImageNet as the core encoder to extract multi-scale feature maps of stage3, stage4, and stage5 from the encoder. In the feature fusion stage, the high-level semantic features of stage4 and stage5 are first upsampled to the same spatial size as the feature map of stage3 through bilinear interpolation, and then feature concatenation is performed in the channel dimension. The concatenated fused features are subjected to local feature parsing through three consecutive convolutional layers to output dual-channel feature maps, which correspond to the two-dimensional plane respectively. and Partial derivative components. Finally, bilinear interpolation is used to restore the dual-channel feature map to its original size from the input image, and pixel-wise L2 normalization is performed to transform it into a continuous unit-direction vector field. .

[0110] (3c) The region-aware orientation field prediction network is optimized using a joint loss function, which combines L1 distance loss and cosine similarity loss, and is expressed as:

[0111]

[0112] in, This refers to the foreground mask region of the flower subject output by the semantic segmentation branch in step 2. and Positions The true direction vector and the predicted vector at that location, The weights of the angle constraint terms, This represents the total number of pixels in the foreground mask region.

[0113] Here, the ground truth vector is the supervision label used in the training phase. It is a direction field label obtained through morphological processing or streamline generation algorithms based on the leaf vein skeleton and the growth morphology of the flower region (such as the radial direction of petals and the direction of leaf veins) output in step 2. It is used to constrain the learning objective of the network. The predicted vector is the direct output of the region-aware direction field prediction network in step 3, that is, the unit direction vector field obtained after VGG-16 feature extraction, multi-scale fusion, convolution parsing, and L2 normalization. This refers to the network's prediction of the orientation for each pixel location.

[0114] The training of the multi-task Attention U-Net network and the region-aware orientation field prediction network was completed using the multimodal floral embroidery pairing dataset EFD. The specific construction method of the dataset is as follows:

[0115] Data Acquisition and Preprocessing: Several sets of original images of natural flowers and corresponding embroidery patterns were collected. All embroidery patterns were tested on an industrial-grade embroidery machine. Standard embroidery thread was used and the color number of the actual embroidery thread used in the processing was recorded. After image registration, geometric alignment and cropping enhancement processing, standardized image pairing data with uniform resolution was obtained.

[0116] Construction of a multimodal ground truth annotation system: For each pair of data, three types of ground truth annotations are completed. The first is semantic and skeleton annotation, which generates four types of high-precision semantic segmentation masks for petals, leaves, flower stems, and stamens, as well as the center line of the topological skeleton of leaf veins. The second is orientation field annotation, which extracts the local texture orientation statistical features of real embroidery products, combines them with manually corrected orientation guide lines in key areas, and fuses them to generate smooth and continuous orientation field ground truths. The third is supporting annotation, which simultaneously generates density field ground truths, color classification ground truths, and vector path files that can be directly used for machining.

[0117] Dataset partitioning: A certain proportion of the labeled dataset is divided into training, validation, and test sets to provide a standardized multi-task learning benchmark for the training and validation of multi-task networks.

[0118] The multi-task Attention U-Net network and the region-aware orientation field prediction network are jointly trained end-to-end. The global total loss function used for training is expressed as follows:

[0119]

[0120] in, To balance the hyperparameters of the gradients for each task, The value is greater than and To enhance the network's ability to extract fine leaf vein skeleton structures; For semantic segmentation branch loss, Extracting branch loss from the leaf vein skeleton. The joint loss is for the orientation field; the specific implementation of the end-to-end joint training method is as follows:

[0121] Pre-training initialization: The multi-task Attention U-Net network and the region perception orientation field prediction network are pre-trained independently. The encoder backbones of the two networks are initialized using ImageNet pre-trained weights. During the pre-training stage, the bottom weights of the encoder are frozen, and only the high-level decoding layer and the task output head are fine-tuned to complete the weight initialization of the two sub-networks.

[0122] Joint fine-tuning optimization: Unfreeze all trainable weights of both networks, and apply the global total loss function as described above. To optimize the objective, end-to-end joint training is performed. During training, a gradient pruning mechanism is used to avoid gradient explosion, and a learning rate cosine annealing strategy is used to accelerate model convergence. Finally, a joint training model with consistent global features for the three types of outputs—semantic segmentation, leaf vein extraction, and orientation field prediction—is obtained.

[0123] In step (4), a precise matching module for embroidery thread colors is constructed. Color clustering and hierarchical determination are performed using semantic segmentation regions as boundaries. The image colors are converged to the actual embroidery thread color numbers available on the embroidery machine. Based on color difference determination, single-layer / double-layer decoupling is performed on each semantic region to obtain a decoupled double-layer structure. A bottom-up same-color processing sequence is established. The specific implementation method is as follows:

[0124] (4a) Embroidery thread color library preprocessing: The more than 1,200 standard physical embroidery thread colors collected are reduced in dimensionality and converged into a core embroidery thread subset of 190 standard colors through perceptual clustering, which serves as the benchmark library for subsequent physical color number matching.

[0125] (4b) Regional color clustering and hierarchical pattern determination: Taking each independent semantic sub-region obtained by semantic segmentation as the boundary, K-Means clustering is performed on the pixels in each sub-region in the HSV color space, with the number of clusters k=2, to obtain two candidate cluster center colors. and Calculate the CIEDE2000 color difference between the two candidate cluster center colors in the CIELAB color space. And set a stratification threshold. =10:

[0126] when When the region is determined to be a single-layer solid color coverage mode, the expected color of all pixels in the region is calculated as the unique primary color, and a single-layer constant high-density stitch fill is performed on the semantic sub-region.

[0127] when At that time: it is determined to be a two-layer light and shadow overlay mode, and the cluster center color with a larger pixel ratio is defined as the bottom primary color. The cluster center color with a smaller proportion is defined as the hierarchical color. These correspond to the base layer area and the surface light and shadow area, respectively.

[0128] (4c) Global feature color locking: Extract the pixels of all leaf vein mask-covered areas in the input image, calculate their mathematical expectation in the CIELAB color space, obtain a globally unified leaf vein base color, lock the processing color number of the leaf vein area, and avoid color breaks in continuous leaf veins.

[0129] (4d) Global color convergence under machine constraints: Let the total number of remaining independent colors in the current full image be... The maximum number of available needles for the target embroidery machine. For constraints, this embodiment is set specifically for the BF-900 embroidery machine. =9, calculate the CIEDE2000 color difference between all independent colors in the entire image in the CIELAB color space, and first merge similar colors with a color difference of less than 5; if the number of colors after merging is still greater than 9, calculate the CIEDE2000 color difference between each pair of independent colors in the entire image, and first merge similar colors with a color difference of less than 5; if the number of colors after merging is still greater than 9, calculate the CIEDE2000 color difference between each Then, using CIEDE2000 color difference as the distance metric, Agglomerative Hierarchical Clustering is used to continuously merge the color clusters with the smallest color difference until the total number of colors in the entire image is strictly equal to the maximum number of available needles on the machine.

[0130] (4e) Entity color matching and processing sequence establishment: The converged full-image colors are projected onto the aforementioned 190 core embroidery thread subsets in the CIELAB color space, and the unique corresponding entity embroidery thread color number is obtained through nearest neighbor search matching. Discrete regions bound with the same entity embroidery thread color number are merged, and the decoupling results of all semantic regions in the full image together constitute a decoupled two-layer structure. Following the physical coverage time sequence of embroidery processing, a bottom-up same-color processing sequence is established.

[0131] Step 5: Build an adaptive needle pattern generation engine. Based on the category of each semantic region, and combined with the density field and orientation field of the corresponding region, generate a discrete needle point sequence for the corresponding region using a needle pattern strategy adapted to the semantics of the region. The specific implementation method is as follows:

[0132] (5a) The adaptive stitch generation engine uses semantic segmentation masks as region boundaries, continuous direction fields of the corresponding regions as the routing reference, and continuous density fields as the basis for stitch density modulation. It employs differentiated stitch strategies for four semantic regions—petals, leaves, stamens, and stems—to generate discrete stitch point sequences for the corresponding regions. Given a two-dimensional connected region... Directional field With density field Generate an ordered set of physical needle placement points: , making the point and The resulting sequence of line segments conforms to both local directional constraints and density coverage requirements.

[0133] (5b) Petal Region Needling Method: A density-sensing ray-filling needleling method based on virtual poles is adopted. First, the principal direction statistics of the orientation field of the petal region are calculated:

[0134]

[0135] The geometric center of the petals is located in the opposite direction to the main direction. Offset outwards to construct a remote virtual pole. :

[0136]

[0137] in, This represents the maximum span of the region in the main direction. For the far-end bias coefficient, in this embodiment, we take... =2.0 to control the degree of divergence; The unit vector is in the main direction. This is the pole. Ray beams are emitted towards the boundaries of the petal regions. For the background layer, global ray filling with a fixed angular spacing is performed; for the foreground layer, the angular spacing of the rays varies with the local density field. Perform dynamic modulation:

[0138]

[0139] in, and These represent the maximum and minimum allowable ray angular spacing of the machine tool, respectively. In this embodiment, they are respectively taken as... =1.5°, =0.3°. Finally, a physical jitter function is introduced at the petal outline boundary to simulate the alternating long and short stitch transition technique in traditional hand embroidery:

[0140]

[0141] in, The needle point is the original geometric boundary. This is the actual point where the needle lands after the shaking. This is the unit direction vector of the current ray; To control the maximum physical disturbance distance of the alternating long and short depths, this embodiment takes... =1.5mm; Let be a uniformly distributed random variable.

[0142] (5c) Leaf Area Stitching: A cooperative continuous stitching technique based on topological decoupling is employed. Based on the extracted leaf vein skeleton, the complete leaf is precisely divided into multiple sub-regions with independent texture directions. To address the issue of exposed veins due to fabric shrinkage, a directional morphological expansion operation is performed on each sub-region towards the central leaf vein.

[0143]

[0144] in, The segmented leaf sub-regions This is the expanded processing area; It is a linear structural element along the normal direction of the leaf vein; To preset the expansion compensation amount, this embodiment takes... =0.3mm. Calculate the principal direction statistics of the orientation field within each region. For the background layer, along Parallel background scan lines are emitted at fixed intervals. For the foreground layer, the system sets a fixed 15° offset angle relative to the background layer for its scanning direction to avoid co-directional fabric cutting and needle trapping effects. The spacing between the foreground scan lines is determined by the density field. Dynamic modulation:

[0145]

[0146] in, and As the machine's permissible maximum needle pitch, in this embodiment, we take... =0.8mm =2.5mm; For the modulation coefficient, in this embodiment, we take... =1.2. After completing the layout of the leaf surface region, the single-stroke continuous generation of leaf veins is initiated. A double offset traversal algorithm based on depth-first search (DFS) is adopted to transform the leaf vein skeleton line segments into an undirected graph based on distance constraints, and DFS traversal is performed on each independent leaf vein tree. During the traversal, the needle not only moves towards the leaf tip, but also turns back along the original path, and injects a very small-scale random two-dimensional spatial offset in the back path to achieve "zero-cutting" continuous processing of complex tree topologies.

[0147] (5d) Stitching technique for the stamen region: A double-layer scanning stitching technique based on orthogonal offset is used. The principal direction of the stamen region is calculated. After laying down a parallel background layer, a 90° orthogonal offset angle is set for the foreground layer. The orthogonal stitching of the upper and lower layers interweaves in a very small space to form a stable cross-shaped physical support structure, completely eliminating the risk of puncturing the base fabric due to dense stitching, and firmly supporting the surface embroidery threads, highlighting the thickness and three-dimensionality of the flower's center. The color gradation and volume in the flower's center area are also achieved through a continuous density field. Dynamic modulation is performed to guide the machine to perform approximation in the dark part of the flower stamen. The densely packed needles apply a tendency to the highlights. The sparse transition between shadows and highlights is achieved by the continuous density field generated in step (1). Automatically defined. Density field value. It is derived from the color depth mapping of the input image; the higher the value, the darker the color (dark area) of that region, requiring a smaller pin spacing (approximation). Dense embroidery is used to create shadows and volume; the lower the value, the brighter the color of the area (highlight), and a larger stitch spacing is used (tending towards) Sparse lines are used to achieve a smooth transition of light and shadow.

[0148] (5e) Flower Stem Region Needling: An adaptive wrapping needle technique based on a central axis graph model is adopted. First, the morphological central axis skeleton of the flower stem region is extracted, and the distance transformation field is calculated to obtain the local dynamic width distributed along the skeleton. A KD-Tree neighborhood search is introduced to construct an undirected graph model. The system performs ordered walks along the connected skeleton nodes and calculates the local tangents and normals in real time. For the background layer, the system strictly generates zigzag wrapping needles that dynamically match the local width along the normal direction. For the foreground layer, the system superimposes secondary-colored wrapping needles on top of the background layer and introduces a dynamic deflection angle of 5°~8° on the surface normal vector to avoid overlapping of high-frequency and dense wrapping needles on the same normal line, which would cause the stitches to break. The step spacing of the surface wrapping needles is also determined by a continuous density field. Dynamic modulation is performed.

[0149] Step 6: Based on the physical occlusion prior of embroidery processing, define the mask union of subsequent processing layers as an absolutely safe connection region. Transform the cross-region connection planning of the same-color processing sequence into a Traveling Salesman Problem with safe mask constraints. Output the optimal global processing path of the same-color layer through the TSP solver. The specific implementation is as follows:

[0150] (6a) Establish the physical coverage sequence of embroidery processing: strictly follow the layering process rule of "first apply the overall base, then add light and shadow, and finally add the structure on top", and plan the semantic region of the whole image and the decoupled double-layer structure as the following bottom-up processing sequence. :

[0151]

[0152] The full-image semantic region comes directly from the output of the multi-task Attention U-Net network in the perception stage of step (2), and is a set of full-domain, fine-grained semantic segmentation masks; the decoupled two-layer structure comes from the hierarchical mode determination of the embroidery thread color accurate matching module in the generation stage of step (4), and is a hierarchical decoupling based on the color gradient within the semantic region. Represents the underlying base layer of the corresponding semantic region, subscript A surface lighting and shadow layer representing the corresponding semantic region;

[0153] (6b) Define the absolutely safe connection area: For the current layer of the same color to be processed, define the union of the masks of all fabric areas that have not yet been embroidered and will be completely covered by the dense layer in the subsequent processing sequence as the absolutely safe connection area. .

[0154] (6c) Construct a TSP (Traveling Salesman Problem) model with security mask constraints: Define the set of discrete sub-regions to be processed in the current same-color layer as:

[0155]

[0156] in, This is the set of discrete sub-regions to be processed within the current layer of the same color. For the first layer within this layer Each area to be embroidered is an independent sub-area. The total number of sub-regions; the discrete sub-regions It is obtained by dividing the connected regions of the same color layer. Each sub-region corresponds to a local embroidery unit that needs to be completed continuously in the embroidery process.

[0157] In this step, the needle entry point for each sub-region is... With the needle exit point The access node is defined as the TSP, the cross-regional movement of the embroidery needle is defined as the Traveling Salesman Path, and the physical transfer cost of the cross-regional needle skipping is defined as the path cost. The path cost function expression is:

[0158]

[0159] in, These are two discrete sub-regions within the same color layer that need to be processed. sub-region The needle exit point; sub-region The needle insertion point; A cross-regional connection path between two points; This represents the machine's wire-cutting command.

[0160] (6d) Optimal Path Finding: A bidirectional greedy algorithm is used to solve the constrained TSP problem described above. Within the absolutely safe connection zone, the sequence of nodes with the minimum connectivity cost is found, generating a globally continuous processing path with the same color layer. The system will only forcibly block the path and insert a machine cut-wire command when two points cannot be connected within the safe mask.

[0161] Step 7: Adaptive constraint optimization of the single-needle step length based on the local curvature of the path is performed to finally generate a vector embroidery path file that can be directly read and executed by the embroidery machine. The specific implementation method is as follows:

[0162] (7a) Single-needle step length adaptive constraint optimization: For the global processing path of the same color layer output in step 6, an adaptive constraint function based on the local path curvature is introduced to dynamically adjust the physical spacing of each needle landing point on the path. The expression of the adaptive constraint function is:

[0163]

[0164] in, and These are the maximum and minimum safe stitch lengths allowed by the hardware of the target embroidery machine, respectively. In this embodiment, we take... =3.0mm =0.5mm; The current routing path in coordinates Local geometric curvature at the location; The curvature penalty coefficient is taken as [value missing] in this embodiment. =0.5. This constraint function enables the machine to advance at high speed with the maximum step size on a long straight path, and automatically shortens the stitch length as the curvature increases at the edge of the petals and at fine turns, so as to achieve high-precision geometric contour approximation. At the same time, it avoids processing defects such as loosening and floating of the embroidery thread due to excessive stitch length, and knots or breakage of the embroidery thread due to excessive stitch length.

[0165] (7b) Machine instruction adaptation and file generation: The optimized needle drop sequence is adapted to machine instructions. According to the standard instruction specifications of the BF-900 computer embroidery machine, the needle drop sequence, corresponding embroidery thread color number, thread cutting instruction, and empty jump instruction are converted into a vector embroidery path file in DST format, thus completing the fully automatic generation from natural flower images to embroidery files that can be processed by the machine.

[0166] The invention will now be demonstrated through specific simulation experiments:

[0167] The specific hardware equipment for this simulation experiment is as follows: the computer has 32GB of RAM, an i7-12700 processor, an RTX3080Ti graphics card with 12GB of video memory; the experimental processing equipment is a BF-900 small computer embroidery machine (single-head 9-needle).

[0168] The specific software environment for this simulation experiment is as follows: Ubuntu 20.04.5 operating system, PyTorch 1.10.0, CUDA 11.3, cuDNN8, and Python 3.7.

[0169] The dataset used in this simulation experiment is the Embroidery-Flowers dataset (EFD). This dataset contains 1000 representative original images of natural flowers and their corresponding embroidery patterns. All finished products were tested using an industrial-grade Brother embroidery machine, and standard embroidery threads were strictly used. The system recorded all physical color codes used during the processing. The dataset is divided into a training set (800 sets), a validation set (100 sets), and a test set (100 sets) in an 8:1:1 ratio. Each data tuple not only contains the original flower image and rendered appearance image, but also includes a complete semantic mask, topological skeleton, density field ground truth, orientation field ground truth, color classification ground truth, and the final vector path file that can be used for machining.

[0170] This invention constructs a multi-dimensional evaluation system, including: color fidelity (average color difference). The evaluation included the actual number of color changes on the machine, field prediction quality (mean absolute error of direction field angle MAE and area smoothness), physical processing quality (needle length distribution compliance rate and number of times the machine cuts threads), and visual and artistic evaluation (inviting 5 senior embroidery designers to conduct subjective scoring on a 5-point Likert scale).

[0171] Simulation Experiment 1: Joint Evaluation of Semantic Topology Awareness and Orientation Field Prediction. Experimental results show that, facing severe overlap of petal edges and abrupt scale changes in extremely fine flower stems, the multi-task Attention U-Net model of this invention can output semantic masks with extremely sharp edges and completely accurate classification. Simultaneously extracted leaf veins not only successfully captured central features but also exhibited excellent topological continuity, without any breaks or burrs. Quantitative results show that the orientation field prediction MAE of the multi-task network of this invention is only 8.3°, significantly better than the 15.7° of the traditional global PCA extraction method. Ablation experiments further confirm that introducing semantic masks to physically isolate and predict overlapping regions greatly alleviates the mutual interference of global gradients, significantly reducing the final MAE error by 22%.

[0172] Simulation Experiment 2: End-to-End Path Generation and Solid Processing Verification. This invention performed complete pattern making and solid machining of various natural flowers with complex semantics and lighting. In the final physical processing product, the embroidery exhibits excellent smooth transitions at the edges of the petals, the solid embroidery threads completely follow the natural texture of plant growth, and the back of the base fabric maintains a very high degree of cleanliness.

[0173] Simulation Experiment 3: In-depth comparison with commercial software and academic baselines. This invention conducts a comprehensive end-to-end comparison of the proposed method with the latest computational graphics top conference algorithm, "Directionality-Aware Design of Embroidery Patterns" proposed by Liu et al. in 2023 (published at Eurographics 2023, hereinafter referred to as the [Liu et al. 2023] algorithm, academic baseline), and the industry benchmark commercial software Wilcom Auto-Digitize (industrial baseline).

[0174] Visual Texture and Multicolor Processing: Faced with the complex multicolor gradients and irregular shapes of real natural flowers, Wilcom software can only divide the image into uniform color blocks and rigidly fill them with monotonous parallel tatami needles, completely losing the natural light and shadow transitions of plant growth. The method of [Liu et al. 2023], in order to forcibly satisfy the usability of its mathematical streamline generation algorithm, has to erode the geometry of the regions, resulting in huge physical gaps (severe white showing) between the semantic regions in the final product. In contrast, the method of this invention not only achieves precise physical isolation of regions through high-precision semantic masks, but also introduces a morphological dilation compensation mechanism to ensure that the texture flow of each petal and leaf is smooth and vivid, and the splicing is full and seamless.

[0175] Physical machinability and wire trimming redundancy: This invention focuses on addressing the pain points of wire trimming in machining by implementing global TSP optimization and hidden safety wiring planning. Experimental results show that the average number of wire trimmings per image generated by the method of this invention is only 74, which is not only far lower than Wilcom's 157, but also significantly better than the [Liu et al. 2023] method (96) without wiring optimization. This optimization completely eliminates messy cross-area jumpers at the source, significantly reducing mechanical wear.

[0176] Simulation Experiment 4: Efficiency Analysis. The end-to-end average inference time of the framework of this invention on the test set is only 28 seconds per image. The specific module time distribution is as follows: perception stage 6 seconds, accurate color matching 3 seconds, geometric stitch generation 12 seconds, and physical path planning and optimization 7 seconds. This computational efficiency fully meets the real-time interaction and preview requirements of industrial-grade CAD software.

[0177] In summary, this invention effectively solves the triple bottlenecks of existing automated embroidery pattern making technology—rigid texture flow, lack of physical texture, and redundant production paths—by unifying and coupling multi-task semantic perception, region-isolated direction field prediction, precise embroidery thread color matching, adaptive hybrid stitch generation, and TSP global path optimization with security mask constraints. Experiments demonstrate that the system outputs a continuously controllable physical field, accurate color reproduction, and extremely high processing feasibility, significantly improving the automation level of the digital manufacturing pipeline.

[0178] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0179] It should be understood that the above description of the embodiments is quite detailed, but this should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims, all of which fall within the scope of protection of this invention. Furthermore, the order of the steps in the above embodiments is only used to illustrate the technical solution of this invention and is not intended to limit the scope of protection of this invention. In practical applications, each step can be adjusted, combined, or executed in parallel as needed without affecting the implementation of the technical solution of this invention. The processing flow shown in the accompanying drawings is not necessarily required to be executed sequentially as shown. In some embodiments, multi-tasking or parallel processing methods can be used to achieve the same or equivalent technical effects.

Claims

1. A method for generating floral embroidery based on semantic perception and field-driven principles, characterized in that, Includes the following steps: Step (1): Input a natural flower image, construct a global continuous density field mapping function based on the physical color space, and generate a continuous density field to guide the density arrangement of surface stitches; Step (2) uses a multi-task Attention U-Net network to extract multi-scale features from the input image. After extracting high-dimensional features through a shared encoder, the fine-grained semantic segmentation map and leaf vein skeleton map of the flower are output synchronously through dual-branch decoding. Step (3): Based on the region mask of the semantic segmentation map, the overlapping regions in the image are physically isolated. The local continuous orientation field of each isolated semantic sub-region is independently predicted by the region-aware orientation field prediction network and then fused to obtain the global continuous orientation field. Step (4): Construct a precise matching module for embroidery thread colors, perform color clustering and hierarchical determination with semantic segmentation regions as boundaries, converge image colors to the physical embroidery thread color numbers available on the embroidery machine, and establish a bottom-up same-color processing sequence. Step (5): Build an adaptive needle generation engine. Based on the category of each semantic region, combine the density field and orientation field of the corresponding region, and use a needle strategy that is adapted to the semantics of the region to generate the discrete needle drop sequence of the corresponding region. Step (6): Based on the physical occlusion prior of embroidery processing, the mask union of subsequent processing layers is defined as an absolutely safe connection area. The cross-region connection planning of the same color processing sequence is transformed into a traveling salesman problem with safe mask constraints. The optimal global processing path of the same color layer is output through the TSP solver. Step (7) is to perform adaptive constraint optimization of the single needle step length based on the local curvature of the path, and finally generate a vector embroidery path file that can be directly read and executed by the embroidery machine.

2. The method for generating floral embroidery based on semantic perception and field-driven principles as described in claim 1, characterized in that: In step (1), given an input image of a natural flower... Global reference canvas background color For any pixel in the image x and y are spatial coordinate parameters; first calculate the normalized global color depth weights: ； Among them, molecules The denominator is the Euclidean distance between the local pixel color and the canvas background color. This is the theoretical maximum contrast distance in the color space. To prevent division by zero by extremely small constants, a global continuous density field mapping function is constructed based on the aforementioned color depth weights: ； in, For Gaussian smoothing operators, This is a Min-Max normalization operation; This refers to the maximum feasible surface density threshold allowed by the embroidery machine hardware.

3. The method for generating floral embroidery based on semantic perception and field-driven principles as described in claim 1, characterized in that: In step (2), the specific implementation of the multi-task Attention U-Net network is as follows: The multi-task Attention U-Net network uses ResNet34 as the encoder backbone. It extracts global color and morphological high-dimensional generalization features of the input natural flower image through a shared deep encoder. The decoding end is set with two parallel task branches, namely the semantic segmentation branch and the leaf vein skeleton extraction branch. After decoding, two types of results are output simultaneously. The semantic segmentation branch outputs fine-grained semantic segmentation masks containing four types of regions: petals, leaves, flower stems, and stamens, which serve as the basis for subsequent adaptive needle generation of region routing control. The extracted leaf vein skeleton outputs a continuous leaf vein topology skeleton map inside the leaf, providing a geometric coordinate reference for the planning of needle direction along the veins and structural expansion compensation in the leaf region. The multi-task Attention U-Net network uses a joint loss function for end-to-end training and optimization. The expression for the joint loss function is as follows: ,in, To balance the hyperparameters of the gradients for each task, For semantic segmentation loss, a combination of cross-entropy loss and Dice loss is used. For the loss of leaf vein skeleton, a weighted binary cross-entropy loss method is adopted.

4. The method for generating floral embroidery based on semantic perception and field-driven principles as described in claim 1, characterized in that: In step (3), the global continuous direction field fusion expression is: ； in, For the first Mask of semantic subregions For pixel-by-pixel Hadamard product, This is a region-aware orientation field prediction network built on the VGG-16 backbone. This represents the total number of semantic subregions. The region-aware orientation field prediction network is optimized using a joint loss function, which combines L1 distance loss and cosine similarity loss, and is expressed as follows: ； in, Foreground mask area, and Positions The true direction vector and the predicted vector at that location, The weights of the angle constraint terms, This represents the total number of pixels in the foreground mask region.

5. The method for generating floral embroidery based on semantic perception and field-driven principles as described in claim 1, characterized in that: In step (4), the specific implementation method of the embroidery thread color accurate matching module is as follows: (4a) Embroidery thread color library preprocessing: The full standard physical embroidery thread color library is reduced in dimensionality through perceptual clustering and converged into a fixed number of core embroidery thread subsets, which serve as the benchmark library for subsequent physical color number matching; (4b) Regional color clustering and hierarchical pattern determination: Taking each independent semantic sub-region obtained by semantic segmentation as the boundary, K-Means clustering is performed on the pixels in each sub-region in the HSV color space, with the number of clusters k=2, to obtain two candidate cluster center colors C1 and C2; the color difference between the two candidate cluster center colors in the CIELAB color space CIEDE2000 is calculated. And set a stratification threshold. Execute the corresponding processing mode based on the color difference results: when When the color is determined to be a single-layer solid color coverage mode, the color mathematical expectation of all pixels in the sub-region is calculated as the unique primary color, and a single-layer constant high-density stitch fill is performed on the semantic region. when When the condition is determined to be a dual-layer light and shadow overlay mode, the cluster center color with a larger pixel proportion is defined as the underlying primary color. The cluster center color with a smaller proportion is defined as the hierarchical color. These correspond to the base layer area and the surface light and shadow area, respectively. (4c) Global feature color locking: Extract the pixels of all leaf vein mask-covered areas in the input image, calculate their mathematical expectation in the CIELAB color space, obtain a globally unified leaf vein base color, lock the processing color number of the leaf vein area, and avoid color breaks in continuous leaf veins. (4d) Global color convergence under machine constraints: based on the maximum number of available needles on the target embroidery machine. To constrain the process, calculate the pairwise color difference between all independent colors in the entire image, and first merge similar colors whose color difference is less than a set threshold; if the number of colors after merging is still greater than a certain threshold, then... Then, using color difference as the distance metric, restricted agglomerative hierarchical clustering is employed to continuously merge color clusters with the smallest color difference until the total number of colors in the entire image equals the maximum number of needles available for the machine. ; (4e) Entity color matching and processing sequence establishment: The converged full image color is projected onto the core embroidery thread subset, and the unique corresponding entity embroidery thread color is obtained through nearest neighbor search matching; the discrete regions bound with the same entity embroidery thread color are merged, and the same color processing sequence is established from bottom to top, following the physical coverage time sequence of embroidery processing.

6. The method for generating floral embroidery based on semantic perception and field-driven principles as described in claim 1, characterized in that: The specific implementation method of step (5) is as follows: The adaptive needle generation engine uses semantic segmentation mask as the region boundary, continuous direction field of the corresponding region as the line reference, and continuous density field as the basis for needle density modulation. Different needle strategies are adopted for four semantic regions: petals, leaves, stamens and stems, to generate discrete needle drop sequence for the corresponding regions. For the petal semantic region, a density-aware ray-filling method based on virtual poles is adopted: First, the main direction statistics of the orientation field of the petal region are calculated, and a far-end virtual pole is constructed by offsetting along the opposite direction of the main direction. Ray beams are emitted from the virtual poles to the boundary of the petal region. For the background layer, global ray filling with a fixed angular spacing is performed. For the foreground layer, the ray angular spacing is dynamically modulated with the local density field. The modulation formula is as follows: ,in, Foreground ray angular spacing, and These are the maximum and minimum permissible ray angular spacing of the machine, respectively. This represents the density field value at the corresponding position; at the petal outline boundary, a physical jitter function is introduced to simulate the alternating transition process of long and short needles. The jitter formula is: ,in, The needle point is the original geometric boundary. This is the actual point where the needle lands after the shaking. Let be the unit direction vector of the current ray. To control the maximum physical disturbance distance of alternating long and short depths. Let be a uniformly distributed random variable; For the semantic region of the leaf, a cooperative continuous needlework method based on topological decoupling is adopted: using the leaf vein skeleton as the dividing line, the complete leaf is divided into multiple sub-regions with independent texture directions. A directional morphological dilation operation is performed on each sub-region towards the central leaf vein. The dilation formula is as follows: ,in The segmented leaf sub-regions This is the processing area after expansion. It is a linear structural element along the normal direction of the leaf veins. To pre-calculate the expansion compensation amount, the background layer generates parallel scan lines with a fixed spacing along the main direction of the region. The scanning direction of the foreground layer is set at a fixed angle relative to the background layer. The spacing of the foreground scan lines is dynamically modulated by the density field, and the modulation formula is: ,in, Foreground scan line spacing, and These represent the minimum and maximum allowable needle spacing of the machine, respectively. The modulation coefficient is used. After the leaf surface area is laid out, the single-line continuous generation of leaf veins is started. A double offset traversal algorithm based on depth-first search is used to transform the leaf vein skeleton line segments into an undirected graph based on distance constraints. A depth-first search traversal is performed on each independent leaf vein tree. For the semantic region of flower stamen, a two-layer scanning needle method based on orthogonal offset is adopted: the background layer generates parallel scan lines with a fixed spacing along the main direction of the region, and the foreground layer is set with a 90° orthogonal offset relative to the background layer. The two paths are superimposed to form a cross support structure; the spacing between scan lines and the spacing between single needle drop points are dynamically modulated by the density field. The needle spacing in the dark area approaches the minimum needle spacing of the machine, and the needle spacing in the highlight area tends to the maximum needle spacing of the machine. For the semantic region of the flower stem, an adaptive zigzag stitch method based on the central axis diagram model is adopted: the morphological central axis skeleton of the flower stem region is extracted, and the distance transformation field is calculated to obtain the local dynamic width distributed along the skeleton; the bottom layer generates zigzag stitches that dynamically match the local width along the skeleton normal direction; the surface stitches introduce a dynamic deflection angle relative to the bottom layer normal vector; and the step spacing of the surface stitches is dynamically modulated by the density field.

7. The method for generating floral embroidery based on semantic perception and field-driven principles as described in claim 1, characterized in that: The specific implementation method of step (6) is as follows: (6a) Establish the physical coverage sequence of embroidery processing: following the layering process of first applying a global base, then overlaying light and shadow, and finally top-pressing the structure, the semantic region of the entire image and the decoupled double-layer structure are planned as a bottom-up processing sequence, expressed as: ； Among them, subscript Represents the underlying base layer of the corresponding semantic region, subscript A surface lighting and shadow layer representing the corresponding semantic region; (6b) Define the absolutely safe connection area: For the current layer of the same color to be processed, define the union of the masks of all fabric areas that have not yet been embroidered and will be completely covered by the dense layer in the subsequent processing sequence as the absolutely safe connection area. ; (6c) Constructing a TSP problem model with security mask constraints: Define the set of discrete sub-regions to be processed in the current same-color layer as... The entry point of each sub-region With the needle exit point The access node is defined as the TSP, the cross-regional movement of the embroidery needle is defined as the Traveling Salesman Path, and the physical transfer cost of the cross-regional needle skipping is defined as the path cost. The path cost function expression is: ； in, These are two discrete sub-regions within the same color layer that need to be processed. sub-region The needle exit point, sub-region The needle insertion point, This refers to a cross-regional connection path between two points. This represents the machine's wire-cutting command; (6d) Optimal path solution: The above-mentioned constrained TSP problem is solved by a bidirectional greedy algorithm. The node sequence with the minimum connectivity cost is found within the absolutely safe connection zone, and a global continuous processing path of the same color layer is generated. The machine cut-off command is inserted only when two points cannot be connected within the safety mask.

8. The method for generating floral embroidery based on semantic perception and field-driven principles as described in claim 1, characterized in that: The specific implementation method of step (7) is as follows: (7a) Single-needle step length adaptive constraint optimization: For the global processing path of the same color layer output in step (6), an adaptive constraint function based on the local path curvature is introduced to dynamically modulate the physical spacing of each needle landing point on the path. The expression of the adaptive constraint function is: ； In the formula, L max With L min These are the maximum and minimum safe stitch lengths allowed by the hardware of the target embroidery machine. The current routing path in coordinates Local geometric curvature at that point This is the curvature penalty coefficient; (7b) Machine instruction adaptation and file generation: The optimized needle drop sequence is adapted to machine instructions. According to the standard instruction specifications of the target embroidery machine, the needle drop sequence, corresponding embroidery thread color number, thread cutting instruction, and empty jump instruction are converted into a vector embroidery path file that can be directly read and executed by the embroidery machine, thus completing the fully automatic generation from natural flower image to embroidery file that can be processed by machine.

9. The method for generating floral embroidery based on semantic perception and field-driven principles as described in claim 1, characterized in that: The training of the multi-task Attention U-Net network and the region-aware orientation field prediction network was completed using the multimodal floral embroidery pairing dataset EFD. The specific construction method of the dataset is as follows: Data Acquisition and Preprocessing: Several sets of original images of natural flowers and corresponding embroidery patterns were collected. All embroidery patterns were tested on an industrial-grade embroidery machine. Standard embroidery thread was used and the color number of the actual embroidery thread used in the processing was recorded. After image registration, geometric alignment and cropping enhancement processing, standardized image pairing data with uniform resolution was obtained. Construction of a multimodal ground truth annotation system: For each pair of data, three types of ground truth annotations are completed. The first is semantic and skeleton annotation, which generates four types of high-precision semantic segmentation masks for petals, leaves, flower stems, and stamens, as well as the center line of the topological skeleton of leaf veins. The second is orientation field annotation, which extracts the local texture orientation statistical features of real embroidery products, combines them with manually corrected orientation guide lines in key areas, and fuses them to generate smooth and continuous orientation field ground truths. The third is supporting annotation, which simultaneously generates density field ground truths, color classification ground truths, and vector path files that can be directly used for machining. Dataset partitioning: A certain proportion of the labeled dataset is divided into training, validation, and test sets to provide a standardized multi-task learning benchmark for the training and validation of multi-task networks.

10. The method for generating floral embroidery based on semantic perception and field-driven principles as described in claim 1, characterized in that: The multi-task Attention U-Net network and the region-aware orientation field prediction network are jointly trained end-to-end. The global total loss function used for training is expressed as follows: ； in, To balance the hyperparameters of the gradients for each task, The value is greater than and To enhance the network's ability to extract fine leaf vein skeleton structures; For semantic segmentation branch loss, Extracting branch loss from the leaf vein skeleton. The joint loss is for the orientation field; the specific implementation of the end-to-end joint training method is as follows: Pre-training initialization: The multi-task Attention U-Net network and the region perception orientation field prediction network are pre-trained independently. The encoder backbones of the two networks are initialized using ImageNet pre-trained weights. During the pre-training stage, the bottom weights of the encoder are frozen, and only the high-level decoding layer and the task output head are fine-tuned to complete the weight initialization of the two sub-networks. Joint fine-tuning optimization: Unfreeze all trainable weights of both networks, and apply the global total loss function as described above. To optimize the objective, end-to-end joint training is performed. During training, a gradient pruning mechanism is used to avoid gradient explosion, and a learning rate cosine annealing strategy is used to accelerate model convergence. Finally, a joint training model with consistent global features for the three types of outputs—semantic segmentation, leaf vein extraction, and orientation field prediction—is obtained.

Images