Embroidery product virtual rendering method and system based on model collaboration

Abstract

The invention relates to the technical field of computer-aided industrial design, in particular to an embroidery product virtual rendering method and system based on model collaboration, and aims to realize generation of a realistic rendering image from a user sketch to a product carrier and an embroidery process integration through cross-model workflow so as to solve the defects in the prior art. The method is suitable for digital design application of the embroidery technology in products such as clothes and accessories. The invention provides the method for generating the sketch to the product rendering graph of the user at low cost, so that the manual training cost can be reduced, a lower-price scheme is provided for traditional practitioners, the personalized requirements of the consumers can be met, the embroidery design efficiency is improved, and the embroidery design period is shortened. The technical threshold of professional design can be reduced, and the universality of creation is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer-aided industrial design, in particular to a virtual rendering method and system for embroidery products based on model collaboration. BACKGROUND

[0002] Under the promotion of individualization and cultural consumption in contemporary society, embroidery, as a traditional craft with both customization and cultural attributes, is gradually being enthusiastically purchased by consumers. However, traditional embroidery often cannot meet the individualized needs of consumers due to its reliance on handcrafters, and consumers cannot see the finished product during the customization process, lacking a reference.

[0003] Existing technologies focus on simple pattern generation and only pay attention to pattern aesthetics, which cannot meet the individualized needs of consumers and lack the presentation of patterns on products. Meanwhile, traditional methods require training of special models for different embroidery categories and fine-tuning, resulting in high training costs. SUMMARY

[0004] Therefore, the present application aims to provide a virtual rendering method and system for embroidery products based on model collaboration to solve the problems in the background art.

[0005] To achieve the above-mentioned purpose, the present application adopts the following technical solutions: The virtual rendering method for embroidery products based on model collaboration comprises the following steps: Obtaining design input data of a user, wherein the design input data includes image data and design intention text description, and the image data is an initial design sketch or a sample style picture; Extracting text features and image features of the design input data and fusing the text features and the image features to obtain a fused feature vector, wherein the text features are extracted based on a text semantic understanding model, and the image features are extracted based on an image feature extraction network; Generating a structured rendering instruction based on the fused feature vector, wherein the structured rendering instruction includes an accurate description of embroidery process parameters required for calling different regions in a design drawing; Searching from a pre-constructed embroidery pattern database set based on the structured rendering instruction to obtain a related database, wherein the embroidery pattern database set includes a plurality of embroidery pattern database sets with classification labels; Constraining a rendering engine with feature data in the related database, inputting the image data in the design input data and the structured rendering instruction into the rendering engine to obtain a virtual rendering picture.

[0006] In an embodiment of the present application, the text features and image features of the design input data are extracted, including: judging the type of the design input data; when the type of the design input data is text, inputting the design input data into a pre-constructed text semantic understanding model to obtain a structured text feature table, wherein the text semantic understanding model is used to perform word segmentation and vectorization on the text to obtain a semantic vector, and perform entity extraction on the semantic vector, and automatically classify and label map the extracted entities to obtain a structured text feature table; when the type of the design input data is image, inputting the design input data into a pre-constructed image feature extraction network to obtain image features, wherein the image features include low-level features representing edge contours, color distribution and basic texture primitives, middle-level features representing local components, composition elements and repetitive patterns, and high-level / semantic features representing overall semantic understanding or style labels of the image content.

[0007] In an embodiment of the present application, the text features and the image features are fused to obtain a fused feature vector, including: capturing an associated area with semantic association with the text features from the image features through an attention mechanism, and performing feature weight enhancement on the associated area to obtain a fused feature vector.

[0008] In an embodiment of the present application, a structured rendering instruction is generated based on the fused feature vector, including: obtaining product carrier surface physical properties, and generating a structured rendering instruction in combination with the product carrier surface physical properties and the fused feature vector, wherein the structured rendering instruction includes scene detailed description, light and position description, product carrier and material description, texture stitch method and physical property description, texture feature and content detailed description, and product texture pattern overall style overall description.

[0009] In an embodiment of the present application, a relevant database is obtained by searching from a pre-constructed embroidery pattern database based on the structured rendering instruction, including: parsing the structured rendering instruction to obtain a query unit representing a plurality of semantic labels, wherein the query unit is an accurate keyword or a semantic keyword; When the query unit is a precise keyword, the query unit is matched with tags of each embroidery pattern database in the pre-constructed embroidery pattern database set, and when a tag of any embroidery pattern database matches the query unit, the tag takes the embroidery pattern database matching the query unit as a relevant database; when there is no tag of any embroidery pattern database matching the query unit, the query unit is matched with elements of each embroidery pattern database in the pre-constructed embroidery pattern database set, and a relevant database is constructed based on all elements matching the query unit. When the query unit is a semantic keyword, the query unit and tags of each embroidery pattern database in the pre-constructed embroidery pattern database set are vectorized and then similarity calculation is performed, and an embroidery pattern database with a similarity greater than a preset similarity threshold is taken as a relevant database; when there is no embroidery pattern database with a similarity greater than the preset similarity threshold, the similarity of the query unit and tags of elements of each embroidery pattern database in the embroidery pattern database set is calculated, and a relevant database is constructed based on all elements with a similarity greater than the preset similarity threshold.

[0010] In an embodiment of the present application, the relevant database includes a stitch database, a pattern database and a material database, wherein the feature data in the relevant database is used to constrain a rendering engine, the image data in the design input data and the structured rendering instruction are input into the rendering engine to obtain a virtual rendering image, including: the image data is parsed into a two-dimensional vector path or a region mask; the two-dimensional vector path or the region mask is mapped into a UV coordinate of a product carrier to determine a bounding box and a filling path of an embroidery area; trace parameters are extracted from the stitch database, scalable vector graphics vector curves or segmented texture masks are extracted from the pattern database, and physical rendering material parameters are extracted from the material database; lighting constraints are constructed, and the two-dimensional vector path or the region mask is rendered in the UV coordinate based on constraints of the trace parameters, constraints of the scalable vector graphics vector curves or the segmented texture masks, constraints of the physical rendering material parameters and the lighting constraints to obtain a virtual rendering image.

[0011] In an embodiment of the present application, the construction method of the embroidery pattern database set includes: an image sample is obtained, wherein the image sample is from historical data or from user uploading; in a construction stage, semantic tags of the image sample are extracted, and the image sample is divided into a plurality of embroidery pattern databases based on the semantic tags to obtain an embroidery pattern database set; In the expansion phase, an image sample uploaded by a user is extracted, a semantic label of the image sample is extracted, a similarity calculation is performed between the semantic label of the image sample and labels of each embroidery pattern database in the embroidery pattern database set, and the image sample is stored in an embroidery pattern database with the highest similarity.

[0012] In an embodiment of the present application, the method further comprises: outputting the virtual rendering image and the tuning options to the user; Upon receiving modified design input data from the user, returning to extracting textual features and image features of the design input data to regenerate a virtual rendering image and returning to outputting the virtual rendering image and the tuning options to the user until no longer receiving modified design input data from the user, wherein the modified design input data is generated based on the tuning options.

[0013] In an embodiment of the present application, the embroidery pattern database set further comprises a user-defined style database.

[0014] The present application also provides a model-based collaborative embroidery product virtual rendering system, comprising: an acquisition module configured to acquire design input data of a user, wherein the design input data comprises image data and a textual description of design intent, and the image data is an initial design sketch or a sample style picture; a feature extraction module configured to extract textual features and image features of the design input data, and fuse the textual features and the image features to obtain a fused feature vector, wherein the textual features are extracted based on a text semantic understanding model, and the image features are extracted based on an image feature extraction network; an instruction generation module configured to generate a structured rendering instruction based on the fused feature vector, wherein the structured rendering instruction comprises an accurate description of embroidery process parameters required to be called in different regions of a design image; a retrieval module configured to retrieve a relevant database from a pre-constructed embroidery pattern database set based on the structured rendering instruction, wherein the embroidery pattern database set comprises a plurality of embroidery pattern databases with classification labels; a rendering module configured to constrain a rendering engine with feature data in the relevant database, input the image data in the design input data and the structured rendering instruction into the rendering engine, and obtain a virtual rendering image.

[0015] The beneficial effects of the present application are: the model cooperation-based embroidery product virtual rendering method and system provided by the present application can realize the generation of a real image from a user sketch to a product carrier and an integrated embroidery process through a cross-model workflow, so as to solve the problems of the prior art and be suitable for the digital design application of the embroidery process in products such as clothes and accessories. The present application provides a method capable of meeting the personalized needs of consumers at low cost and realizing the generation of a product rendering image from a user sketch, which can not only reduce the cost of artificial training and provide a lower-cost solution for traditional practitioners, but also improve the personalized needs of consumers, improve the efficiency of embroidery design, and shorten the embroidery design cycle. The present application can reduce the technical threshold of professional design and improve the universality of creation. BRIEF DESCRIPTION OF DRAWINGS

[0016] The present application will be further described below in conjunction with the drawings and embodiments: Figure 1 is a flowchart of a model cooperation-based embroidery product virtual rendering method according to an embodiment of the present application; Figure 2 is a flowchart of a model cooperation-based embroidery product virtual rendering method according to another embodiment of the present application; Figure 3 is a structural diagram of a model cooperation-based embroidery product virtual rendering system according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] The embodiments of the present application will be described below in conjunction with specific concrete examples, and those skilled in the art can easily understand other advantages and effects of the present application from the disclosure. The present application can also be implemented or applied in different specific embodiments, and the details in the specification can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0018] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present application in a schematic manner, and the diagrams only show the layers related to the present application, not the number of layers, shapes and sizes during actual implementation. The actual implementation of each layer may be a random change, and the layer layout pattern may also be more complex.

[0019] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application, however, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details.

[0020] Figure 1is a flowchart of a model collaboration-based virtual rendering method for embroidery products shown in an embodiment of the present application, as shown in Figure 1 A model collaboration-based virtual rendering method for embroidery products can include steps S110 to S140 in the embodiment: S110, obtaining design input data of a user, wherein the design input data includes image data and design intention text description, and the image data is an initial design sketch or a sample style picture; In the present application, at least two design inputs (text and image) submitted by the user for a specific product carrier (such as clothing, wallet) are obtained through a human-computer interaction interface, and the design inputs include an initial design sketch, a natural language description of design intention, or a sample style picture. Optionally, the design input data can also include the user's historical design scheme. The system stores and records the past design scheme, and the user can perform secondary design and generate rendering based on the historical design scheme.

[0021] Optionally, the system automatically checks and standardizes the input file format and resolution before receiving the input, and automatically converts the resolution and color space (RGB / CMYK). S120, extracting text features and image features of the design input data, and fusing the text features and the image features to obtain a fusion feature vector; generating a structured rendering instruction based on the fusion feature vector, wherein the structured rendering instruction includes an accurate description of embroidery process parameters required for different regions in a design drawing; S121, fusion feature vector When the design input data from the user is obtained, the design intention is analyzed and a structured rendering instruction is generated through a preset design intention analysis module. The design intention analysis module is built-in or calls a multi-modal large language model, and the multi-modal large model has an image feature extraction network, a text semantic understanding model, and a cross-modal fusion unit, etc. It can jointly analyze the received multi-modal design input. The specific process is as follows: S1211, judging the type of the design input data; S1212, when the type of the design input data is text, inputting the design input data into a pre-constructed text semantic understanding model to obtain a structured text feature table, wherein the text semantic understanding model is used for word segmentation and vectorization of text to obtain a semantic vector, and performs entity extraction on the semantic vector, and automatically classifies and labels the extracted entities to obtain a structured text feature table; Specifically, the text semantic understanding model requires the use of, but is not limited to, BERT, RoBERTa or LLaMA fine-tuning based on the Transformer architecture, which can realize intent recognition and entity extraction.

[0022] The model will first analyze the core purpose of the entire sentence text (for example, a handbag with a golden dragon embroidered on it surrounded by red peonies, and the core intent is to design an embroidered handbag). Then the model will perform word segmentation and vectorization on the sentence text to generate semantic vectors, while performing entity extraction and mapping.

[0023] Entity extraction is the process of extracting key information from text to achieve the intent. Entities will be automatically classified and labeled, and finally a structured text feature table will be obtained to achieve clear structured natural language description.

[0024] The structured feature table content includes but is not limited to design theme (core intent), design pattern, pattern position, color description, product position, embroidery needle method, material, style description and unique identification, etc.

[0025] S1213, when the type of the design input data is an image, input the design input data into a pre-constructed image feature extraction network to obtain image features, wherein the image features include low-level features representing edge contours, color distribution and basic texture primitives, middle-level features representing local components, composition elements and repetitive patterns, and high-level / semantic features representing overall semantic understanding or style labels of the image content.

[0026] The image feature extraction network requires the use of a convolutional neural network (CNN) or a visual Transformer (VIT) architecture.

[0027] For the CNN architecture, ResNet or VGGNet is used for hierarchical extraction, mainly to extract low-level edge and texture features and high-level pattern semantic features from user design sketches.

[0028] For the VIT architecture, the global self-attention mechanism is used to obtain the overall composition information, mainly to extract the overall spatial layout, overall atmosphere and texture distribution in the picture from the sample style picture.

[0029] In this application, the network architecture can be dynamically selected according to the type of the input image (design sketch or sample style picture). The image feature extraction network realizes feature recognition and extraction of design sketches and sample style pictures, and puts their features into the database or corresponding database.

[0030] The preprocessed image is fed into the image feature extraction network. As the data flows through the layers of the network, each layer generates an embedding vector, and finally a high-dimensional feature vector is formed.

[0031] The high-dimensional feature vector has a multi-level feature, which is: (1) Low-level features: extracted from the shallow layers of the network, mainly including edge contours, color distribution, basic texture primitives, etc. For user sketches, this corresponds to the basic shapes and lines they sketch; (2) Mid-level features: extracted from the middle layers of the network, including local components of objects, composition elements, and repetitive patterns.

[0032] (3) High-level / semantic features: a highly condensed feature vector is extracted from the deep layers or final output layer of the network, which contains the overall semantic understanding or style label of the image content.

[0033] In addition, for the content of the database, the system will automatically preprocess and realize the vectorization of the database. The system processes each entry of the database through the same image extraction network as the input image processing, converts it into a feature vector, and stores it in a specific vector database (embroidery pattern database).

[0034] In this application, an embroidery pattern database set (constraint library) is constructed in advance to constrain subsequent rendering. In this embodiment, the construction process of the embroidery pattern database set includes: (1) Obtain image samples, wherein the image samples come from historical data or from user uploads; (2) In the construction phase, extract the semantic labels of the image samples, and divide the image samples into multiple embroidery pattern data based on the semantic labels, to obtain an embroidery pattern database set; In the initial stage of constructing the database set, the semantic labels are extracted by the image feature extraction network described above, and then through clustering / classification, etc., by classifying and setting labels for stitches, patterns, styles, and signs, the structured data of embroidery is sorted out, and a database is established respectively, forming a database set, and the database is labeled to realize fast retrieval of specific databases in the database set.

[0035] (3) In the expansion phase, extract image samples from user uploads and extract semantic labels of the image samples, and calculate the similarity of the semantic labels of the image samples with the labels of each embroidery pattern database in the embroidery pattern database set, and store the image samples in the embroidery pattern database with the highest similarity.

[0036] In the expansion phase, the database will also automatically include user-entered example style pictures as data supplements to continuously expand the database.

[0037] In addition, the user's brand elements or customized logo information can be added and stored in the constraint library in the form of pictures, and labeled for use.

[0038] Specifically, for the user's brand elements or customized logo information, the system will generate a specific vector to represent it, and also process it through an image feature extraction network. The system will generate a non-repeating digital code for it, and the content after image network processing will become the label of the element or information together.

[0039] S1214, fuse the text features and the image features to obtain a fused feature vector: In this application, the cross-modal fusion unit is used to fuse image features and language description features. Specifically, the attention mechanism is used to capture the associated regions with semantic correlation from the image features and the text features, and the feature weight of the associated regions is strengthened to obtain a fused feature vector.

[0040] The fusion unit calculates the correlation between the text features and the image features (if the text description is "red peony", the attention mechanism will strengthen the red features of the flower region in the peony image vector; if the text contains "random needlework", the system will retrieve the corresponding needlework texture template in the needlework vector.), to realize the fusion of text semantic information and spatial feature information in the image. After fusion, the unit outputs a unified multi-modal feature vector, which contains complete information of image information and text and image alignment information (i.e. semantic constraints on patterns).

[0041] S122, generating a structured rendering instruction, based on the features and combining the preset product carrier surface physical properties, automatically generating a structured rendering instruction. The instruction contains the accurate description of the embroidery process parameters required to be called in different regions of the design drawing, specifically including: S1221, obtaining the product carrier surface physical properties, and generating a structured rendering instruction based on the product carrier surface physical properties and the fused feature vector, wherein the structured rendering instruction includes detailed description of the product scene, description of light and position, description of product carrier and material, description of needlework and physical properties of texture, detailed description of features and content of texture, and overall description of overall style of product texture pattern.

[0042] Exemplarily, the instruction includes detailed description of the product scene, description of light and position, description of product carrier and material, description of needlework and physical properties of texture, detailed description of features and content of texture, and overall description of overall style of product texture pattern.

[0043] In detail, the structured instruction is a formatted and hierarchically clear data file in JSON format, which allows the rendering engine to better parse and execute. This article cites a structured instruction for reference and explanation, but its content and format are not limited to this.

[0044] { "scene_info": { / / Defines the environment of the rendering scene, such as lighting, camera angle, and background "lighting": "Studio Softbox", / / Defines the light as a studio softbox "camera_angle": "Front-facing, slightly elevated", / / Defines the camera angle as front-facing and slightly elevated "background": "Neutral gray fabric" / / Defines the background as neutral gray fabric }, "product_carrier": { / / Describes the product as a carrier, including its type, material, and color "type": "T-shirt", / / Defines the type as a T-shirt "material": "Cotton", / / Defines the material as cotton "physical_properties": { / / Represents physical properties "surface_texture": "Slightly wrinkled", / / Defines the surface texture as slightly wrinkled "draping": "Soft" / / Defines the draping as soft }, "color": "#FFFFFF" / / Defines the color as white }, "embroidery_elements": [ / / Defines the embroidery elements { "element_id": "e001", / / Defines the element ID001 "source_sketch_region": { / / Defines the source sketch region "shape": "circle", / / Defines the source sketch region as a circle "center_coordinates": [256, 300], / / Defines the center coordinates of the source sketch region [256, 300] "radius": 50 / / Defines the radius of the source sketch area as 50 pixels. }, "design_theme": "Chrysanthemum Emblem", / / Defines the design theme as "Chrysanthemum Emblem". "embroidery_details": { "stitch_db_reference": "RandomStitch", / / Defines the stitch database reference: RandomStitch "thread_properties": { / / Defines the wire properties "material": "Silk", / / Defines the material attribute as silk. "color_palette": ["#4169E1", "#1E90FF", "#87CEFA"], / / Defines a color palette: ["#4169E1", "#1E90FF", "#87CEFA"]; "glossiness": 0.8, / / Defines a glossiness of 0.8. "thickness_mm": 0.5 / / Defines a thickness of 0.5. }, "physical_effects": { / / Defines physical effects "dimensionality": "High (3D effect)", / / Defines the dimension of the physical effect as "High (3D effect)". "density": "90%" / / Defines the needle density as 90%. } }, "spatial_layout": { / / Defines the spatial layout "target_location": "Chest, center", / / Defines the target location for the spatial layout as "Chest center". "rotation_degrees": 0, / / Defines symmetry as 0 "scale": 1.0 / / Defines the scaling ratio as 1.0 (original size)} } ], "overall_style": "Modern minimalist" / / Defines the overall style as "modern minimalist". } S130, based on the structured rendering instructions, a search is performed from a pre-built embroidery pattern database set to obtain a relevant database, wherein the embroidery pattern database set includes multiple embroidery pattern database sets with classification tags; The retrieval process includes: S131, the structured rendering instruction is parsed to obtain query units representing multiple semantic tags, wherein the query unit is a precise keyword or a semantic keyword; First, the system will parse the rendering instructions, dividing them into multiple query units, and performing different searches based on the unit type. Unit types can be divided into precise keywords and semantic keywords. Precise keywords correspond to database tags and can be directly matched. Semantic keywords correspond to design requirements such as style, theme, and abstract design; semantic encoding and vector similarity matching are required during retrieval.

[0045] S132, when the query unit is a precise keyword, the query unit is matched with the tags of each embroidery pattern database in the pre-built embroidery pattern database set. When any tag of an embroidery pattern database matches the query unit, the tag will use the embroidery pattern database that matches the query unit as the relevant database. When no tag of any embroidery pattern database matches the query unit, the query unit is matched with the elements of each embroidery pattern database in the pre-built embroidery pattern database set, and a relevant database is constructed based on all elements that match the query unit. S133, when the query unit is a semantic keyword, the similarity is calculated after vectorizing the tags of the query unit and each embroidery pattern database in the pre-built embroidery pattern database set, and the embroidery pattern database with a similarity greater than a preset similarity threshold is taken as the relevant database; when there is no embroidery pattern database with a similarity greater than the preset similarity threshold, the similarity between the query unit and the tags of the elements in each embroidery pattern database in the embroidery pattern database set is calculated, and the relevant database is constructed based on all elements with a similarity greater than the preset similarity threshold.

[0046] In the above process, for precise keyword searches, such as fields like stitching technique and carrier, direct tag matching is performed. For example, if `stitch_db_reference = "random stitch embroidery"`, the parameter object is retrieved from the stitching technique database. For semantic keyword retrieval, abstract fields such as theme, style, etc. are semantically encoded into semantic vectors, and then similarity calculation is performed to retrieve multiple data resources with higher similarity.

[0047] The system retrieves the corresponding and matching example style pictures input by the user through vector similarity retrieval. The vector similarity calculation method includes but is not limited to cosine similarity calculation (calculating the closeness of vectors in direction) and Euclidean distance calculation (the straight-line distance of two vectors in multidimensional space). After calculating the vector similarity, the system retrieves multiple database feature vectors with higher vector similarity as the intention pointed by the user, which is called in the subsequent.

[0048] Optionally, the system retrieves according to a higher level value when retrieving the vector similarity at the beginning. When the retrieved database feature vectors do not satisfy 3 or less, the system will automatically adjust the retrieval standard value to retrieve more database feature vectors for subsequent calling. If the retrieval standard value is adjusted to a higher value set, the user is reminded to add data in the database or input more example style pictures.

[0049] In addition, whether it is an exact keyword or a semantic keyword, for the case where a specific database is not retrieved, the system can realize separate retrieval, extraction and construction of element labels in different databases into a database.

[0050] S140, constraining the rendering engine with the feature data in the related database, inputting the image data in the design input data and the structured rendering instruction into the rendering engine to obtain a virtual rendering image; Finally, high-fidelity rendering is performed, the embroidery effect rendering engine model is started, the design sketch / image sample and the rendering instruction are input into the rendering engine model and associated with the related database to generate constraints, the stereoscopic sense, gloss and shadow effect of the product under a real physical lighting environment are simulated, and a high-fidelity virtual rendering image of the embroidery product is generated.

[0051] The initial design sketch of the user (as a basic composition), the structured rendering instruction (as a basis for fine rendering), and the feature data called from the database are constrained and input into the rendering engine as a basic composition reference, a basis for fine rendering, and a constraint reference for rendering effect, respectively. Specifically, it includes: S141, the image data is parsed into a two-dimensional vector path or a region mask; S142, the two-dimensional vector path or the region mask is mapped to the UV coordinates of the product carrier to determine the bounding box and the filling path of the embroidery area; S143, extract stitch parameters from the stitch database, extract scalable vector graphics vector curves or segmented texture masks from the pattern database, and extract physically rendered material parameters from the material database; S144, Construct lighting constraints, and render the two-dimensional vector path or region mask in the UV coordinates based on the constraints of the stitch parameters, the constraints of the scalable vector graphics vector curve or the segmented texture mask, the constraints of the physical rendering material parameters and the lighting constraints to obtain a virtual rendering map.

[0052] Specifically, the system will impose constraints such as geometric constraints, material constraints, and lighting constraints.

[0053] Geometric constraints are applied to the macro and micro shapes of embroidered products. In terms of macro layout and outline, constraints are established through the user's initial sketch in S1 and the structured rendering instructions in S2. Specifically, the user's sketch is parsed into a 2D vector path or region mask and mapped onto the UV coordinates of the product carrier. This defines the bounding box and fill path for the embroidery area, ensuring that the final position, size, and overall shape of the embroidery match the user's intentions. Figure 1 To achieve this, at the microscopic level of stitching and texture, the parameters of the stitches are read from the stitch database. In the rendering pipeline, the stitch parameters are used as input to the geometry shader, and SVG vector curves or segmented texture masks are extracted from the pattern database.

[0054] Material constraints are used to ensure that the filaments have realistic optical properties in the rendering. Material parameters retrieved from the database are used to perform specific constraints by querying related PBR material parameters through tags.

[0055] If the system does not find the relevant PBR material parameters, it will issue a reminder that the database content needs to be added.

[0056] Lighting constraints define the overall rendering environment, ensuring the embroidered product has a realistic sense of three-dimensionality and shadow effects. For the scene, ambient light constraints are applied by reading the environment map (HDRI) from the scene database and using it as the global illumination background. For the light source, referring to the "lighting" instruction mentioned earlier, preset light combination parameters (number of light sources, type (point light / parallel light / HDR ambient light), intensity, color temperature, etc.) are extracted. Unless otherwise specified, the rendering engine will randomly select the appropriate light source.

[0057] In addition, users can specify database constraints for the model, rather than simply selecting it through the system, to generate product renderings with specific styles or patterns.

[0058] S150 sends the virtual rendering image to the user and optimizes the virtual rendering image based on the user's feedback.

[0059] Specifically, it includes: S151, the virtual rendering image and optimization options are output to the user; S152, upon receiving modified design input data from the user, return to extracting the text and image features of the design input data to regenerate the virtual rendering image and return to outputting the virtual rendering image and optimization options to the user, until no more modified design input data is received from the user, wherein the modified design input data is generated based on the optimization options.

[0060] In the final feedback step, the generated virtual rendering is shown to the user, along with optimization options. The system receives the user's optimization instructions, returns to step S120, makes local modifications to the structured rendering instructions, and quickly re-renders, achieving iterative optimization of the design.

[0061] Figure 2 This is a flowchart illustrating a model-based collaborative virtual rendering method for embroidery products according to another embodiment of this application, as shown below. Figure 2 As shown, this application first receives multimodal design input from an external source, then parses the intent of the multimodal design input to generate rendering instructions; based on the multimodal design input, it searches an embroidery database to obtain relevant elements. The rendering instructions are then input into the rendering engine, and relevant element constraints are introduced to generate a rendered image. The rendered image is input to the user's end and optimized based on user feedback, finally resulting in a rendered image that matches the user's intent.

[0062] This invention discloses a virtual rendering method for embroidery products based on model collaboration. This application achieves the generation of a realistically rendered image integrating user sketches and embroidery techniques through a cross-model workflow, overcoming the shortcomings of existing technologies. It is applicable to the digital design applications of embroidery techniques in clothing, accessories, and other products. This application provides a low-cost method for meeting consumers' personalized needs and generating product renderings from user sketches. It reduces manual training costs, provides a lower-cost solution for traditional practitioners, improves the ability to meet consumers' personalized needs, enhances embroidery design efficiency, and shortens the embroidery design cycle. It lowers the technical threshold for professional design and enhances the universality of creation.

[0063] like Figure 3 As shown, this application also provides a virtual rendering system for embroidery products based on model collaboration, including: The acquisition module is used to acquire the user's design input data, wherein the design input data includes image data and text description of design intent, and the image data is an initial design sketch or a sample style image; The feature extraction module is configured to extract text features and image features of the design input data, and fuse the text features and the image features to obtain a fused feature vector; The instruction generation module is configured to generate a structured rendering instruction based on the fused feature vector, wherein the structured rendering instruction comprises an accurate description of embroidery process parameters required to be called in different regions of a design drawing. The retrieval module is configured to retrieve a relevant database from a pre-constructed embroidery pattern database set based on the structured rendering instruction, wherein the embroidery pattern database set comprises a plurality of embroidery pattern databases with classification labels. The rendering module is configured to constrain a rendering engine with feature data in the relevant database, input image data in the design input data and the structured rendering instruction into the rendering engine, and obtain a virtual rendering image.

[0064] The present application provides a virtual rendering system for embroidery products based on model collaboration, which realizes the generation of a real-sense rendering image integrating a product carrier and an embroidery process from a user sketch through a cross-model workflow, so as to solve the problems in the prior art and be suitable for the digital design application of the embroidery process in products such as clothes and accessories. The present application provides a method capable of meeting the personalized needs of consumers at low cost and realizing the generation of a product rendering image from a user sketch, which can not only reduce the cost of artificial training and provide a lower-price solution for traditional practitioners, but also improve the satisfaction of consumers' personalized needs, improve the efficiency of embroidery design, and shorten the embroidery design cycle. The present application can reduce the technical threshold of professional design and improve the universality of creation.

[0065] The computer readable storage medium in the embodiment can be understood by those skilled in the art that all or part of the steps of the above-mentioned method embodiments can be completed by a computer program related hardware. The foregoing computer program can be stored in a computer readable storage medium. The program executes the steps of the above-mentioned method embodiments when executed; and the foregoing storage medium includes ROM, RAM, magnetic disc or optical disc and various storage program codes.

[0066] The electronic terminal provided in the embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected with the processor and the transceiver and complete communication between each other. The memory is used for storing a computer program, the communication interface is used for communication, and the processor and the transceiver are used for running the computer program, so that the electronic terminal executes each step of the above method.

[0067] In the present embodiment, the memory can comprise a Random Access Memory (RAM) and can also include a non-volatile memory such as at least one disk memory.

[0068] The processor described above can be a general processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0069] In the above-described embodiments, although the present application has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art in light of the foregoing description. Embodiments of this application are intended to embrace all such alternatives, modifications and variations as can fall within the scope of the appended claims.

[0070] The above-described embodiments are merely illustrative for the principles and effects of the present application, and are not intended to limit the present application. Any modification or change made by any person skilled in the art without departing from the spirit and scope of the present application shall be covered by the claims of the present application.

Claims

1. A model-based collaborative embroidery product virtual rendering method, characterized in that, The method comprises the steps of: obtaining design input data of a user, wherein the design input data comprises image data and design intention text description, and the image data is an initial design sketch or a sample style picture; extracting text features and image features of the design input data, and fusing the text features and the image features to obtain a fused feature vector, wherein the text features are extracted based on a text semantic understanding model, and the image features are extracted based on an image feature extraction network; generating a structured rendering instruction based on the fused feature vector, wherein the structured rendering instruction comprises an accurate description of embroidery process parameters required to be called in different regions of a design drawing; searching a relevant database from a pre-constructed embroidery pattern database set based on the structured rendering instruction, wherein the embroidery pattern database set comprises a plurality of embroidery pattern database sets with classification labels; constraining a rendering engine with feature data in the relevant database, inputting the image data in the design input data and the structured rendering instruction into the rendering engine, and obtaining a virtual rendering drawing.

2. The model-based collaborative embroidery product virtual rendering method according to claim 1, characterized in that, The step of extracting text features and image features of the design input data comprises: judging a type of the design input data; when the type of the design input data is text, inputting the design input data into a pre-constructed text semantic understanding model to obtain a structured text feature table, wherein the text semantic understanding model is used for word segmentation and vectorization of the text to obtain a semantic vector, entity extraction of the semantic vector, automatic classification and label mapping of the extracted entities, and obtaining of the structured text feature table; when the type of the design input data is image, inputting the design input data into a pre-constructed image feature extraction network to obtain image features, wherein the image features comprise low-level features representing edge contours, color distribution and basic texture primitives, middle-level features representing local components, composition elements and repetitive patterns, and high-level / semantic features representing overall semantic understanding or style labels of image content.

3. The model-based collaborative embroidery product virtual rendering method according to claim 1, characterized in that, The step of fusing the text features and the image features to obtain a fused feature vector comprises: capturing, through an attention mechanism, an associated region having a semantic association with the text features from the image features, and performing feature weight reinforcement on the associated region to obtain the fused feature vector.

4. The model-based collaborative embroidery product virtual rendering method according to claim 1, characterized in that, The step of generating a structured rendering instruction based on the fused feature vector comprises: obtaining physical properties of a product carrier surface, and generating a structured rendering instruction in combination of the physical properties of the product carrier surface and the fused feature vector, wherein the structured rendering instruction comprises a detailed description of a product scene, a light and position description, a product carrier and material description, a description of a needle method and physical properties of a texture, a detailed description of features and content of the texture, and an overall description of an overall style of a product texture pattern.

5. The model-based collaborative embroidery product virtual rendering method according to claim 1, wherein, The step of searching a relevant database from a pre-constructed embroidery pattern database set based on the structured rendering instruction comprises: The structured rendering instruction is parsed to obtain a query unit representing a plurality of semantic tags, wherein the query unit is an accurate keyword or a semantic keyword; When the query unit is an accurate keyword, the query unit is matched with the tags of each embroidery pattern database in the pre-constructed embroidery pattern database set, and when the tag of any embroidery pattern database matches the query unit, the embroidery pattern database with which the tag matches the query unit is taken as a relevant database; when there is no tag of any embroidery pattern database matching the query unit, the query unit is matched with the elements of each embroidery pattern database in the pre-constructed embroidery pattern database set, and a relevant database is constructed based on all elements matching the query unit; When the query unit is a semantic keyword, the query unit and the tags of each embroidery pattern database in the pre-constructed embroidery pattern database set are vectorized and then similarity calculation is performed, and an embroidery pattern database with a similarity greater than a preset similarity threshold is taken as a relevant database; when there is no embroidery pattern database with a similarity greater than the preset similarity threshold, the similarity of the query unit and the tags of the elements of each embroidery pattern database in the embroidery pattern database set is calculated, and a relevant database is constructed based on all elements with a similarity greater than the preset similarity threshold.

6. The model-based collaborative embroidery product virtual rendering method according to claim 1, wherein, The relevant database includes a stitch database, a pattern database and a material database, wherein the feature data in the relevant database is used to constrain the rendering engine, the image data in the design input data and the structured rendering instruction are input into the rendering engine to obtain a virtual rendering image, including: The image data is parsed into a two-dimensional vector path or a region mask; The two-dimensional vector path or the region mask is mapped to the UV coordinates of a product carrier to determine the bounding box and the filling path of an embroidery area; Stitch parameters are extracted from the stitch database, scalable vector graphics vector curves or segmented texture masks are extracted from the pattern database, and physical rendering material parameters are extracted from the material database; Lighting constraints are constructed, and the two-dimensional vector path or the region mask is rendered in the UV coordinates based on the constraints of the stitch parameters, the constraints of the scalable vector graphics vector curves or the segmented texture masks, the constraints of the physical rendering material parameters and the lighting constraints to obtain a virtual rendering image.

7. The model-based collaborative embroidery product virtual rendering method according to claim 1, wherein, The construction method of the embroidery pattern database set includes: An image sample is obtained, wherein the image sample comes from historical data or is uploaded by a user; In the construction stage, semantic tags of the image sample are extracted, and the image sample is divided into a plurality of embroidery pattern databases based on the semantic tags to obtain an embroidery pattern database set; In the expansion stage, an image sample uploaded by a user is extracted, semantic tags of the image sample are extracted, similarity calculation is performed between the semantic tags of the image sample and the tags of each embroidery pattern database in the embroidery pattern database set, and the image sample is stored in the embroidery pattern database with the highest similarity.

8. The model-based collaborative embroidery product virtual rendering method according to claim 1, wherein, Further comprising: outputting the virtual rendering and the tuning options to the user; when receiving modified design input data from the user, returning to extracting the textual features and the image features of the design input data to regenerate a virtual rendering and returning to outputting the virtual rendering and the tuning options to the user until no longer receiving modified design input data from the user, wherein the modified design input data is generated based on the tuning options.

9. The model-based collaborative embroidery product virtual rendering method according to claim 1, wherein, The embroidery pattern database set further includes a user-defined style database.

10. A model-based collaborative embroidery product virtual rendering system applied to a model-based collaborative embroidery product virtual rendering method according to any one of claims 1-9, characterized in that, The method comprises: an acquisition module configured to acquire design input data of a user, wherein the design input data comprises image data and a textual description of design intention, and the image data is an initial design sketch or a sample style picture; a feature extraction module configured to extract textual features and image features of the design input data, and fuse the textual features and the image features to obtain a fused feature vector; an instruction generation module configured to generate a structured rendering instruction based on the fused feature vector, wherein the structured rendering instruction comprises an accurate description of embroidery process parameters required to be called in different regions of a design drawing; a retrieval module configured to retrieve a relevant database from a pre-constructed embroidery pattern database set based on the structured rendering instruction, wherein the embroidery pattern database set comprises a plurality of embroidery pattern databases with classification labels; a rendering module configured to constrain a rendering engine with feature data in the relevant database, input the image data in the design input data and the structured rendering instruction into the rendering engine, and obtain a virtual rendering.

Images