Product design generation method, system, equipment and medium

Abstract

The invention provides a product design generation method, system and device based on a knowledge graph and a text graph model and a medium, and the method comprises the steps: receiving an object input by a user, and converting the input object into a root node of a to-be-constructed knowledge graph; constructing a knowledge graph based on the root node; extracting attribute values of leaf nodes in the knowledge graph, and expanding the attribute values through a large language model to generate image generation cue words; and generating a design corresponding to the object through a text graph model based on the image generation cue word. Through deep combination of the knowledge graph and the text graph model, the problem that a traditional AI generation method only depends on a single cue word and lacks systematic creative expansion capability is solved, and the design generation process is more hierarchical, logical and controllable. Automatic creative divergence is supported, high-quality and controllable cue words are dynamically generated in combination with a knowledge graph, a user is supported to automatically generate new design schemes at different levels, and inspiration exploration is more efficient.

Description

Technical Field

[0001] The present disclosure relates to the technical field of product design generation, and in particular, to a product design generation method, system, device, and medium based on a knowledge graph and a text-to-image model. Background Art

[0002] With the development of technology, product design is gradually evolving from traditional manual drawing and modeling tools to intelligent, data-driven design methods. Computer Aided Design (CAD) software (such as Rhino3D, 3DMax, Blender, C4D, and Keyshot) has been widely used in the product development process. These tools rely on technologies such as geometric modeling and physical rendering to provide designers with high-precision visualization capabilities. However, they are mainly used to achieve the digital presentation of established ideas and still have certain limitations in inspiring innovative inspiration and expanding the design solution space.

[0003] In recent years, the application of artificial intelligence (AI)-driven generation technologies in the field of product design has gradually emerged. In particular, the rapid development of Generative Adversarial Networks (GANs) and Diffusion Models has provided designers with new creative exploration tools. Diffusion models can generate high-quality images from random noise by gradually denoising and restoring, and show great advantages in style consistency and detail performance. However, most of the existing text-to-image technologies rely on a single prompt to guide generation and lack the ability to expand thinking. In the actual product design process, designers usually need to conduct multi-dimensional and multi-level creative expansion around a core concept. For example, designers may want to explore design possibilities based on different materials, different styles, different functions, etc. However, traditional prompt-driven generation is difficult to decouple these dimensions and is even less able to form a systematic creative expansion path.

[0004] Existing image generation techniques have made some progress in modifying specific dimensions of images. For example, methods based on Generative Adversarial Networks (GANs) and control methods based on Diffusion Models. Methods based on GANs use GANs and their variants to introduce specific control vectors into the generator, enabling manipulation of certain specific dimensions of the generated images. For example, StyleGAN performs hierarchical control over high-level features (such as overall shape) and low-level features (such as texture details) of images through style vectors, thus achieving adjustment of a single dimension. However, the control of such methods is usually limited to global characteristics and lacks support for specific local or fine-grained dimensions (such as only adjusting the texture of a certain object). In addition, the control process requires a large amount of training data, and the interpretability of the latent space of the model is weak, making it difficult to meet the high-degree-of-freedom modification requirements of designers in specific dimensions. Control methods based on diffusion models have performed well in high-quality image generation in recent years. Models such as Stable Diffusion allow images to be generated through text prompts and combine technologies such as ControlNet to control the generated images in specific dimensions. For example, designers can partially achieve modification of the target dimension by inputting prompts such as "change the color of the object" or "adjust the texture". However, the control granularity of this method is limited, the generation effect depends on the accuracy of the prompts, and changes in other dimensions often occur when modifying a single dimension, making it difficult to achieve completely independent control. Additionally, with the development and related research of large language models (LLMs), there have been more and more works combining LLMs for image generation or modification. These methods can often achieve excellent results on a single image.

[0005] However, design is a process of divergent thinking. Designers usually hope to conduct hierarchical expansion based on a certain concept (such as starting from "chair" and exploring design possibilities of different styles, different materials, and different functions). Current AI generation methods only support "random exploration" or "local changes" based on similarity, lacking a systematic concept expansion path, resulting in poor divergence and logic of design schemes. Or for an AI-generated image, designers cannot trace which concepts or logics the model is based on to obtain this design, making it difficult to adjust and optimize. Existing methods are difficult to support further exploration of new design directions based on existing solutions, that is, they cannot extract information from the generated results and promote the next round of creative generation.

[0006] Therefore, existing AI generation techniques are difficult to meet the requirements of multi-dimension, systematization, controllability, and interpretability in the product design process. There is an urgent need for a new method to combine knowledge graphs and diffusion models to build an iterative, inspiration-driven design generation system to enhance designers' creative exploration capabilities. Summary of the Invention

[0007] The present disclosure provides a product design generation method, system, device, and storage medium based on a knowledge graph and a text-to-image model.

[0008] According to one aspect of the present disclosure, there is provided a product design generation method based on a knowledge graph and a text-to-image model, including:

[0009] Receiving an object input by a user and converting the input object into a root node of a knowledge graph to be constructed; wherein the object includes text and / or an image;

[0010] Constructing the knowledge graph based on the root node; wherein the knowledge graph is stored in the form of a triple of node-dimension-attribute, the node represents the object, the dimension represents a specific attribute classification of the object, the attribute is a specific description item under the dimension, and the knowledge graph forms a tree structure starting from the root node;

[0011] Extracting the attribute values of the leaf nodes in the knowledge graph and generating an image generation prompt by expanding the attribute values through a large language model;

[0012] Generating a design corresponding to the object based on the image generation prompt through a text-to-image model.

[0013] In the method according to some embodiments of the present disclosure, the method further includes:

[0014] Visualizing the tree structure of the knowledge graph and the generated design to generate a visual interface with the object as the root node, including multi-level dimensions and attributes.

[0015] In the method according to some embodiments of the present disclosure, any leaf node in the knowledge graph is used as a new root node, and an image generation prompt is generated based on the new root node to expand the hierarchy of the knowledge graph.

[0016] In the method according to some embodiments of the present disclosure, when the object is an image, the receiving the object input by the user and converting the input object into a root node of the knowledge graph to be constructed includes:

[0017] Analyzing the image through the text-to-image model or the large language model, extracting the main object in the image and generating a corresponding text description, and converting the text description into a root node of the knowledge graph to be constructed.

[0018] In the method according to some embodiments of the present disclosure, the constructing the knowledge graph based on the root node includes:

[0019] Construct the knowledge graph through the large language model based on the root node and the preset question template;

[0020] Store the knowledge graph in a lightweight text representation format with a hierarchical structure.

[0021] In the method according to some embodiments of the present disclosure, the preset question template adopts a parameterized input method, and defines the knowledge graph construction rule through the [Object, M, N] triple;

[0022] Where Object is the root node name, M is the number of dimension classifications, and N is the minimum number of attribute items under each dimension classification.

[0023] In the method according to some embodiments of the present disclosure, the storing the knowledge graph in a lightweight text representation format with a hierarchical structure includes:

[0024] Store the knowledge graph in JSON format, where the JSON format includes the root node, the dimensions, and the attribute values under each dimension to form a data structure with a hierarchical relationship.

[0025] According to another aspect of the present disclosure, there is provided a product design generation system based on a knowledge graph and a text-to-image model, the system includes:

[0026] A receiving module that receives an object input by a user and converts the input object into the root node of the knowledge graph to be constructed; wherein, the object includes text and / or an image;

[0027] A construction module that constructs the knowledge graph based on the root node; wherein, the knowledge graph is stored in the form of a node-dimension-attribute triple, the node represents the object, the dimension represents a specific attribute classification of the object, and the attribute is a specific description item under the dimension, and the knowledge graph forms a tree structure starting from the root node;

[0028] An expansion module that extracts the attribute values of the leaf nodes in the knowledge graph and generates image generation prompt words by expanding the attribute values through a large language model;

[0029] A generation module that generates a design corresponding to the object through a text-to-image model based on the image generation prompt words.

[0030] According to another aspect of the present disclosure, there is provided a computer device including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the above-mentioned method is implemented.

[0031] According to another aspect of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0032] The present disclosure proposes a product design generation method, system, device and medium based on a knowledge graph and a text-to-image model. The method includes: receiving an object input by a user and converting the input object into the root node of a knowledge graph to be constructed; wherein the object includes text and / or an image; constructing the knowledge graph based on the root node; wherein the knowledge graph is stored in the form of a triple of node-dimension-attribute, the node represents the object, the dimension represents a specific attribute classification of the object, and the attribute is a specific description item under the dimension, and the knowledge graph forms a tree structure starting from the root node; extracting the attribute values of the leaf nodes in the knowledge graph, and generating an image generation prompt by expanding the attribute values through a large language model; generating a design corresponding to the object through the text-to-image model based on the image generation prompt. By deeply integrating the knowledge graph and the text-to-image model, the problem that traditional AI generation methods only rely on a single prompt and lack systematic creative expansion ability is solved, making the design generation process more hierarchical, logical and controllable. And it supports automatic creative divergence. By combining the large language model and the knowledge graph, high-quality and controllable prompts are dynamically generated, supporting designers to automatically generate new design schemes at different levels and making inspiration exploration more efficient. Thus, the deficiencies of existing AI generation technologies in aspects such as creative divergence, fine-grained control and iterative optimization are effectively solved.

[0033] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0035] Figure 1 is the first process schematic diagram according to an exemplary embodiment of the present disclosure;

[0036] Figure 2 is the second process schematic diagram according to an exemplary embodiment of the present disclosure;

[0037] Figure 3 is the third process schematic diagram according to an exemplary embodiment of the present disclosure;

[0038] Figure 4 is the visualization effect schematic diagram according to an exemplary embodiment of the present disclosure;

[0039] Figure 5It is a block diagram of a product design generation system based on a knowledge graph and a text-to-image model according to an exemplary embodiment of the present disclosure;

[0040] Figure 6 It is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. Detailed implementation manners

[0041] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0042] To more clearly illustrate the technical solutions of the embodiments of this specification, the accompanying drawings required for the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are merely some examples or embodiments of this specification. For those of ordinary skill in the art, without creative efforts, this specification can also be applied to other similar scenarios based on these drawings. Unless obvious from the language context or otherwise stated, the same reference numerals in the figures represent the same structure or operation.

[0043] Referring to "embodiments" in this application means that specific features, structures, or characteristics described in connection with the embodiments can be included in at least one embodiment of this application. The phrase appears in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described in this application can be combined with other embodiments.

[0044] In recent years, the application of artificial intelligence-driven generation technologies in the field of product design has gradually emerged. Especially with the rapid development of generative adversarial networks and diffusion models, new creative exploration tools have been provided for designers. The diffusion model can generate high-quality images from random noise by gradually denoising and restoring, and shows great advantages in terms of style consistency and detail performance. However, most of the existing text-to-image technologies rely on a single prompt to guide generation and lack the ability of thinking expansion. In the actual product design process, designers usually need to conduct multi-dimensional and multi-level creative expansion around a core concept. For example, designers may hope to explore design possibilities based on different materials, different styles, different functions, etc. However, traditional prompt-driven generation is difficult to decouple these dimensions, and even more unable to form a systematic creative expansion path. For the above reasons, such as Figure 1 、 Figure 2 andFigure 3 As shown in the figure, this embodiment provides a product design generation method based on a knowledge graph and a text-to-image model, including:

[0045] S101. Receive the object input by the user and convert the input object into the root node of the knowledge graph to be constructed; wherein, the object includes text and / or image.

[0046] In the method according to some embodiments of the present disclosure, the object input by the user is accepted as the root node for performing the next step of constructing the knowledge graph. In this step, a specific product object name is usually obtained, denoted as Object.

[0047] In the method according to some embodiments of the present disclosure, when the object is an image, step S101 includes:

[0048] Analyze the image through a text-to-image model or a large language model, extract the main object in the image and generate a corresponding text description, and convert the text description into the root node of the knowledge graph to be constructed.

[0049] If the user's input is in text form, it can be directly recorded. If the user provides a picture input, a text-from-image model (e.g., Blip text-from-image model) or a multimodal large language model (e.g., ChatGPT4o) can be used to automatically generate the main object in the image and record it, thereby completing the construction of the initial node (i.e., the root node) of the knowledge graph.

[0050] In a specific example, obtain the input from the designer. The designer provides the product to be designed this time (e.g., "chair")

[0051] Text input: The designer provides the product name Name, and the initial input object = Name.

[0052] Image input: The designer provides a product design drawing Image. First, use the Blip text-from-image model to determine the initial object, i.e., object = Blip I2T (Image).

[0053] S102. Construct a knowledge graph based on the root node; wherein, the knowledge graph is stored in the form of a triple of node-dimension-attribute. The node represents an object, the dimension represents a specific attribute classification of the object, and the attribute is a specific description item under the dimension. The knowledge graph forms a tree structure starting from the root node.

[0054] In one example, the knowledge graph, as a structured form of knowledge representation, is intended to describe things (entities) and their relationships so that computers can understand and reason. It is usually presented as a graph structure. The structured information of the knowledge graph is stored through triples (Subject-Predicate-Object), which include nodes, edges and attributes, respectively. The nodes represent entities (such as people, places, companies, products, etc.), the edges represent the relationships between entities (such as "is a friend", "located", "belongs to"), and the attributes describe the characteristics of the entities (such as name, date of birth, location, etc.). In this embodiment, the idea of ​​knowledge graph is adopted to construct a node-dimension-attribute triple. Specifically, the node represents the product entity of the object input in step S101. It is the object to be explored in combination with the knowledge graph. The dimension represents a specific angle for this entity, and this dimension will be expanded later (such as the material, color, style, etc. of the product), and the attribute is a specific representation under a specific dimension. This step will use a large language model (such as ChatGPT4o) to build a knowledge graph, so that this step can give full play to the thinking expansion ability of the large language model to expand the designer's thinking and explore inspiration more efficiently.

[0055] S103, extracting the attribute values ​​of the leaf nodes in the knowledge graph, and expanding the attribute values ​​through the large language model to generate image generation prompt words.

[0056] In one embodiment, the knowledge graph constructed by step S102 is a tree structure, and the specific attributes of each dimension are recorded on the leaf nodes of the tree. In order to generate images for better reference by designers, this step performs a text-to-image generation operation. In a text graph, the quality of the prompt words often has a great impact on the generation results. In actual operation, if designers are required to manually design prompt words, the effect may be poor. Therefore, a large language model is used to expand the content on the leaf nodes to form image generation prompt words that are more in line with the requirements of text generation images.

[0057] S104, generating prompt words based on the image and generating a design corresponding to the object through a text-based graph model.

[0058] This embodiment proposes a product design generation method based on a knowledge graph and a text-to-image model. By constructing a multi-level knowledge graph, using a large language model to expand prompts, and combining the multi-dimensional control ability of a diffusion model, it realizes the intelligent generation and iterative optimization of product design. First, this embodiment solves the problems of designers in creative divergence and multi-dimensional control. By constructing a hierarchical knowledge graph of features such as style, material, and function, it provides structured guidance for the generation process, thereby enhancing the logic and controllability of generation. Second, by dynamically expanding prompts through a large language model, this embodiment can automatically generate high-quality and accurately described prompts based on the nodes of the knowledge graph, enabling users to obtain design solutions that meet their needs without manually adjusting complex text inputs. In addition, this embodiment combines a text-to-image model or a diffusion model for intelligent image generation, which not only meets the actual needs of product design but also can be iteratively optimized without deviating from the initial creative direction, ultimately providing a more creative, systematic, and efficient product design generation method.

[0059] In the method according to some embodiments of the present disclosure, as Figure 4 shown, the product design generation method based on a knowledge graph and a text-to-image model of the present disclosure further includes:

[0060] S105. Visualize the tree structure of the knowledge graph and the generated design to generate a visual interface with the object as the root node, including multi-level dimensions and attributes.

[0061] The knowledge graph constructed in the foregoing steps will be a tree structure with Object as the root node, including at least three layers. Among them, the 2k-th layer, k ∈ N * is the thinking dimension when expanding the knowledge graph, and the (2k + 1)-th layer, k ∈ N * is some specific attributes in the 2k-th dimension. Through the method of this embodiment, users can not only quickly generate multiple design solutions in the initial stage of inspiration but also carry out systematic creative expansion in combination with the knowledge graph, thereby enhancing the intelligent level and creative exploration efficiency of product design.

[0062] In the method according to some embodiments of the present disclosure, step S102 includes:

[0063] S1021. Construct a knowledge graph through a large language model based on the root node and a preset question template.

[0064] S1022. Store the knowledge graph in a lightweight text representation format with a hierarchical structure.

[0065] In a specific embodiment, storing the knowledge graph in a lightweight text representation format with a hierarchical structure includes:

[0066] The knowledge graph is stored in JSON format, where the JSON format includes a root node, dimensions, and attribute values under each dimension to form a hierarchical data structure.

[0067] In one example, JSON, as a lightweight text representation format with a hierarchical structure, is completely independent of any programming language and is an ideal data exchange language. For ease of storage and further development, this embodiment stores the knowledge graph in JSON format, and the specific format is as follows:

[0068]

[0069] "Object" in the format is the root node, Dimensions_X represents X dimensions of divergent thinking, and several attribute_X values under each Dimensions_X are several specific attributes under that dimension.

[0070] In a specific embodiment, the preset question template adopts a parameterized input method, and defines the knowledge graph construction rule through the [Object, M, N] triple. Where Object is the root node name, M is the number of dimension classifications, and N is the minimum number of attribute items under each dimension classification.

[0071] In one example, this step constructs the knowledge graph with the help of the large language model ChatGPT. To ensure that the structure obtained in the above steps can be obtained, a preset question template is designed. By submitting this template to the large language model, a JSON text that meets the requirements in both format and content can be obtained.

[0072] The design of this question template is as follows:

[0073] Here is an idea to combine the knowledge graph and the text graphmodel. For example, given an initial prompt word and an initial image, we need to generate a knowledge graph based on the initial object. Each node of the graph generates a new image after bringing new and different information. All the images generated finally form a graph structure, which is similar to divergent thinking for images.

[0074] Your current task is to generate a corresponding knowledge graph for a given object. For the main object, you need to diverge from M perspectives, and each perspective provides N specific different words or phrases under this dimension. For example, I give the requirement: ["Mobile Phone", 3, 3], and I expect to get a knowledge graph represented by JSON like this:

[0075] {

[0076] "Mobile Phone": {

[0077] "Material": ["Glass", "Aluminum", "Plastic"],

[0078] "Style": ["Modern", "Vintage", "Futuristic",],

[0079] "Usage": ["Gaming Phone", "Business Phone", "PhotographyPhone"]

[0080] }

[0081] }

[0082] Your task is to return a JSON text with the structure above. Please note that the JSON text above is only a format requirement. You can express your own content. All text should be in English. You only need return the JSON text. Here are my requirements:

[0083] [Object, M, N]

[0084] In the preset template, the idea of the method of this embodiment is first introduced to the large language model so that it can understand the purpose and generate a response that better meets the expected effect. To ensure that a response in the desired format can be obtained, a specific example is provided in the template, requiring ChatGPT to follow the relevant format and clearly specifying the relevant requirements, such as writing in English and only replying with JSON without other descriptions. Express the final requirement with a formula: [Object, M, N]. Where Object is the initial object obtained in the above steps, M represents the number of dimensions, and N represents how many specific attributes are provided for each dimension. By integrating the requirements into three parameters and condensing them into this formula, future development can be facilitated. Of course, the above question template can also be in Chinese, which will not be elaborated here.

[0085] Modify the three parameters in the question template in the above steps as needed and send them to the large language model to obtain the generated JSON document, thus completing the construction of a three-layer knowledge graph. This step can also be implemented in the form of an API interface.

[0086] In the method according to some embodiments of the present disclosure, after steps S102 and S103, the method of the present disclosure further includes:

[0087] S1031. Use any leaf node in the knowledge graph as a new root node, and generate an image generation prompt based on the new root node to expand the hierarchy of the knowledge graph.

[0088] Specifically, in step S102, the knowledge graph was constructed only once for the initial input object, resulting in a three-layer tree structure with relevant entries or images generated based on them on the nodes. It should be noted that for each leaf node, it can itself be used as another initial input to generate another knowledge graph with it as the core. Therefore, the knowledge graph obtained in step S102 can be continuously expanded, and each leaf node can be extended again. Similarly, the newly added leaf nodes after expansion can also be expanded again as needed, and the iteration continues. This method of hierarchical iteration using the structure of the knowledge graph can automatically recommend new design directions through the large language model. Designers can select new expansion paths based on the existing generated images, making the creative exploration more logical and traceable.

[0089] The attribute nodes in the knowledge graph are just simple words or phrases. To apply them to the text-to-image model, it is best to further expand the prompt words to achieve better generation effects. The text prompt words were expanded through the large language model ChatGPT. The second prompt word expansion question template was designed as follows:

[0090] Your task now is to write a prompt word for text-generated images based on the keywords I give. The keywords are given in JSON, which contains the keywords and the model used by the text-generated image. You need to give appropriate prompt words based on the text-generated image model. This is an example:

[0091] {

[0092] "KeyWords": ["Mobile Phone", "Futuristic"],

[0093] "Model": "StableDiffusionXL"

[0094] }

[0095] For this JSON, your answer might be “A sleek futuristic mobile phone with a transparent holographic display, glowing neon edges, and a minimalist design, set in a high-tech environment with ambient cyberpunk lighting, highly detailed textures”.

[0096] All text should be in English. You only need to return the prompt. Here is your task:

[0097] {

[0098] "KeyWords": ["Object", "attribute_i"],

[0099] "Model": "model_name"

[0100] }

[0101] The request for expanding the prompt is also written into a JSON file. The value of "KeyWords" is the keyword to be expanded, which comes from a leaf node and a root node in the knowledge graph, that is, a specific attribute and the initial object name. The value of "Model" is the name of the large model planned to be used in the subsequent text-to-image generation step. Since different large models have their own prompt writing specifications, this step can further improve the writing level of the prompt.

[0102] Obtain the result of expanding the prompt

[0103] Change the main body name and related attributes in the above template, select the model name, and then send the task to ChatGPT to obtain a complete set of prompts for text-to-image generation.

[0104] Obtain the text-to-image result

[0105] Provide the expanded prompt above to the text-to-image large model StableDiffusion to generate the product design drawing corresponding to this node.

[0106] Knowledge graph expansion

[0107] For each attribute, a new knowledge graph can be generated as a new object, thereby expanding the depth and breadth of the original indication graph. This step only requires changing the "Object" value in the prompt word template in the above step to a specific attribute on a certain leaf node.

[0108] Visual page layout

[0109] The knowledge graph finally forms a multi-layer tree structure, with relevant attributes or generated images on the nodes. Designers can see the thinking links of any node, which helps to expand the designers' thinking.

[0110] Combined with the above implementation method, a specific example is introduced below:

[0111] Step S1: Obtain the designer's input. The designer provides the product to be designed this time (for example, "chair")

[0112] Text input: The designer provides the product name Name, and the initial input object = Name.

[0113] Image input: The designer provides a product design drawing Image, and first uses the Blip image-to-text model to determine the initial object.

[0114] object = Blip I2T (Image)

[0115] Step S2: Modify the knowledge graph question template to:

[0116] Here is an idea to combine the knowledge graph and the text graph model. For example, given an initial prompt word and an initial image, we need to generate a knowledge graph based on the initial object. Each node of the graph generates a new image after bringing new and different information. All the images generated finally form a graph structure, which is similar to divergent thinking for images.

[0117] Your current task is to generate a corresponding knowledge graph fora given object.For the main object,you need to diverge from M perspectives,and each perspective provides N specific different words or phrases underthis dimension.For example,I give the requirement:["Mobile Phone",3,3],and Iexpect to get a knowledge graph representedby JSON like this:

[0118] {

[0119] "Mobile Phone":{

[0120] "Material": ["Glass", "Aluminum", "Plastic"],

[0121] "Style": ["Modern", "Vintage", "Futuristic",],

[0122] "Usage": ["Gaming Phone", "Business Phone", "PhotographyPhone"]

[0123] }

[0124] }

[0125] Your task is to return a JSON text with the structure above.Pleasenote that the JSON text above is only a format requirement.You can expressyour own content.All text should be in English.You only need return the JSONtext.Here are my requirements:

[0126] [“Chair”,3,3]

[0127] The key to modifying the question template lies only in the final question formula. Here, the modification [“Chair”,3,3] means thinking from 3 different dimensions based on the “Chair” object, and giving 3 different attributes for each dimension. Submitting the task to ChatGPT4o, the JSON-formatted reply is as follows:

[0128]

[0129]

[0130] For each of the obtained attributes, you can choose to perform text-to-image generation operations, thus entering step S3, or you can perform further expansion, thus entering step S4. If you enter step S3, you need to expand the prompt word for text-to-image generation. Modify the question template in step S31 as follows:

[0131] Your task now is to write a prompt word for text-generated images based on the keywords I give. The keywords are given in JSON, which contains the keywords and the model used by the text-generated image. You need to give appropriate prompt words based on the text-generated image model. This is an example:

[0132] {

[0133] "Key Words":["Mobile Phone","Futuristic"],

[0134] "Model":"StableDiffusionXL"

[0135] }

[0136] For this JSON, your answer might be “A sleek futuristic mobile phone with a transparent holographic display, glowing neon edges, and a minimalist design, set in a high-tech environment with ambient cyberpunk lighting, highly detailed textures”.

[0137] All text should be in English. You only need to return the prompt. Here is your task:

[0138] {

[0139] "Key Words": ["Chair", "Vintage"],

[0140] "Model": "Flux"

[0141] }

[0142] Send the above task to ChatGPT4o to get the following expanded response:

[0143] A vintage wooden chair with intricate carvings, aged leather upholstery, and an antique finish, set in a cozy, dimly lit Victorian-style room – highly detailed, cinematic lighting

[0144] Input this prompt into StableDiffusion to obtain the product design drawing with this attribute. If you enter step S4, that is, to recreate the knowledge graph for a certain attribute (for example, to expand the knowledge graph of the “Vintage” retro style), you can modify the last formula in the knowledge graph question template to ["Vintage", 3, 2], indicating to diverge the thinking for this attribute, requiring two different attribute descriptions to be given from three dimensions. Send the task to ChatGPT4o to get the following JSON text:

[0145]

[0146] Thus, the secondary iteration of the overall knowledge graph is completed. For each node, whether it is generated during the first construction of the knowledge graph or obtained through subsequent expansion, it can be arbitrarily selected for operation in step S3 and step S4, and finally a divergent tree-like text / image set is constructed. The effect is as Figure 4 shown.

[0147] In this embodiment, by deeply integrating the knowledge graph with the text-to-image model, the problem that traditional AI generation methods only rely on a single prompt word and lack systematic creative expansion ability is solved, making the design generation process more hierarchical, logical, and controllable. This method supports automated creative divergence, uses the LLM combined with the knowledge graph to dynamically generate high-quality and controllable prompt words, supports designers to automatically generate new design schemes at different levels, makes the inspiration exploration more efficient, and provides a brand-new intelligent tool for product design.

[0148] The product design generation method based on the knowledge graph and the text-to-image model provided in this embodiment can be executed in intelligent terminals, computer terminals, network devices, chips, chip modules, or similar computing devices.

[0149] Corresponding to the product design generation method based on the knowledge graph and the text-to-image model introduced above, this embodiment also provides a product design generation system based on the knowledge graph and the text-to-image model. The following will be introduced separately. Specifically, as Figure 5 shown, the product design generation system based on the knowledge graph and the text-to-image model of this embodiment includes:

[0150] Receiving module 1, which receives the object input by the user and converts the input object into the root node of the knowledge graph to be constructed; wherein, the object includes text and / or image;

[0151] Construction module 2, which constructs a knowledge graph based on the root node; wherein, the knowledge graph is stored in the form of a triple of node-dimension-attribute, the node represents the object, the dimension represents the specific attribute classification of the object, the attribute is the specific description item under the dimension, and the knowledge graph forms a tree-like structure starting from the root node;

[0152] Expansion module 3, which extracts the attribute values of the leaf nodes in the knowledge graph and generates image generation prompt words by expanding the attribute values through a large language model;

[0153] Generation module 4, which generates a design corresponding to the object through the text-to-image model based on the image generation prompt words.

[0154] It should be noted that the embodiment of the product design generation system based on the knowledge graph and the text-to-image model provided in the embodiments of the present application and the embodiment of the product design generation method based on the knowledge graph and the text-to-image model are based on the same inventive concept.

[0155] It should be noted that the product design generation system based on the knowledge graph and the text-to-image model in this embodiment can be, for example: a single chip, a chip module or an electronic device, or it can also be a chip or a chip module integrated in an electronic device. Regarding the various devices and products described in the above embodiments and the various modules / units they contain, they can be software modules / units, hardware modules / units, or they can also be partly software modules / units and partly hardware modules / units. For example, for the various devices and products applied to or integrated in a chip, the various modules / units they contain can all be implemented in the form of hardware such as circuits, or at least some of the modules / units can be implemented in the form of software programs that run on the processor integrated inside the chip, and the remaining (if any) part of the modules / units can be implemented in the form of hardware such as circuits; for the various devices and products applied to or integrated in a chip module, the various modules / units they contain can all be implemented in the form of hardware such as circuits, and different modules / units can be located in the same component (such as a chip, a circuit module, etc.) or different components of the chip module, or at least some of the modules / units can be implemented in the form of software programs that run on the processor integrated inside the chip module, and the remaining (if any) part of the modules / units can be implemented in the form of hardware such as circuits; for the various devices and products applied to or integrated in a terminal, the various modules / units they contain can all be implemented in the form of hardware such as circuits, and different modules / units can be located in the same component (such as a chip, a circuit module, etc.) or different components inside the terminal, or at least some of the modules / units can be implemented in the form of software programs that run on the processor integrated inside the terminal, and the remaining (if any) part of the modules / units can be implemented in the form of hardware such as circuits.

[0156] It is a schematic structural diagram of an electronic device provided in an embodiment of the present invention. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the product design generation method based on the knowledge graph and the text-to-image model. Figure 6 The displayed electronic device 30 is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.

[0157] Such as Figure 6As shown, the electronic device 30 may be presented in the form of a general computing device. For example, it may be a server device. The components of the electronic device 30 may include, but are not limited to: at least one of the above-mentioned processors 31, at least one of the above-mentioned memories 32, and a bus 33 that connects different system components (including the memory 32 and the processor 31).

[0158] The bus 33 includes a data bus, an address bus, and a control bus.

[0159] The memory 32 may include volatile memory, such as random access memory (RAM) 321 and / or cache memory 322, and may further include read-only memory (ROM) 323.

[0160] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324. Such program modules 324 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0161] The processor 31 executes various functional applications and data processing by running computer programs stored in the memory 32, such as the product design generation method based on the knowledge graph and text-to-image model of the present disclosure.

[0162] The electronic device 30 may also communicate with one or more external devices 34 (such as a keyboard, a pointing device, etc.). Such communication may be carried out through an input / output (I / O) interface 35. Moreover, the device 30 for model generation may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 36. As Figure 6 shown, the network adapter 36 communicates with other modules of the device 30 for model generation through the bus 33. It should be understood that although not shown in the figure, other hardware and / or software modules may be used in combination with the device 30 for model generation, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0163] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, such a division is merely exemplary and not mandatory. In fact, according to the embodiments of the present invention, the features and functions of two or more of the above-mentioned units / modules may be embodied in one unit / module. Conversely, the features and functions of one unit / module described above may be further divided and embodied by multiple units / modules.

[0164] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the steps in the product design generation method based on a knowledge graph and a text-to-image model.

[0165] Among them, more specifically, the readable storage medium can include but is not limited to: portable disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories, optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0166] In a possible implementation manner, the present invention can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps in the product design generation method based on a knowledge graph and a text-to-image model.

[0167] Among them, the program code for executing the present invention can be written in any combination of one or more programming languages. The program code can be executed entirely on the user device, partially on the user device, executed as an independent software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0168] It should be understood that the various forms of processes shown above can be used, steps can be reordered, added, or deleted. For example, the steps recorded in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in this disclosure can be achieved. This embodiment does not limit this here.

[0169] It should be understood that the "system", "device", "unit", and / or "module" used in this embodiment is a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0170] In this embodiment, a flowchart is used to illustrate the operations performed by the system according to the embodiments of this embodiment. It should be understood that the previous or subsequent operations do not necessarily need to be executed precisely in sequence. On the contrary, they can be executed in reverse order or processed simultaneously for each step. At the same time, other operations can also be added to these processes, or one or several steps of operations can be removed from these processes.

[0171] As shown in this embodiment, unless the context clearly indicates an exceptional situation, words such as "a", "an", "one", and / or "the" are not specifically singular and can also include the plural. Generally speaking, the terms "including" and "comprising" only indicate the inclusion of the clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0172] The definitions included in this embodiment. As used herein, terms such as "have", "may have", "include", or "may include" indicate the existence of corresponding functions, operations, elements, etc. of this embodiment, and do not limit the existence of one or more other functions, operations, elements, etc. In addition, it should be understood that as used herein, the term "include" or "have" indicates the existence of features, numbers, steps, operations, elements, components, or combinations thereof described in the specification, and does not exclude the existence or addition of one or more other features, numbers, steps, operations, elements, components, or combinations thereof.

[0173] The definition of "and / or" in this embodiment. As used herein, terms such as "A or B", "at least one of A and / or B", or "one or more of A and / or B" include any and all combinations of the words listed therewith. For example, "A or B", "at least one of A and / or B", or "one or more of A and / or B" means (1) including at least one A, (2) including at least one B, or (3) including both at least one A and at least one B.

[0174] The definitions of first and second in this embodiment. The descriptions such as first and second that appear in this embodiment are only for illustration and to distinguish the described objects, without an order, and do not represent a special limitation on the number of devices in this embodiment, and cannot constitute any limitation to this embodiment. For example, the first element may be referred to as the second element without departing from the scope of the present disclosure. Similarly, the second element may be referred to as the first element.

[0175] Although the specific embodiments of the present invention have been described above, those skilled in the art should understand that this is only an example, and the protection scope of the present invention is defined by the appended claims. Without departing from the principles and essence of the present invention, those skilled in the art can make various changes or modifications to these embodiments, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A product design generation method based on knowledge graph and cultural graph model, characterized in that: The method comprises: Receive an object input by a user, and convert the input object into a root node of a knowledge graph to be constructed; wherein the object includes text and / or an image; The knowledge graph is constructed based on the root node; wherein the knowledge graph is stored in the form of a node-dimension-attribute triple, the node represents the object, the dimension represents a specific attribute classification of the object, the attribute is a specific description item under the dimension, and the knowledge graph forms a tree structure starting from the root node; Extracting attribute values ​​of leaf nodes in the knowledge graph, and expanding the attribute values ​​through a large language model to generate image generation prompt words; Generate prompt words based on the image and generate a design corresponding to the object through a text graph model.

2. The product design generation method based on the knowledge graph and the cultural graph model according to claim 1, characterized in that: The method further comprises: The tree structure of the knowledge graph and the generated design are visualized to generate a visualization interface with the object as the root node and including multiple levels of dimensions and attributes.

3. The product design generation method based on the knowledge graph and the cultural graph model according to claim 1 or 2, characterized in that: The method further comprises: Any leaf node in the knowledge graph is used as a new root node, and an image generation prompt word is generated based on the new root node to expand the hierarchy of the knowledge graph.

4. The product design generation method based on the knowledge graph and the cultural graph model according to claim 1, characterized in that: When the object is an image, the step of receiving the object input by the user and converting the input object into a root node of the knowledge graph to be constructed includes: The image is analyzed by the text graph model or the large language model, the main object in the image is extracted and a corresponding text description is generated, and the text description is converted into a root node of the knowledge graph to be constructed.

5. The product design generation method based on the knowledge graph and the cultural graph model according to claim 1, characterized in that: The step of constructing the knowledge graph based on the root node includes: Based on the root node and the preset question template, construct the knowledge graph through the large language model; The knowledge graph is stored in a hierarchical lightweight text representation format.

6. The product design generation method based on the knowledge graph and the cultural graph model according to claim 5 is characterized in that: The preset question template adopts parameterized input mode and defines the knowledge graph construction rules through [Object, M, N] triples; Where Object is the root node name, M is the number of dimension categories, and N is the minimum number of attribute items under each dimension category.

7. The product design generation method based on the knowledge graph and the cultural graph model according to claim 5, characterized in that: The storing of the knowledge graph in a hierarchical lightweight text representation format includes: The knowledge graph is stored in JSON format, wherein the JSON format includes the root node, the dimensions, and the attribute values ​​under each dimension to form a data structure with a hierarchical relationship.

8. A product design generation system based on knowledge graph and cultural graph model, characterized in that: The system comprises: A receiving module, an object input by a user, and converting the input object into a root node of a knowledge graph to be constructed; wherein the object includes text and / or an image; A construction module, constructing the knowledge graph based on the root node; wherein the knowledge graph is stored in the form of a node-dimension-attribute triple, the node represents the object, the dimension represents a specific attribute classification of the object, the attribute is a specific description item under the dimension, and the knowledge graph forms a tree structure with the root node as the starting point; An expansion module extracts the attribute values ​​of leaf nodes in the knowledge graph and expands the attribute values ​​through a large language model to generate image generation prompt words; A generation module generates a prompt word based on the image and generates a design corresponding to the object through a text graph model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images