Method and device for performing generative ai-based co-design for industrial designers and engineers

Abstract

According to an embodiment of the present disclosure, a method performed by a computing device may be provided for performing a generative AI-based co-design for industrial designers and engineers, the method comprising the steps of: inputting a reference design for a product to a topological optimization model to acquire an output topological optimization design; and inputting the topological optimization design to a rendering model to acquire an output rendered image.

Description

Generative AI-based co-design execution method and device for industrial designers and engineers

[0001] The present disclosure relates to a generative AI-based co-design method and apparatus for industrial designers and engineers, and more specifically, to a generative AI-based co-design method and apparatus for industrial designers and engineers that generates a concept design of a product, reconstructs a 3D shape from an image, and explores the design.

[0002] Automotive wheel design is a crucial factor influencing a vehicle's appearance and performance, making a balance between aesthetic and engineering requirements essential. The conventional wheel design process, which involves the sequential progression of creative styling by exterior designers and structural analysis by engineers, has been time-consuming and costly due to repetitive modifications, making it difficult to find the optimal balance.

[0003] Recently, research on AI-based design assistance has become active due to the advancement of foundation models. However, while text-based design generative models such as DALL-E and Midjourney can be helpful in providing visual inspiration, they have limitations in applying to the entire design process because they fail to consider engineering constraints and lack connectivity with 3D modeling.

[0004] The prior art literature related to this is as follows.

[0005] (Patent Document 0001) Republic of Korea Registered Patent Publication No. 10-2707647 (September 20, 2024)

[0006] The embodiments of the present disclosure are intended to satisfy both the creative intent of the designer and the engineering performance requirements.

[0007] The embodiments of the present disclosure aim to effectively achieve a balance of creativity and functionality by utilizing generative AI technology to automate the design exploration process and simultaneously integrating engineering performance analysis.

[0008] The embodiments of the present disclosure aim to combine topology optimization and generative AI in the automotive wheel design process.

[0009] The embodiments of the present disclosure aim to rapidly convert a 2D concept design into a 3D shape by utilizing 3D model generation technology specialized for wheels.

[0010] The embodiments of the present disclosure aim to significantly improve the efficiency of collaboration between designers and engineers by enabling detailed evaluation from the initial stage of determining the design concept.

[0011] The embodiments of the present disclosure aim to reduce subjectivity in design selection and optimize the balance between aesthetic requirements and functional performance based on data through style evaluation and engineering performance analysis using deep learning models.

[0012] The embodiments of the present disclosure aim to provide an efficient design process by integrating creative design exploration and performance evaluation, as well as ultimately provide a product development process through collaboration between AI and humans.

[0013] The problems of the present disclosure are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0014] An embodiment of the present disclosure may provide a generative AI-based co-design method for industrial designers and engineers, performed by a computing device, comprising the steps of: inputting a reference design for a product into a topology optimization model to obtain an output topology optimization design; and inputting the topology optimization design into a rendering model to obtain an output rendering image.

[0015] An embodiment of the present disclosure may provide a generative AI-based co-design method for industrial designers and engineers, further comprising: a process of generating a topology-optimized design according to topology optimization conditions by applying topology optimization to the reference design; and a process of training the topology-optimized model with training data configured as inputs, the reference design and topology optimization conditions, and the topology-optimized design as output.

[0016] An embodiment of the present disclosure can provide a generative AI-based co-design method for industrial designers and engineers, wherein the process of generating a topology-optimized design according to topology optimization conditions by applying topology optimization to the reference design is characterized by applying piece topology optimization after carving the reference design and generating the topology-optimized design through rotational post-processing.

[0017] An embodiment of the present disclosure can provide a generative AI-based co-design method for industrial designers and engineers, wherein the topology optimization model includes a pre-trained encoder and decoder and a U-net located between the encoder and decoder, and during the training process of the topology optimization model, only the U-net is trained and updated.

[0018] An embodiment of the present disclosure can provide a generative AI-based co-design method for industrial designers and engineers, characterized in that the rendering model is a pre-trained artificial intelligence model that receives the topology optimization design and style text for the product as input and outputs a realistic rendering image of the product.

[0019] An embodiment of the present disclosure may provide a generative AI-based co-design method for industrial designers and engineers, performed by a computing device, comprising the steps of: inputting a rendering design image of a product into a depth model to obtain an output depth map image; and inputting the depth map image into a three-dimensional shape reconstruction model to obtain an output 3D mesh shape model.

[0020] An embodiment of the present disclosure may provide a generative AI-based co-design method for industrial designers and engineers, further comprising: a process of acquiring a depth map image paired with a rendering design image; a process of augmenting the paired rendering design image and the depth map image; a process of preprocessing the augmented rendering design image and the depth map image; and a process of training a depth model with training data configured with the preprocessed rendering design image as input and the preprocessed depth map image as output.

[0021] An embodiment of the present disclosure can provide a generative AI-based co-design method for industrial designers and engineers, characterized in that the training data includes all of the images in which padding is applied to the product object image, the background image is replaced with another background image, the background image is replaced with a solid color background, noise is added to the product object image or background image, the scale of the product object image or background image is adjusted, or blur is added to the product object image or background image during the preprocessing process.

[0022] An embodiment of the present disclosure can provide a generative AI-based co-design method for industrial designers and engineers, characterized in that the depth model is a Marigold model based on a stable diffusion architecture fine-tuned using the preprocessed rendering design image and the preprocessed depth map image as training data.

[0023] An embodiment of the present disclosure may provide a generative AI-based co-design method for industrial designers and engineers, wherein the process of inputting the depth map image into a three-dimensional shape reconstruction model to obtain an output 3D mesh shape modeling includes: obtaining a first area point of the product located parallel to a plane perpendicular to the central axis from the depth map image when the product has a rotational symmetry structure with respect to the central axis; obtaining a second area point of the product located parallel to the central axis; obtaining an entire area point for the product by combining the first area point and the second area point; and obtaining a three-dimensional mesh shape modeling of the product based on the entire area point.

[0024] An embodiment of the present disclosure may provide a generative AI-based co-design method for industrial designers and engineers, performed by a computing device, comprising: a process of inputting a plurality of rendering design images of a product into a depth model to obtain a plurality of outputted depth map images; a process of inputting the plurality of depth map images into a feature extraction model to obtain a plurality of outputted feature embeddings; a process of clustering the plurality of feature embeddings and then sampling them by cluster; a process of inputting the rendering design images sampled by cluster into a performance evaluation model to obtain outputted structural performance; a process of inputting the rendering design images sampled by cluster into a style evaluation model to obtain outputted style scores; and a process of exploring the sampled rendering designs by structural performance or style based on the structural performance and style scores of the rendering design images sampled by cluster.

[0025] An embodiment of the present disclosure can provide a generative AI-based co-design method for industrial designers and engineers, characterized in that the performance evaluation model is trained using structural performance measured by structural analysis of a 3D mesh shape model generated based on a rendering design image sampled per cluster as training data.

[0026] An embodiment of the present disclosure can provide a generative AI-based co-design method for industrial designers and engineers, characterized in that the style evaluation model is trained using product design images and product style keywords as training data, or is trained using image-text pairs unrelated to the product.

[0027] An embodiment of the present disclosure may provide a generative AI-based co-design execution device for industrial designers and engineers, comprising a process, wherein the process comprises: a process of inputting a reference design for a product into a topology optimization model to obtain an output topology optimization design; and a process of inputting the topology optimization design into a rendering model to obtain an output rendering image.

[0028] An embodiment of the present disclosure may provide a generative AI-based co-design execution device for industrial designers and engineers, comprising a process, wherein the process comprises: a process of inputting a rendering design image of a product into a depth model to obtain an output depth map image; and a process of inputting the depth map image into a three-dimensional shape reconstruction model to obtain an output 3D mesh shape model.

[0029] An embodiment of the present disclosure may provide a generative AI-based co-design execution device for industrial designers and engineers, comprising a process, wherein the process comprises: a process of inputting a plurality of rendering design images for a product into a depth model to obtain a plurality of outputted depth map images; a process of inputting the plurality of depth map images into a feature extraction model to obtain a plurality of outputted feature embeddings; a process of clustering the plurality of feature embeddings and then sampling them by cluster; a process of inputting the rendering design images sampled by cluster into a performance evaluation model to obtain outputted structural performance; a process of inputting the rendering design images sampled by cluster into a style evaluation model to obtain outputted style scores; and a process of searching for the sampled rendering designs by structural performance or style based on the structural performance and style scores of the rendering design images sampled by cluster.

[0030] The means for solving the problem of the present disclosure are not limited to the means for solving the problem described above, and means for solving the problem not mentioned will be clearly understood by those skilled in the art to which the present disclosure belongs from the present specification and the attached drawings.

[0031] According to one embodiment of the present disclosure, the embodiment of the present disclosure provides an effect that satisfies both the creative intent of a designer and the engineering performance requirements.

[0032] Embodiments of the present disclosure provide the effect of effectively achieving a balance of creativity and functionality by utilizing generative AI technology to automate the design exploration process and simultaneously integrating engineering performance analysis.

[0033] An embodiment of the present disclosure provides the effect of combining topology optimization and generative AI in an automotive wheel design process.

[0034] An embodiment of the present disclosure provides the effect of rapidly converting a 2D concept design into a 3D shape by utilizing 3D model generation technology specialized for wheels.

[0035] The embodiments of the present disclosure enable detailed evaluation from the initial stage of determining the design concept, thereby providing the effect of significantly improving the efficiency of collaboration between designers and engineers.

[0036] Embodiments of the present disclosure provide the effect of reducing subjectivity in design selection and optimizing the balance between aesthetic requirements and functional performance based on data through style evaluation and engineering performance analysis using deep learning models.

[0037] The embodiments of the present disclosure provide the effect of not only providing an efficient design process by integrating creative design exploration and performance evaluation, but ultimately providing a product development process through collaboration between AI and humans.

[0038] The effects of the present disclosure are not limited to those described above, and unmentioned effects will be clearly understood by those skilled in the art from the present specification and the accompanying drawings.

[0039] FIG. 1 is a block diagram of a generative AI-based co-design execution device for industrial designers and engineers according to an embodiment of the present invention.

[0040] FIG. 2 is a flowchart of a generative AI-based co-design method for industrial designers and engineers according to an embodiment of the present invention.

[0041] FIG. 3 is a functional block diagram of a generative AI-based co-design execution device for industrial designers and engineers according to an embodiment of the present invention.

[0042] FIG. 4 is an overall conceptual diagram of a generative AI-based co-design execution framework for industrial designers and engineers according to an embodiment of the present invention.

[0043] FIG. 5 is a flowchart of a method for generating a product concept design according to an embodiment of the present invention. FIG. 6 is a conceptual diagram of a method for generating a design using topology optimization according to an embodiment of the present invention.

[0044] FIG. 7 is a conceptual block diagram of a topology optimization model according to an embodiment of the present invention.

[0045] FIG. 8 is a conceptual block diagram of a rendering model according to an embodiment of the present invention.

[0046] FIG. 9 is a flowchart of a 3D shape reconstruction method according to an embodiment of the present invention.

[0047] Figure 10 is a conceptual diagram of generating a depth map image paired with a product image.

[0048] FIG. 11 is a conceptual block diagram of a depth model according to an embodiment of the present invention.

[0049] Figure 12 is an example of augmenting a depth map image paired with a product image.

[0050] Figure 13 is an example of constructing training data by preprocessing depth map images paired with product images.

[0051] Figure 14 is an example of the cognitive process of the Marigold model.

[0052] Figure 15 is an example comparing the performance of the Marigold model (baseline), the Depth Anything model (baseline), and the finely tuned Marigold model.

[0053] FIG. 16 is a flowchart of a method for obtaining a three-dimensional shape according to an embodiment of the present invention.

[0054] FIG. 17 is an example of the process of acquiring a first region according to an embodiment of the present invention.

[0055] FIG. 18 is an example of the process of acquiring a second region according to an embodiment of the present invention.

[0056] FIG. 19 is an example of a process of combining regions according to an embodiment of the present invention.

[0057] FIG. 20 is an example of scaling and alignment of a first region and a second region according to an embodiment of the present invention.

[0058] FIG. 21 is an example of a process for generating a three-dimensional mesh shape of a product according to an embodiment of the present invention.

[0059] Figure 22 is an example of a three-dimensional mesh shape of a product generated from a product image through the S200 process.

[0060] FIG. 23 is a flowchart of a design search method according to an embodiment of the present invention. FIG. 24 is a conceptual diagram of a design search method according to an embodiment of the present invention.

[0061] Figure 25 is an example of a design space based on low-dimensional feature embeddings for a product design.

[0062] Figure 26 is an example of product design clustering.

[0063] Figure 27 is an example of design sampling from clustered product designs.

[0064] Figure 28 is a flowchart of a method for style-engineering dual evaluation of a product.

[0065] Figure 29 is an example of converting a 3D shape model from a mesh type to a NURBS type.

[0066] Figure 30 is an example of evaluating structural performance through structural analysis of a three-dimensional shape model.

[0067] FIG. 31 is a conceptual diagram of a performance evaluation model according to an embodiment of the present invention.

[0068] Figure 32 shows the predicted value of structural performance from the product design image using a performance evaluation model according to an embodiment of the present invention.

[0069] FIG. 33 is a conceptual diagram of a style evaluation model according to an embodiment of the present invention.

[0070] Figure 34 shows score values ​​for each style keyword from a product design image using a style evaluation model according to an embodiment of the present invention.

[0071] FIG. 35 shows an example of a detailed exploration of a product design according to an embodiment of the present invention.

[0072] FIG. 36 is a conceptual diagram of a stable diffusion model with dual (Style, Engineering Performance) constraints applied according to another embodiment of the present invention.

[0073] FIG. 37 is a detailed structural diagram of a surrogate model according to an embodiment of the present invention.

[0074] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In describing the embodiments, technical details that are well known in the art to which the present disclosure belongs and are not directly related to the present disclosure will be omitted. This is intended to convey the essence of the present disclosure more clearly without obscuring it by omitting unnecessary explanations.

[0075] The embodiments described in this specification are intended to clearly explain the concept of this disclosure to those skilled in the art to which this disclosure belongs. Therefore, this disclosure is not limited to the embodiments described in this specification, and the scope of this disclosure should be interpreted to include modifications or variations that do not deviate from the concept of this disclosure.

[0076] The terms used in this specification have been selected to be as widely used as possible, taking into account their functions in this disclosure; however, these terms may vary depending on the intent of those skilled in the art to which this disclosure pertains, case law, or the emergence of new technology. However, if a specific term is defined and used with an arbitrary meaning, the meaning of that term will be described separately. Accordingly, the terms used in this specification should be interpreted based on their actual meaning and the content throughout this specification, rather than merely their names.

[0077] The drawings attached to this specification are intended to facilitate the explanation of the present disclosure, and the shapes depicted in the drawings may be exaggerated as necessary to aid in understanding the present disclosure; therefore, the present disclosure is not limited by the drawings.

[0078] In this specification, each of the phrases such as “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “at least one of A, B, or C” may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0079] In this specification, if it is determined that a detailed description of known components or functions related to this disclosure could obscure the essence of this disclosure, such detailed description will be omitted as necessary. Furthermore, numbers used in the description of this specification (e.g., First, Second, etc.) are merely identifiers to distinguish one component from another.

[0080] Furthermore, the suffixes "part" and "substance" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification, and do not inherently possess distinct meanings or roles.

[0081] That is, the embodiments of the present disclosure are provided to make the present disclosure complete and to inform those skilled in the art of the scope of the present disclosure, and the invention of the present disclosure is defined only by the scope of the claims. Throughout the specification, the same reference numerals refer to the same components.

[0082] Terms such as “first” and / or “second” may be used to describe various components, but said components shall not be limited by said terms. For the sole purpose of distinguishing one component from another, for example, without departing from the scope of rights according to the concept of the present disclosure, the first component may be named the second component, and similarly, the second component may be named the first component.

[0083] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. Conversely, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Other expressions describing the relationship between components, such as "between" and "exactly between," or "adjacent to" and "directly adjacent to," should be interpreted in the same way.

[0084] In the drawings, each block of the process flow diagrams and combinations of the flow diagrams can be executed by computer program instructions. Since these computer program instructions can be loaded into the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, the instructions executed through the processor of the computer or other programmable data processing equipment create means to perform the functions described in the flow diagram block(s). Since these computer program instructions can also be stored in computer-available or computer-readable memory that can be directed toward the computer or other programmable data processing equipment to implement the function in a specific way, the instructions stored in computer-available or computer-readable memory can also produce a manufactured item containing means of instruction to perform the function described in the flow diagram block(s). Since computer program instructions can be loaded onto a computer or other programmable data processing equipment, instructions that execute a computer or other programmable data processing equipment by performing a series of operation steps on the computer or other programmable data processing equipment to create a process executed by the computer may also provide steps for executing the functions described in the flowchart block(s).

[0085] Additionally, device-readable storage media may be provided in the form of non-transitory storage media. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0086] Additionally, each block may represent a module, segment, or part of code containing one or more executable instructions for executing a specified logical function(s). It should also be noted that in some alternative execution examples, the functions mentioned in the blocks may occur out of order. For instance, two blocks described in succession may actually be executed substantially simultaneously, or the blocks may be executed in reverse order depending on the corresponding function. For instance, actions performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically; one or more of these actions may be executed in a different order, omitted, or one or more other actions may be added.

[0087] As used in this disclosure, the term “unit” refers to a software or hardware component such as a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC). A “unit” performs specific roles but is not limited to software or hardware. A “unit” may be configured to reside in an addressable storage medium or may be configured to run one or more processors. Accordingly, according to some embodiments, a “unit” includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and “units” may be combined into a smaller number of components and “units” or further separated into additional components and “units.” In addition, the components and 'parts' may be implemented to utilize one or more CPUs within the device or secure multimedia card. Furthermore, according to various embodiments of the present disclosure, the 'parts' may include one or more processors.

[0088] The operating principles of the present disclosure will be described in detail below with reference to the attached drawings. In describing the present disclosure below, specific descriptions of related known functions or configurations will be omitted if it is determined that such detailed descriptions would unnecessarily obscure the essence of the present disclosure. Furthermore, the terms described below are defined in consideration of their functions in the present disclosure, and these may vary depending on the intentions or practices of the user or operator. Therefore, their definitions should be based on the content throughout this specification.

[0089]

[0090] FIG. 1 is a block diagram of a generative AI-based co-design execution device for industrial designers and engineers according to an embodiment of the present invention.

[0091] Referring to FIG. 1, a generative AI-based co-design execution device (1000) for industrial designers and engineers according to one embodiment of the present invention (hereinafter referred to as the "computing device") may be composed of one or more combinations of a memory (1100) that communicates with each other via a bus, a user interface output unit (1200), a processor (1300), a storage (1400), a communication unit (1500) connected to a network, and a user interface input unit (1600). The processor (1300) may be a semiconductor device that executes programs or processing instructions stored in the memory (1100) or storage (1400).

[0092] Programs or processing instructions may store a generative AI-based co-design execution method for industrial designers and engineers according to one embodiment of the present invention, and a combination of one or more of the steps (processes) constituting the method. A processor (1300) may execute a generative AI-based co-design execution method for industrial designers and engineers according to one embodiment of the present invention, and a combination of one or more of the steps (processes) constituting the method.

[0093] A generative AI-based co-design execution method for industrial designers and engineers according to an embodiment of the present invention is described below. For convenience of explanation, each process is described based on being performed by a processor (1300), but is not limited thereto and can be applied using instructions stored in memory (1100) or storage (1400).

[0094] FIG. 2 is a flowchart of a generative AI-based co-design execution method for industrial designers and engineers according to an embodiment of the present invention. FIG. 3 is a functional block diagram of a generative AI-based co-design execution device for industrial designers and engineers according to an embodiment of the present invention. FIG. 4 is an overall conceptual diagram of a generative AI-based co-design execution framework for industrial designers and engineers according to an embodiment of the present invention.

[0095] Referring to FIG. 2, a generative AI-based co-design method for industrial designers and engineers according to one embodiment of the present invention may include one or more combinations of a product concept design generation process (S100), a 3D shape reconstruction process (S200), and a design exploration process (S300).

[0096] A computing device (1000) can perform one or more combinations of processes S100 to S300 of a generative AI-based co-design execution method for industrial designers and engineers according to one embodiment of the present invention.

[0097] Referring to FIG. 3, in various embodiments, the product design generation model (100), 3D shape reconstruction model (200), and design exploration model (300) are elements that constitute the processor (1300), and each of the models (100, 200, 300) may be the entity that performs the S100 process, the S200 process, and the S300 process, respectively.

[0098] In various embodiments, the design generation model (100), 3D shape reconstruction model (200), and design exploration model (300) are elements that constitute the program itself or processing instructions stored in memory (1100) or storage (1400), and each of the models (100, 200, 300) may include code for executing each of the S100 process, S200 process, S300 process.

[0099] Referring to FIG. 4, process S100 corresponds to stage 1, process S200 corresponds to stage 2, and process S300 corresponds to stage 3. Processes S100 through S300 may be performed sequentially, but are not limited thereto, and each process may be performed separately.

[0100] The S100 process is explained in detail below.

[0101] FIG. 5 is a flowchart of a method for generating a product concept design according to an embodiment of the present invention. FIG. 6 is a conceptual diagram of a method for generating a design using topology optimization according to an embodiment of the present invention. FIG. 7 is a conceptual block diagram of a topology optimization model according to an embodiment of the present invention. FIG. 8 is a conceptual block diagram of a rendering model according to an embodiment of the present invention.

[0102] In the S100 process, the computing device (1000) can perform a product concept design generation process (S100). Below, detailed processes that can be performed in the S100 process are described.

[0103] In various embodiments, the computing device (1000) can generate a product concept design by the design generation model (100).

[0104] Referring to FIG. 5, the S100 process may include one or more combinations of the following: a process of obtaining optimization data (S110), a process of obtaining a product design based on the optimization data (S120), and a process of obtaining a rendering image of the product (S130).

[0105] In the S110 process, the computing device (1000) can perform the process of acquiring topology optimization data of the product.

[0106] Referring to FIG. 6, in various embodiments, a computing device (1000) may obtain a reference design for a product and apply a topology optimization technique to the reference design to generate topology optimization data.

[0107] Here, the product is an object having a three-dimensional shape and may be an object having a symmetrical structure when viewed from at least one direction. Examples include a wheel or steering wheel of a car, a propeller of a ship or airplane, etc.

[0108] A reference design is a two-dimensional image of a product that can have a symmetrical structural pattern when viewed from a specific direction. Here, the symmetrical structural pattern can include point symmetry, where the original shape overlaps when rotated 180 degrees around a point, and rotational symmetry, where the original shape overlaps when the object is rotated by a specific angle.

[0109] In various embodiments, the computing device (1000) can generate symmetric topology optimization data by applying a piece topology optimization technique to a reference design. In other words, the computing device (1000) can generate engineering patterns based on the reference design.

[0110] Specifically, the computing device (1000) can divide a reference design into pieces of various sizes based on a plurality of division setting values. In this case, the reference design can be divided into a plurality of pieces based on a center that is a point of symmetry.

[0111] For example, the division setting value can be from 3 to 25, and preferably from 4 to 13. For example, when the division setting value is 4, the reference design can be divided into 4 pieces by cutting at 90-degree intervals from the center. For example, when the division setting value is 12, the reference design can be divided into 12 pieces by cutting at 30-degree intervals from the center.

[0112] A fragment of a reference design can be expressed as a 'fragment reference design'. Multiple fragment reference designs may include fragments divided into 3 parts, 4 parts, and fragments divided into various values.

[0113] In various embodiments, the computing device (1000) can generate multiple piece topology optimization data by applying a topology optimization technique to a piece reference design by adjusting parameter values ​​of multiple optimization conditions.

[0114] Multiple optimization conditions may include one or more combinations of C1 (Similarity) condition, C2 (Load Ratio) condition, and C3 (Volume Ratio) condition. The parameter value of the C1 (Similarity) condition may be set to one of 0.0005, 0.05, 0.5, or 5; the parameter value of the C2 (Load Ratio) condition may be set to one of 0, 0.1, 0.2, 0.3, or 0.4; and the parameter value of the C3 (Volume Ratio) condition may be set to one of (0.7, 0.8, 0.9, 1.0, 1.1) / piece.

[0115] Fragment topology optimization data may be a fragment reference design to which a topology optimization technique has been applied, a fragment topology optimization design, a topology-optimized fragment image, or a fragment topology optimization image. Fragment topology optimization data may include multiple topology-optimized fragment images as a combination of multiple conditions.

[0116] In various embodiments, the computing device (1000) can generate topology optimization data for a complete product by applying rotation post-processing to the piece topology optimization data.

[0117] Here, topology optimization data may be a topology-optimized image of the entire product formed by combining images that have been rotationally symmetrical through rotational post-processing of fragment images. This can be expressed as a topology-optimized overall design, or an overall topology-optimized design, or a topology-optimized overall image, or an overall topology-optimized image, or a term with the word 'overall' removed.

[0118] In the S120 process, the computing device (1000) can perform the process of learning an artificial intelligence model based on optimized data.

[0119] A computing device (1000) can train an artificial intelligence model with training data generated based on optimization data. Here, the training data may be composed of a reference design (e.g., product image) and values ​​of multiple optimization conditions as input values, and a (entire) topology optimization design (e.g., topology optimization image) as output values.

[0120] Referring to Fig. 7, the artificial intelligence model may be a topology optimization model (TOM) that receives a reference design and inputs values ​​of multiple optimization conditions, and outputs a topology optimization design.

[0121] The Topology Optimization Model (TOM) can be composed of an encoder (E), a decoder (D), and a U-Net.

[0122] An encoder is a part of a neural network that transforms input data (e.g., a reference design) into a latent space or a feature space. Generally, an encoder can perform feature extraction functions to extract important information from input data, dimensionality reduction functions to compress high-dimensional input data into a low-dimensional space, and data representation learning functions to learn the inherent patterns and structures of the input data. The encoder of the present invention is a pre-trained model, and its weights are not updated during the training process of the Topology Optimization Model (TOM) and remain in a fixed state.

[0123] A decoder is a part of a neural network that converts a latent representation generated by an encoder into the original data (e.g., a topology optimization design) or a desired output form. Generally, a decoder can perform a data restoration function that reconstructs encoded data into its original form, and a function that generates a desired output based on the latent representation. The decoder of the present invention is a pre-trained model, and its weights are not updated during the training process of the Topology Optimization Model (TOM) and remain in a fixed state.

[0124] U-Net is a Convolutional Neural Network (CNN) architecture for image segmentation designed to achieve high performance even with a small amount of training data. U-Net consists of a symmetric encoder-decoder structure and is divided into a contracting path, an expansive path, and an output layer.

[0125] The shrinkage path (encoder) can be composed of successive convolutional layers that perform two nxn (e.g., 3x3) convolution operations and apply a ReLU activation function after each convolution, and max pooling that uses mxm (e.g., 2x2) pooling to reduce the spatial dimension of the feature map by half and double the number of channels, and this process is repeated to increase the depth of the network. Through this, abstract features can be progressively extracted from input images, and more complex patterns can be learned by increasing the number of channels while reducing spatial resolution.

[0126] The expansion path (decoder) can be composed of upsampling, which increases the resolution of feature maps using transpose convolution or upconvolution; skip connections that concatenate corresponding feature maps extracted from the contraction path; and successive convolutional layers applying two nxn (e.g., 3x3) convolutions and ReLU, repeating this process to restore the original input size. Through this, it integrates details obtained from the contraction path while restoring resolution, and combines positional and contextual information to provide accurate segmentation.

[0127] Skip Connection directly connects the feature maps extracted at each step of the shrinking path to the corresponding steps of the expanding path, mitigating the vanishing gradient problem that occurs as the network deepens and preserving details. The output layer is connected to the output of the expanding path, and finally, an oxo (e.g., 1x1) convolution is used to adjust the number of output channels to the desired number of classes.

[0128] In various embodiments, the topology optimization model (TOM) is connected in the order of an encoder, a U-net, and a decoder, and the encoder receives values ​​of a reference design and a plurality of optimization conditions, and the decoder can output a topology optimization design.

[0129] The encoder receives a 2D image of a product, which serves as a reference design, as input and encodes it, and can output the encoded reference design as an image embedding (Zref) in the latent space. The image embedding is represented as a feature vector containing important information about the product image. Additionally, the encoder receives multiple optimization conditions as input and encodes them, and can output them as a text embedding in the latent space. The text embedding is represented as a feature vector containing important information about the optimization conditions. Here, the multiple optimization conditions refer to boundary conditions for applying topology optimization and can be expressed as numerical values ​​or parameter values.

[0130] The encoded reference design, image embedding (Zref), and the predicted topological optimization design (Zt) with added noise (D) are concatenated in the channel direction, and the combined tensor is input into the UNet in the form [Zref, Zt]. The text embedding is inserted at the attention layer of the UNet, enabling the topological optimization model to recognize the corresponding condition.

[0131] The Topology Optimization Model (TOM) calculates a loss function by determining the difference between the predicted topology optimization design and the topology optimization design of the training data, and is trained to update the weights of the UNet to minimize this loss. Note that the encoder and decoder are pre-trained and are no longer updated. The pre-trained decoder takes the final output of the UNet as input and can reconstruct the topology-optimized image.

[0132] In the S130 process, the computing device (1000) can perform the process of generating a topology optimization design of an optimized product according to conditions by applying an artificial intelligence model based on optimization data.

[0133] Here, the artificial intelligence model is a topology optimization model (TOM) learned through the aforementioned S120 process, and the learned topology optimization model (TOM) receives a reference design and optimization conditions as input and outputs a topology-optimized shape design.

[0134] Topology optimization shape design refers to topology optimization design and represents an image to which topology optimization is applied according to optimization conditions for the product shape. Here, the topology optimization image for the product has the same format or form as the reference image, is a two-dimensional image of a specific direction of the product, and can have an engineering pattern with rotational symmetry.

[0135] In the process of S140, the computing device (1000) can perform the process of generating a rendered image of the product by realistically rendering the design of the product.

[0136] Referring to FIG. 8, the design of the product may refer to an image having an engineering pattern, the topology optimization design or topology optimization image (Img_TO) described above. Additionally, the rendering image of the product (Img_Ren) may be a photorealistic rendering image, and the photorealistic rendering image may be an image that expresses a combination of at least one or two of depth, volume, surface material, surface texture, lighting, shadow, reflection, background, and color (brightness, contrast, saturation).

[0137] In various embodiments, the computing device (1000) can generate a realistic rendering image (Img_Ren) from a topology-optimized image (Img_TO) using a rendering model (RM). In other words, the computing device (1000) can convert a topology-optimized image, which is a planar image, into a three-dimensional and realistic image. Here, the rendering model (RM) may include one or a combination of a preprocessing unit and a stable diffusion model.

[0138] The input data of the rendering model (RM) may include a topology-optimized image of the product and text containing requirements. The requirements may include information on at least one or a combination of two of style, depth, volume, surface material, surface texture, lighting, shadows, reflections, background, and color (brightness, contrast, saturation). Here, the text containing the requirements may be a requirement prompt or a style prompt.

[0139] The processing unit receives a topology-optimized image (Img_TO), which is a binary (grayscale) image, as input and can perform pre-processing steps such as changing the image size or adding color values ​​(e.g., silver, etc.). However, pre-processing is not strictly necessary for the topology-optimized image and may not be performed depending on the stable diffusion model.

[0140] The Stable Diffusion Model receives preprocessed topology-optimized images and requirements text via a prompt and can generate realistic rendering images (Img_Ren). The realistic rendering image (Img_Ren) is an image that renders the product design in a realistic and lifelike manner, and it can generate rendering images with various appearances depending on the requirements information.

[0141] The preprocessed topology-optimized image may be an image resized to be suitable as an input for the stable diffusion model and modified to colors appropriate for the product. The stable diffusion model is pre-trained and generates a new rendered image by receiving the image requiring rendering and the requirement text as prompt-based input.

[0142] An embodiment of the present invention introduces a piece topology optimization suitable for a symmetrical product through the above-described S100 process to collect symmetric piece topology optimization data (e.g., an image of a piece of the product), generates overall topology optimization data of the product (e.g., an image of the entire product) through rotational post-processing of the piece topology optimization data, trains a topology optimization model (TOM) trained based on the topology optimization data, and the trained topology optimization model (TOM) can receive the product image as input and output topology optimization data.

[0143] In addition, the present invention can generate a realistic product rendering image by using a 2D product image, which is topology optimization data, and text of requirements through digital rendering.

[0144] In summary, topology optimization patterns (topology optimization images) can be generated using a conditionally generative model (Topology Optimization Model (TOM) based on optimization conditions), allowing for the exploration of various shapes based on the generated topology optimization images within the design space. Additionally, Image-to-Image technology enables the rapid rendering of various visual concept designs based on the generated patterns. Through this process, designers can efficiently review various shape options and styles at the initial concept design stage without the time-consuming process of manual coloring.

[0145] The S200 process is explained below.

[0146] FIG. 9 is a flowchart of a 3D shape reconstruction method according to an embodiment of the present invention. FIG. 10 is a conceptual diagram of generating a depth map image paired with a product image. FIG. 11 is a conceptual block diagram of a depth model according to an embodiment of the present invention. FIG. 12 is an example of augmenting a depth map image paired with a product image. FIG. 13 is an example of constructing training data by preprocessing a depth map image paired with a product image. FIG. 14 is an example of the recognition process of a Marigold model. FIG. 15 is an example comparing the performance of a Marigold model (baseline), a Depth Anything model (baseline), and a fine-tuned Marigold model. FIG. 16 is a flowchart of a 3D shape acquisition method according to an embodiment of the present invention. FIG. 17 is an example of the acquisition process of a first region according to an embodiment of the present invention. FIG. 18 is an example of the acquisition process of a second region according to an embodiment of the present invention. FIG. 19 is an example of the combination process of regions according to an embodiment of the present invention. FIG. 20 is an example of scaling and alignment of a first region and a second region according to an embodiment of the present invention. FIG. 21 is an example of a process for generating a three-dimensional mesh shape of a product according to an embodiment of the present invention. FIG. 22 is an example of a three-dimensional mesh shape of a product generated from a product image through process S200.

[0147] In the S200 process, the computing device (1000) can perform a 3D shape reconstruction process (S200). Below, detailed processes that can be performed in the S200 process are described.

[0148] In various embodiments, the computing device (1000) can reconstruct the 3D shape of a product by the 3D shape reconstruction model (200).

[0149] In various embodiments, the 3D shape reconstruction model (200) relates to an Image-to-3D deep learning model that reconstructs an image of a product into a 3D shape model, and can provide a structurally high level of completeness for the 3D model of the product. Through this, the designer can quickly convert a 2D concept design image into a 3D shape model and examine various 3D characteristics.

[0150] In the case of conventional technology, Text-to-3D generation models have the problem of ignoring structural characteristics by focusing only on visual validity, making precise structural modeling difficult, and the structural completeness decreases when texturing is applied. In contrast, the 3D shape reconstruction model (200) according to the embodiment of the present invention creates a 3D shape model based on a rendering image of the product, so it is possible to create a 3D model of the product with precise structure, and the structural completeness may not decrease even when texturing is applied.

[0151] The S200 process may include one or more combinations of the following: a process of acquiring a rendering image (S210), a process of augmenting a rendering image (S220), a preprocessing process (S230), a process of training a depth model (S240), a process of acquiring a depth map (S250), and a process of acquiring a 3D shape (S260). For example, the S200 process may consist of a combination of different processes in the training process and the inference process.

[0152] The 3D shape reconstruction model (200) can reconstruct the 3D shape of a product by performing one or more combinations of processes S210 to S260. Each process is described in detail below.

[0153] In the S210 process, the computing device (1000) can obtain a rendered image.

[0154] In various embodiments, the computing device (1000) may be stored in storage (1400), generated during the S100 process, or obtain a rendered image (Img_Ren) from a reference 3D model (M_3D).

[0155] Referring to FIG. 10, in various embodiments, a computing device (1000) can obtain a 3D modeling (M_3D) configured in a mesh form and generate a rendering image (Img_Ren) and a depth map image (Depth map) by setting a background image as an image plane in the 3D modeling.

[0156] Here, the rendering image (Img_Ren) and the depth map image can be mapped to form pairs and are images extracted in the same direction for the 3D modeling of the product.

[0157] The rendered image (Img_Ren) can be a 2D image of the product. Alternatively, the rendered image (Img_Ren) can be an RGB image of the product or an RGBA image of the product. Here, an RGBA image is a color model in digital images that uses four channels: Red, Green, Blue, and Alpha. RGB represents the basic components of color, and Alpha is an additional channel that controls transparency. The Alpha channel represents transparency or opacity and determines how the pixels of the image overlap with the background or other images.

[0158] The depth map image may be a depth map image having depth information about the product extracted from a 3D model of the product.

[0159] Here, an image can refer to an image dataset, meaning one or more images. Even when expressed as "image" in the following description, it includes the meaning of an image dataset.

[0160] In the S220 process, the computing device (1000) can augment the rendered image.

[0161] Referring to FIG. 11, in various embodiments, the computing device (1000) can augment a rendered image using a data augmentation model (220). Additionally, the computing device (1000) can augment a rendered image and a depth map image using the data augmentation model (220).

[0162] Image augmentation refers to increasing the diversity of images by utilizing existing images and employing various techniques, without actually collecting new images, in order to increase the number of images.

[0163] Referring to FIG. 12, the data augmentation model (220) can generate a new augmented rendering image by rotating the rendering image (Img_Ren) by a certain angle. The augmented rendering image may have its position or orientation changed while maintaining the content of the original rendering image. Additionally, the data augmentation model (220) can generate a new augmented depth map image by rotating the depth map image paired with the rendering image. Here, the rotation angle may be set randomly or selected evenly within a specific range. For example, rotations between -30 degrees and +30 degrees, rotations of +45 degrees, etc., may be selected.

[0164] Referring to FIG. 12, the data augmentation model (220) can combine two or more rendering images to generate a new augmented image. Additionally, the data augmentation model (220) can also perform a combination transformation on the depth map image paired with the rendering image to generate a new augmented depth map image.

[0165] An augmented rendering image (or depth map image) may be generated by linearly combining the pixel values ​​of at least two existing rendering images with weights, by cropping a portion of one image and pasting it onto another, or by combining multiple images into one large image to create an image containing objects at various scales and positions. This can generate data with new patterns and characteristics through the combination of various existing rendering images, and

[0166] Referring to FIG. 12, the data augmentation model (220) can generate a new augmented rendering image by converting the colors of an existing rendering image into two colors (Bi-color) or by applying a specific color combination. Additionally, the data augmentation model (220) can generate a new augmented depth map image by applying a dichromatic technique to a depth map image paired with the rendering image.

[0167] An augmented rendering image (or depth map image) may be created by converting the pixel values ​​of an image into two values ​​(black and white) and then combining them, or by applying a specific color filter to the image to leave only two major colors.

[0168] In various embodiments, the data augmentation model (220) may apply one or more combinations of rotation, combine, and bi-color techniques to a rendering image dataset (or depth map image dataset) to generate a new augmented image. Each of the rotation, combine, and bi-color techniques helps the model to be trained in the future to make accurate predictions in various situations through spatial transformation, data mixing, and color transformation.

[0169] In various embodiments, the data augmentation model (220) can generate augmented image data pairs by applying the same augmentation technique to a pair of rendering images and depth map images.

[0170] In the S230 process, the computing device (1000) can perform a preprocessing process.

[0171] Referring to FIG. 11, in various embodiments, the computing device (1000) may perform a preprocessing process on an augmented rendering image using a preprocessor (230). Additionally, the computing device (1000) may perform a preprocessing process on an augmented depth map image using a preprocessor (230).

[0172] Referring to FIG. 13, in various embodiments, the preprocessor (230) can generate a preprocessed image by applying various preprocessing to the augmented image. Here, the preprocessed image may have a constant image size.

[0173] In various embodiments, the preprocessor (230) may perform a preprocessing process on one or both of the product object (obj) image and the background (bg) image within the augmented image to generate a preprocessed image.

[0174] In various embodiments, referring to X1 in FIG. 13, the preprocessor (230) can apply padding around the product object, and specifically, can generate an image with padding applied to at least one of the left, bottom, right, and top of the product object.

[0175] In various embodiments, referring to X2 in FIG. 13, the preprocessor (230) can replace the background with another image to generate an image in which a product object image is combined on another background image.

[0176] In various embodiments, referring to X3 in FIG. 13, the preprocessor (230) can change the background to a different solid color, remove the background, or add noise to the background to generate an image.

[0177] In various embodiments, referring to X4 in FIG. 13, the preprocessor (230) can enlarge or reduce the scale of the product object, or blur the product object to generate a blurred image.

[0178] In various embodiments, the preprocessor (230) can generate an image by simultaneously applying one or more of the techniques for X1 to X4 described above.

[0179] In various embodiments, the preprocessor (230) may apply the same preprocessing technique to the augmented depth map images (D1, D2, D3, D4) paired with the augmented rendering image. Alternatively, in various embodiments, the preprocessor (230) may apply different preprocessing techniques to the augmented depth map images (D1, D2, D3, D4) paired with the augmented rendering image. Infinitely many pairs of preprocessed rendering images and preprocessed depth map images can be obtained by using various preprocessing techniques.

[0180] Through this, the computing device (1000) can obtain training data configured by using a preprocessed rendering image as input data and labeling a preprocessed depth map image paired with the rendering image as output data.

[0181] In the S240 process, the computing device (1000) can learn a depth map.

[0182] Referring to FIG. 11, in various embodiments, the computing device (1000) can train the depth model (240) using training data.

[0183] The depth model (240) may be an artificial intelligence model or a deep learning model that, when it receives an image (x) of a product, outputs a depth map image containing depth information about the product. For example, the depth model (240) may be a Vision Transformers for Dense Prediction (DPT) model, such as the Depth Anything (2024, TicTok) model, or a Stable diffusion architecture-based model, such as the Marigold (2024, ETH Zurich) model.

[0184] Referring to Fig. 14, the Marigold model basically has a stable diffusion architecture structure and has a structure in which the entire model is trained by mapping a rendered image (x) and a depth map image (d) onto a latent space using a latent encoder, and then training a latent diffusion U-Net in the direction of gradually removing noise after adding sample noise. Here, the latent encoder may not be trained further as it has already been trained.

[0185] When a Marigold model fine-tuned using training data receives a rendered image (x) as input, it outputs a depth map image based on the input rendered image (x).

[0186] Referring to FIG. 15, the depth model (240) according to an embodiment of the present invention may use any one of the Marigold model without additional training from the existing trained model (baseline), the Depth Anything model without additional training from the existing trained model (baseline), or the fine-tuned Marigold model, but preferably the fine-tuned Marigold model may be used. This is because when comparing the depth map images, which are the results of the three models described above, using the validation data (Validation data) that was left for evaluation in the rendering image and depth map image preprocessed in process S230, it was seen that the result of the fine-tuned Marigold model was most similar to the validation data.

[0187] In the S250 process, the computing device (1000) can obtain a depth map.

[0188] Referring to FIG. 11, in various embodiments, the computing device (1000) can generate a depth map image from a rendering image (x) using a depth model (240).

[0189] Here, the rendering image (x) may be one of the rendering image stored in storage (1400), the rendering image extracted from the 3D model of the product, the augmented rendering image of process S220, and the preprocessed rendering image of process S230.

[0190] Here, the depth model (240) may be a Marigold model that has been fine-tuned and trained in the S240 process, a Marigold model without additional training (baseline), or a depth-anything model (baseline).

[0191] In the S260 process, the computing device (1000) can obtain a 3D shape of the product.

[0192] In various embodiments, the computing device (1000) can obtain a three-dimensional shape of a product using a depth map image.

[0193] Referring to FIG. 16, the S260 process may include one or more combinations of the first region (spoke) acquisition process (S261), the second region (rim) acquisition process (S262), the region combination process (S263), and the 3D mesh shape acquisition process (S264). The computing device (1000) may perform one or more combinations of the above processes.

[0194] In process S261, the computing device (1000) can acquire a first area (spoke) for the product.

[0195] In various embodiments, when the product has a rotationally symmetric structure with respect to a central axis, the first region of the product may refer to a region of the product located parallel to a plane perpendicular to the central axis. Alternatively, the first region of the product may refer to a portion of the product viewed from the central axis of the product.

[0196] In various embodiments, the first region of the product represents a three-dimensional shape when the product is viewed from a specific direction. In an embodiment of the present invention, the first region of the product refers to the spoke region of an automobile wheel.

[0197] A car wheel consists of a rim portion that is connected to the tire around the circumference of the wheel, a hub portion that is connected to the car body through a hole in the center of the wheel, a disc portion that is a round plate between the hub and the rim, and spoke portions that consist of spokes connecting the rim and the hub.

[0198] Referring to FIG. 17, in various embodiments, the computing device (1000) can calculate all points (P_tot) corresponding to the entire product based on a depth map image (d) of the product. In other words, it can extract all points (P_tot) corresponding to the entire product represented on the depth map image (d).

[0199] Referring to FIG. 17, in various embodiments, the computing device (1000) can filter first area points (P1, spoke points) corresponding to a first area (spoke) among all previously extracted points (P_tot). Here, the first area point (spoke point) may be a point on the surface of an object where a vector from the medial axis of the product object toward the product object meets the surface.

[0200] Through this, the computing device (1000) can reconstruct the shape of the product object using the central axis of the product object and the spoke lengths from the center point on each central axis to the spoke points, and can generate a surface mesh by connecting the spoke points.

[0201] Referring to FIG. 17, in various embodiments, the computing device (1000) can obtain symmetric spoke points for the entire angle range by performing rotational symmetry processing on spoke points (S1) of a predetermined angle range when the first region has rotational symmetry as a characteristic of the product. That is, among the first region points (P1, spoke points), points (S1, spoke points) belonging to a predetermined angle range (90 degrees, 60 degrees, 45 degrees, 30 degrees, etc.) with respect to the central axis are extracted, and after generating points (P1', spoke points) of the entire angle by rotationally symmetrically copying them with respect to the central axis, the points can be connected to generate a surface mesh.

[0202] Through this, the embodiment of the present invention can perfectly realize rotational symmetry in the first region (spoke) of the product. However, the process of extracting points within a predetermined angle range and rotating them is not necessarily required if there is no rotational symmetry region on the product.

[0203] In process S262, the computing device (1000) may acquire a second region (rim) of the product. Process S262 may be performed when the product shape has a geometric structure that is close to a torus or ring shape, and may not be performed when it does not have a ring shape.

[0204] In various embodiments, when the product has a rotationally symmetric structure with respect to a central axis, the second region of the product may refer to a region of the product located parallel to the central axis. Alternatively, the second region of the product may refer to a region of the product located in a radial direction with respect to the central axis of the product.

[0205] In various embodiments, the computing device (1000) can retrieve a second area shape (rim shape) for a product stored in storage (1400) based on the size of a first area (spoke shape).

[0206] Referring to FIG. 18, in various embodiments, a computing device (1000) can extract second area cross-section points (S2, rim cross-section points) through cross-sections from the central axis to the side in a reference 3D model of a product. The second area cross-section points (S2, rim cross-section points) may include a second-1 area cross-section point (S2-1) corresponding to the side of the product and a second-2 area cross-section point (S2-2) corresponding to the top surface of the product.

[0207] In various embodiments, the computing device (1000) can obtain second-1 area points (P2-1) corresponding to 360 degrees by rotating the second-1 area cross-section points (S2-1) symmetrically around a central axis (z-axis). Additionally, the computing device (1000) can obtain second-2 area points (P2-2) corresponding to 360 degrees by rotating the second-2 area cross-section points (S2-2) symmetrically around a central axis (z-axis).

[0208] In various embodiments, the computing device (1000) can generate a second-1 area mesh shape (P2-1) having a side ring shape corresponding to the side of the product and a second-2 area mesh shape (RS2-2) having a top ring shape corresponding to the top surface of the product and a second-2 area mesh shape (RS2-2) having a mesh surface and a side ring shape corresponding to the top surface of the product based on second-2 area points (P2-2).

[0209] In process S263, the computing device (1000) can perform a process of combining regions. Process S263 may be performed when process S262 is performed, and may not be performed when process S262 is not performed.

[0210] In process S263, the computing device (1000) can perform a process of combining the first region and the second region with each other.

[0211] In various embodiments, the computing device (1000) can combine a first area point and a second area point to generate an entire area point of the product, and generate an entire area shape of the product based on the entire area point of the product.

[0212] In various embodiments, the computing device (1000) can combine the first area mesh shape and the second area mesh shape to generate the entire area mesh shape of the product.

[0213] Referring to FIG. 19, in process S263, when the computing device (1000) combines a first area corresponding to the top surface of the product and a second area corresponding to the side of the product, the first area deformed through a preset offset deformation can be combined with the second area.

[0214] The pre-set offset variation may be one of a positive offset where the center of the first region is designed to be higher than the center of the height of the second region, a zero offset where it is designed to be the same, or a negative offset where it is designed to be lower.

[0215] Referring to FIG. 20, in process S263, the computing device (1000) can perform a scaling process that changes the size of at least one of the first region or the second region when combining the first region and the second region, and a process that aligns the positions of the first region and the second region.

[0216] In the S264 process, the computing device (1000) can perform a 3D mesh shape acquisition process.

[0217] Referring to FIG. 21, in process S264, the computing device (1000) can convert the combined first region and second region (point or shape) into a regular grid (Discretization into regular grid) and then extract mesh data using a Marching Cubes algorithm. The mesh data is a three-dimensional model of a product, meaning a surface represented as a mesh.

[0218] Here, discretization into a regular grid is a process of dividing a 3D object into small cells (regular grids), thereby converting a continuous 3D space into a discontinuous regular grid. Then, the marching cube algorithm extracts mesh data by determining whether a boundary exists based on the value (1 or 0) within the cube corresponding to the cell of the regular grid.

[0219] Referring to FIG. 21, in process S264, the computing device (1000) can perform an optimization process on the mesh data to finally generate a three-dimensional mesh shape (Generated mesh).

[0220] Mesh optimization is a process of reducing mesh complexity, smoothing surfaces, and improving processing speed and quality to efficiently handle 3D model mesh data. A mesh consists of a set of triangles or polygons representing the surface of a 3D object, and mesh optimization is the process of applying various techniques to improve the resolution, quality, and rendering efficiency of this mesh.

[0221] In various embodiments, mesh optimization may include one or more combinations of: a Laplacian Smoothing algorithm that makes the mesh surface smoother, Reducing Mesh Counts that reduces the number of triangles (polygons) in the mesh, and Watertight Post-Processing that makes the mesh surface free of holes or discontinuities.

[0222] Referring to FIG. 21, in process S264, the computing device (1000) can perform a scale change process to match the size of the actual product to the three-dimensional mesh shape.

[0223] For example, if the product is a car wheel, the computing device (1000) can scale the top surface size (e.g., 19 inches) and side height (e.g., 8.5 inches) of the 3D mesh shape for the car wheel to match the actual car wheel size.

[0224] Referring to FIG. 22, a computing device (1000) can generate a three-dimensional mesh shape model from a rendering image using a 3D shape reconstruction model (200). Although the rendering image is a two-dimensional image of a first area (top surface, spoke area) of the product, that is, an image of a part of the product, a three-dimensional mesh shape model of the entire product can be generated from the rendering image.

[0225] In various embodiments, 3D shape reconstruction for a product may mean structure-recognized precise 3D shape reconstruction for product design, and here, may mean structure-recognized precise 3D shape reconstruction for automobile wheel design.

[0226] The 3D shape reconstruction model (200) can propose a model applicable to products having a symmetrical structure, such as a wheel. For example, 3D spoke points in the form of spokes can be extracted using a depth map from a 2D concept rendering image of the upper surface of a car wheel, and 3D rim points can be extracted from the reference mesh shape modeling of the wheel. Then, the spoke and rim point cloud data can be integrated to create a complete wheel shape, and then reconstructed into a 3D mesh model using a Marching cube algorithm. Additionally, an optimization process can be applied to generate a final 3D mesh shape model.

[0227] The S300 process is explained in detail below.

[0228] FIG. 23 is a flowchart of a design search method according to an embodiment of the present invention. FIG. 24 is a conceptual diagram of a design search method according to an embodiment of the present invention. FIG. 25 is an example of a design space based on low-dimensional feature embeddings for a product design. FIG. 26 is an example of product design clustering. FIG. 27 is an example of design sampling from clustered product designs. FIG. 28 is a flowchart of a method for style-engineering dual evaluation of a product. FIG. 29 is an example of converting 3D shape modeling from a mesh type to a NERBS type. FIG. 30 is an example of evaluating structural performance through structural analysis of 3D shape modeling. FIG. 31 is a conceptual diagram of a performance evaluation model according to an embodiment of the present invention. FIG. 32 shows predicted values ​​of structural performance from a product design image using a performance evaluation model according to an embodiment of the present invention. FIG. 33 is a conceptual diagram of a style evaluation model according to an embodiment of the present invention. FIG. 34 shows score values ​​by style keyword from a product design image using a style evaluation model according to an embodiment of the present invention. FIG. 35 shows an example of a detailed exploration of a product design according to an embodiment of the present invention.

[0229] In the S300 process, the computing device (1000) can perform a design exploration process (S300). Below, detailed processes that can be performed in the S300 process are described.

[0230] In various embodiments, the computing device (1000) can perform a search for designs by the design search model (300). Through this, the design search can be performed through an integrated style and engineering evaluation of the product.

[0231] Referring to FIGS. 23 and 24, the S300 process may include one or more combinations of the design sampling and simulation process (S310), the style-engineering dual evaluation process (S320), and the design exploration process (S330). The computing device (1000) may perform one or more combinations of these processes.

[0232] In the S310 process, the computing device (1000) can perform design sampling and simulation.

[0233] Referring to FIG. 25, a computing device (1000) can generate feature embeddings from images of a product. The images of the product may be RGB or RGBA images, two-dimensional images of the product, and rendering images generated through the image generation method described above.

[0234] In the process of S310, in various embodiments, the computing device (1000) can generate depth map images of the product using a depth model (240) (e.g., Depth estimator, Marigold model (baseline), Depth Anything model (baseline), fine-tuned Marigold model, etc.).

[0235] Subsequently, the computing device (1000) can generate feature embeddings of depth maps from depth map images using a feature extraction model (2510) (e.g., Pre-trained Vision Transformer (ViT), CNN, etc.). The feature embeddings of depth maps can be expressed as depth map features, depth features, depth map feature embeddings, etc. Extracting feature embeddings from depth map images has advantages because features can be extracted only from the shape information of the product.

[0236] In various embodiments, the computing device (1000) may exclude the depth model (240) and directly generate feature embeddings from images of the product using a feature extraction model (2510). Extracting feature embeddings from images of the product has the advantage of being able to extract features from the product's texture, color, and same-surface level information.

[0237] In an embodiment of the present invention, extracting feature embeddings from a depth map image is preferable to extracting feature embeddings from a product image. Below, the explanation will continue based on extracting feature embeddings from a depth map image.

[0238] In the process of S310, in various embodiments, the computing device (1000) can visualize feature embeddings using t-distributed Stochastic Neighbor Embedding (t-SNE).

[0239] Feature embeddings represent high-dimensional data as dense vectors in a fixed low-dimensional space (usually 2 or 3 dimensions). t-SNE reduces the dimensions of embedded vectors to a lower dimension to visually demonstrate how they are arranged relative to each other in high-dimensional space. Furthermore, by utilizing the characteristic that data points close (similar) in high-dimensional space are also placed close in low-dimensional space, it provides data similarity in high-dimensional space through relationships or clusters of data in the low-dimensional space.

[0240] Looking at the t-distribution probabilistic embedding graph in Fig. 25, it was confirmed that the distribution of feature embeddings extracted from the depth map has a dense distribution depending on the volume (or area) of the product shape, which confirms that sampling is possible depending on the geometric characteristics of the product.

[0241] In the process of S310, in various embodiments, the computing device (1000) can perform clustering of product designs or feature embeddings.

[0242] In process S310, the computing device (1000) can perform a process to determine the optimal number of clusters (k) in clustering. Specifically, the Elbow Method can be used.

[0243] Referring to Fig. 26, the Elbow Method for Optimal k for determining the optimal number of clusters (k) is a method that selects the optimal number of clusters by finding an inflection point (Elbow) where the sum of squared distances within clusters (SSD) decreases as the number of clusters increases, but the rate of decrease decreases. Here, the number of clusters (k) was determined to be 9 because the inflection point occurs at 9.

[0244] In the process of S310, in various embodiments, the computing device (1000) may perform sampling to extract sample data that well reflects the representativeness of the data within each clustered cluster. Specifically, Latin Hypercube Sampling (LHS) may be used.

[0245] LHS is a method used to efficiently sample in high-dimensional space. It is a method that obtains samples that accurately reflect the distribution of data by dividing the space into intervals in each dimension and extracting samples evenly from all intervals.

[0246] Referring to Fig. 27, sampled depth maps and product images of Cluster 1 and Cluster 7 are shown. As the sample data of Cluster 1 shows that the wheel spoke shape has a spiral shape, and the sample data of Cluster 7 shows that the wheel spoke shape consists of approximately 5 spokes, it can be seen that the data is well clustered according to the geometric shape of the automobile wheel (product).

[0247] In the S320 process, the computing device (1000) can perform a style-engineering dual evaluation.

[0248] Referring to FIG. 28, the S320 process may include one or more combinations of the following: a product image acquisition process (S321), a preprocessing process for CAD design (S322), a simulation process for performance evaluation (S323), a performance evaluation model learning process (S324), a performance evaluation process (S325), a product image acquisition process (S326), a style keyword acquisition process (S327), a style evaluation model learning process (S328), and a style evaluation process (S329). The computing device (1000) may perform one or more combinations of the above processes.

[0249] In the S321 process, the computing device (1000) can perform the process of acquiring a product image.

[0250] In various embodiments, the computing device (1000) may obtain a product image stored in storage (1400) or extracted from a reference 3D model. The product image may have an RGB or RGBA format for the product, be a two-dimensional image of the product, and may be a rendered image of the product.

[0251] In the S322 process, the computing device (1000) can perform a preprocessing process for CAD design.

[0252] In various embodiments, the computing device (1000) can reconstruct a 3D shape model by performing the S100 process and the S200 process. Here, the 3D shape model for the product refers to a 3D mesh shape model in which the surface of the shape is composed of polygons.

[0253] Referring to FIG. 29, the computing device (1000) can convert a 3D mesh shape model into a simulable 3D shape model. Here, if the simulable 3D shape model is expressed as NURBS (Non-Uniform Rational B-Splines), it can be expressed as a 3D NURBS shape model.

[0254] NURBS is a mathematical model used to smoothly represent complex 3D shapes by defining curves and surfaces as control points. NURBS is widely used, particularly in CAD systems, for modeling 3D shapes.

[0255] In the S323 process, the computing device (1000) can perform a simulation process for performance evaluation.

[0256] Referring to FIG. 30, in various embodiments, a computing device (1000) may operate a simulator for performance evaluation based on 3D shape modeling. Here, the 3D shape modeling may be at least one of the 3D mesh shape modeling and 3D NURBS shape modeling described above.

[0257] A simulator is software capable of modal analysis of 3D shape models, and can evaluate the structural performance of a product through simulation. The target of structural performance evaluation may have various parameters (e.g., mass of the model and natural frequencies by mode) that can evaluate the engineering performance of the product.

[0258] In the case according to an embodiment of the present invention, FunctionBay’s Recurdyn2024 software was used as the simulator, and modal analysis was performed through structural analysis simulation of the model using this, and the structural performance of the automobile wheel was evaluated.

[0259] Here, the mesh type is automatically generated tetrahedral elements (Solid4), with a minimum element of 1 mm and a maximum element of 100 mm. The material properties were selected from Aluminum alloy 6061-T6, which is primarily used for automotive wheels, suspension components, and body frames and weighs approximately one-third the weight of steel; the Young's Modulus is 68.9 GPa (10,000 ksi), and the Poisson's Ratio is 0.33. The simulator calculates the model's mass and the natural frequencies of 13 modes to produce the final output (refer to the lower output values ​​in Fig. 30).

[0260] In the 324 process, the computing device (1000) can perform the learning process of the performance evaluation model.

[0261] Referring to FIG. 31, in various embodiments, a computing device (1000) can train a performance evaluation model (CEM) using pairs of product images and structural performance evaluation targets (e.g., vibration frequency and mass of a wheel) as training data. Through this, the trained performance evaluation model (CEM) can receive product images as input and output structural predicted performance (e.g., predicted vibration frequency and predicted mass of a wheel) for the product in the image.

[0262] The performance evaluation model (CEM) may include a feature extraction model (ViT) that extracts features by embedding them from product images, and a surrogate model that receives the feature embeddings as input and outputs structural prediction performance for the product.

[0263] In the S325 process, the computing device (1000) can perform a performance evaluation process.

[0264] Referring to FIG. 32, in various embodiments, a computing device (1000) receives a product image using a learned performance evaluation model (CEM) and outputs structural prediction performance for the product (e.g., predicted frequency and predicted mass of a wheel).

[0265] In the S326 process, the computing device (1000) can perform the product image acquisition process.

[0266] In various embodiments, the computing device (1000) may obtain a product image stored in storage (1400) or extracted from a reference 3D model. The product image may have an RGB or RGBA format for the product, be a two-dimensional image of the product, and may be a rendered image of the product.

[0267] In the S327 process, the computing device (1000) can perform the process of acquiring style keywords.

[0268] In various embodiments, the computing device (1000) may obtain style keywords stored in storage (1400) or input from a user. Style keywords may be adjectives containing an external and sensory evaluation of a product, such as Modern, Futuristic, Complex, Dynamic, Luxury, Polished, Rugged, Sculpted, Bold, etc.

[0269] In the S328 process, the computing device (1000) can perform the learning process of the style evaluation model.

[0270] Referring to FIG. 33, in various embodiments, the style evaluation model (SEM) may be a deep learning model that enables multimodal understanding by learning text and images simultaneously, and as an example, it may be CLIP (Contrastive Language-Image Pretraining).

[0271] CLIP learns images and text (style keywords) simultaneously to understand the relationship between the two data, so it can predict text from images or images from text.

[0272] In various embodiments, for embodiments of the present invention, a style evaluation model (SEM) can be trained using large-scale image-text pairs collected from the internet regardless of the product.

[0273] In various embodiments, in the case of an embodiment of the present invention, product images and style keywords are provided to a user, and by allowing the user to select a style keyword that suits the product, a style evaluation model (SEM) can be trained using product image and style keyword pairs as training data.

[0274] The Style Evaluation Model (SEM) is trained using a Vision Encoder (e.g., ResNet, Vision Transformer (ViT)) to process images and a Text Encoder (e.g., Transformer, GPT-family language models) to process text. After generating image embedding vectors and text embedding vectors through the Vision Encoder and Text Encoder, respectively, the model is trained using a Contrastive Loss function to compare the similarity between the two vectors, thereby making correct matches in image-text pairs as close as possible and moving incorrect matches far apart.

[0275] In the S329 process, the computing device (1000) can perform a style evaluation process.

[0276] Referring to FIG. 34, in various embodiments, a computing device (1000) can output a score for each style keyword suitable for a product (or expressed as a score for each style, style score, etc.) using a learned style evaluation model (CEM). Specifically, when the learned style evaluation model (CEM) receives a product image as input, it can output a score for each style keyword that describes the product in the product image well.

[0277] For example, the scores by style keyword for the first product image in Fig. 34 may be output as Futuristic: 0.39, Elegant: 0.28, Sporty: 0.16, Sculpted: 0.07, Classic: 0.06. Here, the style keyword with the highest score may be the representative style keyword of the product.

[0278] In the S330 process, the computing device (1000) can perform design exploration.

[0279] Referring to FIG. 35, in various embodiments, a computing device (1000) can represent sample points for a product in a low-dimensional space (2D or 3D) using style keywords for the product and structural prediction performance for the product (e.g., mass, frequency, etc.).

[0280] In various embodiments, the computing device (1000) can filter products by representative style keywords and search for products based on structural prediction performance for the filtered products. For example, as shown in FIG. 35, car wheel images having a style classified as Luxury can be filtered, and wheel images having desired structural prediction performance (mass and frequency) can be selected from among them.

[0281] In the S300 process, an embodiment of the present invention can comprehensively evaluate the engineering performance and style of a product using concept design images of the product. Feature embeddings are obtained from approximately 10,000 concept design images of the product generated through the S100 process, and clustering is performed using these embeddings. Additionally, 2,500 representative design points are extracted from a total of 10,000 design spaces using the LHS technique.

[0282] Each design point is reconstructed into a mesh-type 3D shape model through the S200 process, and the structural performance of the wheel is evaluated using FunctionBay's RecurDyn structural analysis simulation. During this process, the wheel's frequency and mass are primarily analyzed. Based on this data, a surrogate model (performance evaluation model) is constructed to provide a model capable of predicting performance when a concept design image is input. Additionally, a CLIP model (style evaluation model) is utilized to assess the similarity between style keywords and product designs. Finally, by performing a combined engineering performance evaluation and style evaluation of the product design's shape, designs can be filtered based on either style or engineering performance, providing the effect of searching for a desired design from the filtered options.

[0283] FIG. 36 is a conceptual diagram of a stable diffusion model with dual (Style, Engineering Performance) constraints applied according to another embodiment of the present invention.

[0284] As part of the rendering model (RM) of the S100 process or the design exploration model (300) of the S300 process, a process of generating and optimizing a design by simultaneously considering the designer's intent and engineering performance is illustrated.

[0285] Referring to FIG. 36, the computing device (1000) can receive a reference design containing a realistic style and a text prompt such as 'A photo of car wheel'. The reference design is converted into a reference latent vector through an encoder.

[0286] The Denoising U-Net of the stable diffusion model takes a noisy latent vector and a text prompt as input, predicts the noise, and generates a predicted latent vector for the original image.

[0287] In this process, the predicted latent vector is input into a loss function that evaluates two constraints. Here, the similarity loss reflects the designer's style intent by evaluating how similar the predicted latent vector is to the original reference latent vector. Additionally, the engineering loss evaluates the engineering performance (e.g., mass, stiffness, frequency, etc.) of the predicted latent vector in real time using a surrogate model pre-trained in processes such as S324.

[0288] The two loss values ​​mentioned above are combined into a total loss to guide the prediction process of the Denoising U-Net. In other words, Generative AI goes beyond simply generating images to output an optimized Generated Design that resembles the designer's style while simultaneously satisfying target engineering performance (e.g., weight minimization, frequency maximization). This allows the designer's creative intent and the engineer's engineering requirements to be considered simultaneously within a single generation process.

[0289] The computing device can calculate the engineering loss and style loss constituting the total loss based on a predetermined method. The engineering loss can be calculated as a weighted sum of Objective 1 and Objective 2. Here, Objective 1 represents a term that minimizes the mass predicted by the surrogate model, and Objective 2 represents a term that induces the predicted frequency to be greater than a set minimum frequency. The style loss is calculated based on the cosine similarity between the predicted latent vector and the latent vector of the reference style, and by minimizing this loss, the generated design follows the style of the reference design. These loss values ​​are used in the gradient calculation of Fig. 36 to influence the direction and degree of correction of the generated image.

[0290] FIG. 37 is a detailed structural diagram of a surrogate model according to an embodiment of the present invention. It shows in detail the process of predicting scalar values ​​corresponding to the engineering performance of a product by receiving 2D image and 2.5D depth information as input.

[0291] Specifically, a 2D image is input into a pre-trained depth estimator to generate a predicted depth image containing depth information of the product. At this time, the predicted depth image can be used as 2.5D information.

[0292] The original 2D image (RGB) and the 2.5D predicted depth image can be combined through a modified input channel and input into a backbone network (ResNet34). Feature vectors passed through the backbone network and multiple fully connected layers are finally output as scalar values, which are key engineering performance indicators such as the 3D model mass of the product, the first-order rim shape mode, and the first-order spoke shape mode. At this time, the surrogate model can be used to calculate the engineering loss described in Fig. 36 in real time.

[0293] The aforementioned surrogate model acts as a key element in calculating engineering loss within the first stage stable diffusion model, which can guide generative AI to understand engineering constraints and generate an optimized design.

[0294] In the denoising process of the diffusion model, the predicted noise (epsilon) can be updated by reflecting the gradient calculated by the total loss of these two loss functions. Through this, it can go beyond simply removing noise and play a role in modifying the image to optimize the design toward engineering goals while maintaining style.

[0295]

[0296] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0297] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

1. A generative AI-based co-design execution method for industrial designers and engineers, performed by a computing device, A process of inputting a reference design for a product into a topology optimization model to obtain an output topology optimization design; A generative AI-based co-design method for industrial designers and engineers, comprising the process of inputting the above-mentioned topology optimization design into a rendering model to obtain an output rendering image.

2. In Paragraph 1, A process of generating a topology-optimized design according to topology optimization conditions by applying topology optimization to the above reference design; A generative AI-based co-design method for industrial designers and engineers, further comprising the process of training the topology optimization model with training data configured as inputs for the above reference design and topology optimization conditions, and configured as outputs for the above topology optimization design.

3. In Paragraph 1, The process of generating a topology-optimized design according to topology optimization conditions by applying topology optimization to the above reference design is, A generative AI-based co-design method for industrial designers and engineers, characterized by applying piece topology optimization after carving the above reference design and generating the above topology optimized design through rotation post-processing.

4. In Paragraph 1, A generative AI-based co-design method for industrial designers and engineers, characterized in that the topology optimization model includes a pre-trained encoder and decoder and a U-net located between the encoder and decoder, and during the training process of the topology optimization model, only the U-net is trained and updated.

5. In Paragraph 1, A generative AI-based co-design method for industrial designers and engineers, characterized in that the rendering model is a pre-trained artificial intelligence model that receives the topology optimization design and style text for the product as input and outputs a realistic rendering image of the product.

6. In Paragraph 1, A process of inputting a rendering design image of a product into a depth model to obtain an output depth map image; A generative AI-based co-design method for industrial designers and engineers, further comprising the process of inputting the depth map image into a 3D shape reconstruction model to obtain an output 3D mesh shape model.

7. In Paragraph 6, A process of acquiring the depth map image paired with the rendering design image above; The process of augmenting the above-mentioned paired rendering design image and depth map image; A process of preprocessing the above-mentioned augmented rendering design image and depth map image; The above-mentioned preprocessed rendering design image is configured as input, and the above-mentioned preprocessed A generative AI-based co-design method for industrial designers and engineers, further comprising the process of training the depth model with training data configured with a depth map image as output.

8. In Paragraph 7, A generative AI-based co-design method for industrial designers and engineers, characterized in that the depth model is a Marigold model based on a stable diffusion architecture fine-tuned using the preprocessed rendering design image and the preprocessed depth map image as training data.

9. In Paragraph 6, The process of inputting the above depth map image into a 3D shape reconstruction model to obtain the output 3D mesh shape modeling is, If the above product has a rotationally symmetric structure with respect to a central axis, the process of obtaining a first region point of the above product located parallel to a plane perpendicular to the central axis from the depth map image; A process of acquiring a second area point of the product located parallel to the central axis; A process of obtaining a total area point for the product by combining the first area point and the second area point; A generative AI-based co-design method for industrial designers and engineers, comprising the process of obtaining a three-dimensional mesh shape model of the product based on the entire area points.

10. A generative AI-based co-design execution device for industrial designers and engineers comprising a processor, wherein the processor, A process of inputting a reference design for a product into a topology optimization model to obtain an output topology optimization design; A generative AI-based co-design execution device for industrial designers and engineers, comprising the process of inputting the above-mentioned topology optimization design into a rendering model to obtain an output rendering image.

Images