Clothing simulation method and device

Abstract

A clothing simulation method and device are disclosed. The clothing simulation method comprises: obtaining appearance information of clothing; predicting a pattern parameter corresponding to the clothing from the appearance information of the clothing; and generating one or more clothing patterns on the basis of the pattern parameter.

Description

Costume simulation method and device

[0001] The following embodiments relate to a clothing simulation method and apparatus.

[0002] Clothing can be expressed in various forms depending on the characteristics and style of the fabric. Clothing trends can change over time and across regions, and designing and / or developing new garments that align with these trends can require significant time and effort from the designer. Therefore, various methods are being considered to facilitate the design and / or development of clothing.

[0003] According to one embodiment, a garment simulation method comprises: a step of obtaining external shape information of a garment; a step of predicting a pattern parameter corresponding to the garment from the external shape information of the garment; and a step of generating one or more garment patterns based on the pattern parameter.

[0004] The above pattern parameter may include correlations between multiple points corresponding to the appearance of one or more of the garment patterns.

[0005] The step of predicting the pattern parameters may include the step of predicting the pattern parameters using a first neural network model trained to predict the corresponding pattern parameters corresponding to each of the appearance information of a plurality of garments.

[0006] The first neural network model above may be trained using pattern parameters corresponding to clothing patterns included in the training data and appearance information of the clothing simulated by the clothing patterns.

[0007] The step of predicting the pattern parameters may include: a step of classifying one or more types of clothing patterns corresponding to the appearance information of the clothing according to the category of the clothing; and a step of predicting pattern parameters according to the one or more types of clothing patterns.

[0008] The step of generating the above-mentioned garment pattern may include: a step of extracting at least one first parameter corresponding to an individual garment pattern from the pattern parameters; and a step of determining the relative positions of a plurality of points corresponding to the shape of the individual garment pattern based on the first parameter.

[0009] The above correlation may include the distance between the above multiple points.

[0010] The above clothing simulation method may further include the step of obtaining a vector corresponding to at least one of the category of the clothing or the type of the clothing. The step of predicting the pattern parameters may include the step of generating parameters of a clothing pattern corresponding to at least one of the category or the type based on the vector and the pattern parameters.

[0011] The step of predicting the pattern parameters may further include the step of predicting whether at least some of the one or more garment patterns are generated from the appearance information of the garment.

[0012] The above clothing simulation method further includes the step of obtaining the body size of a user or avatar, and the step of predicting the pattern parameter may include the step of predicting the pattern parameter based further on the body size.

[0013] The appearance information of the above-mentioned garment may include at least one image or prompt among a technical drawing image, which is an image rendering the result of a simulation of wearing one or more of the above-mentioned garment patterns, a sketch image of the above-mentioned garment, a captured image of the above-mentioned garment, a depth image of the above-mentioned garment, or an image of the above-mentioned garment being worn by an avatar or a person.

[0014] The step of acquiring the above-mentioned external shape information may include the step of acquiring a prompt corresponding to the above-mentioned external shape information; and the step of converting the prompt into an image feature by a diffusion model.

[0015] The above clothing simulation method may further include the step of generating a first texture image corresponding to one or more clothing patterns from the external shape information of the clothing; and the step of generating the clothing with the texture reflected by reflecting the first texture image to the one or more clothing patterns or the clothing.

[0016] The above clothing simulation method may further include the step of obtaining a connection relationship between at least some of the points corresponding to the shape of a first clothing pattern among the one or more clothing patterns and at least some of the points corresponding to the shape of a second clothing pattern among the one or more clothing patterns; and the step of determining a sewing relationship between the first clothing pattern and the second clothing pattern based on the connection relationship.

[0017] The above-described garment simulation method further includes the step of generating an image in which at least one of the shape, texture, or color of the garment is modified, and the step of predicting the pattern parameter may include the step of predicting the pattern parameter from the modified image.

[0018] The above clothing simulation method may further include the step of generating a second texture image from the modified image; and the step of generating the clothing with the texture reflected by reflecting the second texture image on one or more clothing patterns or the clothing.

[0019] The step of generating the second texture image may include at least one of the following: generating a second texture image including at least one of a raw texture image generated by cropping a portion of the transformed image or a material map based on physically based rendering (PBR) when the texture is included in the transformed image; or generating the second texture image by applying the characteristics of the fabric classified in the transformed image to the texture of the transformed image.

[0020] The step of generating the modified image may include: converting the external shape information of the garment into a canny image in the form of a canny edge; and inputting the canny image into a diffusion model to generate the modified image.

[0021] The above clothing simulation method may further include the step of simulating the wearing of one or more generated clothing patterns on an avatar.

[0022] According to one embodiment, an electronic device comprises one or more processors including a processing circuit; and one or more memories for storing instructions, wherein when the instructions are executed individually or integrally by the one or more processors, the electronic device is configured to acquire appearance information of a garment, predict pattern parameters of one or more garment patterns from the appearance information of the garment, and generate one or more garment patterns based on the pattern parameters.

[0023] FIG. 1 is a diagram illustrating a clothing simulation method according to one embodiment.

[0024] FIG. 2 is a flowchart illustrating a clothing simulation method according to one embodiment.

[0025] FIG. 3 is a drawing for explaining the type of external shape information of a garment according to one embodiment.

[0026] FIG. 4a is a diagram illustrating the operation of a first neural network model according to one embodiment.

[0027] FIG. 4b is a diagram illustrating the structure of a first neural network model according to one embodiment.

[0028] FIG. 5 is a diagram illustrating key points according to one embodiment.

[0029] FIG. 6 is a diagram illustrating a method for extracting key points using sewing relationships according to one embodiment.

[0030] FIG. 7 is a diagram illustrating a method for extracting key points from an image of a clothing pattern according to one embodiment.

[0031] FIG. 8a is a diagram for explaining pattern parameters calculated based on key points according to one embodiment.

[0032] FIG. 8b is a diagram illustrating a method for training a neural network model regardless of the category of clothing according to one embodiment.

[0033] FIG. 8c is a diagram illustrating a method in which a neural network model is trained differently for each category of clothing according to one embodiment.

[0034] FIG. 9 is a diagram illustrating a method for normalizing shape data of a clothing pattern according to one embodiment.

[0035] FIG. 10a is a diagram illustrating a method for generating a clothing pattern based on pattern parameters when the two-dimensional point list and the pattern parameters have a linear relationship according to one embodiment.

[0036] FIG. 10b is a diagram illustrating a method for generating a clothing pattern based on pattern parameters when the two-dimensional point list and pattern parameters have a non-linear relationship according to one embodiment.

[0037] FIG. 11 illustrates a diagram for explaining the relationship between pattern parameters and key points according to one embodiment.

[0038] FIG. 12 is a flowchart illustrating a clothing simulation method reflecting a first texture image according to one embodiment.

[0039] FIG. 13 is a drawing for explaining a method of generating a three-dimensional garment from a prompt according to one embodiment.

[0040] FIG. 14 is a flowchart illustrating a clothing simulation method for generating a three-dimensional clothing from a prompt according to one embodiment.

[0041] FIG. 15 is a diagram illustrating a method for converting a prompt into an input image according to one embodiment.

[0042] FIG. 16 is a drawing for explaining a method of generating a three-dimensional garment by an image of a garment modified according to one embodiment.

[0043] FIG. 17 is a drawing for explaining a method of generating an image of a modified garment according to one embodiment.

[0044] FIG. 18 is a drawing for explaining a method of generating a second texture image by deforming a garment according to one embodiment.

[0045] FIG. 19a is a drawing for explaining a method of generating a second texture image from a modified image according to one embodiment.

[0046] FIG. 19b is a drawing for explaining a method of generating a second texture image from a modified image according to one embodiment.

[0047] FIG. 19c is a drawing for explaining a method of generating a second texture image using a texture generator according to one embodiment.

[0048] FIG. 20 is a drawing for explaining a method of generating a texture image according to one embodiment.

[0049] FIG. 21 is a drawing for explaining a clothing simulation method according to one embodiment.

[0050] FIG. 22 is a diagram illustrating a method for predicting pattern parameters based on a user's body size according to one embodiment.

[0051] FIG. 23 is a drawing for explaining a method of generating a three-dimensional garment by reflecting a second texture image generated based on the type of fabric classified from a modified image according to one embodiment.

[0052] FIG. 24 is a drawing for explaining a method of classifying fabric types in a modified image according to one embodiment.

[0053] FIG. 25 is a drawing for explaining a method of generating a second texture image from an image modified based on the type of fabric classified according to one embodiment.

[0054] FIG. 26 is a diagram for explaining the process of wearing a 3D garment on an avatar, which is generated by a second texture image based on the type of fabric classified in the appearance information of the garment according to one embodiment.

[0055] FIG. 27 is a block diagram of an electronic device for performing a clothing simulation according to one embodiment.

[0056] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, actual implementations are not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or substitutions included in the technical concept described by the embodiments.

[0057] In relation to the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of the noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise.

[0058] In this document, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0059] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish a component from another component and do not limit the components in other aspects (e.g., importance or order). For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0060] Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0061] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, terms such as "comprising" or "having" are intended to specify the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0062] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0063] As used in this specification, the term "clothing" may refer to a unit used by a fashion company when designing. For example, if an avatar is wearing a top, bottoms (skirt or pants), a scarf, a bag, and socks, each of the top, bottoms, scarf, bag, and socks may correspond to clothing. Clothing may be understood to encompass not only garments but also all clothing-related items that can be worn on the body, such as accessories, bags, shoes, etc.

[0064] As used in this specification, the term "avatar" may refer to any type of three-dimensional object that serves as the object for which clothing is worn or placed within a virtual space. This is not limited to typical human figures and can be defined as a broad concept that includes body forms of various sizes, mannequins, torsos, as well as biological forms or abstract geometric structures. Accordingly, "avatar" should be broadly interpreted to mean any three-dimensional mesh or geometric shape having a surface capable of physically interacting with virtual clothing.

[0065] 'Clothing patterns' may be 2D patterns corresponding to each body part that constitutes a 3D garment. The 2D patterns may be virtual 2D patterns modeled as a set of multiple polygons for the simulation of the 3D garment. The 2D patterns include multiple pattern pieces, and each of the multiple pattern pieces may be modeled as a polygonal mesh based, for example, on the body shape of a 3D avatar. The polygonal mesh may include multiple polygons (e.g., triangles or squares, etc.).

[0066] In one embodiment, the garment pattern may be composed of a mesh containing multiple polygons. Depending on the embodiments, the mesh may be modeled in various ways. For example, the vertices of the polygons included in the mesh may be point masses having mass, and the sides of the polygons may be represented as springs having elasticity connecting that mass. Accordingly, the three-dimensional garment may be modeled, for example, by a Mass-Spring Model. Depending on the physical properties of the fabric used, the springs may have resistance values ​​for, for example, stretch, shear, and bending. Alternatively, the mesh may be modeled by a strain model. The polygons included in the mesh may be modeled, for example, as triangles, or as polygons of quadrilateral or larger. In some cases, when a 3D volume needs to be modeled, the mesh can be modeled as a 3D polyhedron.

[0067] The vertices of the polygon(s) contained in the mesh can move due to external forces, such as gravity, and internal forces, such as stretch, shear, and bending. By calculating the external and internal forces to determine the force applied to each vertex, the displacement velocity and motion of each vertex can be obtained. The movement of the garment can be simulated through the movement of the vertices of the polygon(s) constituting the mesh in each time motion. For example, when a garment composed of a polygon mesh is worn on a 3D avatar, a natural 3D virtual garment based on the laws of physics can be realized. The vertices of the polygon(s) contained in the mesh can move according to the action of external forces, such as gravity, and internal forces such as stretch, shear, and bending. By calculating external and internal forces to determine the force applied to each vertex, the displacement and velocity of movement of each vertex can be obtained. Additionally, the movement of virtual clothing can be simulated through the movement of the vertices of the mesh's polygons at each time step. By fitting a 2D pattern composed of polygonal meshes onto a 3D avatar, a natural-looking 3D virtual garment based on the laws of physics can be realized.

[0068] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0069]

[0070] FIG. 1 is a diagram illustrating a clothing simulation method according to one embodiment. Referring to FIG. 1, a diagram (100) is shown illustrating the process in which an electronic device according to one embodiment receives external shape information (101) of a clothing, generates pattern parameters (130) for a clothing pattern, and generates a clothing pattern (140) from the pattern parameters (130).

[0071] The appearance information (101) is information about the appearance of the clothing, and may include, for example, text-type information, image-type information, numeric-type information, or various combinations thereof.

[0072] According to one embodiment, the appearance information (101) of the garment may refer to information describing the appearance of the garment. The appearance information (101) of the garment may be one or more. If there is one or more appearance information (101) of the garment, their types may be identical or different. The appearance information (101) of the garment may be an image type and / or a prompt type. The appearance information (101) of the garment may be at least one image among a technical drawing image, a sketch image of the garment, a captured image of the garment, a depth image of the garment, or an image of the garment worn by an avatar or a person, such as an image rendering one or more garment patterns of the garment as shown in FIG. 3 below, but is not necessarily limited thereto. The appearance information (101) of the garment may be user input information, such as a prompt or text describing the shape of the garment.

[0073] Pattern parameters (130) are data inferred (or predicted) by a neural network model (110) for the generation of a clothing pattern (140). For convenience of explanation, parameters with defined physical meanings are exemplified in FIG. 1, but as will be described later, parameters based on the correlation between feature points corresponding to the appearance of the clothing pattern (140) may be broadly included. Feature points corresponding to the appearance may be predetermined according to the clothing pattern. According to one embodiment, the number of feature points corresponding to the appearance may be predetermined according to the type of clothing pattern, and identification information may be assigned to each feature point. For example, the identification information may include information on the type of clothing pattern, information on line segments within the corresponding clothing pattern, information on sequence numbers within the corresponding line segment, etc. Feature points include points sampled along the appearance of the clothing pattern. According to an embodiment, feature points may include normalized points such as incline points and / or curve points of the corresponding clothing pattern, and points sampled in a predetermined number between the normalized points along the appearance of the clothing pattern. Alternatively, the feature points may include points sampled along the outline of the garment pattern, excluding at least some of the normalized points.

[0074] The pattern parameter (130) may be defined as the correlation (e.g., distance) between feature points. According to one embodiment, the pattern parameter (130) may include the correlation between feature points according to the type of clothing pattern. For example, a first group of feature points corresponding to a first clothing pattern may be defined, and a second group of feature points corresponding to a second clothing pattern may be defined. The pattern parameter (130) may include the correlation between feature points belonging to the first group of feature points and the correlation between feature points belonging to the second group of feature points.

[0075] In the following, feature points may be referred to as key points, and feature points and / or key points may be understood as a concept encompassing core key points and sampling points.

[0076] The electronic device can predict pattern parameters (130) corresponding to the garment based on the external shape information (101) of the garment. The electronic device can predict pattern parameters (130) using a first neural network model (110). The first neural network model (110) corresponds to an artificial neural network and may be called an "Image to parameter model" in that it predicts pattern parameters from the external shape information (101) of the garment. According to one embodiment, the first neural network model (110) may be trained to predict pattern parameters (130) according to the type of garment, or it may be trained to predict pattern parameters (130) regardless of the type of garment. The inference pipeline may vary depending on the training method of the neural network model (110), and more specific embodiments will be described later.

[0077] The first neural network model (110) may be trained using a database (DB) that includes a pair of pattern parameters (130) corresponding to a three-dimensional garment (or an avatar image wearing a three-dimensional garment) (150).

[0078] For example, given a situation in which clothing patterns included in multiple 3D garments are provided, appearance information of the garments can be generated by simulating the wearing of the clothing patterns. Additionally, pattern parameters can be extracted from the clothing patterns. Through this process, training data for a first neural network model (110) including pairs of appearance information and pattern parameters can be constructed. Alternatively, the training data may include pairs of pattern parameters extracted from actual clothing patterns and appearance information of a garment actually manufactured using those patterns. The structure and operation of the first neural network model (110) will be explained in more detail with reference to FIGS. 4a and 4b below.

[0079] According to one embodiment, an electronic device can identify the type of a given garment pattern for constructing training data. The type of garment pattern may include, for example, types and / or attributes of detailed patterns such as a front body pattern, a back body pattern, and a sleeve pattern, but is not necessarily limited thereto. The electronic device can extract key points for each type of garment pattern. For example, the electronic device may extract key points using sewing information. The electronic device may also extract key points using a neural network model that detects key points at the image level. Alternatively, the electronic device may designate points specified by a user in each garment pattern as key points. The electronic device can extract pattern parameters based on the key points. The method by which the electronic device extracts key points is described in more detail with reference to FIGS. 5 to 7 below.

[0080] The electronic device can generate a garment pattern (140) based on pattern parameters (130). A specific methodology for generating a garment pattern (140) from pattern parameters (130) will be described later.

[0081] An electronic device can generate a three-dimensional garment (150) using a garment pattern (140). For example, the electronic device can generate a three-dimensional garment (150) by simulating the garment pattern (140) being worn on an avatar. The electronic device can automatically determine the sewing relationship between the garment patterns (140). The electronic device can obtain a connection relationship between at least some of the feature points corresponding to the shape of the first garment pattern and at least some of the feature points corresponding to the shape of the second garment pattern. For example, the electronic device can determine that a connection relationship exists between some feature points of the first garment pattern and some feature points of the second garment pattern based on sewing data between the first type of garment pattern and the second type of garment pattern. The electronic device can obtain the connection relationship in a deterministic manner by utilizing a lookup table or a transformation matrix, or obtain the connection relationship in a statistical manner by utilizing a separately trained artificial neural network. The electronic device can determine the sewing relationship between the first garment pattern and the second garment pattern based on the connection relationship. The electronic device can generate a three-dimensional garment (150) based on the sewing relationship.

[0082] The electronic device may generate clothing patterns of various sizes (161, 163, 165, 167) to be worn on avatars, such that they are dependent on the different body sizes of the avatars wearing the three-dimensional clothing (150) as in the drawing (160). According to one embodiment, the electronic device may use a neural network model that takes into account different body sizes.

[0083] According to one embodiment, the electronic device may use an auto-fitting function for a clothing pattern generated based on an avatar of a specific size to fit a corresponding avatar.

[0084]

[0085] FIG. 2 is a flowchart illustrating a clothing simulation method according to one embodiment. Referring to FIG. 2, the process of generating a clothing pattern through steps (210) to (230) is illustrated by an electronic device (e.g., the electronic device (2700) of FIG. 27) that performs clothing simulation according to one embodiment.

[0086] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0087] In step (210), the electronic device obtains the appearance information of the garment. As previously described, the type of appearance information of the garment may include the various images and / or prompts mentioned above. The types of images included in the appearance information of the garment are described in more detail with reference to FIG. 3 below.

[0088] Additionally, the Garment Category may include, for example, at least one of a T-shirt, blouse, shirt, blazer, trousers, shorts, culottes, skirt, one-piece dress, dress, jacket, or coat, but is not necessarily limited thereto. In addition to clothing, the Garment Category may encompass all clothing accessories such as hats, socks, underwear, swimwear, and bags.

[0089] According to an embodiment, the electronic device may generate a three-dimensional garment by converting an input prompt into an input image. A method for the electronic device to generate a garment pattern and / or a three-dimensional garment from a prompt and a method for converting a prompt into an input image will be explained in more detail with reference to FIGS. 13 and 15 below.

[0090] In step (220), the electronic device predicts pattern parameters corresponding to the garment based on the appearance information of the garment obtained in step (210). The electronic device may predict the pattern parameters using a first neural network model trained to predict pattern parameters according to the appearance information of the garment. The first neural network model may include, for example, a previously trained ResNet50 and / or classifier, but is not necessarily limited thereto. The first neural network model may be trained using pattern parameters corresponding to garment patterns included in the training data and appearance information of the garment simulated by the corresponding garment pattern. The operation and structure of the first neural network model according to one embodiment will be explained in more detail with reference to FIGS. 4a and 4b below.

[0091] In step (220), the method by which the electronic device predicts pattern parameters is as follows.

[0092] The electronic device may input external shape information of the garment to a first neural network model and obtain pattern parameters output from the first neural network model. The pattern parameters may include data for generating one or more garment patterns. As previously described, the pattern parameters may include correlations between key points predefined for each garment pattern.

[0093] According to one embodiment, an electronic device can classify clothing categories based on external shape information of the clothing. In this case, the electronic device can predict pattern parameters using a neural network model for each clothing category. The types of clothing patterns included in the clothing may differ depending on the clothing category. The neural network model for each clothing category can be trained to predict pattern parameters corresponding to the types of clothing patterns included in the clothing of the corresponding category.

[0094] According to an embodiment, the electronic device may predict pattern parameters by modifying the type of external shape information of the garment. A method for the electronic device to predict pattern parameters by modifying the type of external shape information of the garment is explained in more detail with reference to FIGS. 16 to 18 below.

[0095] In step (220), if the body size of a user or avatar is input, the electronic device can predict pattern parameters based on the input body size. The method of the electronic device predicting pattern parameters based on the input body size is explained in more detail with reference to Fig. 22 below.

[0096] In step (230), the electronic device generates one or more clothing patterns based on the pattern parameters predicted in step (220). For example, the electronic device may generate clothing patterns based on key points of clothing patterns extracted from the pattern parameters.

[0097] For example, an electronic device can predict the locations of key points for each garment pattern from pattern parameters and generate a garment pattern based on this.

[0098] According to one embodiment, the pattern parameter may include the correlation of key points for each garment pattern. The electronic device may determine the appearance of the corresponding garment pattern by adjusting the positions of the corresponding key points based on the correlation (e.g., distance) of the key points for each garment pattern.

[0099] According to one embodiment, the operation of generating a garment pattern from pattern parameters can be formulated as a problem in which the relative positions of key points corresponding to the garment pattern are used as variables, and the variables (relative positions) are controlled in a direction that reduces the difference between the correlation calculated based on the relative positions of the key points and the correlation included in the pattern parameters. According to one embodiment, weights may be assigned to the key points for each garment pattern. The weights of the key points for each garment pattern may be referenced in the form of a lookup table or inferred by a separately trained artificial neural network. By processing the problem formulated with consideration of the weights, the electronic device can improve computational efficiency and / or accuracy.

[0100] For example, an electronic device can predict the locations of multiple sampling points. At least some of the multiple sampling points may correspond to core key points. By predicting multiple sampling points in addition to the core key points, the electronic device can accurately predict the shape of the garment pattern, especially when the distance between the core key points is curved. The electronic device can realistically depict the curved lines of the garment pattern by using sampling points sampled in a predetermined manner (e.g., evenly).

[0101] A more specific method for an electronic device to generate one or more clothing patterns based on pattern parameters is explained in more detail with reference to FIGS. 10a to 10c below. The relationship between pattern parameters and key points is explained in more detail with reference to FIG. 11 below.

[0102] According to one embodiment, a garment pattern may be generated as a parametric pattern, but is not necessarily limited thereto. A parametric pattern may refer to a pattern that includes parameters that can be directly set and / or manipulated by the user. A parametric pattern may be a pattern matched with various parameters to enable the creation of various shapes and styles by adjusting specific parameters in the garment design. A parametric pattern may be, for example, a pattern in which the positions of points serving as the basis for pattern generation are determined by pattern parameter(s) (130) representing the dimensions of each part of the garment pattern. The user can easily create various designs by changing variables such as the length, width, and curvature of the parametric pattern, for example. Additionally, when using a parametric pattern, various different styles of dresses can be created using a single garment pattern.

[0103] According to an embodiment, the electronic device may generate a three-dimensional garment using one or more garment patterns generated in step (230). The electronic device may additionally generate a texture image and generate a three-dimensional garment by reflecting the texture image. The method of generating a three-dimensional garment by the electronic device by reflecting the texture image is described in more detail with reference to FIGS. 12 and FIGS. 20 below. At this time, the texture image may be generated from the garment or may be generated from an image of the garment modified.

[0104] According to one embodiment, the electronic device may generate physical property data from the external shape information of the garment. The electronic device may provide a physical simulation of the garment by applying the physical property data to the generated garment pattern. For example, the external shape information of the garment may be external shape information in which the garment is worn on a user or avatar of a predefined body shape and / or a predefined pose. The electronic device may use an artificial neural network trained to predict physical property data from the external shape information of the garment worn on a predefined body shape and / or pose (e.g., the shape of the fabric stretched in correspondence with a specific body part).

[0105] In the following, a texture image generated based on appearance information of a garment including various types of images and / or prompts may be referred to as the "first texture image," and a texture image generated based on an image of a modified garment may be referred to as the "second texture image."

[0106] A method for an electronic device to generate a second texture image from an image of a modified garment is explained in more detail with reference to FIGS. 18 and FIGS. 19a to 19c below.

[0107] According to one embodiment, the electronic device may classify the type of fabric in a modified image and generate a three-dimensional garment by reflecting a second texture image generated based on the classified type of fabric. The method of generating a three-dimensional garment by reflecting a second texture image generated based on the type of fabric classified by the electronic device will be explained in more detail with reference to FIGS. 23 to 26 below.

[0108]

[0109] FIG. 3 is a drawing for explaining the types of images according to the external shape information of a garment according to one embodiment. Referring to FIG. 3, types of images according to the external shape information (101) of a garment according to one embodiment are illustrated.

[0110] The image according to the external shape information (101) of the garment is an image input to an electronic device, and may be, for example, a technical drawing image (310) which is an image rendering the result of a simulation of wearing one or more garment patterns, a sketch image (320) which sketches the garment, a captured image (330) which actually takes a picture of the garment, a depth image (not shown in the drawing) of the garment, and / or a captured image (340) which takes a picture of a person (or avatar) wearing the garment, but is not necessarily limited thereto. The image according to the external shape information (101) of the garment may further include a schematic image which schematically represents the garment.

[0111] Here, the ‘technical drawing image (310)’ may refer to a drawing that accurately depicts the design, structure, and details of the garment. The technical drawing image (310) may provide information necessary for the actual production process, for example, including accurate dimensions and proportions regarding the size and position of each part of the garment. The technical drawing image (310) may include detailed design elements such as pleats, buttons, zippers, etc. The technical drawing image (310) may visually represent the appearance of the garment when actually worn. The electronic device may convert the technical drawing image (310) into an image of the actual garment being worn, for example, by a diffusion simulation model and / or a physical simulation model.

[0112] In one embodiment, convenience can be provided when designing a garment by generating a garment pattern and / or a three-dimensional garment from various types of images according to the external shape information (101) of the garment.

[0113]

[0114] FIG. 4a is a diagram illustrating the operation of a first neural network model according to one embodiment. Referring to FIG. 4a, a process is illustrated in which a first neural network model (110) according to one embodiment outputs a pattern parameter (130) using one or more external shape information (101) of a garment as input.

[0115] One or more appearance information (101) of a garment may be, for example, a technical drawing image, sketch image, photographic image, depth image and / or an image of the garment being worn by a person (or avatar), a prompt, or a combination thereof, as described above through FIG. 3. For example, if the appearance information (101) of the garment is a sketch image, the electronic device may input the sketch image into the first neural network model (110). If the appearance information (101) of the garment is a color image, the electronic device may use the color image as is, or convert it into a black-and-white sketch image and use it. If the appearance information (101) of the garment is a prompt, the electronic device may use the prompt as is, or convert the prompt into image-type information and use it. The category of the garment included in the one or more appearance information (101) of the garment may be one or multiple.

[0116] The first neural network model (110) may include, for example, a garment type classifier (410) and a parameter predictor (430). For convenience of explanation, an embodiment including a garment type classifier (410) and a parameter predictor according to the garment type has been illustrated, but the first neural network model (110) may be implemented as an integrated predictor covering multiple garment types.

[0117] The first neural network model (110) can classify the category of clothing included in the appearance information (101) of clothing by the clothing type classifier (410). At this time, the category of clothing may be, for example, at least one of a T-shirt, blouse, shirt, blazer, pants, shorts, skirt-pants, skirt, one-piece dress, dress, jacket, or coat, or a combination thereof.

[0118] The first neural network model (110) can predict pattern parameters (130) corresponding to the clothing for each category of clothing classified by the clothing type classifier (410). The electronic device can predict pattern parameters (130) for each category of clothing classified by the clothing type classifier (410) using a parameter predictor (430), which is a pre-trained model. The parameter predictor (430) may include a T-shirt parameter predictor, a skirt parameter predictor, and a pants parameter predictor for each category of clothing, and may further include parameter predictors for other categories of clothing.

[0119] The first neural network model (110) can be trained using pair data between a set of parameters of clothing patterns used in the clothing and appearance information of an image in which the clothing is worn. At this time, the appearance information of the clothing may be, for example, a sketch image, a color image, or a prompt.

[0120] The first neural network model (110) may be a neural network model in the form of a Multi-Layer Perceptron (MLP) that is composed of, for example, an input layer, a hidden layer, and an output layer, and processes inputs through an activation function in each layer, but is not necessarily limited thereto.

[0121]

[0122] FIG. 4b is a diagram illustrating the structure of a first neural network model according to one embodiment. Referring to FIG. 4b, an example of the structure of a first neural network model (110) for predicting pattern parameters (130) from the external shape information (101) of a garment according to one embodiment is illustrated. In FIG. 4b, the external shape information (101) of the garment may be, for example, a technical drawing image, but is not necessarily limited thereto and may be other types of information such as a prompt.

[0123] The first neural network model (110) may include, for example, a backbone network (e.g., ResNet50 (450)), a style classifier (460), and a parameter predictor (470).

[0124] The backbone network primarily performs feature extraction, and an additional layer (e.g., a style classifier (460)) that performs final classification based on the extracted features may be connected afterwards. For example, a model such as ResNet50 (450) can be used as a backbone network in image classification tasks. ResNet50 (450) can extract important features from the appearance information of the clothing and pass the extracted features to the style classifier (460) to perform final classification for each type of clothing. Through this process, complex image data can be processed effectively and classification results with high accuracy can be obtained.

[0125] ResNet50 (450) is a type of residual network that uses skip connections and is a deep learning model consisting of 50 layers. A skip connection is a connection that skips layers of the network, and through the skip connection, the vanishing gradient problem can be solved and deeper networks can be effectively trained. ResNet50 (450) can be used in various computer vision tasks such as image recognition, object detection, and feature extraction, for example.

[0126] The style classifier (460) can classify the category and / or style of the clothing included in the appearance information (101) of the clothing, just like the clothing type classifier (410) of FIG. 4a. The clothing styles may include, for example, Classic style, Minimal style, Feminine style, Vintage style, Hippie style, Retro style, etc., but are not necessarily limited thereto.

[0127] The first neural network model (110) can predict pattern parameters (130) for each category (or style) of clothing classified by the style classifier (460). The electronic device can predict pattern parameters (130) for each category (style) of clothing classified by the style classifier (460) using a parameter predictor (470), which is a pre-trained model. The parameter predictor (470) may be the parameter predictor (430) described above through FIG. 4a. At this time, the dataset for the first neural network model (110) may be pair data of a technical drawing image, which is the external shape information (101) of the clothing, and pattern parameters (130).

[0128] According to an embodiment, the electronic device may transform the external shape information (101) of a garment (e.g., a technical drawing image) into external shape information of a different form of garment by a diffusion model, and learn a first neural network model (110) by the transformed image.

[0129] The parameter predictor (470) may analyze the two-dimensional shape of the clothing pattern to extract key points and set the distances between the key points as pattern parameters (130). Refer to FIGS. 5 to 7 below for a method of extracting key points.

[0130]

[0131] FIG. 5 is a diagram illustrating key points according to one embodiment. Referring to FIG. 5, key points extracted from clothing patterns (501, 503, 505) according to one embodiment are shown.

[0132] According to one embodiment, an electronic device may extract corner point(s) from one or more garment patterns according to one or more types of garment patterns for the construction of training data. The electronic device may label the corner points through sewing matching. An index may be assigned to the corner points by the labeling. The electronic device may extract key points by uniformly sampling the labeled corner points for each edge segment of one or more garment patterns. For example, the electronic device may acquire and / or extract key points from an image of an actual garment pattern, or may acquire and / or extract key points from digital pattern data.

[0133] According to one embodiment, the electronic device can identify one or more types of clothing patterns corresponding to the appearance information of the clothing for each category of clothing. The electronic device can extract key points for each of the one or more classified types of clothing patterns.

[0134] The garment patterns (501) may be the front or back pattern of the garment top. The garment pattern (503) may be a sleeve pattern. The garment pattern (505) may be a cuff pattern corresponding to the sleeve end decoration or sleeve end pattern of a top, such as a dress shirt or blouse, for example.

[0135] The key points indicated by circled letters in Fig. 5 may refer to feature points used when recognizing or tracking each part of a clothing pattern. A feature point refers to a point that is easily identifiable in a clothing pattern and may mainly correspond to the edge, corner point, or unique part of the clothing pattern (e.g., a part that deviates from the general pattern shape). The feature points may correspond to points on the clothing pattern that correspond to key locations of the 3D avatar when the 3D clothing is fitted onto the 3D avatar. For example, the feature points may correspond to points on the clothing pattern that correspond to at least one of the following: both arms, both wrists, left and right torsos, both shoulders, head, neck, both legs, left and right lower body, both ankles, armpits, groin, pelvis, buttocks, abdomen, chest, both hands, both feet, both elbows, both knees, both fingertips, between the fingers, both backs of the hands, both insteps, both toes, or both heels of the 3D avatar.

[0136] The key points of the clothing patterns (501, 503, 505) may consist of an appropriate number of points to express the characteristics of the pattern. There may be 9 key points in the clothing pattern (501), ranging from 0 to 8. There may be 6 key points in the clothing pattern (503), ranging from 0 to 5. Additionally, there may be 4 key points in the clothing pattern (505), ranging from 0 to 3.

[0137] For example, in the front (or back) pattern (501) of the top, key points may include, for example, the side neck point, the shoulder point, the end point of the armhole, etc. In the sleeve pattern (503), key points may include, for example, the sleeve cap, the end point of the sleeve cap, the two end points of the bottom of the sleeve, etc. In the cuff pattern (505), key points may include the four corners of the square of the pattern, etc. In this way, key points may include points that serve as references for parameters when converting the garment pattern into parameters.

[0138] The key points illustrated in FIG. 5 may be referred to as core key points, and to extract pattern parameters, the electronic device may additionally sample points between the core key points. The pattern parameters may include the correlation (e.g., distance) between key points including the core key points and the sampling points.

[0139]

[0140] FIG. 6 is a diagram illustrating a method for extracting key points using sewing relationships according to one embodiment. Referring to FIG. 6, an electronic device according to one embodiment finds a sewing-possible pattern (630) for each line segment through sewing matching from key points (610) of a garment pattern, and a diagram (600) showing the result (650) of extracting key points from the sewing-possible pattern (630) according to the sewing relationships of the pattern is shown.

[0141] The electronic device can select one or more clothing patterns as target patterns according to the type of one or more clothing patterns, and can search for and extract corner points indicated by original characters from the target patterns. Here, 'target pattern' may refer to a clothing pattern from which key points (610) are to be extracted.

[0142] In FIG. 6, each of the corner points may be labeled with an index corresponding to a corresponding body part (e.g., Body Back, Body Front, Body Side, Arm, Shoulder, Neck, Head) or empty space. At this time, each of the corner points may be labeled with, for example, a circular character for sewing matching (e.g., , , .. , ) or different colors may be displayed.

[0143] Since the deformation forms of the garment can be very diverse, in one embodiment, corner points representing a representative form or shape to be parameterized in the garment or garment pattern can be extracted.

[0144] The electronic device may, for example, select a garment pattern placed at a placement point disclosed in Patent Registration Publication No. 2014-0108451, filed by the applicant and registered on July 16, 2020, as the target pattern. For example, in the case of a front pattern, the electronic device may select a garment pattern placed at a placement point on the front of the body panel (Body_Front) as the target pattern.

[0145] The electronic device can search for corner points in a clothing pattern, for example, by setting points where the cosine value is greater than 0.9 based on left and right points. In this case, if the clothing pattern is a left-right symmetrical pattern as shown in FIG. 6, the electronic device can extract corner points for only half of the clothing pattern, including corner points that are symmetrical to each other.

[0146] The electronic device can extract key points by assigning an index to each corner point through sewing relationships. The electronic device can label corner points through sewing matching for one or more garment patterns. For example, the electronic device can determine all possible combinations of garment patterns that can be sewn together for each line segment after selecting the types of patterns that can be sewn together for each line segment based on the front pattern. The electronic device can extract key points by matching the sewing relationships of the pattern data corresponding to each garment pattern to determine which corner points each pattern's line segment is composed of. The corner points can be labeled with patterns (630) that can be sewn together for each line segment.

[0147] Additionally, the electronic device may construct a dataset in which the shape information of the garment and pattern parameters are defined in pairs for training the key point predictors (740, 750, 760) described later through FIG. 7 below. The electronic device may generate the dataset by extracting key points by sampling corner points labeled by edge segments of one or more garment patterns and points between corner points in a predetermined manner (e.g., uniformly between corner points). Here, 'edge segment' may refer to a part of the garment pattern corresponding to a corner point where the cosine value is greater than 0.9 based on an edge or left / right point.

[0148] The electronic device can represent the shape of the garment as an array of two-dimensional positions of key points as shown in the drawing (610). At this time, the key points may include a core key point and key points.

[0149] After extracting core key points, the electronic device can extract edge key points that have equal spacing between the extracted core key points.

[0150] The numbers listed in the drawing (630) (e.g., '1', '5', '10', '15') may be values ​​that are empirically selected to determine how many key points are extracted from the edges.

[0151] Drawing (650) is a drawing showing key points extracted from various types of top garments corresponding to the garment pattern shown in drawing (610). For example, since the top garment is not a swimsuit, core key points 0 and 1 in drawing (630) may exist in the same location. Also, in the case where the sleeve shape is raglan, since there is no shoulder line, core key points 4 and 5 may exist in the same location.

[0152] The electronic device can calculate the two-dimensional positional relationship of key points using pattern parameters.

[0153]

[0154] FIG. 7 is a diagram illustrating a method for extracting key points from an image of a clothing pattern according to one embodiment. Referring to FIG. 7, a diagram (700) is shown illustrating the process of an electronic device according to one embodiment extracting key points (747, 757, 767) from a pattern image (710). The pattern image (710) may be, for example, a two-dimensional image of an actual clothing pattern, but is not necessarily limited thereto and may be other types of information such as digital pattern data.

[0155] The key point predictors (740, 750, 760) illustrated in FIG. 7 may be neural networks trained to predict core key points of corresponding clothing patterns according to the type of clothing. The core key points of clothing patterns can be understood as an example of a pattern parameter (130 in FIG. 1).

[0156] The electronic device can classify the type of pattern image (701) by inputting external information of the garment, such as the pattern image (710), into the neural network (701). The types of the pattern image (710) may include, for example, a T-shirt, a blouse, a shirt, a blazer, pants, shorts, skirt-pants, a skirt, a dress, a jacket, and a cord, but are not necessarily limited thereto.

[0157] At this time, the neural network (701) may include a backbone network (e.g., ResNet50 (720)) and a pattern type classifier (730). ResNet50 (720) can extract features from a pattern image (710). The pattern type classifier (730) can classify the type of the pattern image (710) as one of a T-shirt, a skirt, or pants from the features extracted by ResNet50 (720).

[0158] The electronic device can input the classified type of the pattern image (710) into a key point predictor (740) of the corresponding type to extract key points (747) corresponding to each pattern. For example, if the classified type of the pattern image (710) is a T-shirt, the electronic device can input the pattern image (710) into a key point predictor (740) corresponding to the T-shirt to extract key points (747) corresponding to the T-shirt (each pattern of the T-shirt). The key point predictor (740) includes, for example, a front body keypoint predictor, a back body keypoint predictor, and a sleeve keypoint predictor, and can extract key points (747) corresponding to each pattern constituting the T-shirt.

[0159] Alternatively, if the classified type of the pattern image (710) is a skirt, the electronic device may input the pattern image (710) into a key point predictor (750) corresponding to the skirt to extract key points of the skirt. The key point predictor (750) corresponding to the skirt includes, for example, a front body keypoint predictor and a back body keypoint predictor, and can extract key points (757) corresponding to each pattern constituting the skirt.

[0160] Alternatively, if the classified type of the pattern image (710) is pants, the electronic device can input the pattern image (710) into a key point predictor (760) corresponding to the pants to extract the key points (767) of the pants.

[0161] At this time, the key point predictors (740, 750, 760) corresponding to each type of the pattern image (710) can be trained by a dataset in which, for example, the appearance information of the garment and core key points corresponding to the appearance of the garment pattern are defined as pairs. At this time, the dataset can be generated by, for example, key points extracted based on the sewing information of the garment pattern, and / or key points manually extracted by the user from the garment pattern or pattern image. For example, if there is no sewing information in the garment pattern, that is, if the garment pattern is not sewn with another garment pattern, key points directly extracted (set) by the user from the pattern image can be used as a dataset for training the key point predictors (740, 750, 760). The key point predictors (740, 750, 760) can be trained by the dataset to extract key points from the pattern image (710).

[0162] According to one embodiment, the electronic device can extract core key points of an input clothing pattern using a key point predictor (740, 750, 760) and build training data by sampling between the core key points a predetermined number corresponding to the clothing pattern.

[0163]

[0164] FIG. 8a is a diagram for explaining pattern parameters calculated based on key points according to one embodiment.

[0165] Referring to FIG. 8a, a normalized pattern parameter (810, 820) calculated based on the distance between key points (831) in a garment pattern (e.g., front pattern of a top (801)) according to one embodiment, and a diagram (830) for explaining the relationship between the normalized pattern parameter (810, 820) and the key points (831, 833, 835) are shown. For example, in order to indicate the front length in the front pattern of a top (801), the length of the line (810) connecting the side neck point and the bottom point corresponding to the key point can be normalized. Additionally, in order to indicate the length across the shoulder in the front pattern of a top (801), the length of the line (820) connecting the two end points of the shoulder corresponding to the key point in the front pattern of a top (801) can be normalized.

[0166] As previously explained, the electronic device can calculate the length (or distance value) of a line (837) formed by connecting any key points (833, 835) included in the clothing pattern shown on the right side of the drawing (830) as pattern parameters, in other words, as various values ​​that can be expressed as connection relationships between key points (831) of the clothing pattern shown on the left side of the drawing (830) as well as normalized pattern parameters (810, 820).

[0167] In other words, pattern parameters may include not only parameters in which specific physical meanings (e.g., front length, across shoulder, etc.) are defined, such as the front pattern (801) of Fig. 8a, but also parameters in which specific physical meanings are not defined and the data itself has meaning (e.g., data-driven parameters), such as the garment pattern of Fig. 830. Data-driven parameters may include correlations between multiple points corresponding to the shape of the garment pattern. For example, the correlation may include distances between multiple points.

[0168] The electronic device may parameterize all possible pairs of key points (831) of the clothing pattern illustrated on the left side of the drawing (830). Data-driven parameters may include parameters in which physical meaning is defined.

[0169] In one embodiment, as shown in FIG. 8b or FIG. 8c below, data-driven parameters can be predicted through an artificial neural network, and then clothing patterns can be generated based on the data-driven parameters.

[0170] Hereinafter, with reference to FIG. 8b, an example of a neural network model being trained regardless of the category of clothing is described, and with reference to FIG. 8c, an example of a neural network model being trained differently for each category of clothing is described.

[0171]

[0172] FIG. 8b is a diagram illustrating a method for training a neural network model regardless of the category of clothing according to one embodiment.

[0173] An electronic device according to one embodiment may obtain pattern parameters (856) from a parameter predictor (855) based on the external shape information of the garment. The electronic device may extract or generate parameters (858) of a garment pattern corresponding to the category and / or type of the garment from the pattern parameters (856) based on a vector (857) embedding the category and / or type of the garment.

[0174] For example, when the electronic device receives external information of a garment (e.g., an image (851) and / or a prompt (850)), it can transform the external information of the garment (e.g., an image (851) and / or a prompt (850)) into another type of external information (e.g., a schematic of the garment (853)) by a diffusion model (852). The electronic device can input the schematic of the garment (853) transformed by the diffusion model (852) into a neural network model (e.g., a parameter predictor (855)).

[0175] The parameter predictor (855) can predict data-driven parameters (856) based on a schematic diagram (853) of the garment. At this time, the electronic device can generate or extract pattern parameters (858) corresponding to the category and / or type of the garment among the data-driven parameters (856) using a vector (857) mapped to the category and / or type (850) of the garment. The prompt (850) may include, for example, a garment category (e.g., T-Shirt) and / or a garment type (e.g., Set-In indicating a general type with flat sleeve ends in the garment), but is not necessarily limited thereto.

[0176] The category and / or type (850) of the clothing may be information obtained by user input such as a prompt, or information generated from other types of appearance information such as an image.

[0177] The vector (857) may be, for example, a one-hot vector representing a parameter used corresponding to a category and / or type of clothing, but is not necessarily limited thereto. The electronic device may derive a pattern parameter (858), which is a T-shirt set-in parameter, from a data-based parameter (856) by the vector (857).

[0178] According to an embodiment, the electronic device may further include a neural network model that generates pattern parameters (858) from data-based parameters (856) and vectors (857).

[0179] Additionally, the parameter predictor (855) can further predict from the schematic diagram (853) of the garment whether various patterns, such as sleeves, exist, that is, whether the garment includes any pattern. Based on the prediction result regarding the existence of the pattern, the electronic device can determine whether to generate the corresponding pattern during the pattern generation process.

[0180] The parameter predictor (855) can predict not only the data-based parameters (856) but also what pattern(s) should be created according to the appearance of the input clothing, and output the generated pattern based on the prediction results. For example, the parameter predictor (855) can output a probability value between 0 and 1 corresponding to each of the possible candidate patterns.

[0181] According to an embodiment, the electronic device may receive additional input of the body size (854) of the user or avatar along with external shape information of the clothing (e.g., image (851)). The parameter predictor (855) may predict data-based parameters (856) from shape data of one or more clothing patterns generated based on the body size (854). In this case, the parameter predictor (855) may be a neural network model trained to predict pattern parameters (858) by category of clothing and / or type of clothing according to the body size (854).

[0182] The electronic device can generate a clothing pattern based on pattern parameters (858).

[0183]

[0184] FIG. 8c is a diagram illustrating a method in which a neural network model is trained differently for each category of clothing according to one embodiment.

[0185] Unlike in FIG. 8b, in FIG. 8c, the electronic device may include an artificial neural network trained differently for each category of clothing (e.g., a trouser / skirt parameter predictor (874) and a T-shirt parameter predictor (877)). In this case as well, a vector (881, 882) corresponding to the sub-category and / or sub-type of each clothing may be used.

[0186] According to one embodiment, when external shape information of a garment (e.g., image (871)) is input to an electronic device, the external shape information of the garment (e.g., image (851)) can be transformed into another type of external shape information (e.g., schematic diagram of the garment (853)) by a diffusion model (852). The electronic device can input the schematic diagram of the garment (853) transformed by the diffusion model (852) into an artificial neural network (e.g., trouser / skirt parameter predictor (874) and T-shirt parameter predictor (877)) that is trained differently for each category of the garment.

[0187] For example, if it is known that the appearance information of the garment is for a T-shirt, the electronic device can predict a data-driven parameter (878) using an artificial neural network (e.g., T-shirt parameter predictor (877)) corresponding to the category of the garment (e.g., T-shirt). In this case, the electronic device can generate (or extract) a pattern parameter (879) corresponding to the garment category and / or garment type among the data-driven parameters (878) using a vector (882) corresponding to the garment category and / or garment type. In this case, if it is known that the appearance information of the garment is not a T-shirt but trousers or a skirt, the electronic device can predict a data-driven parameter (875) using an artificial neural network (e.g., trousers or skirt parameter predictor (874)) corresponding to the category of the garment (e.g., trousers or skirt). At this time, the electronic device can generate (or extract) a pattern parameter (876) corresponding to the corresponding garment category and / or garment type among the data-based parameters (875) using a vector (881) corresponding to the garment category and / or garment type.

[0188]

[0189] FIG. 9 is a diagram illustrating a method for normalizing shape data of a garment pattern according to one embodiment. Referring to FIG. 9, a diagram (900) is shown illustrating a method for normalizing shape data of garment patterns (910, 920, 930) to generate a garment pattern from data-driven pattern parameters according to one embodiment.

[0190] For example, if the body size of a user or avatar is not entered, the electronic device can predict pattern parameters corresponding to one or more normalized clothing patterns. The electronic device can generate one or more clothing patterns corresponding to the pattern parameters using a list of 2D points transformed through normalization for each type of one or more clothing patterns.

[0191] To represent the lines between each key point of the clothing patterns (910, 920, 930), the electronic device may normalize the shape (or form) of the clothing patterns (910, 920, 930) as shown in drawings (940, 950, 960) by sampling a predetermined number of points from the clothing patterns (910, 920, 930). The two-dimensional positions of the sampled points may be represented as two-dimensional coordinates with the key point of the 0th index in each clothing pattern (910, 920, 930) as the origin.

[0192] The electronic device can, for example, normalize the shape data of the front pattern (or back pattern) (910) as shown in the drawing (940) by sampling 55 points for the front pattern (or back pattern) (910) of the image and representing the 2D positions of the 55 sampling points as 2D coordinates with 110 values ​​of (x, y position).

[0193] Alternatively, the electronic device may normalize the shape data of the sleeve pattern (920) as shown in the drawing (950) by sampling 65 points for the sleeve pattern (920) and representing the 2D positions of the 65 sampling points as 2D coordinates with 130 values ​​of (x, y position).

[0194] At this time, the k-th point, represented by a circled character in each garment pattern (910, 920, 930), can represent the same position in the garment. For example, the 4th point of the front pattern of the top (or back pattern of the top) (910) can represent the shoulder endpoint.

[0195] As described above, when the shape data of each clothing pattern (910, 920, 930) is normalized, clothing patterns of the same type can be defined by the same number of 2D coordinates. In this case, for each of the clothing patterns that are of the same type but have different shapes (e.g., clothing patterns corresponding to tops, or a pattern in the form of a blouse with one ribbon and another in the form of a shirt), a point at the same index (e.g., the i-th point) may be data of the same topology representing the same location.

[0196] As described above, the electronic device can predict pattern parameters from shape data of one or more normalized clothing patterns by sampling a preset number of vertices from one or more clothing patterns.

[0197] The electronic device may, for example, generate one or more garment patterns corresponding to pattern parameters using a 2D point list transformed through normalization for each type of one or more garment patterns. In this case, the 2D point list may include information expressing the 2D positions of sampling points corresponding to each garment pattern as 2D coordinates.

[0198]

[0199] FIG. 10a is a diagram illustrating a method for generating a clothing pattern based on pattern parameters when the two-dimensional point list and the pattern parameters have a linear relationship according to one embodiment.

[0200] Referring to FIG. 10a, a garment pattern of a specific type (e.g., front body) can be converted into a 2D point list containing the same number of 2D points through the normalization of the aforementioned shape data. At this time, the matrix of the 2D point list is It can be expressed as.

[0201] The electronic device can extract pattern parameters using a two-dimensional point list of the garment pattern. The matrix of pattern parameters is It can be expressed as.

[0202] At this time, the method by which the electronic device converts the pattern parameter into a list of two-dimensional points corresponding to the pattern parameter is as follows.

[0203] For example, a matrix of a point list ( ) and matrix of pattern parameters( If ) is a linear relationship, a transformation matrix that converts pattern parameters into a 2D point list ( ) can be expressed as shown in mathematical formula 1 below.

[0204]

[0205] Here, It could be.

[0206] The electronic device can generate a garment pattern using pattern parameters extracted using a two-dimensional point list of the garment pattern.

[0207]

[0208] FIG. 10b is a diagram illustrating a method for generating a clothing pattern based on pattern parameters when the two-dimensional point list and pattern parameters have a non-linear relationship according to one embodiment.

[0209] For example, the matrix of the aforementioned point list (through FIG. 10a) ) and matrix of pattern parameters( If ) is in a non-linear relationship, the electronic device uses a second neural network model (1030) such as FIG. 10b to form a matrix of pattern parameters ( )(1010) is the matrix of the point list( It can be converted into )(1050). At this time, the second neural network model (1030) corresponds to an artificial neural network, and through supervised learning, the matrix of pattern parameters ( )(1010) is the matrix of the point list( It may be a regression model trained to convert to )(1050), but is not necessarily limited to this.

[0210]

[0211] FIG. 11 illustrates a diagram for explaining the relationship between pattern parameters and key points according to one embodiment. Referring to FIG. 11, key points (1130) of a clothing pattern derived from pattern parameters (1110) according to one embodiment are illustrated.

[0212] For example, a specific type of clothing pattern (e.g., front body) can all be converted into a list of two-dimensional points of the same number through the normalization of the shape data described above.

[0213] The electronic device uses a set of pair data between previously configured key points and pattern parameters (1110) to convert the pattern parameters (1110) into key points (1130) using a conversion matrix ( ) can be produced.

[0214] The electronic device is a conversion matrix ( through ) Key points (k) (1130) of the clothing pattern can be generated from pattern parameters (p) (1110) as shown above.

[0215] For example, a dataset of key points (e.g., the 2D (x,y) locations of key points) as a matrix Represented as such, the data set of pattern parameters (1110) is a matrix It can be expressed as.

[0216] Transformation matrix ( ) is a matrix of point lists( ) and matrix of pattern parameters( As a linear transformation matrix between, It can be expressed as follows.

[0217] The aforementioned mathematical formula 1 is a transformation matrix that minimizes the error ( It may correspond to ).

[0218]

[0219] FIG. 12 is a flowchart illustrating a method for simulating clothing by reflecting a first texture image according to one embodiment. Referring to FIG. 12, an electronic device according to one embodiment can generate a three-dimensional clothing through steps (1210) to (1250).

[0220] In step (1210), the electronic device can classify the category of the clothing included in the appearance information of the clothing.

[0221] In step (1220), the electronic device can predict a pattern parameter corresponding to the garment from the appearance information of the garment according to the category of the garment classified in step (1210).

[0222] In step (1230), the electronic device can generate a first texture image corresponding to one or more garment patterns from the appearance information of the garment. The method by which the electronic device generates the first texture image is as follows.

[0223] The electronic device can generate a first texture image from a prompt using a pre-learned texture generator, for example, as illustrated in FIG. 19c below.

[0224] Alternatively, the electronic device may generate a first texture image by applying the type of fabric classified from the appearance information of the garment to the texture of the appearance information of the garment. Here, 'fabric' may refer to a basic cloth or fabric used to make clothes. The fabric may be a factor that determines the texture, durability, comfort, and appearance of the garment. The fabric may include various types of materials, such as, for example, cotton, wool, silk, polyester, and / or various other blended fabrics, and may have unique characteristics and uses. The fabric used for the garment may vary depending on the use and style of the garment.

[0225] Electronic devices can predict (classify) the type of fabric in the appearance information of a garment by calculating the similarity between the text-encoded value of the fabric name and the image-encoded value of the fabric using a neural network model (e.g., the CLIP (Contrastive Language-Image Pre-training) model). Here, the CLIP model may correspond to an Artificial Intelligence (AI) model that learns images and text together to understand the relationship between images and text and apply it to various tasks. Through multimodal learning that learns images and text simultaneously, the CLIP model can generate powerful representations that combine the two types of information: images and text. Furthermore, the CLIP model can be trained using a zero-shot learning technique, which utilizes a pre-trained model to achieve high performance on new tasks without additional training. Zero-shot learning can correspond to a learning method in which an AI model possesses the ability to perform classification or prediction on new classes that it has not previously learned. Artificial intelligence (AI) models can identify new categories without labeled data through zero-shot learning techniques.

[0226] Alternatively, if the external information of the garment includes a texture, the electronic device may generate a first texture image comprising at least one of a raw texture image generated by cropping a portion of the external information of the garment or a material map based on Physically Based Rendering (PBR). The electronic device may generate a raw texture image by cropping a portion of the external information of the garment. In this case, the raw texture image may include textures such as transparency, roughness, and sparkle according to the type of fabric included in the external information of the garment, for example.

[0227] An electronic device may generate at least one of a first texture image or a physically based rendering (PBR)-based material map from a raw texture image. Physically based rendering (PBR) may correspond to a technology that simulates the visual properties of a material (or substance) based on actual physical properties. A physically based rendering (PBR)-based material map can enable realistic texture and material representation in clothing design. A physically based rendering (PBR)-based material map may include, for example, at least one of an albedo (or base color), which is a texture representing the inherent color of an object and does not include shading or shadows caused by light; metallicity, which indicates whether the material is metallic or non-metallic; roughness, which represents the roughness of the surface and controls the way light is reflected; or a normal map, which adds fine details to the surface to provide a more realistic texture.

[0228] In step (1240), the electronic device can generate one or more clothing patterns based on the pattern parameters predicted in step (1220).

[0229] In step (1250), the electronic device can create a three-dimensional garment by reflecting the first texture image created in step (1230) onto one or more garment patterns created in step (1240). The electronic device can create a three-dimensional garment with a texture reflected by reflecting the first texture image onto one or more garment patterns or a three-dimensional garment.

[0230] The electronic device can put the 3D clothing created in step (1250) on the avatar. For example, the electronic device can simulate putting one or more created clothing patterns on the avatar. In this specification, "putting on" can be understood as a process of combining pattern information or clothing patterns by a computer program to put a 3D clothing object on a 3D avatar.

[0231]

[0232] FIG. 13 is a drawing for explaining a method of generating a three-dimensional garment from a prompt according to one embodiment. Referring to FIG. 13, a drawing (1300) is shown illustrating the process of generating a three-dimensional garment by reflecting a texture image onto a garment pattern using an electronic device according to one embodiment.

[0233] The electronic device may receive a prompt (1310) and input it into a neural network model (1320). The neural network model (1320) may convert the prompt (1310) into an input image (1330). The neural network model (1320) may be the aforementioned diffusion model, but is not necessarily limited thereto. The input image (1330) may be at least one of a technical drawing image, a sketch image, a photographic image corresponding to the clothing, or an image of the clothing being worn by an avatar or a person, but is not necessarily limited thereto. According to an embodiment, instead of the image itself being decoded, it may be input to a pattern predictor (1340) and / or a texture generator (1360) in the form of intermediate data containing image features.

[0234] The electronic device can predict pattern parameters corresponding to the clothing included in the input image (1330) using a pattern predictor (1340) and generate one or more clothing patterns (1350) based on the pattern parameters. The electronic device may, for example, generate one or more clothing patterns (1350) from the input image (1330) using the first neural network model (110) described above, but is not necessarily limited thereto.

[0235] The electronic device can create a three-dimensional garment (1380) using one or more garment patterns (1350).

[0236] Additionally, the electronic device can generate (1360) a first texture image (1370) corresponding to one or more clothing patterns (1350) from an input image (1330). Although not illustrated in the drawing, the electronic device can generate physical data corresponding to one or more clothing patterns (1350) from an input image (1330).

[0237] The electronic device can generate a three-dimensional garment (1380) with a texture by reflecting a first texture image (1370) onto one or more garment patterns (1350). Although not shown in the drawing, the electronic device can generate a three-dimensional garment with a physical property by reflecting physical property data onto one or more garment patterns (1350).

[0238]

[0239] FIG. 14 is a flowchart illustrating a clothing simulation method for generating a three-dimensional clothing from a prompt according to one embodiment. Referring to FIG. 14, an electronic device according to one embodiment can put a three-dimensional clothing on an avatar through steps (1410) to (1490).

[0240] In step (1410), the electronic device may receive a prompt. The 'prompt' may correspond to a command or question that the user inputs to provide information to an artificial intelligence model or neural network model and to elicit a desired answer or result. The artificial intelligence model or neural network model may identify the user's intent through the prompt and generate an appropriate response that matches the user's intent.

[0241] In step (1420), the electronic device can convert the prompt into one or more input images. The electronic device can convert the prompt into input images by, for example, a diffusion model. The method by which the electronic device converts the prompt into input images by a diffusion model is described in more detail with reference to FIG. 15 below.

[0242] In step (1430), the electronic device can classify the categories of input clothing included in one or more input images.

[0243] In step (1440), the electronic device can generate an image that modifies at least one of the shape, texture, or color of the garment.

[0244] In step (1450), the electronic device can predict pattern parameters corresponding to the clothing from the modified image generated in step (1440) according to the category of clothing classified in step (1430).

[0245] In step (1460), the electronic device can generate a second texture image from the modified image generated in step (1440).

[0246] In step (1470), the electronic device can generate one or more clothing patterns based on the pattern parameters predicted in step (1450).

[0247] In step (1480), the electronic device can create a three-dimensional garment by reflecting a second texture image onto one or more garment patterns created in step (1470).

[0248] In step (1490), the electronic device can put the 3D clothing created in step (1480) on the avatar and display the result of the wearing.

[0249]

[0250] FIG. 15 is a diagram illustrating a method for converting a prompt into an input image according to one embodiment. Referring to FIG. 15, the process of converting a prompt (1510) according to one embodiment into an input image (1550) by a diffusion model (1530) is illustrated.

[0251] The electronic device can, for example, convert the prompt (1510) into an input image (1550) by a diffusion model (1530).

[0252] The diffusion model (1530) can generate visual content (e.g., input image (1550)) through a description of a specific clothing model or clothing pattern included in the prompt (1510).

[0253] The diffusion model (1530) is a type of generative AI model that can generate new data (e.g., input image (1550)) through a process of adding noise to data (e.g., prompt (1510)) and removing it. The diffusion model (1530) can use a forward process that gradually adds noise to the data and a reverse process that removes the noise to restore the original data.

[0254] The diffusion model (1530) can, for example, gradually restore an image from random noise to finally generate a high-quality image (e.g., input image (1550)), and this process can be carried out step by step using a Markov chain.

[0255]

[0256] FIG. 16 is a drawing for explaining a method of generating a three-dimensional garment by a modified image of a garment according to one embodiment. Referring to FIG. 16, a drawing (1600) is shown illustrating a process of generating a three-dimensional garment by reflecting a second texture image (1680) generated from a modified image (1640) of a garment included in an input image (1601) of one embodiment onto a garment pattern (1650) or a garment (1630) based on the garment pattern (1650).

[0257] The electronic device may receive a prompt (1610) and input it into a neural network model (1620). The neural network model (1620) may convert the prompt (1610) into an input image (1601). The neural network model (1620) may be, for example, the diffusion model (1530) described above through FIG. 15, but is not necessarily limited thereto. The input image (1601) may be at least one of a technical drawing image, a sketch image, a photographic image, a depth image, or an image of the clothing worn by an avatar or a person, or a combination thereof, but is not necessarily limited thereto.

[0258] The electronic device can generate a modified image (1640) by modifying (1630) at least one of the shape, texture, or color of the garment included in the input image (1601). The method by which the electronic device modifies (1630) the garment included in the input image (1601) is described in more detail with reference to FIG. 17 below.

[0259] The electronic device can predict pattern parameters from a modified image (1640) and generate one or more clothing patterns (1650) based on the pattern parameters. The electronic device can generate one or more clothing patterns (1650) from the modified image (1640) using, for example, the first neural network model (110) described above, but is not necessarily limited thereto. The electronic device can generate a three-dimensional clothing (1660) using one or more clothing patterns (1650).

[0260] Additionally, the electronic device may generate (1670) a second texture image (1680) corresponding to one or more clothing patterns (1650) from a modified image (1640). For example, if the modified image (1640) contains a texture, the electronic device may generate a second texture image (1680) that includes at least one of a raw texture image generated by cropping a portion of the modified image (1640) or a material map based on physically based rendering (PBR). Alternatively, the electronic device may generate a second texture image (1680) that includes characteristics of a fabric classified from the modified image (1640).

[0261] The electronic device can generate a textured 3D garment (1690) by reflecting a second texture image (1680) onto a 3D garment (1660).

[0262]

[0263] FIG. 17 is a drawing for explaining a method for generating images of a modified garment according to one embodiment. Referring to FIG. 17, modified images (1715, 1725, 1735) generated by an electronic device according to one embodiment modifying various forms of a garment included in the external shape information (1710, 1720, 1730) of the garment are shown.

[0264] The electronic device can generate modified images (1715, 1725, 1735) by converting the shape information of the garment (1710, 1720, 1730) into a Canny image in the form of a Canny edge, and then inputting the Canny image into a Canny ControlNet. Here, the Canny image may be an image represented by an edge corresponding to the outline of the image.

[0265] In cases where texture information is not included in the image, such as in a technical drawing image or a sketch image, the electronic device can obtain a texture image through a conversion process. Therefore, if a texture image exists separately, or if the texture is included in the appearance information of the garment, the electronic device may not perform the image transformation process.

[0266]

[0267] FIG. 18 is a drawing for explaining a method of generating a second texture image by deforming a garment according to one embodiment. Referring to FIG. 18, a drawing (1800) is shown for explaining a process in which an electronic device according to one embodiment deforms a garment included in the external shape information (1810, 1820, 1830, 1840) of various types of garments to generate a deformed image, and generates a second texture image from the deformed image.

[0268] The electronic device can easily generate various designs by generating an image that modifies at least one of the shape, texture, or color of the garment included in the garment's external information (1810, 1820, 1830, 1840). The electronic device can generate an image that modifies at least one of the shape, texture, or color of the garment by, for example, various image generation and transformation techniques based on a diffusion model.

[0269] Here, various image generation and transformation techniques may include, for example, i2i (Image-to-Image), Canny ControlNet, and recoloring techniques, but are not necessarily limited thereto. i2i (Image-to-Image) is an image transformation technique that can generate a new image (e.g., a finished garment image) from garment shape information (e.g., a sketch image). Canny ControlNet can detect the outlines of an image by utilizing edge information detected by the Canny Edge Detection technique. The recoloring technique is a technique that changes the color of the garment shape information; for example, it can convert a black-and-white image into a color image or change a specific color to another color.

[0270] For example, if the appearance information of the garment is an image that does not contain texture information, such as a technical drawing image (1810) or a sketch image (1820), the electronic device can generate a converted image (1815, 1825) with texture through conversion by the image generation and conversion technique described above. If the appearance information of the garment is an image (1830) of the actual garment, the electronic device can generate a converted image (1835) that converts the shape and texture of the garment included in the image (18730) of the actual garment.

[0271] Alternatively, if the electronic device is an image (1840) of a person wearing the actual garment, it can generate a converted image (1845) in which only the color of the garment's appearance information is changed by a recoloring technique.

[0272] According to an embodiment, the electronic device may not proceed with the process of generating a converted image with a texture through conversion if the texture image exists separately or if the appearance information of the garment is an image containing a texture, such as an image of the actual garment (1830) or an image of a person wearing the actual garment (1840).

[0273]

[0274] FIG. 19a is a drawing for explaining a method of generating a second texture image from a modified image according to one embodiment.

[0275] Referring to FIG. 19a, an electronic device according to one embodiment may generate a modified image (1920) by modifying at least one of the shape, texture, or color of a garment included in the appearance information of various types of garments (e.g., input image (1910)). The electronic device may generate a second texture image (1960) from the modified image (1920). The second texture image (1960) may include at least one of a raw texture image (1940) or a physically based rendering (PBR) based material map (1950).

[0276] More specifically, if the modified image (1920) contains a texture, the electronic device may generate a second texture image comprising at least one of a raw texture image or a physically based rendering (PBR) based material map generated by cropping a portion of the modified image. The electronic device may generate a raw texture image (1940) by cropping (1930) a portion of the modified image (1920) (e.g., a patch). Additionally, the electronic device may generate a physically based rendering (PBR) based material map (1950) based on the raw texture image (1940). The electronic device can, for example, generate (1940) a second texture image (1960) by combining a raw texture image (1940) with a second texture image (1965) and a physically based rendering (PBR) based material map (1950) by a fabric diffusion technique that converts the texture of a two-dimensional image into a three-dimensional clothing model in high quality.

[0277] The method by which an electronic device generates a material map (1950) based on physically based rendering (PBR) is explained in more detail with reference to Fig. 20 below.

[0278]

[0279] FIG. 19b is a drawing for explaining a method of generating a second texture image from a modified image according to one embodiment.

[0280] Referring to FIG. 19b, an electronic device according to one embodiment can generate a modified image (1925) by modifying at least one of the shape, texture, or color of a garment included in the appearance information of various types of garments (e.g., input image (1915)). The electronic device can generate a raw texture image (1945) by cropping (1935) a portion of the modified image (1925) (e.g., a patch).

[0281] The electronic device may generate (1955) a second texture image (1965) and / or a physically based rendering (PBR) based material map (1975) based on a raw texture image (1945). The electronic device may generate (1955) the second texture image (1965) and / or a physically based rendering (PBR) based material map (1975) from the raw texture image (1945) by, for example, a fabric diffusion technique that converts the texture of a two-dimensional image into a three-dimensional garment model in high quality.

[0282] The fabric diffusion technique can automatically convert the textures and patterns of a 2D garment image into a 3D garment model. The fabric diffusion technique can generate various texture maps, such as reflection, roughness, normal, and metallicity texture maps, for example, to accurately re-illuminate and render a second texture image (1965) and / or a physically based rendering (PBR) based material map (1975) under various lighting conditions.

[0283]

[0284] FIG. 19c is a diagram illustrating a method for generating a second texture image using a texture generator according to one embodiment. Referring to FIG. 19c, a user interface (UI) screen (1980) of a texture generator according to one embodiment is shown.

[0285] The electronic device can generate a texture image from a prompt using a texture generator. For example, the user can select "Plain" as the fabric type and "Camouflage" as the content type in the option field displayed on the user interface (UI) screen (1980) of the texture generator. Additionally, the user can set the pattern size to "1". Additionally, the user can enter "a rainbow color" in the text prompt input field.

[0286] The texture generator can generate a texture image (1990) that reflects data (e.g., selected options and / or prompts) entered on a user interface (UI) screen (1980).

[0287]

[0288] FIG. 20 is a drawing for explaining a method for generating a texture image according to one embodiment. Referring to FIG. 20, a drawing (2000) showing the process of an electronic device generating a text image according to one embodiment is shown.

[0289] The electronic device can generate a texture image (e.g., a transparent print image (2055)) and a physically based rendering (PBR) based material map (2045) using a captured image (2005) of a portion of the actual garment (2001). In this case, the captured image (2005) may correspond to a cropped image of the portion of the actual garment (2001) that is to be converted into fabric.

[0290] The electronic device can input the captured image (2005) into a print generator (2020) to generate a normalized print image (2030). Additionally, the electronic device can input the captured image (2005) into a texture generator (2010) to generate a texture image (2015). This process may be referred to as the ‘(a) text and print generation process’.

[0291] The electronic device can generate not only texture images but also material maps (2045) based on physically based rendering (PBR). The electronic device can obtain a transparent print image (2055) by adding an alpha channel to a normalized print image (2030). At this time, the transparent print image (2055) may include RGBA (Red, Green, Blue, Alpha). Here, RGBA is one of the color representation methods and may correspond to the abbreviations for Red, Green, Blue, and Alpha. Red may represent the red component (channel) of the color, Green may represent the green component (channel) of the color, and Blue may represent the blue component (channel) of the color. Additionally, Alpha may be a component (channel) representing transparency. RGBA can be primarily used for digital texturing. For example, the electronic device can provide various visual effects by controlling color and transparency through RGBA. Alternatively, the electronic device can make specific parts transparent through the alpha channel.

[0292] The electronic device can input a texture image (2015) into a physically based rendering (PBR) generator (2040) to generate a high-quality physically based rendering (PBR) based material map (2045). This process can be called the ‘(b) physically based rendering (PBR) based material map generation process’.

[0293] The electronic device may apply a physically based rendering (PBR)-based material map (2045) and a transparent printed image (2055) to a target mesh (2060) of a three-dimensional garment of any shape. The electronic device may generate a textured mesh (2070) by applying the physically based rendering (PBR)-based material map (2045) to the target mesh (2060) by tiling and / or overlaying. Tiling may correspond to a method of arranging tiles of the same shape to fill a given space without overlapping or gaps. Tiling may include periodic tiling in which the same pattern is repeated, non-periodic tiling in which the pattern is not repeated, and symmetric tiling constructed using reflection symmetry, rotational symmetry, translational symmetry, etc. The electronic device may apply a physically based rendering (PBR)-based material map (2045) to a target mesh (2060) by periodic tiling, non-periodic tiling, or symmetric tiling. The overlaying technique may correspond to a method of maximizing visual effects by overlapping multiple layers. The electronic device may reflect textures by overlaying the physically based rendering (PBR)-based material map (2045) onto the target mesh (2060). This process may be referred to as the '(c) tiling and overlaying process'.

[0294] The electronic device can generate a rendering image (2080) corresponding to a textured mesh (2070) by a relighting technique that simulates clothing in various lighting environments by changing the lighting conditions of a 3D model. This process can be called the '(d) relighting process'.

[0295]

[0296] FIG. 21 is a diagram illustrating a clothing simulation method according to one embodiment. Referring to FIG. 21, a process is illustrated in which a 3D clothing generated based on a prompt input by a user according to one embodiment is fitted onto avatars of various sizes having different body sizes through auto-fitting.

[0297] The electronic device can convert a prompt into the appearance information of the garment, as shown in the drawing (2110). For example, let's assume the user enters a prompt such as "Category of garment: T-shirt, Text prompt: V-neck, Short sleeve, Slim fit, Cropped". In this case, the electronic device can input the prompt into the aforementioned diffusion model to generate an image (2115) corresponding to the prompt.

[0298] The electronic device can predict pattern parameters from an image (2115), as shown in drawing (2120), and generate one or more clothing patterns based on the pattern parameters. In addition, the electronic device can generate texture images corresponding to one or more clothing patterns from the image (2115), as shown in drawing (2130).

[0299] The electronic device can combine one or more clothing patterns and texture images to output a three-dimensional clothing with a texture reflected, such as in drawing (2140). The electronic device can display the result of wearing the three-dimensional clothing with a texture reflected, such as in drawing (2143), on avatars having different body sizes. Alternatively, the electronic device can display the result of wearing the three-dimensional clothing with a texture reflected, such as in drawing (2141), on an avatar having a body size corresponding to the appearance information of the clothing.

[0300]

[0301] FIG. 22 is a diagram illustrating a method for predicting pattern parameters based on a user's body size according to one embodiment. Referring to FIG. 22, a diagram (2200) is shown illustrating a process in which an electronic device according to one embodiment receives external shape information of a garment (e.g., input image (2201)), generates pattern parameters (2230) for a garment pattern, and generates a three-dimensional garment (2250) from the pattern parameters (2230) through a garment pattern (2240). At this time, the garment pattern (2240) may correspond to a parametric pattern, but is not necessarily limited thereto.

[0302] The electronic device may receive the body size (2220) of a user or avatar along with external shape information of the clothing (e.g., input image (2201)). The electronic device may predict pattern parameters (2230) from shape data of one or more clothing patterns generated based on the body size (2220). At this time, the electronic device may predict the pattern parameters (2230) using a first neural network model (2210) trained to predict pattern parameters according to the type of input image (2201) based on the body size (2220).

[0303] The electronic device can generate a garment pattern (2240) based on pattern parameters (2230). The garment pattern (2240) may be a data-based parametric pattern. The electronic device can generate a three-dimensional garment (2250) using the garment pattern (2240).

[0304]

[0305] FIG. 23 is a drawing for explaining a method of generating a three-dimensional garment by reflecting a second texture image generated based on the type of fabric classified from a modified image according to one embodiment.

[0306] Referring to FIG. 23, an electronic device according to one embodiment can predict pattern parameters (2330) by inputting appearance information of a garment (e.g., technical drawing image (2310)) into a first neural network (2320). The first neural network (2320) may be trained to predict pattern parameters (2330) from the appearance information of a garment (technical drawing image (2310)) according to the category of the garment (e.g., dress) classified from the appearance information of the garment (e.g., technical drawing image (2310)).

[0307] The electronic device can generate one or more clothing patterns (2340) based on pattern parameters (2330).

[0308] Additionally, the electronic device can generate an image (2350) that modifies at least one of the shape, texture, or color of a garment (e.g., a dress) included in the appearance information of the garment (e.g., a technical drawing image (2310)).

[0309] The electronic device can classify the characteristics (2360) of the fabric in the modified image (2350). Here, the characteristics (2360) of the fabric may include, for example, the type of fabric and / or the fiber material of the fabric. The type of fabric may include, for example, chambray, a light plain weave fabric; Oxford, a thick basket weave fabric; chiffon, a thin and transparent fabric; and Clip Jacquard, a type of Jacquard fabric in which some threads are clipped (cut or removed) during the process of weaving a complex pattern to highlight specific parts, but is not necessarily limited thereto. The fiber material may include, for example, cotton, wool, silk, polyester and / or various other blended fabrics, but is not necessarily limited thereto. The method of classifying the characteristics (2360) of the fabric in the image (2350) modified by the electronic device is explained in more detail with reference to Fig. 24 below.

[0310] The electronic device may extract or generate a texture image (second texture image) (2370) from a modified image (2350). At this time, the electronic device may generate a texture image (2370) by applying a unique texture (or texture image) (2365) sampled according to the characteristics (2360) of the fabric classified earlier together with the texture image extracted from the modified image (2350). For example, if the characteristics (2360) of the fabric are classified as "chiffon," the electronic device may generate a texture image (2370) by applying a unique texture (or texture image) (2365) sampled from thin, transparent, and flowing chiffon together with the modified image (2350). A method for generating a texture image (2370) by applying a sampled unique texture (or texture image) (2365) together with a modified image (2350) of an electronic device is explained in more detail with reference to FIG. 25 below.

[0311] The electronic device may generate a three-dimensional garment (2390) by applying a texture image (2370) that reflects the characteristics (2360) of a fabric including a sampled unique texture image (2365) to one or more garment patterns (2340). The electronic device may display only the three-dimensional garment (2390), or may display the state in which the three-dimensional garment (2390) is worn on an avatar, as shown in FIG. 23.

[0312]

[0313] FIG. 24 is a drawing for explaining a method of classifying fabric types from a modified image according to one embodiment. Referring to FIG. 24, a drawing (2400) showing fabric types classified from modified images (2410, 2430) by an electronic device according to one embodiment is shown.

[0314] For example, the electronic device can predict (classify) the type of fabric of the modified image (2410, 2430) as a fabric name such as "Satin" and "Chiffon" by calculating the similarity between the text-encoded value of the fabric name and the image-encoded value of the fabric image (e.g., modified image (2410, 2430)) using the aforementioned CLIP model.

[0315] The names of the fabrics are, for example: "Boucle", "Canvas", "Challis", "Chambray", "Oxford", "Chiffon", "Clip jacquard", "Corduroy", "Crepe", "CDC", "Crepe knit", "Crochet", "Denim", "Dewspo", "Dobby", "Dobby mesh", "Double knit", "Interlock", "Double weave", "Eyelet", "Flannel", "Fleece", "French terry", "Gauze", "Double gauze", "Georgette", "ITY", "Matte jersey", "Jacquard", "Brocade", "Jacquard knit", "Jersey", "Lace", "Loop terry", "Low gauge knit", “Melton”, “Boiled”, It may include, but is not necessarily limited to, "Memory", "Mesh", "Tulle", "Neoprene", "Scuba", "Organza", "Ottoman", "PVC", "Pique", "Plaid", "Plain", "Pointelle", "Polar fleece", "Ponte", "Poplin", "Quilted knit", "Rib", "Ripstop", "Satin", "Seersucker", "Sherpa", "TRS", "Taffeta", "Tricot", "Tweed", "Twill", "Tyvek", "Vegan fur", "Vegan leather", "Vegan suede", "Velour", "Velvet", "Velvet", "Velveteen", "Voile", "Waffle", etc.

[0316] Here, the CLIP model can be considered an Artificial Intelligence (AI) model capable of learning images (e.g., images of fabric) and text (e.g., fabric names) together to understand the relationship between fabric images and fabric names and apply it to various tasks. Through multimodal learning that simultaneously learns fabric images and fabric names, the CLIP model can generate powerful representations that combine the two pieces of information: fabric images and fabric names.

[0317] Furthermore, the CLIP model can be trained using zero-shot learning techniques, which utilize a pre-trained model to achieve high performance on new tasks without additional training. Zero-shot learning refers to a training method that enables an artificial intelligence (AI) model to perform classification or prediction on new classes that it has not previously learned. Through zero-shot learning techniques, the AI ​​model can identify new categories even without labeled data.

[0318]

[0319] FIG. 25 is a drawing for explaining a method of generating a second texture image from an image modified based on the type of fabric classified according to one embodiment.

[0320] Referring to FIG. 25, a drawing (2500) is shown illustrating second texture images (2530, 2535, 2570, 2575) and physically based rendering (PBR) based material maps (2540, 2580) generated from images (2510, 2550) modified by an electronic device according to one embodiment.

[0321] The electronic device can generate second texture images (2530, 2535, 2570, 2575) by applying the characteristics of the fabric classified in the modified images (2510, 2550) to the texture of the modified images (2510, 2550).

[0322] The electronic device can generate second texture images (2530, 2535, 2570, 2575) and physically based rendering (PBR) based material maps (2540, 2580) by reflecting a texture image (2520, 2560) sampled according to the characteristics of the fabric (e.g., type of fabric) classified from the deformed image (2510, 2550) onto a texture extracted from the deformed image (2530).

[0323] At this time, the type of fabric classified from the modified image (2510) is "Chiffon", and the type of fabric classified from the modified image (2550) may be "Satin".

[0324] For example, the electronic device can extract a style from an image (2510, 2550) transformed using two IP-Adapters in a diffusion model and extract a layout from a sampled texture image (2520, 2560) to use.

[0325] In addition, the electronic device can also generate physically based rendering (PBR) based material maps (2540, 2580) along with texture images.

[0326]

[0327] FIG. 26 is a diagram for explaining the process of wearing a 3D garment on an avatar, which is generated by a second texture image based on the type of fabric classified in the appearance information of the garment according to one embodiment.

[0328] Referring to FIG. 26, an electronic device according to one embodiment can predict pattern parameters (2630) by inputting the external shape information (2610) of the garment into a first neural network (2620).

[0329] The appearance information (2610) of the clothing may be an image of the actual clothing, an image of the clothing worn by a person, or a prompt entered by the user, but is not necessarily limited thereto.

[0330] The first neural network (2620) may be trained to predict pattern parameters (2630) from the appearance information (2610) of the clothing according to the category of the clothing (e.g., T-shirt) classified from the appearance information (2610) of the clothing.

[0331] The electronic device can generate one or more clothing patterns (2640) based on pattern parameters (2630).

[0332] Additionally, the electronic device can classify the characteristics (2670) of the fabric from the external information (2610) of the garment. Here, the characteristics (2670) of the fabric may include, for example, the type of the fabric (textile) and / or the fiber material of the fabric.

[0333] The electronic device can extract or generate a texture image (2650) from the appearance information (2610) of the garment. At this time, the electronic device can reflect the characteristics (2670) of the fabric classified earlier into the texture image (2650). The electronic device can generate a three-dimensional garment (2690) by applying the texture image (2650) reflecting the characteristics (2670) of the fabric to one or more garment patterns (2640). The electronic device may display only the three-dimensional garment (2690), or may display the state in which the three-dimensional garment (2690) is worn on an avatar, as shown in FIG. 26.

[0334]

[0335] FIG. 27 is a block diagram of an electronic device for performing a clothing simulation according to one embodiment. Referring to FIG. 27, an electronic device (2700) according to one embodiment includes one or more processors (2710) and memory (2730). The electronic device (2700) may further include an output device (2750).

[0336] One or more processors (2710), memory (2730), and output devices (2750) can be connected to each other via a communication bus (2705). The output device (2750), indicated by a dashed line in FIG. 27, may be optionally included depending on the type of electronic device (2700).

[0337] The electronic device (2700) may be a PC (Personal Computer), a user device (User Equipment) such as a smartphone, a server, and / or a cloud server or cloud computing model that provides SaaS (Software as a Service) services.

[0338] One or more processors (2710) include processing circuitry.

[0339] The memory (2730) stores instructions executed by one or more processors (2710). When the instructions are executed individually or collectively by one or more processors (2710), the electronic device (2700) enables the aforementioned clothing simulation method to be performed.

[0340] The electronic device (2700) obtains external shape information of the garment. The electronic device (2700) predicts pattern parameters of one or more garment patterns from the external shape information of the garment. The electronic device (2700) generates one or more garment patterns based on the pattern parameters. The electronic device (2700) can generate a three-dimensional garment by one or more garment patterns.

[0341] The output device (2750) can output (display) a three-dimensional garment and / or an avatar wearing the three-dimensional garment generated by one or more processors (2710).

[0342] In addition, the memory (2730) can store various information generated during the processing of one or more processors (2710) described above. According to one embodiment, the memory (2730) can store a program implementing the clothing simulation method described above through FIGS. 1 to 26. In addition, the memory (2730) can store various data and programs. The memory (2730) may include volatile memory or non-volatile memory. The memory (2730) may store various data by having a large-capacity storage medium such as a hard disk.

[0343] Additionally, one or more processors (2710) may perform at least one method or an algorithm corresponding to at least one method described above through FIGS. 1 to 26. One or more processors (2710) may be a data processing device implemented in hardware having a circuit having a physical structure for executing desired operations. For example, the desired operations may include code or instructions included in a program. One or more processors (2710) may be composed of, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a NPU (Neural Network Processing Unit). For example, the electronic device (2700) implemented in hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, an Application-Specific Integrated Circuit (ASIC), and a Field Programmable Gate Array (FPGA).

[0344] One or more processors (2710) can execute a program and control an electronic device (2700). The program code executed by one or more processors (2710) can be stored in memory (2730).

[0345] Additionally, an electronic device (2700) according to one embodiment can receive data from a user through an input / output device (I / O) and output generated data. For example, the electronic device (2700) can receive user input (e.g., prompt, input image, selection for clothing pattern) through various types of input / output devices (e.g., user interface (UI), pen, mouse, etc.). The electronic device (2700) can be connected to an external device (e.g., personal computer or network) through an input / output device and exchange data.

[0346] An electronic device (2700) according to one embodiment may further include other components not illustrated. For example, the electronic device (2700) may further include a communication module that provides a function for the electronic device (2700) to communicate with another electronic device or another server via a network. Also, for example, the electronic device (2700) may further include other components such as a transceiver, various sensors, a database, etc.

[0347]

[0348] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0349] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or instruct the processing unit independently or collectively. Software and / or data may be stored on any type of machine, component, physical device, virtual equipment, computer storage medium, or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on computer-readable recording media.

[0350] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may store program instructions, data files, data structures, etc., either individually or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0351] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0352] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based thereon. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0353] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

1. Step of acquiring appearance information of the costume; A step of predicting a pattern parameter corresponding to the garment from the external shape information of the garment; and A step of generating one or more clothing patterns based on the above pattern parameters A costume simulation method including 2. The above pattern parameter is A correlation between multiple points corresponding to the external shape of one or more of the above-mentioned clothing patterns, Costume simulation method.

3. In Paragraph 1, The step of predicting the above pattern parameters A step of predicting the pattern parameters using a first neural network model trained to predict the corresponding pattern parameters corresponding to each of the appearance information of multiple garments. A costume simulation method including 4. In Paragraph 3, The above first neural network model A clothing simulation method learned using pattern parameters corresponding to clothing patterns included in training data and appearance information of clothing simulated by said clothing patterns.

5. In Paragraph 1, The step of predicting the above pattern parameters A step of classifying one or more types of clothing patterns corresponding to the appearance information of the clothing according to the categories of the clothing; and A step of predicting pattern parameters for each of the above one or more types of clothing patterns A costume simulation method including 6. In Paragraph 1, The step of generating the above clothing pattern A step of extracting at least one first parameter corresponding to an individual garment pattern from the above pattern parameters; and A step of determining the relative positions of a plurality of points corresponding to the external shape of the individual garment pattern based on the first parameter. A costume simulation method including 7. In Paragraph 2, The above correlation includes the distance between the above plurality of points, Costume simulation method.

8. In Paragraph 1, A step of obtaining a vector corresponding to at least one of the category of the above clothing or the type of the above clothing. Includes more, The step of predicting the above pattern parameters A step of generating parameters of a clothing pattern corresponding to at least one of the categories or types based on the above vector and the above pattern parameters. A costume simulation method including 9. In Paragraph 1, The step of predicting the above pattern parameters A step of predicting whether at least some of the one or more garment patterns are generated from the appearance information of the garment. A costume simulation method including further 10. In Paragraph 1, Step of obtaining the body size of the user or avatar Includes more, The step of predicting the above pattern parameters A step of predicting the pattern parameters based further on the above body size A costume simulation method including 11. In Paragraph 1, The appearance information of the above costume A clothing simulation method comprising at least one image or prompt among a technical drawing image, which is an image rendering the result of wearing simulation of one or more clothing patterns, a sketch image of the clothing, a captured image of the clothing, a depth image of the clothing, or an image of the clothing being worn by an avatar or a person.

12. In Paragraph 1, The step of acquiring the above external shape information A step of obtaining a prompt corresponding to the above external shape information; and A step of converting the above prompt into image features by a diffusion model A costume simulation method including 13. In Paragraph 1, A step of generating a first texture image corresponding to one or more garment patterns from the appearance information of the garment; and A step of generating the garment with a reflected texture by reflecting the first texture image onto one or more garment patterns or the garment. A costume simulation method including further 14. In Paragraph 1, A step of obtaining a connection relationship between at least some of the points corresponding to the appearance of a first garment pattern among the one or more garment patterns and at least some of the points corresponding to the appearance of a second garment pattern among the one or more garment patterns; and A step of determining the sewing relationship between the first garment pattern and the second garment pattern based on the above connection relationship. A costume simulation method including further 15. In Paragraph 1, A step of generating an image in which at least one of the shape, texture, or color of the above garment is modified Includes more, The step of predicting the above pattern parameters A step of predicting the pattern parameters from the modified image above A costume simulation method including 16. In Paragraph 15, A step of generating a second texture image from the above-described modified image; and A step of generating the garment with the reflected texture by reflecting the second texture image onto one or more of the garment patterns or the garment. A costume simulation method including further 17. In Paragraph 16, The step of generating the second texture image above If the texture is included in the modified image, a step of generating a second texture image comprising at least one of a raw texture image generated by cropping a portion of the modified image or a material map based on physically based rendering (PBR); or A step of generating the second texture image by applying the characteristics of the fabric classified in the above-described modified image to the texture of the above-described modified image. A costume simulation method comprising at least one of the following.

18. In Paragraph 15, The step of generating the above-mentioned modified image A step of converting the external shape information of the above garment into a Canny image in the form of a Canny edge; and The step of generating the modified image by inputting the above Canny image into a diffusion model A costume simulation method including 19. In Paragraph 1, Step of simulating the fitting of one or more of the above-generated clothing patterns onto an avatar A costume simulation method including further 20. In electronic devices, One or more processors including processing circuits; and One or more memories that store instructions Includes, When the above instructions are executed individually or integrally by the one or more processors, the electronic device, Obtain the appearance information of the costume, Predicting pattern parameters of one or more garment patterns from the appearance information of the above garment, and Generating one or more clothing patterns based on the above pattern parameters, Electronic device.

Images