Clothing template simulation method and device using neural network model

Abstract

The invention provides a costume simulation method and device. The clothing simulation method comprises the following steps: obtaining template information of each template of clothing, wherein the template information comprises information about sample points extracted from each template; predicting stitching information about the template based on an embedding vector of each of the templates obtained by applying the template information to a template embedding model trained to estimate correlation between input template information; and generating a simulation result of the garment based on the stitching information.

Description

Technical Field

[0001] Embodiments described herein relate to automatically placing clothing patterns on a three-dimensional (3D) avatar for simulation. Background Art

[0002] When a garment is worn on a person, it looks three-dimensional (3D), but it is more like a two-dimensional (2D) object because it is a combination of fabric pieces cut according to a 2D pattern. Because fabric (the material of the garment) is flexible, the shape of the fabric may change depending on the body shape or movements of the person wearing the garment. In addition, different fabrics may have different physical properties (e.g., strength, elasticity, and shrinkage). Due to such differences, garments made from the same shape pattern will behave, look, and feel differently when worn by a 3D avatar.

[0003] In the apparel industry, computer-based clothing simulation technology is widely used to develop realistic clothing designs. During the clothing simulation process, users usually manually arrange clothing patterns of clothing in appropriate positions on a three-dimensional (3D) avatar. For users who lack professional knowledge of clothing design or simulation, such pattern arrangement can be time-consuming and difficult for them. Summary of the invention

[0004] Technical methods to solve problems

[0005] Embodiments relate to the arrangement of simulated garment patterns performed on a computing device using a neural network model. Pattern information indicating the configuration of each of the patterns is received. The pattern information is applied to the neural network model to extract features from the configuration of each of the patterns. Arrangement points where the patterns are placed relative to a three-dimensional (3D) virtual image on which the garment is placed are predicted by processing the extracted features. At least a subset of the patterns are arranged on the predicted arrangement points. Patterns from the arrangement points are spliced ​​into the garment and placed on the 3D virtual image. The garment is simulated on the 3D virtual image. Embodiments may provide a technique for improving the accuracy of the model by training the model, the model outputting an embedding vector of each pattern from the pattern information of each pattern using the loss of arrangement information about the pattern and the loss of stitching information about the pattern. However, technical aspects are not limited to the above aspects, and other technical aspects may also exist.

[0006] In one or more embodiments, a clothing simulation method is provided, the clothing simulation method comprising: obtaining pattern information of each pattern of clothing, the pattern information comprising information about sample points extracted from each of the patterns; predicting stitching information about the pattern based on an embedding vector of each of the patterns obtained by applying the pattern information to a pattern embedding model trained to estimate the correlation between input pattern information; and generating a simulation result of the clothing based on the stitching information.

[0007] In one or more embodiments, the clothing simulation method may further include extracting a predetermined number of sample points from each of the patterns of the clothing, and adjusting the positions of the sample points based on the positions of predetermined types of points contained in each of the patterns.

[0008] In one or more embodiments, obtaining the pattern information of each of the patterns may include obtaining information about the sample points of the target pattern based on the length of an outline from a reference point of the target pattern in the pattern of the garment to each sample point extracted from the target pattern.

[0009] In one or more embodiments, the template embedding model may include a transformer encoder, and predicting the stitching information may include obtaining information indicating a stitching line of the sample pair extracted from the sample by applying the embedding vector pairs corresponding to the sample pairs extracted from the sample to the transformer decoder.

[0010] In one or more embodiments, the template embedding model may include: a first converter encoder, which is trained to estimate the correlation between sample points extracted from the same sample based on input sample information; and a second converter encoder, which is trained to estimate the correlation between samples based on the encoded data of each of the samples obtained from the embedding vector of the sample point output by the first converter encoder.

[0011] In one or more embodiments, predicting the stitching information may include: obtaining encoded data of the first sample by applying the sample information of the first sample to the first converter encoder, including an embedding vector for each sample point corresponding to the first sample; obtaining encoded data of the second sample by applying the sample information of the second sample to the first converter encoder, including an embedding vector for each sample point corresponding to the second sample; obtaining embedding vectors of the first sample and the second sample by applying the encoded data of the first sample and the encoded data of the second sample to the second converter encoder; and predicting stitching information about the sample pair of the first sample and the second sample by applying the embedding vectors of the first sample and the second sample to the converter decoder.

[0012] In one or more embodiments, the stitching information may include a pair of lines stitched together within the pattern.

[0013] In one or more embodiments, the stitching information may further include information indicating a stitching direction of the line.

[0014] In one or more embodiments, the information indicating the stitching lines of the template pair may include information about sample points extracted from a first template in the template pair and corresponding to the start and end points of the stitching lines contained in the first template, and information about sample points extracted from a second template in the template pair and corresponding to the start and end points of the stitching lines contained in the second template.

[0015] In one or more embodiments, the template embedding model may be trained based on a loss regarding the difference between the predicted stitching information and the ground truth.

[0016] In one or more embodiments, predicting the stitching information about the pattern may include predicting the arrangement information and the stitching information about the pattern based on the embedding vector of each of the patterns, and generating the simulation result of the garment may include generating the simulation result of the garment based on the stitching information and the arrangement information.

[0017] In one or more embodiments, the template embedding model may be trained based on a loss on the difference between the predicted stitching information and the ground truth and a loss on the difference between the predicted arrangement information and the ground truth.

[0018] In one or more embodiments, the information about the sample point may include at least one of the following information: information indicating the position of the sample point within the pattern, information indicating the position of the sample point within the garment, information indicating the positional relationship between sample points adjacent to each other, and information indicating the type of the sample point.

[0019] In one or more embodiments, a clothing simulation device is provided, comprising a processor, wherein the processor is configured to: obtain pattern information of each pattern of clothing, the pattern information comprising information about sample points extracted from each of the patterns; predict stitching information about the pattern based on an embedding vector of each of the patterns obtained by applying the pattern information to a pattern embedding model trained to estimate the correlation between input pattern information; and generate a simulation result of the clothing based on the stitching information. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] These and / or other aspects, features and advantages of the present invention will become apparent and more easily understood from the following description of embodiments taken in conjunction with the accompanying drawings, in which:

[0021] Figure 1 It is a flow chart of automatically arranging clothing patterns on a 3D virtual image according to one embodiment.

[0022] Figure 2 is a simplified diagram showing sample points of a template and information indicating the positions of the sample points within the template according to one embodiment.

[0023] Figure 3 is a simplified diagram showing sample point information according to one embodiment.

[0024] Figure 4A and Figure 4B is a simplified diagram showing stitching information and an automatic matching stitching part according to one embodiment.

[0025] Figure 5 is a block diagram showing the structure of a neural network model for clothing simulation for predicting stitching information according to one embodiment.

[0026] Figure 6 is a block diagram showing the structure of a neural network model for predicting stitching information and arrangement information according to one embodiment.

[0027] Figure 7 is a simplified diagram showing a layout point and a layout plate associated therewith according to one embodiment.

[0028] Figure 8 is a schematic diagram of an automatic arrangement device according to an embodiment. DETAILED DESCRIPTION

[0029] The description of the specific structure or function of the embodiment is only for example, and the embodiment can be implemented in different forms. The actual implementation is not limited to the specific embodiment disclosed, and the scope of this specification includes all changes within the technical ideas described by the embodiment, their equivalents and even their substitutes.

[0030] In the description with reference to the accompanying drawings, like or related components may be labeled with like reference numerals. Unless the context clearly indicates otherwise, a term in the singular may refer to one or more corresponding components.

[0031] In the content, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", "at least one of A, B or C", etc. may include any or all possible combinations of the items listed in the phrase.

[0032] The terms "first", "second", "first", "second", etc. are only used to distinguish one component from other components and do not limit other aspects (e.g., importance or order) of the other components. For example, the first component can be named the second component, and similarly, the second component can also be named the first component.

[0033] When it is said that one (e.g., first) component is "coupled" or "connected" to another (e.g., second) component, whether or not terms such as "functionality" or "communicability" are present, it means that the component is connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0034] In the absence of special instructions in the content, the singular forms "a", "an", and "the" are intended to include the plural forms as well. In this specification, the terms "include / comprise" and / or "include / comprises" are used to express the existence of the stated features, numbers, steps, operations, elements, and / or parts, but do not exclude the possibility of the existence or additional existence of one or more other features, numbers, steps, operations, elements, parts, and / or their combinations.

[0035] In the absence of other definitions, all terms, including technical or scientific terms used herein, have the common meanings understood by ordinary technicians in the field to which the examples belong. Commonly used terms as defined in dictionaries should be understood as the meanings in the relevant technical content, and should not be interpreted as idealized or overly formalized meanings if they are not clearly defined in this specification.

[0036] In the process of describing the examples with reference to the drawings, the same components are marked with the same reference numerals and repeated descriptions are omitted. In the process of describing the example embodiments, when it is determined that the detailed description of the relevant well-known structure or function will cause unnecessary confusion to the present disclosure, its detailed description is omitted.

[0037] Figure 1 It is a flow chart of automatically arranging clothing patterns on a 3D virtual image according to one embodiment.

[0038] Reference Figure 1 The automatic layout device may obtain pattern information of each pattern of the garment 110. The pattern information may include information about sample points extracted from each pattern.

[0039] A pattern as described herein refers to a design or configuration of a two-dimensional (2D) fabric generated and processed by a computer program. A pattern can be a part of a garment placed on a 3D avatar. In this article, the term "pattern" and the term "clothing pattern" are used interchangeably.

[0040] The pattern can be modeled as a combination of meshes. The mesh can be triangular, wherein each of the three vertices of the mesh is assigned a mass (e.g., a point mass), and the edges of the mesh are represented by elastic springs connecting these masses. Such a pattern can be modeled by a mass-spring model, etc., wherein each spring has a corresponding resistance value in terms of stretching, shearing, and bending, etc., according to the physical properties of the fabric used. Each vertex can move according to external forces such as gravity and internal forces generated by stretching, shearing, and bending. By calculating the external and internal forces to obtain the force applied to each vertex, the displacement and movement speed of each vertex can be determined. In addition, the collective movement of the vertices of the mesh at each time step can be used to simulate the time and space movement of virtual clothing including the mesh. By draping a 2D virtual clothing pattern formed by a triangular mesh on a 3D virtual image, a natural-looking 3D virtual clothing can be implemented based on the laws of physics. In other embodiments, the mesh can have a shape other than a triangle (e.g., a quadrilateral).

[0041] There may be a plurality of patterns for making a garment. Each pattern of a garment may refer to each of the plurality of patterns for making a garment. The pattern information of each pattern obtained in 110 may include pattern information of each pattern of the garment. For example, when the pattern of the garment includes a first pattern and a second pattern, the pattern information of each pattern may include pattern information of the first pattern and pattern information of the second pattern.

[0042] The pattern information may include information about sample points extracted (or sampled) from the pattern of the garment. For example, when the pattern of the garment includes a first pattern and a second pattern, the pattern information of the first pattern may include information about sample points extracted from the first pattern, and the pattern information of the second pattern may include information about sample points extracted from the second pattern.

[0043] The pattern may be a 2D object, and one or more one-dimensional (1D) sample points may be extracted from the 2D space corresponding to the pattern. For example, a predetermined number of sample points may be extracted from the contour (or edge) of the pattern. For example, points (e.g., vertices, segment points, notches, etc.) that are determined to be important for clothing implementation in the pattern may be extracted as sample points.

[0044] According to one embodiment, a clothing simulation method may include extracting a predetermined number of sample points from each sample of clothing, and adjusting the positions of the sample points based on the positions of the predetermined types of points contained in each sample. For example, a predetermined number of sample points may be extracted from a reference point (e.g., a vertex located at the leftmost and uppermost part of the sample) along the contour line of the sample at predetermined intervals. When the total length of the contour line of the sample is N, and the number of sample points to be extracted is "30", sample points may be extracted from the reference point along the contour line at intervals of N / 30. Reference points may also be extracted as sample points. When there are points of a predetermined type (e.g., vertices, segmentation points, gaps, etc.) around the sample point, the position of the sample point may be adjusted to the position of the corresponding point.

[0045] One embodiment, according to 110, may include obtaining information about sample points of a target pattern based on the length of a contour line from a reference point of the target pattern in the clothing pattern to each sample point extracted from the target pattern. The target pattern may refer to any pattern in the clothing pattern.

[0046] According to one embodiment, the information about the sample point may include at least one of the following information: information indicating the position of the sample point within the pattern, information indicating the position of the sample point within the garment, information indicating the positional relationship between sample points adjacent to each other, and information indicating the type of the sample point. Specific examples of the information about the sample point will be described in detail below.

[0047] According to one embodiment, the information about the sample point may include information indicating the position of the sample point within the pattern. For example, the information indicating the position of the sample point within the pattern may be determined based on the length of a contour line from a reference point of the pattern to the sample point. The length of the contour line from the reference point to the sample point may include the length of the contour line extending from the reference point to the sample point along a predetermined direction (e.g., clockwise or counterclockwise). In other words, operation 110 may include obtaining information about the sample point of the target pattern based on the length of the contour line from the reference point of the target pattern in the pattern of the garment to each sample point extracted from the target pattern. The target pattern may refer to any one of the patterns of the garment. The information indicating the position of the sample point within the pattern will be described in detail below.

[0048] The automatic layout device may predict 120 stitching information about the pattern of the garment based on the embedding vector of each pattern. Each pattern is obtained by applying the pattern information to a pattern embedding model trained to estimate the correlation between input pattern information.

[0049] The stitching information is information indicating the stitching relationship between the patterns used to make the garment, and may include, for example, a pair of lines (stitching lines) stitched together within the pattern of the garment. For example, in order to make the garment, when a first line as part of the outline of the first pattern and a second line as part of the outline of the second pattern are stitched together, the stitching information may include a pair of lines consisting of the first line and the second line. For example, the pair of lines stitched together may include information indicating the two endpoints of each line (for example, the two endpoints include a start point and an end point). The stitching information may include information indicating the stitching direction of the lines stitched together. The stitching information will be described in detail below.

[0050] The neural network model, according to an embodiment, may include a template embedding model. The template embedding model may be a training model for outputting an embedding vector for each template from the input template information of each template. For example, the template embedding model may include a transformer encoder trained to estimate the correlation between input data. The embedding vector for each template may be data generated based on the correlation between templates estimated from the template information input to the template embedding model.

[0051] One embodiment, according to 120, may include prediction stitching information. The stitching information may include information about indicating a stitching line of a template pair by applying an embedding vector pair corresponding to the template pair extracted from the template to a converter decoder. The embedding vector pair corresponding to the template pair may include embedding vectors of the templates included in the template pair. For example, when the template pair includes a first template and a second template, the embedding vector pair corresponding to the template pair may include an embedding vector of the first template and an embedding vector of the second template.

[0052] Information indicating stitching lines of a sample pair may be output from a converter decoder. Information indicating stitching lines may include sample points of a sample. Stitching lines of a sample pair may correspond to lines stitched together between the sample pair. Stitching lines of a sample pair may be specified by sample points contained in the sample pair. For example, information indicating stitching lines of a sample pair may include information about sample points extracted from a first sample in the sample pair and corresponding to the start and end points of stitching lines contained in the first sample, and information about sample points extracted from a second sample in the sample pair and corresponding to the start and end points of stitching lines contained in the second sample. Stitching lines and stitching information will be described in detail below.

[0053] The embedding vector pair corresponding to each combination of two sample combinations that can be extracted from the sample can be input into the converter decoder. There may also be sample pairs in which there are no lines sewn together. For example, in clothing, when the first sample is sewn only with the second sample, there are no seams in the other sample pairs except the first sample and the second sample. Based on the reliability of the information indicating the seams of the sample pair output from the converter decoder, it can be determined whether there are seams in the sample pair. For example, when the reliability of the information indicating the seams output from the sample pair consisting of the first sample and the second sample is greater than or equal to a threshold, it can be determined that there are seams between the first sample and the second sample. For example, when the reliability of the information indicating the seams output from the sample pair consisting of the first sample and the third sample is greater than or equal to a threshold, it can be determined that there are no seams between the first sample and the third sample.

[0054] One embodiment, according to 120, may include: applying the sample information of the first sample to the first converter encoder to obtain the encoded data of the first sample, including the embedding vector of each sample point corresponding to the first sample; applying the sample information of the second sample to the first converter encoder to obtain the encoded data of the second sample, including the embedding vector of each sample point corresponding to the second sample; applying the encoded data of the first sample and the encoded data of the second sample to the second converter encoder to obtain the embedding vector of the first sample and the embedding vector of the second sample; and applying the embedding vector of the first sample and the embedding vector of the second sample to the converter decoder to predict the stitching information of the first sample and the second sample pair. The training and inference operations of the model including the first converter encoder, the second converter encoder and the converter decoder will be described in detail below.

[0055] The automatic layout device may generate 130 a simulation result of the garment based on the stitching information. The simulation result may include 3D shape data of the garment generated by stitching the pattern based on the stitching information. The simulation result may be provided in the form of projection on a 2D screen through an output device such as a display.

[0056] According to an embodiment, a clothing simulation method may include predicting arrangement information to generate a clothing simulation result.

[0057] For example, one embodiment may include predicting 120 the layout information and stitching information of each pattern based on the embedding vector of each pattern. In addition, one embodiment may also include generating 130 a simulation result of the garment based on the stitching information and the layout information. The operation of predicting the stitching information and the layout information will be described in detail below.

[0058] Figure 2 is a simplified diagram showing sample points of a template and information indicating the positions of the sample points within the template according to one embodiment.

[0059] Reference Figure 2 The sample points extracted from the template 200 may include a first sample point 210 , a second sample point 220 , and a third sample point 230 .

[0060] As described above, the information indicating the position of the sample point within the template 200 may be determined based on the length of the contour line from the reference point 201 (i.e., the vertex located at the leftmost and uppermost part of the template 200) to the sample point. For example, the information indicating the position of the sample point within the template 200 may be determined based on a parameter (hereinafter referred to as a length parameter) having a ratio of the length of the contour line extending from the reference point 201 to the sample point in a counterclockwise (or clockwise) direction to the length of the entire contour line of the template 200.

[0061] For example, the length of the contour line extending counterclockwise from the reference point 201 to the first sample point 210 may correspond to the length of the first line 202. When the length of the first line 202 is 0.1 of the length of the entire contour line of the template 200, the length parameter of the first sample point 210 may be determined to be 0.1.

[0062] For example, the length of the contour line extending from the reference point 201 to the second sample point 220 in the counterclockwise direction may correspond to the sum of the lengths of the first line 202 and the second line 203. When the sum of the lengths of the first line 202 and the second line 203 is 0.4 of the length of the entire contour line of the template 200, the length parameter of the second sample point 220 may be determined to be 0.4.

[0063] For example, the length of the contour line extending from the reference point 201 to the third sample point 230 in the counterclockwise direction may correspond to the sum of the lengths of the first line 202, the second line 203, and the third line 204. When the sum of the lengths of the first line 202, the second line 203, and the third line 204 is 0.5 of the length of the entire contour line of the template 200, the length parameter of the third sample point 230 may be determined to be 0.5.

[0064] For example, the reference point 201 may also be a sample point of the template 200. For the reference point 201, the length parameter may be determined to be zero.

[0065] Figure 3 is a simplified diagram showing sample point information according to one embodiment.

[0066] Reference Figure 3 , the sample point information 300 may be information about any one point extracted from the sample. The above information about the sample points extracted from the sample may correspond to a set of sample point information 300 corresponding to each sample point extracted from the sample. In other words, the sample information may include a set of sample point information 300 corresponding to each sample point extracted from the sample. For example, when a first sample point and a second sample point are extracted from a first sample, the sample information of the first sample may include sample point information 300 corresponding to the first sample point and sample point information 300 corresponding to the second sample point.

[0067] The sample point information 300 may include relative position information 310 of the sample point and the center point of the template. The relative position information 310 of the sample point and the center point of the template may correspond to information indicating the position of the sample point within the template. For example, the relative position information 310 of the sample point and the center point of the template may be 2D coordinate data based on the center point of the template.

[0068] The sample point information 300 may include world coordinate (eg, world position) information 320 of the sample point. The world coordinate information 320 of the sample point may correspond to information indicating the position of the sample point within the clothing.

[0069] The sample point information 300 may include bounding box information 330. The bounding box information 330 is information about the size of the template, and may include height and width values ​​of a rectangle containing the template.

[0070] The sample point information 300 may include cosine conversion information 340 of a length parameter (eg, lengthParam) and sine conversion information 350 of a length parameter. The length parameter may be the same as described above with reference to Figure 2 The length parameter corresponds to the above.

[0071] The sample point information 300 may include segment length information 360 .

[0072] The sample point information 300 may include sample point type information 370. The type of the sample point may include, for example, a segment type, a gap type, and a vertex type.

[0073] Figure 4A and Figure 4B is a simplified diagram showing stitching information and an automatic matching stitching part according to one embodiment.

[0074] Reference Figure 4A , shows an example of stitching information 400 of a pattern pair including a pattern m and a pattern n according to one embodiment.

[0075] The stitching information 400 may include information 410 for indicating stitching lines of the pattern pair. The information 410 for indicating stitching lines of the pattern pair may include a length parameter of a start point of a stitching line included in pattern m (e.g., StartLengthParamm), a length parameter of an end point of a stitching line included in pattern m (e.g., EndLengthParamm), a length parameter of a start point of a stitching line included in pattern n (e.g., StartLengthParamn), and a length parameter of an end point of a stitching line included in pattern n (e.g., EndLengthParamn).

[0076] The stitching information 400 may include information 420 for indicating the stitching direction of the stitching line. The information 420 for indicating the stitching direction of the stitching line may include information (e.g., Directionm) for indicating the stitching direction of the stitching line included in the pattern m and information (e.g., Directionn) for indicating the stitching direction of the stitching line included in the pattern n. The stitching direction is a direction connecting the start point and the end point of the stitching line along the contour line, and may be determined as a clockwise direction, a counterclockwise direction, or the like.

[0077] For example, refer to Figure 4B , when the information indicating the sewing direction of the sewing line included in the pattern m 430 indicates the clockwise direction, the line 433 determined by connecting the contour lines from the start point 431 to the end point 432 of the sewing line included in the pattern m 430 in the clockwise direction may correspond to the sewing line included in the pattern m 430. When the information indicating the sewing direction of the sewing line included in the pattern m 430 indicates the counterclockwise direction, the line 434 determined by connecting the contour lines from the start point 431 to the end point 432 of the sewing line included in the pattern m 430 in the counterclockwise direction may correspond to the sewing line included in the pattern m 430.

[0078] For example, when information indicating the sewing direction of the sewing line included in the pattern n 440 indicates a counterclockwise direction, a line 443 determined by connecting contour lines in a counterclockwise direction from a start point 441 to an end point 442 of the sewing line included in the pattern n 440 may correspond to the sewing line included in the pattern n 440.

[0079] In order to make a garment, pattern m 430 and pattern n 440 can be sewn together along the stitching line so that the starting point 431 of the stitching line included in pattern m 430 and the starting point 441 of the stitching line included in pattern n 440 are sewn together, and the ending point 432 of the stitching line included in pattern m 430 and the ending point 442 of the stitching line included in pattern n 440 are sewn together.

[0080] Figure 5 is a block diagram showing the structure of a neural network model for clothing simulation for predicting stitching information according to one embodiment. Hereinafter, the neural network model for clothing simulation may be referred to as a clothing simulation model.

[0081] The clothing simulation model 500 may be a neural network model for predicting or outputting data so as to generate clothing simulation results according to the pattern information of each pattern of the clothing. For example, the data for generating the clothing simulation results may include stitching information 502.

[0082] Reference Figure 5 , the clothing simulation model 500 may include a pattern embedding model 510. As described above, the pattern embedding model 510 may be a training model for outputting an embedding vector of each pattern from input pattern information of each pattern.

[0083] The template embedding model 510 may include: a point converter encoder 520, which is trained to estimate the correlation between sample points extracted from the same sample according to input sample information; and a template converter encoder 540, which is trained to estimate the correlation between the samples according to the encoded data 531 and 532 of each sample obtained from the embedding vector of the sample point output by the point converter encoder 520. The point converter encoder 520 may correspond to the first converter encoder described above. The template converter encoder 540 may correspond to the second converter encoder described above.

[0084] The point converter encoder 520 may include a neural network that is trained to estimate the correlation between sample points extracted from the template based on the information about the sample points included in the template information about the template. The point converter encoder 520 may estimate the correlation between the sample points based on the information about the sample points included in the input template information, and may output an embedding vector corresponding to each sample point. The information about the sample points of the template may include sample point information 501 corresponding to each sample point extracted from the corresponding template.

[0085] The encoded data 531 and 532 of the template may be obtained from the embedding vector of each sample point extracted from the same template output by the point converter encoder 520. For example, the encoded data 531 and 532 of the template may be data generated by connecting or adding the embedding vectors corresponding to the sample points extracted from the corresponding templates.

[0086] For example, sample point information 501 corresponding to each sample point extracted from the first sample may be input to the first point converter encoder 521. An embedding vector corresponding to each sample point extracted from the first sample may be obtained from the first point converter encoder 521, and encoded data 531 of the first sample may be obtained from the embedding vector.

[0087] For example, the sample point information 501 corresponding to each sample point extracted from the second template may be input to the second point converter encoder 522. An embedding vector corresponding to each sample point extracted from the second template may be obtained from the second point converter encoder 522, and encoded data 532 of the second template may be obtained from the embedding vector.

[0088] exist Figure 5 , a first dot converter encoder 521 and a second dot converter encoder 522 that process the template information of each template are shown in separate configurations. However, this illustrates a logical structure of the dot converter 520, wherein the dot converter 520 performs operations in units of template information of each template and includes a hardware configuration of the dot converter 520 that is the same as the number of templates. However, the physical structure of the dot converter 520 is not limited thereto. For example, the template information of each template may be sequentially input to the dot converter 520, and the encoded data 531 and 532 of each template may be obtained through a serial operation. For example, the template information of all templates input to the dot converter 520 may be processed in parallel in units of the template information of each template to obtain the encoded data 531 and 532 of each template.

[0089] The template converter encoder 540 may include a neural network that is trained to estimate the correlation between templates based on the encoded data 531 and 532 of each template. The template converter encoder 540 may estimate the correlation between templates based on the encoded data 531 and 532 of each input template, and may output embedding vectors 551 and 552 corresponding to each template.

[0090] The transformer decoder 560 may include a neural network that is trained to estimate the stitching line of the pattern pair based on the embedding vector pair 553 of the input pattern pair. The transformer decoder 560 may output stitching information 502 of the pattern pair from the embedding vector pair 553 of the input pattern pair. For example, the stitching information 502 output from the transformer decoder 560 may be the same as Figure 4A Corresponding to the stitching information 400 in.

[0091] The embedding vector pairs of all pattern pairs that can be formed by combining the patterns of the garment may be input to the converter decoder 560. For example, when the garment includes "n" patterns, "nC2" different embedding vector pairs corresponding to "nC2" different pattern pairs may be respectively input to the converter decoder 560. As described above, some pattern pairs may not have lines that are stitched together. When the embedding vector pairs of the pattern pairs in which there are no lines stitched together are input to the converter decoder 560, information indicating that there are no stitches may be output. Alternatively, the stitching information 502 may be output together with the reliability score, and the stitching information 502 may be filtered by performing a post-processing operation on the output of the converter decoder 560. For example, stitching information 502 having a reliability score less than a threshold value may be determined to be not a stitching line and may be excluded from the stitching information 502.

[0092] The garment simulation model 500 according to one embodiment may be trained based on the ground truth about the stitching information 502. The garment simulation model 500 may be trained based on a loss about the difference between the predicted stitching information 502 and the ground truth. The garment simulation model 500 may be trained to reduce the loss about the difference between the predicted stitching information 502 and the ground truth.

[0093] For example, the loss regarding the difference between the predicted stitching information 502 and the ground truth may include the loss regarding the difference between the information indicating the predicted stitching line and the ground truth. The information indicating the stitching line may include sample point information corresponding to the start point of the stitching line included in the template and sample point information corresponding to the end point. For example, the information indicating the stitching line may include at least one of the length parameters of the start point and the end point of the stitching line included in the template and the world coordinates of the start point and the end point of the stitching line included in the template.

[0094] For example, the loss regarding the difference between the predicted stitching information 502 and the ground truth may include a loss regarding the difference between the information indicating the stitching direction and the ground truth.

[0095] For example, the loss regarding the difference between the predicted stitching information 502 and the ground truth may include a loss regarding the difference between the ground truth and the shape of the stitching line generated as a simulation result based on the predicted stitching information 502. For example, the loss may be included, which is defined as having a smaller value as the intersection over union (IoU) increases, where the IoU indicates the degree of overlap between the shape of the stitching line generated as a simulation result based on the predicted stitching information 502 and the shape of the stitching line included in the ground truth simulation result.

[0096] Figure 6 is a block diagram showing the structure of a neural network model for predicting stitching information and arrangement information according to one embodiment.

[0097] Reference Figure 6 The clothing simulation model 600 may include a neural network model for predicting stitching information and arrangement information. For example, the data used to generate the clothing simulation result may include not only stitching information but also arrangement information.

[0098] According to an embodiment, the clothing simulation model 600 may include a pattern embedding model 610, an automatic layout model 620, and an automatic stitching model 630. The pattern embedding model 610 may be Figure 5 Corresponding to the template embedding model 510 in .

[0099] The automatic stitching model 630 may be a model for predicting stitching information of a pattern of a garment based on the embedding vector of each pattern output from the pattern embedding model 610. For example, the automatic stitching model 630 may include the above-mentioned Figure 5 The converter decoder 560.

[0100] The automatic arrangement model 620 may be a model that predicts arrangement information of a template based on an embedding vector of each template output from the template embedding model 610 .

[0101] Here, the arrangement information may refer to a point where a 2D pattern is arranged on an object that can wear a 3D virtual garment. For example, the arrangement information may include information about a preset arrangement board or arrangement point on an object that can wear a 3D virtual garment. In addition, the arrangement information may also include information about a pattern arranged on an arrangement point, where an arrangement point is one or more points on a arrangement board on an object that can wear a 3D virtual garment.

[0102] The object may include a 3D object. Specifically, the object may be an object that can wear 3D virtual clothing and may correspond to a 3D virtual image. In addition, the object may be an object that can wear 3D virtual clothing and may include a hanger, a human model, a mannequin, and furniture.

[0103] The arrangement points described herein refer to the initial position of the associated fragment relative to the body part of the 3D avatar before the fragment is connected to one or more adjacent fragments via a seamline. For simulation or visual presentation, when interacting with the 3D avatar, the garment can be spliced ​​by moving the fragments of the garment from the arrangement points to the splicing points. In one embodiment of the 3D avatar, there are 109 arrangement points.

[0104] The arrangement information may include at least one of the name of the arrangement point and the 3D position of the arrangement point. The clothing simulation model 600 may output the arrangement point information in the form of the 3D position coordinates of the arrangement point, the offset and / or the name of the arrangement point. However, the arrangement point information output from the clothing simulation model 600 is not necessarily limited to this. Here, the offset may correspond to a value indicating how much the arrangement point is moved on the x, y, and z axes relative to a specific point on the 3D position coordinates.

[0105] Based on the arrangement information output from the automatic arrangement model 630, the pattern of the garment may be arranged on the object.

[0106] The automatic arrangement model 630 can determine the initial arrangement position of the 2D pattern within the object based on the arrangement information about the predicted part of the object on which the 2D pattern needs to be arranged. The simulation result can include the 3D shape of the garment, wherein the 2D pattern is arranged at a specific position on the object according to the arrangement information, and the garment is sewn together according to the sewing information for wearing.

[0107] The clothing simulation model 600 according to one embodiment can be trained based on ground truth about arrangement information. The clothing simulation model 600 can be trained based on the loss of the difference between the predicted arrangement information and the ground truth. Various different training algorithms (including supervised training algorithms) can be used to train based on the output data of the clothing simulation model 600 and the ground truth.

[0108] According to one embodiment, the embedding vector of each template output from the template embedding model 610 can be used to predict stitching information and arrangement information. The template embedding model 610 can be trained based on a loss about the difference between the predicted stitching information and the ground truth and a loss about the difference between the predicted arrangement information and the ground truth.

[0109] Figure 7 is a simplified diagram showing a layout point and a layout plate associated therewith according to one embodiment.

[0110] Figure 77 is a simplified diagram showing arrangement points 730 and arrangement plates 750 associated therewith according to one embodiment. For example, there may be 730 arrangement points associated with the 3D avatar 710. The arrangement points 730 are points where 2D patterns are arranged on or above the body parts of the 3D avatar 710. The 2D patterns are arranged on the arrangement points and then spliced ​​onto the 3D avatar 710 or worn on the 3D avatar 710 to perform clothing simulation.

[0111] The name of the arrangement point 730 may correspond to the classification name of the arrangement board 750 to which the arrangement point 730 is assigned. For example, the name of the arrangement point 730 may be a major classification name obtained by broad classification of the arrangement board 750 including the arrangement point 730, or may be a minor classification name obtained by detailed classification of the arrangement board 750 having the major classification name.

[0112] The layout plate includes one or more layout points, and the one or more layout points correspond to each body part reflected by the body shape and posture of the 3D virtual image 710. The layout plate 750 can correspond to the body part of the 3D virtual image 710. Each body part layout plate can be assigned to both hands, both feet, both elbows, both knees, both arms, both wrists, left and right sides of the body, etc. For example, the body part layout plate can be formed in the shape of a 3D column around the corresponding body part. The 3D column can be shaped as a cylinder, an elliptical column, and a polygonal prism, etc. In addition to the 3D column, the body part layout plate can also be formed in other shapes.

[0113] Arrangement plate 750 can be divided into body arrangement plate, arm arrangement plate, wrist arrangement plate, shoulder arrangement plate, leg arrangement plate, ankle arrangement plate, lower body arrangement plate, neck arrangement plate, and head arrangement plate by rough classification. Body arrangement plate can include two sub-arrangement plates: left body arrangement plate and right body arrangement plate, or front body arrangement plate and rear body arrangement plate. Arm arrangement plate can include sub-arrangement plates for left arm, right arm and both arms. Wrist arrangement plate can include sub-arrangement plates for left wrist, right wrist and both wrists. Shoulder arrangement plate can include sub-arrangement plates for left shoulder, right shoulder and both shoulders. Leg arrangement plate can include sub-arrangement plates for left leg, right leg and both legs or front leg and back leg. Ankle arrangement plate can include sub-arrangement plates for left ankle, right ankle and both ankles. The lower body layout plate may include sub-layout plates for the left lower body and the right lower body, or the front lower body and the back lower body. The neck layout plate may include sub-layout plates for the left neck, the right neck, and the double neck. The head layout plate may include sub-layout plates divided according to three directions of the head (e.g., left, right, and vertical directions, or X (horizontal), Y (vertical), and Z (depth) directions).

[0114] The name of the arrangement point 730 may correspond to the category name of the arrangement board 750 to which the arrangement point 730 belongs based on the arrangement board 750. For example, the name of the arrangement point 730 may be a general category name of the arrangement board including the arrangement point 730, or may be a category name of a detailed arrangement board belonging to a main category of the arrangement board.

[0115] The placement board 750 may refer to a board including placement points corresponding to each body part that reflects the body shape and posture of the 3D virtual image 710. The placement board 750 is a placement board corresponding to each body part of the three-dimensional virtual image (such as both hands, both feet, both elbows, both knees, both arms, both wrists, left and right torsos, etc.) ("arrangement board for each body part"). For example, the arrangement board for each body part may be configured in the form of a three-dimensional pillar surrounding the corresponding body part. The three-dimensional pillar may include a cylinder, an elliptical pillar, a polygonal pillar, etc. The placement board for each body part may be composed of various different shapes other than a three-dimensional pillar, but is not necessarily limited to this.

[0116] The placement board 750 is, for example, a body placement board, an arm placement board, a wrist placement board, a shoulder placement board, a leg placement board, and an ankle placement board (it can be divided into major categories such as ankle placement board, lower body placement board, neck placement board, and head placement board, but is not necessarily limited to this).

[0117] The torso layout plate may consist of two detailed layout plates (e.g., a left torso layout plate and a right torso layout plate, or a front torso layout plate and a rear torso layout plate). For example, the arm layout plate may consist of a detailed layout plate for the left arm and the right arm. For example, the wrist placement plate may consist of a detailed placement plate for the left wrist and the right wrist. For example, the shoulder layout plate may consist of a detailed layout plate for the left shoulder and the right shoulder. For example, the leg layout plate may consist of a detailed layout plate for the left leg and the right leg or the front leg and the rear leg.

[0118] For example, the automatic layout model ( Figure 6 The automatic layout model 620 in the example may output layout information indicating that the target template corresponds to any one of the layout points 730. Based on the layout information, the target template may be arranged on the 3D virtual image 710, so that the virtual image 710 is arranged at a layout point corresponding to a specific position (e.g., a center point) of the target template. Alternatively, the target template may be arranged on the 3D virtual image 710, so that the virtual image 710 is arranged at the center of a layout board in which a layout point corresponding to a specific position (e.g., a center point) is arranged.

[0119] For example, the automatic layout model ( Figure 6 The automatic layout model 620 in the embodiment may output layout information indicating that the target template corresponds to a layout plate, such as a front body (e.g., Body_Front) or a back body (e.g., Body_Back). Based on the layout information, the target template may be arranged at the center of the corresponding layout plate.

[0120] Figure 8 is a schematic diagram of an automatic arrangement device according to an embodiment.

[0121] Reference Figure 8 According to an embodiment, a clothing simulation device (eg, an automatic arrangement device) 800 may include a processor 801, a memory 803, and an input / output device (I / O) 805. The clothing simulation device 800 may include a processor 801, a memory 803, and an input / output device (I / O) 805. Figures 1 to 7 The device of the described clothing simulation method. Hereinafter, the clothing simulation device 800 may be referred to as "device". The communication interface, the processor 801, the memory 803 and the I / O device 805 may be connected to each other via a communication bus. The communication interface may obtain the pattern information of each pattern of the clothing, and the pattern information includes information about the sample points extracted from each pattern.

[0122] According to an embodiment, the processor 801 may execute the above Figures 1 to 7At least one step of the clothing simulation method described in . For example, the processor 801 may perform at least one of the following steps: obtaining pattern information of each pattern of the clothing, the pattern information including information about sample points extracted from each pattern; predicting stitching information about the pattern of the clothing based on the embedding vector of each pattern obtained by applying the pattern information to a pattern embedding model trained to estimate the correlation between input pattern information; and generating a simulation result of the clothing based on the stitching information.

[0123] The memory 803 according to an embodiment may be a volatile memory or a non-volatile memory, and may store the above-mentioned Figures 1 to 7 The memory 803 may store data related to the clothing simulation method described in the description. For example, the memory 803 may store data generated during the execution of the clothing simulation method or store data required for the execution of the clothing simulation method. For example, the memory 803 may store sample information of each sample. For example, the memory 803 may store the weight of at least one layer of a neural network included in the clothing simulation model.

[0124] The apparatus 800 according to an embodiment may exchange data with an external device (e.g., a personal computer or a network) or a user through an input / output device 805. For example, the apparatus 800 may receive pattern information of each pattern through the input / output device 805 and output a clothing simulation result.

[0125] According to one embodiment, the memory 803 may store the Figures 1 to 7 The processor 801 may execute the program stored in the memory 803 and control the apparatus 800. The program code executed by the processor 801 may be stored in the memory 803.

[0126] The device 800 according to an embodiment may further include other components not shown. For example, the device 800 may include a communication module, which provides the device 800 with a function of communicating with other electronic devices or other servers through a network. Some or all of the clothing simulation models may be stored in an external memory. The device 800 may communicate with the clothing simulation models stored in the external memory through the communication module to obtain sewing information and / or arrangement information of the clothing pattern. In addition, the device 800 may further include other components, such as a transceiver, various sensors, and a database.

[0127] The above-described device can be implemented by hardware components, software components, and / or a combination of hardware components and software components. For example, the device and components described in the embodiment can be embodied using one or more general-purpose computers or special-purpose computers such as processors, controllers, arithmetic logic units (arithmetic logic units, ALU), digital signal processors (digital signal processors), microcomputers, field programmable gate arrays (field programmable gate arrays, FPGAs), programmable logic units (programmable logic units, PLUs), microprocessors, or any other device capable of executing and responding to instructions (instructions). The processing device can execute an operating system (OS) and one or more application software executed in the operating system. In addition, the processing device responds to the execution of the software to access, store, operate, process and generate data. For ease of understanding, a method with only one processing device is described, but a person of ordinary skill in the art should understand that the processing device can include multiple processing elements (processing element) and / or multiple types of processing elements. For example, the processing device can include multiple processors or a processor and a controller. Furthermore, other processing configurations such as parallel processors can also be included.

[0128] Software can include a computer program, code, instructions, or a combination of more than one of these, that causes a processing device to operate in a desired manner, or that, alone or collectively, commands a processing device. Software and / or data can be permanently or temporarily embody any type of device, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave for interpretation by a processing device or for providing commands or data to a processing device. Software is distributed on computer systems connected by a network and can be stored or executed in a distributed manner. Software and data can be stored in one or more computer-readable storage media.

[0129] The method according to the embodiment is embodied in the form of a program command that can be executed by a variety of computer means and recorded in a computer-readable medium. The computer-readable medium can include program commands, data files, data structures, etc. in a single or combined form. The program instructions recorded in the medium can be instructions specially designed and constructed to implement the embodiment, or instructions that ordinary technicians in the field of computer software can use based on known ones. The computer-readable recording medium can include magnetic media such as hard disks, floppy disks, and tapes; optical media similar to CD-ROMs, DVDs, etc.; magneto-optical media similar to floptical disks, and hardware devices specially constructed for storing and executing program commands similar to read-only memories (ROMs), random access memories (RAMs), flash memories, etc. Examples of program instructions include not only machine language codes generated by compilers, but also high-level language codes that can be executed by computers by using interpreters, etc.

[0130] To perform the operations of the embodiments, the hardware device may be configured in such a manner that the operations are implemented with one or more software modules, and vice versa.

[0131] In summary, the embodiments are described through limited drawings, and a person skilled in the art can make various changes and modifications based on the description. For example, the described techniques are performed in a different order from the described method, and / or the described systems, structures, devices, circuits and other components are combined or combined in a different form from the described method, or replaced or substituted by other components or equivalents, and appropriate results can also be obtained.

[0132] Accordingly, other embodiments, other examples, and their equivalents are within the scope of the appended claims.

Claims

1. A clothing simulation method, comprising the following steps: Obtaining pattern information of each pattern of clothing, wherein the pattern information includes information about sample points extracted from each of the patterns; predicting stitching information about the template based on an embedding vector of each of the templates obtained by applying the template information to a template embedding model trained to estimate correlations between input template information; as well as Based on the stitching information, a simulation result of the garment is generated.

2. The clothing simulation method according to claim 1 further comprises the following steps: Extracting a predetermined number of sample points from each of the samples of the garment; as well as The positions of the sample points are adjusted based on the positions of the predetermined type of points included in each of the templates.

3. The clothing simulation method according to claim 1, wherein: The obtaining of the pattern information of each of the patterns includes obtaining information on the sample points of the target pattern based on the length of a contour line from a reference point of the target pattern in the pattern of the garment to each sample point extracted from the target pattern.

4. The clothing simulation method according to claim 1, wherein: The template embedding model includes a transformer encoder, and The predicting the stitching information includes obtaining information indicating a stitching line of the pattern pair extracted from the pattern by applying an embedding vector pair corresponding to the pattern pair extracted from the pattern to a transformer decoder.

5. The clothing simulation method according to claim 1, wherein: The template embedding model includes: a first transformer encoder, the first transformer encoder being trained to estimate the correlation between sample points extracted from the same sample according to the input sample information; and A second transformer encoder is trained to estimate correlations between samples based on encoded data of each of the samples obtained from an embedding vector of a sample point output by the first transformer encoder.

6. The clothing simulation method according to claim 5, wherein: The predicting of the stitching information includes: Obtaining encoded data of the first sample, including an embedding vector for each sample point corresponding to the first sample, by applying the sample information of the first sample to the first transformer encoder; Obtaining encoded data of the second sample by applying the sample information of the second sample to the first transformer encoder, including an embedding vector for each sample point corresponding to the second sample; Obtaining an embedding vector for the first sample and an embedding vector for the second sample by applying the encoded data for the first sample and the encoded data for the second sample to the second transformer encoder; and Stitching information about a pattern pair of the first pattern and the second pattern is predicted by applying the embedding vector of the first pattern and the embedding vector of the second pattern to a transformer decoder.

7. The clothing simulation method according to claim 1, wherein: The stitching information includes a pair of lines that are stitched together within the pattern.

8. The clothing simulation method according to claim 7, wherein: The stitching information also includes information indicating a stitching direction of the thread.

9. The clothing simulation method according to claim 4, wherein: The information indicating the seam lines of the pattern pair includes: information about sample points extracted from a first sample of the sample pair and corresponding to start and end points of a stitch line included in the first sample; and Information about sample points extracted from a second template of the template pair and corresponding to start points and end points of a stitch line included in the second template.

10. The clothing simulation method according to claim 1, wherein: The template embedding model is trained based on a loss on the difference between the predicted stitching information and the ground truth.

11. The clothing simulation method according to claim 1, wherein: The predicting stitching information about the template includes predicting arrangement information and the stitching information about the template based on the embedding vector of each template, and The generating the simulation result of the garment includes generating the simulation result of the garment based on the stitching information and the arrangement information.

12. The clothing simulation method according to claim 11, wherein: The template embedding model is trained based on a loss on the difference between the predicted stitching information and the ground truth and a loss on the difference between the predicted arrangement information and the ground truth.

13. The clothing simulation method according to claim 1, wherein: The information about the sample point includes at least one of the following information: information indicating the location of the sample point within the pattern of the garment; information indicating the location of the sample point within the garment; Information indicating the positional relationship between adjacent sample points; as well as Information indicating the type of the sample point.

14. A non-transitory computer-readable storage medium having stored therein instructions, which, when executed by a processor, cause the processor to perform the method according to claim 1.

15. A clothing simulation device, comprising: one or more processors; as well as a memory having instructions stored thereon, which, when executed by the one or more processors, cause the one or more processors to: Obtaining pattern information of each pattern of clothing, wherein the pattern information includes information about sample points extracted from each of the patterns; predicting stitching information about the template based on an embedding vector of each of the templates obtained by applying the template information to a template embedding model trained to estimate correlations between input template information; as well as Based on the stitching information, a simulation result of the garment is generated.

Images