Using Artificial Intelligence for UV Mapping on 3D Objects

Abstract

The present invention discloses using artificial intelligence for UV mapping on 3D objects. Various embodiments herein describe systems and techniques for generating seams for 3D models. The techniques include: generating one or more inputs for one or more trained machine learning models based on the 3D model; providing the one or more inputs to the one or more trained machine learning models; receiving seam prediction data generated based on the one or more inputs from the one or more trained machine learning models; and placing one or more predicted seams on the 3D model based on the seam prediction data.

Description

Background

[0001] Field of Various Embodiments

[0002] The present invention relates to the field of waste treatment, and more particularly to an emergency emission supervision system for waste incineration power generation, an analysis system, and a user terminal device.

[0003] Description of Related Art

[0004] 3D graphics applications enable users to design and generate 3D models for various applications, including 3D model-based video games, special effects, design visualization, and printing physical objects. Typically, one or more 2D images are used to texture the 3D model via a UV mapping process. The UV mapping process typically involves defining a set of seams at which the 3D model should be split or divided into one or more parts. The one or more parts are unfolded or flattened into a single 2D image (two-dimensional UV space) so that textures can be drawn on the 2D image. In this way, the textures on the 2D image are applied or "mapped" to the 3D model.

[0005] In UV mapping, improper placement of seams on the 3D model can cause several problems. In particular, if too many seams, too few seams, or incorrectly placed seams are defined for a given 3D model, the resulting 2D texture image may contain several problems. As described in detail below, these problems include low layout efficiency, distortion, visual artifacts, and seams that do not lie at semantic boundaries.

[0006] Regarding low layout efficiency, when the 3D model is flattened, the flattened segments are arranged in a single 2D image. The size and shape of each flattened segment affect how the flattened segments are laid out on the 2D image. Depending on the arrangement, a certain amount of blank space will be reserved around the flattened segments arranged on the 2D image. However, a large amount of blank space and thus non-optimal use of the space on the 2D image are undesirable. In particular, the larger the 2D image, the more computer storage space is required to store the 2D image, the more network bandwidth is required to transmit the 2D image, and the more memory is utilized when the computing device renders the 3D model with the texture. Therefore, an efficient layout that minimizes the blank space in the 2D image would be more desirable.

[0007] Regarding distortion, when applying a texture on a 2D image to a 3D model, the 2D image may be stretched or compressed. For example, if an image of a world map is placed on a sphere, a single rectangular segment will cause the image to stretch along the equator and compress along the poles. The more seams added, the more segments are created, thus reducing the amount of distortion when applying the 2D image. However, using a large number of additional segments affects the layout efficiency of the 2D image and is therefore generally not the best solution for reducing distortion. Referring to the sphere example, dividing the 3D model into a single rectangular segment will result in less blank space in the 2D image compared to dividing the 3D model into several non-rectangular segments.

[0008] Regarding visual artifacts, when applying a texture on a 2D image to a 3D model, visual artifacts or visual discontinuities may exist around the location of the seams. Therefore, it is undesirable to place seams in visually prominent areas. For example, when an image of a world map is placed on a sphere, there may be lines, distortion, or other visual artifacts at the location where the edges of the 2D image are joined together. Adding additional seams to reduce distortion will increase the number of places where such visual artifacts are prominent.

[0009] In addition, seams divide the 3D model into one or more flattened segments. Placing seams such that the 3D model is divided into logical parts (i.e., along semantic boundaries) enables an artist to identify and edit the textures of different parts of the model. For example, for a human body model, dividing the model into segments corresponding to the arms, legs, head, and torso will allow the artist to identify and edit the textures of each part of the body. However, incorrect or undesirable seam placement may result in the 3D model being divided into too many segments, segments with more than one logical semantic correspondence, or segments with no logical semantic correspondence to the 3D model. Referring to the human body model example, although dividing the model into additional segments may result in less distortion, the additional segments may not semantically correspond to the underlying 3D model of the human body. Therefore, it will be difficult for the artist to determine which parts of the body should be placed on which segments, or at which location on each segment and in which orientation the body parts should be drawn.

[0010] As mentioned above, there is a need in the art for more efficient and robust techniques for generating UV mapping seams for 3D models. Summary of the Invention

[0011] One embodiment of the present application describes a computer-implemented method for generating a set of seam predictions for a three-dimensional (3D) model. The method includes: generating one or more representations of the 3D model based on the 3D model as inputs to one or more trained machine learning models; generating a set of seam predictions associated with the 3D model by applying the one or more trained machine learning models to the one or more representations of the 3D model, wherein each seam prediction included in the set of seam predictions identifies a different seam along which the 3D model can be cut; and placing one or more seams on the 3D model based on the set of seam predictions.

[0012] Compared with previous methods, at least one advantage of the disclosed technology is that, unlike previous methods, the computer system automatically generates seams for a 3D model that take into account semantic boundaries and seam positions while minimizing distortion and reducing the number of segments required to protect semantic boundaries. Additionally, using trained machine learning models allows the computer system to generate seams based on learned best practices (i.e., based on imperceptible criteria extracted during the machine learning model training process). BRIEF DESCRIPTION OF THE DRAWINGS

[0013] To be able to understand the above-described features of the various embodiments in detail, the inventive concept briefly outlined above may be described more specifically with reference to the various embodiments, some of which are illustrated in the drawings. However, it should be noted that the drawings only show typical embodiments of the inventive concept and should therefore in no way be considered as limiting the scope, and there are other equally effective embodiments.

[0014] Figure 1 is a schematic diagram showing a computing system configured to implement one or more aspects of the present disclosure.

[0015] Figure 2 is according to various embodiments of the present disclosure Figure 1 a more detailed illustration of a 3D modeling application and a machine learning model.

[0016] Figure 3 is according to various embodiments of the present disclosure by Figure 1 a flowchart of the method steps for predictive seam generation performed by a 3D modeling application.

[0017] Figure 4 is according to various embodiments of the present disclosure by Figure 1 a flowchart of the method steps for predictive seam generation using a 2D image performed by a 3D modeling application.

[0018] Figure 5 is according to various embodiments of the present disclosure byFigure 1 Flowchart of method steps for predictive seam generation represented by a graph, performed by a 3D modeling application.

[0019] Figure 6 Illustration of a predictive seam generation process using 2D images according to various embodiments of the present disclosure.

[0020] Figure 7 Illustration of a predictive seam generation process represented by a graph according to various embodiments of the present disclosure. Detailed Description

[0021] In the following description, numerous specific details are set forth to provide a more thorough understanding of the various embodiments. However, it will be apparent to one of ordinary skill in the art that the inventive concept may be practiced without one or more of these specific details.

[0022] Figure 1 A computing device 100 configured to implement one or more aspects of the present disclosure is shown. As shown, the computing device 100 includes an interconnect (bus) 112 that connects one or more processing units 102, an input / output (I / O) device interface 104 coupled to one or more input / output (I / O) devices 108, a memory 116, a storage device 114, and a network interface 106 connected to a network 110.

[0023] The computing device 100 includes a server computer, a desktop computer, a laptop computer, a smart phone, a personal digital assistant (PDA), a tablet computer, or any other type of computing device configured to receive input, process data, and optionally display images, and is adapted to practice one or more embodiments. The computing device 100 described herein is illustrative, and any other technically feasible configuration falls within the scope of the present disclosure.

[0024] The processing unit 102 includes any suitable processor implemented as: a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), an artificial intelligence (AI) accelerator such as a tensor processing unit (TPU), any other type of processing unit, or a combination of different processing units, such as a CPU configured to operate in conjunction with a GPU. Generally, the processing unit 102 can be any technically feasible hardware unit capable of processing data and / or executing software applications. Additionally, in the context of the present disclosure, the computing elements shown in the computing device 100 may correspond to physical computing systems ( For example , systems in a data center), or may be virtual computing implementations executed within a computing cloud.

[0025] In one embodiment, the I / O device 108 includes devices capable of providing input, such as a keyboard, a mouse, a touch-sensitive screen, etc., and devices capable of providing output, such as a display device. Additionally, the I / O device 108 may include devices capable of receiving both input and providing output, such as a touch screen, a Universal Serial Bus (USB) port, etc. The I / O device 108 may be configured to receive various types of input from an end user ( For example , the designer) of the computing device 100, and also provide various types of output to the end user of the computing device 100, such as a displayed digital image or digital video or text. In some embodiments, one or more of the I / O devices 108 are configured to couple the computing device 100 to the network 110.

[0026] The network 110 includes any technically feasible type of communication network that allows for the exchange of data between the computing device 100 and an external entity or device, such as a web server or another networked computing device. For example, the network 110 may include a Wide Area Network (WAN), a Local Area Network (LAN), a wireless (WiFi) network, and / or the Internet, etc.

[0027] The storage device 114 includes non-volatile storage devices for applications and data, and may include fixed or removable disk drives, flash memory devices, and CD-ROM, DVD-ROM, Blu-ray Disc, HD-DVD, or other magnetic, optical, or solid-state storage devices. The 3D modeling application 118 and the machine learning model 120 may be stored in the storage device 114 and loaded into the memory 116 when executed.

[0028] The memory 116 includes Random Access Memory (RAM) modules, flash memory cells, or any other type of memory cells or combinations thereof. The processing unit 102, the I / O device interface 104, and the network interface 106 are configured to read data from and write data to the memory 116. The memory 116 includes various software programs executable by the processing unit 102 and application data associated with the software programs, including the 3D modeling application 118 and the machine learning model 120. The 3D modeling application 118 and the machine learning model 120 are described in further detail below with respect to Figure 2 .

[0029] Figure 2 is a more detailed illustration of the Figure 1 3D modeling application 118 and the machine learning model 120 according to various embodiments of the present disclosure. As shown, the 3D modeling application 118 includes, but is not limited to, a 3D model 220, a preprocessing engine 222, a seam visualization engine 224, a postprocessing engine 226, and a graphical user interface 228.

[0030] The 3D modeling application 118 automatically generates a set of one or more predicted seams for the 3D model 220 using one or more models of the machine learning model 120. Each predicted seam indicates a possible seam along which the 3D model 220 can be cut and flattened. In some embodiments, the 3D modeling application 118 also includes tools and features for generating or modifying 3D models, such as the 3D model 220. In other embodiments, the 3D modeling application 118 can be an application or tool separate from the application for generating or modifying the 3D model 220 and can receive the 3D model 220 for generating the predicted seams.

[0031] In operation, the preprocessing engine 222 generates 3D model data 230 based on the 3D model 220 and provides the 3D model data 230 as input data to the machine learning model 120. In some embodiments, the 3D model data 230 includes one or more representations of the 3D model 220.

[0032] In one or more embodiments, the preprocessing engine 222 generates the 3D model data 230 by rendering a set of 2D images based on the 3D model 220. Each 2D image in the set of 2D images depicts a different perspective of the 3D model 220. Different perspectives of the 3D model can be any combination of different scales, viewing angles, viewing positions, 3D model poses, and / or spatial deformations of the 3D model. Additionally, for the portion of the 3D model 220 that is visible in the depicted perspective, each 2D image can correspond to different types of information associated with the 3D model 220. Different types of 3D model information can be, for example, normal vectors, 3D positions, wireframes, Gaussian curvatures, shape indices, curvatures, sphericity ratios, or any other type of intrinsic measure of the model geometry.

[0033] The 2D image corresponding to the normal vector information represents the normal vector of each vertex of the 3D model 220 visible from the perspective of the 2D image in red, green, and blue (RGB) values. Interpolate positions between vertices on the 3D model in the 2D image. The 2D image corresponding to the 3D position information represents the 3D position of each rendered pixel of the 3D model from the perspective of the 2D image in RGB values. The 2D image corresponding to the wireframe information represents the lines corresponding to the edges of the 3D model 220 visible from the perspective of the 2D image in grayscale values. The 2D image corresponding to the principal curvature represents the extreme bending (i.e., local minimum and maximum) of the surface of the 3D model at each point in hue saturation value (HSV) values. Additionally, Gaussian curvature, shape index, curvature, and sphericity ratio can be derived from the principal curvature values. The 2D image corresponding to the Gaussian curvature information represents the likelihood of being mapped to a plane while minimizing distortion in HSV values. The 2D image corresponding to the shape index information represents five distinct local curvature families in HSV values. The 2D image corresponding to the curvature information represents the pure local curvature based on a weighted neighborhood in HSV values. The 2D image corresponding to the sphericity ratio information represents the proximity of the local curvature to a sphere in HSV values.

[0034] In one or more embodiments, the preprocessing engine 222 generates 3D model data 230 by generating a graph representation of the 3D model 220. The graph representation includes a set of vertices corresponding to the vertices of the 3D model 220 and a set of edges corresponding to the edges of the 3D model 220. In some embodiments, each vertex of the graph representation includes additional information associated with the corresponding vertex of the 3D model 220, such as the normal vector of the corresponding vertex, the 3D position information of the corresponding vertex, and the likelihood that the corresponding vertex is mapped to a plane while minimizing distortion.

[0035] In some embodiments, the preprocessing engine 222 divides the 3D model 220 into multiple groups. Each group does not occlude or intersect with other groups in the multiple groups. For example, a 3D model of a car can be divided into a first group including the car wheels and a second group including the car body. The preprocessing engine 222 uses any of the techniques discussed above to generate corresponding 3D model data 230 for each of the multiple groups.

[0036] In an embodiment, the preprocessing engine 222 divides the 3D model 220 into a set of connected components (c) to divide the 3D model 220 into multiple groups. A connected component can include a set of vertices and edges of the 3D model 220 that are connected to each other. For example, for a 3D model of a car, the edges and vertices forming the right front wheel of the car will be connected to each other, but will not be connected to the edges and vertices forming the left front wheel of the car or the edges and vertices forming the car body. The preprocessing engine 222 can divide the 3D model of the car into five connected components: the right front wheel, left front wheel, right rear wheel, left rear wheel, and the car body.

[0037] For each connected component c[i] in the set c, the preprocessing engine 222 adds the component to a new group G. For each remaining component c[j] in the set c, the preprocessing engine 222 determines whether c[i] and c[j] cross and whether c[i] and c[j] occlude each other. If c[i] and c[j] do not cross and do not occlude each other, the preprocessing engine 222 adds the component c[j] to the group G and removes the component c[j] from the set c. This process is repeated for each remaining component in the set c until the set is empty.

[0038] The machine learning model 120 receives 3D model data 230 from the 3D modeling application 118 and generates prediction seam data 240 indicative of one or more predicted seams for the 3D model 220 based on the 3D model data 230. If the 3D model data 230 corresponds to a particular part of the 3D model 220 and / or a particular perspective of the 3D model 220, the predicted seams generated by the machine learning model 120 correspond to the particular part of the 3D model 220 and / or the particular perspective of the 3D model 220.

[0039] The machine learning model 120 includes one or more trained machine learning models, such as model 210(1) and model 210(2). Although Figure 2 two machine learning models are shown, the machine learning model 120 may include any number of machine learning models. Each machine learning model in the machine learning model 120 can be any technically feasible machine learning model. In some embodiments, the machine learning model 120 includes a recurrent neural network (RNN), a convolutional neural network (CNN), a deep neural network (DNN), a deep convolutional network (DCN), a deep belief network (DBN), a restricted Boltzmann machine (RBM), long short-term memory (LSTM) cells, gated recurrent units (GRU), a generative adversarial network (GAN), a self-organizing map (SOM), and / or other types of artificial neural networks or components of artificial neural networks. In other embodiments, the machine learning model 120 includes functions that perform clustering, principal component analysis (PCA), latent semantic analysis (LSA), Word2vec, and / or another unsupervised, semi-supervised, reinforcement, or self-supervised learning technique. In some embodiments, the machine learning model 120 includes a neural network (shallow or deep), a regression model, a support vector machine, a decision tree, a random forest, a gradient boosting tree, a naive Bayes classifier, a Bayesian network, a hierarchical model, and / or a combined model.

[0040] In some embodiments, the machine learning model 120 includes a trained machine learning model that is trained to receive a 2D image of a 3D model and generate data indicative of predicted seams on the 2D image. For example, the data can indicate which pixels of the 2D image correspond to the predicted seams.

[0041] In some embodiments, the machine learning model 120 includes a trained machine learning model that is trained to receive a graph representation corresponding to a 3D model and generate data indicative of which vertices and / or edges of the graph representation correspond to predicted seams.

[0042] In some embodiments, the machine learning model 120 includes a trained machine learning model that is trained to receive a volumetric representation corresponding to a 3D model and generate data indicative of which voxels of the volumetric representation correspond to predicted seams.

[0043] In some embodiments, the machine learning model 120 includes a trained machine learning model that is trained to receive point cloud data of a 3D model and generate data indicative of which points in the point cloud representation correspond to predicted seams.

[0044] In some embodiments, the machine learning model 120 includes a trained machine learning model that is trained to receive a parametric representation corresponding to a 3D model and generate data indicative of which parameter values in the parametric representation correspond to predicted seams.

[0045] Additionally, in some embodiments, the machine learning model 120 includes one or more trained machine learning models that receive model data that includes an initial set of predicted seams and generate an improved or adjusted set of predicted seams, for example, by removing predicted seams that do not meet a threshold probability value, reducing the predicted seams to a specified thickness, connecting two or more predicted seams to fill gaps, smoothing the predicted seams, removing isolated vertices, or adjusting the predicted seams based on model symmetry.

[0046] In some embodiments, the output data generated by the machine learning model 120 includes probability values associated with the predicted seams. The probability values can indicate the likelihood that a pixel, vertex, edge, voxel, or point corresponds to a seam of the 3D model 220. For example, a probability value of 0 can be associated with a pixel, vertex, or edge to indicate that the pixel, vertex, or edge is unlikely to correspond to a seam, while a probability value of 1 can be associated with a pixel, vertex, edge, voxel, or point to indicate that the pixel, vertex, edge, voxel, or point is likely to correspond to a seam. In some embodiments, the probability value can be one of two binary values, for example, indicating a seam or not a seam. In some embodiments, the probability value can be a range of values, for example, to indicate a percentage likelihood.

[0047] The models of the machine learning model 120 can be trained to receive different types of input data, generate different types of output data, and / or be trained with different models and training hyperparameters and weights. As an example, the model 210(1) can be trained to receive 2D images depicting views of a 3D model and generate an output indicating predicted seams for the views of the 3D model depicted in the 2D image. The model 210(2) can be trained to receive graph data corresponding to a 3D model and generate an output indicating which edges of the graph are predicted to correspond to seams. As another example, the model 210(1) can be trained to receive model data corresponding to a 3D model and generate an output indicating predicted seams, while the model 210(2) can be trained to receive predicted seam data corresponding to one or more predicted seams for a 3D model and generate an output indicating one or more improved predicted seams for the 3D model. In some embodiments, one or more models of the machine learning model 120 are selected based on the type of input received from the 3D modeling application 118.

[0048] In some embodiments, different training datasets are used to train the models of the machine learning model 120. Each training dataset can correspond to a different style of 3D model, such as organic, man-made, human, automotive, animal, or other categories or model types. One or more models of the machine learning model 120 can be selected based on the style of the 3D model 220.

[0049] In some embodiments, the training data provided by the user is used to train one or more models of the machine learning model 120. For example, the user can provide a training set of 3D models that the user previously created and for which seams were defined. Thus, one or more models are trained to generate predicted seams based on preferences and other criteria learned from the user's previous work. One or more models of the machine learning model 120 can be selected based on the user's use of the 3D modeling application 118.

[0050] In some embodiments, multiple models of the machine learning model 120 receive the same input provided by the 3D modeling application 118. Thus, the machine learning model 120 can generate multiple sets of predicted seams based on a single input.

[0051] The seam visualization engine 224 receives the predicted seam data 240 generated by the machine learning model 120 and processes the predicted seam data 240. In some embodiments, processing the predicted seam data 240 includes aggregating or combining multiple sets of predicted seam data. In some embodiments, the multiple sets of predicted seam data correspond to the same part of the 3D model 220 or the same perspective of the 3D model 220. For example, multiple inputs provided to the machine learning model 120 may correspond to the same perspective of the 3D model 220 but include different information related to the 3D model 220. A corresponding set of predicted seam data may be generated based on each input. As another example, the machine learning model 120 may generate multiple sets of predicted seam data for each input provided by the 3D modeling application 118. The seam visualization engine 224 aggregates the multiple sets of predicted seam data to generate a single set of predicted seams for a part of the 3D model 220 or the 3D model 220 at that perspective. In one or more embodiments, the seam visualization engine 224 aggregates the multiple sets of predicted seam data by identifying predicted seams that are the same or within a threshold distance of each other among the multiple sets of predicted seam data or removing predicted seams of predicted seam data that do not belong to a threshold number of sets.

[0052] In some embodiments, the multiple sets of predicted seam data correspond to different parts of the 3D model 220 or different perspectives of the 3D model 220. The seam visualization engine 224 combines the multiple sets of predicted seam data. In one or more embodiments, combining the multiple sets of predicted seam data includes identifying and merging predicted seams that overlap or connect among the multiple sets of predicted seam data to generate a single set of predicted seams for the 3D model 220. After generating the single set of predicted seams, the predicted seams are placed on the 3D model 220. In one or more embodiments, combining the multiple sets of predicted seam data includes, for each set of predicted seam data, placing one or more predicted seams on the 3D model 220 based on that set of predicted seam data. The seam visualization engine 224 may place the predicted seams on the 3D model 220 in a manner similar to that discussed below. After the predicted seams from the multiple sets of predicted seam data are placed on the 3D model 220, the 3D model 220 includes a single set of predicted seams.

[0053] In some embodiments, processing the predicted seam data 240 includes placing one or more predicted seams on the 3D model 220 based on the predicted seam data 240. In some embodiments, the predicted seam data 240 indicates positions on the 3D model 220 where seams can be placed. The seam visualization engine 224 places one or more predicted seams onto the 3D model 220 based on the positions indicated by the predicted seam data 240. In some embodiments, the predicted seam data 240 indicates one or more edges and / or one or more vertices of the 3D model 220 that are predicted to correspond to seams. The seam visualization engine 224 places one or more predicted seams onto the 3D model 220 based on the one or more edges and / or one or more vertices.

[0054] In some embodiments, each edge and / or vertex of the 3D model 220 is associated with a probability value indicating the likelihood that the edge or vertex corresponds to a seam. Placing one or more predicted seams onto the 3D model 220 includes updating the probability values of one or more edges and / or one or more vertices of the 3D model 220 based on the predicted seam data 240.

[0055] In some embodiments, the predicted seam data 240 includes corresponding probability values associated with each vertex of a graph representation of the 3D model 220. Placing one or more predicted seams onto the 3D model 220 includes, for each vertex of the graph representation, determining the corresponding vertex of the 3D model 220 and updating the probability value of the corresponding vertex of the 3D model 220 based on the probability value associated with the vertex of the graph representation.

[0056] In some embodiments, the predicted seam data 240 includes corresponding probability values associated with each edge of a graph representation of the 3D model 220. Placing one or more predicted seams onto the 3D model 220 includes, for each edge of the graph representation, determining the corresponding edge of the 3D model 220 and updating the probability value of the corresponding edge of the 3D model 220 based on the probability value associated with the edge of the graph representation.

[0057] In some embodiments, the predicted seam data 240 includes a plurality of 2D images depicting the 3D model 220, and for each 2D image of the plurality of 2D images, indicates predicted seams on the 2D image. The seam visualization engine 224 projects the predicted seams onto the 3D model 220 based on the predicted seams indicated in the plurality of 2D images.

[0058] In some embodiments, to project the predicted seam onto the 3D model 220, the seam visualization engine 224 assigns a probability value q[w]=0 to each vertex w of the 3D model 220. In some embodiments, each 2D image corresponds to a view v of the 3D model 220 captured from a camera position k[v], and each pixel [i,j] of the image represents the probability that the pixel corresponds to a seam on the 3D model 220 (e.g., a probability p[i,j]==0 does not correspond to a seam, while a probability p[i,j]==1 corresponds to a seam).

[0059] For each pixel [i,j], the seam visualization engine 224 calculates a vector originating from the camera position k[v] and passing through the pixel. The seam visualization engine 224 then calculates the intersection points t,n (if any) between the vector and the 3D model 220, where t represents the position of the intersection point and n represents the normal of the intersection point.

[0060] If there is an intersection between the vector and the 3D model 220, then the seam visualization engine 224 determines the vertex w of the 3D model 220 closest to the intersection point and updates the probability value q[w] of that vertex to the average of the probability p[i,j] and q[w]. In some embodiments, the seam visualization engine 224 weights the probability value q[w] with the intersection normal n and the intersection distance |t–k[v]|. After the probability values of the vertices of the 3D model 220 have been updated based on multiple 2D images, the seam visualization engine 224 assigns a probability value to each edge [wi,wj] of the 3D model based on the probability values associated with the vertices of the edge (e.g., p[wi] and p[wj]). In one embodiment, the probability value assigned to the edge is the larger of the probability values assigned to the vertices of the edge.

[0061] Once the predicted seam has been placed on the 3D model, the post-processing engine 226 improves or adjusts the predicted seam, such as by removing predicted seams that do not meet a threshold probability value, reducing the predicted seam to a specified thickness, connecting two or more predicted seams to fill gaps, smoothing the predicted seam, removing isolated vertices, and adjusting the predicted seam based on model symmetry.

[0062] In some embodiments, to remove predicted seams that do not meet a threshold probability value, the post-processing engine 226 determines whether a vertex or edge of the 3D model 220 is associated with a probability value below the threshold probability value. If the vertex or edge is associated with a probability value below the threshold probability value, then the post-processing engine 226 updates the probability value to a value indicating that the edge or vertex does not correspond to a seam, such as a probability value of zero.

[0063] In some embodiments, to adjust the predicted seam based on model symmetry, the post - processing engine 226 analyzes the 3D model 220 to identify vertices that match symmetrically left - to - right, e.g., where the left - hand side and the right - hand side are symmetric. For any pair of symmetric vertices, the post - processing engine 226 replaces the probability value associated with each vertex with the average of two probability values.

[0064] In some embodiments, to fill or reduce gaps in the predicted seam, the post - processing engine 226 identifies a portion of the predicted seam where the probability values associated with the vertices in that portion of the predicted seam are lower than a threshold probability value. The post - processing engine 226 adjusts the probability values of the vertices in that portion of the predicted seam to the threshold probability value.

[0065] In some embodiments, the predicted seam has multiple edge thicknesses, e.g., including multiple parallel edges of the 3D model 220. The post - processing engine 226 adjusts the predicted seam so that it has a single edge thickness. For example, if the predicted seam includes three parallel edges, the post - processing engine 226 adjusts the probability values associated with two of the edges to zero to remove those edges from the predicted seam. Additionally, the post - processing engine 226 may also remove any edges that connect the remaining edge to the removed edges from the predicted seam. In some embodiments, the post - processing engine 226 selects the edge in the middle of a set of parallel edges as the remaining edge. In other embodiments, the post - processing engine 226 selects the edge associated with the highest probability value as the remaining edge.

[0066] In some embodiments, the post - processing engine 226 removes stray or isolated vertices from the set of predicted seams. The post - processing engine 226 identifies vertices with probability values greater than the threshold probability value but whose removal would not affect the topology of the connected seam. The post - processing engine 226 adjusts the probability values associated with the isolated vertices to zero to remove them from the set of predicted seams.

[0067] In some embodiments, to smooth the predicted seam, the post - processing engine 226 moves one or more vertices in the predicted seam to adjacent vertices. In some embodiments, the post - processing engine 226 moves a vertex to an adjacent vertex if the move would decrease the angle between consecutive edges in the predicted seam. Decreasing the angle between the edges in the predicted seam changes the seam to a smoother form that still closely approximates the original path of the predicted seam.

[0068] In some embodiments, the post - processing engine 226 uses the machine - learning model 120 to adjust or improve the predicted seams. The post - processing engine 226 generates 3D model data 230 based on the predicted seams and provides the 3D model data 230 to the machine - learning model 120. In one embodiment, the post - processing engine 226 generates a graphical representation of the 3D model 220 that includes a set of predicted seams. The post - processing engine 226 provides the graphical representation to the machine - learning model 120. The machine - learning model 120 adjusts and improves the set of predicted seams in the graphical representation. For example, each edge of the graphical representation can be associated with a probability value indicating the likelihood that the edge is a seam. The machine - learning model 120 adjusts or improves the set of predicted seams by adjusting the probability values associated with the edges of the graphical representation.

[0069] The graphical user interface 228 displays the 3D model 220. In some embodiments, the graphical user interface 228 includes controls for rotating, zooming in and out, and otherwise viewing the 3D model 220 from different perspectives. After one or more predicted seams are placed on the 3D model 220, the graphical user interface 228 is updated to display the 3D model 220 with its predicted seams.

[0070] In some embodiments, each predicted seam is associated with a probability value. The graphical user interface 228 displays a visual indication of the probability value associated with each predicted seam. For example, the graphical user interface 228 can use a series of colors to correspond to a range of probability values and display each predicted seam using the color corresponding to the probability value associated with the predicted seam. As another example, when the user places the cursor on a predicted seam or selects a predicted seam, the graphical user interface 228 can display the probability value associated with that predicted seam.

[0071] In some embodiments, the graphical user interface 228 includes graphical controls that enable the user to adjust one or more predicted seams. In some embodiments, the graphical user interface 228 includes tools and controls that allow the user to add seams to the 3D model 220, modify predicted seams, remove predicted seams, or merge two or more predicted seams. In some embodiments, after the user has adjusted one or more predicted seams, the 3D model 220 with the adjusted seams is stored or otherwise provided to the machine - learning model 120 for training an additional machine - learning model or improving the trained machine - learning model.

[0072] In some embodiments, the graphical user interface 228 includes tools and controls that allow a user to change configuration settings associated with the preprocessing engine 222, the machine learning model 120, the seam visualization engine 224, and / or the postprocessing engine 226. As an example, the graphical user interface 228 may include graphical controls for adjusting the input generated by the preprocessing engine 222, such as the number of inputs, the type of input, information related to the 3D model 220 used in generating the input, or whether the 3D model 220 is divided into multiple groups. As another example, the graphical user interface 228 may include graphical controls for selecting a 3D model style from a plurality of 3D model styles. Each 3D model style may correspond to a different machine learning model, such as a 3D model of an organic, man-made, human, automotive, animal, or other style used to train different machine learning models.

[0073] In some embodiments, the graphical user interface 228 updates the predicted seams displayed on the 3D model 220 based on how the changed configuration settings affect the predicted seams. For example, the graphical user interface 228 may include controls for specifying or adjusting a threshold probability value. The postprocessing engine 226 may hide, discard, or remove predicted seams not associated with probability values above the threshold probability value. After adjusting the threshold probability value using the graphical user interface 228, the postprocessing engine 226 updates the predicted seams for the 3D model 220 based on the adjusted threshold probability value. The graphical user interface 228 updates the display of the 3D model 220 with the updated set of predicted seams.

[0074] Figure 3 is a flowchart of method steps for generating predicted seams performed by a Figure 1 3D modeling application 118 according to various embodiments of the present disclosure. Although the method steps are described in connection with the Figure 1 and Figure 2 system, those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of the present disclosure.

[0075] In step 302, the preprocessing engine 222 processes the 3D model 220 to generate one or more inputs for the machine learning model 120. The generation of the one or more inputs is performed in a manner similar to that disclosed above with respect to the preprocessing engine 222 and as further described below with respect to Figure 4 and Figure 5 the further description.

[0076] In some embodiments, the preprocessing engine 222 generates one or more inputs by rendering a set of 2D images based on the 3D model 220. Each 2D image in the set of 2D images depicts a different perspective of the 3D model 220. The different perspectives of the 3D model can be any combination of different scales, viewing angles, viewing positions, and / or 3D model poses. Additionally, for the portion of the 3D model 220 that is visible in the depicted perspective, each 2D image can correspond to different types of information associated with the 3D model 220.

[0077] In some embodiments, the preprocessing engine 222 generates one or more inputs by generating a graphical representation of the 3D model 220. The graphical representation includes a set of nodes corresponding to the vertices of the 3D model 220 and a set of edges corresponding to the edges of the 3D model 220. In some embodiments, each node of the graphical representation includes additional information associated with the corresponding vertex of the 3D model 220, such as the normal vector of the corresponding vertex, the 3D position information of the corresponding vertex, and the likelihood that the corresponding vertex is mapped to a plane while minimizing distortion.

[0078] In some embodiments, the preprocessing engine 222 divides the 3D model 220 into multiple groups. Each group does not occlude or intersect with other groups in the multiple groups. For example, a 3D model of a car can be divided into a first group including the car wheels and a second group including the car body. As another example, a 3D model of a human can be divided into a group including the head, a group including the limbs, and a group including the torso. The preprocessing engine 222 uses any of the techniques discussed above to generate corresponding 3D model data 230 for each of the multiple groups.

[0079] In step 304, the 3D modeling application 118 applies the machine learning model 120 to one or more inputs. In some embodiments, applying the machine learning model 120 includes providing one or more inputs to one or more models of the machine learning model 120. The machine learning model 120 receives one or more inputs and generates output data indicating one or more predicted seams for the 3D model 220 based on the one or more inputs. If the 3D model data 230 corresponds to a particular portion of the 3D model 220 and / or a particular perspective of the 3D model 220, then the predicted seams correspond to the particular portion of the 3D model 220 and / or the particular perspective of the 3D model 220.

[0080] In step 306, the 3D modeling application 118 receives output data from the machine learning model 120. The output data indicates one or more predicted seams for the 3D model 220 based on one or more inputs provided to the machine learning model 120 in step 304. In some embodiments, the one or more inputs include 2D images of the 3D model 220, and the output data received from the machine learning model 120 indicates predicted seams on the 2D images. In some embodiments, the one or more inputs include a graphical representation corresponding to the 3D model 220, and the output data received from the machine learning model 120 indicates which edges of the graphical representation correspond to the predicted seams.

[0081] In step 308, the seam visualization engine 224 processes the output data to generate a set of predicted seams for the 3D model 220. The processing of the output data is performed in a manner similar to that disclosed above with respect to the seam visualization engine 224 and as further described below with respect to Figure 4 and Figure 5 In some embodiments, processing the output data includes aggregating or combining multiple sets of output data to generate a set of combined predicted seams for the 3D model 220. In some embodiments, processing the output data includes placing one or more seams onto the 3D model 220 based on the output data.

[0082] In some embodiments, the output data indicates positions on the 3D model 220 that are predicted to correspond to seams. The seam visualization engine 224 places one or more predicted seams onto the 3D model 220 based on the positions indicated by the output data. For example, the output data may indicate predicted seams on a 2D image depicting the 3D model 220, and the seam visualization engine 224 projects the predicted seams onto the 3D model 220 based on the 2D image. As another example, the output data may indicate one or more edges and / or one or more vertices of the 3D model 220 that are predicted to be parts of seams, and the seam visualization engine 224 places one or more predicted seams onto the 3D model 220 based on the one or more edges and / or one or more vertices.

[0083] In step 310, the post - processing engine 226 improves the set of predicted seams. The improvement of the set of predicted seams is performed in a manner similar to that disclosed above with respect to the post - processing engine 226 and as further described below with respect to Figure 4 and Figure 5 In some embodiments, the post - processing engine 226 improves the set of predicted seams by: removing predicted seams that do not meet a threshold probability value, reducing the predicted seams to a specified thickness, connecting two or more predicted seams to fill gaps, smoothing the predicted seams, removing isolated vertices, or adjusting the predicted seams based on model symmetry.

[0084] In some embodiments, the post - processing engine 226 improves the set of predicted seams by using the machine - learning model 120. The post - processing engine 226 generates input data based on the predicted seams and provides the input data to the machine - learning model 120. As an example, the post - processing engine 226 may generate a graphical representation of the 3D model 220 that includes the set of predicted seams. The post - processing engine 226 provides the graphical representation to the machine - learning model 120. The machine - learning model 120 adjusts and improves the set of predicted seams in the graphical representation.

[0085] In step 312, the 3D modeling application 118 applies the predicted seams to the 3D model 220. In some embodiments, the 3D modeling application 118 applies the predicted seams in response to receiving user input accepting the predicted seams via the graphical user interface 228.

[0086] In some embodiments, applying the predicted seams to the 3D model 220 includes splitting the 3D model along the predicted seams and flattening the resulting segments of the 3D model 220 into 2D texture images. In other embodiments, applying the predicted seams to the 3D model 220 includes storing the 3D model 220 with the predicted seams and / or providing the 3D model 220 with the predicted seams to a different application or tool that performs 2D texture image generation.

[0087] Figure 4 is a flowchart of method steps for generating predicted seams using 2D images performed by a Figure 1 3D modeling application according to various embodiments of the present disclosure. Although the method steps are described in connection with the Figure 1 and Figure 2 system, those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of the present disclosure.

[0088] In step 402, the pre - processing engine 222 renders the 3D model 220 into a set of 2D images. Rendering the 3D model 220 into a set of 2D images is performed in a manner similar to that disclosed above with respect to the pre - processing engine 222. In some embodiments, each 2D image in the set of 2D images depicts a different perspective of the 3D model 220. Different perspectives of the 3D model may be any combination of different scales, viewing angles, viewing positions, and / or 3D model poses. Additionally, for the portion of the 3D model 220 that is visible in the depicted perspective, each 2D image may correspond to different types of information associated with the 3D model 220. In some embodiments, the pre - processing engine 222 divides the 3D model 220 into multiple non - overlapping and non - intersecting groups. The pre - processing engine 222 generates a corresponding set of 2D images for each group of the multiple groups.

[0089] In step 404, the 3D modeling application 118 applies the machine learning model 120 to the 2D images in the set of 2D images. In some embodiments, applying the machine learning model 120 to a 2D image includes providing the 2D image to one or more models of the machine learning model 120. The machine learning model 120 receives the 2D image from the 3D modeling application 118 and generates output data indicative of a predicted seam on the 2D image for the 3D model 220 based on the 2D image.

[0090] In step 406, the 3D modeling application 118 receives the output data indicative of the predicted seam on the 2D image from the machine learning model 120. For example, for each pixel in the 2D image, the output data may indicate whether the pixel is predicted to correspond to a seam on the 3D model 220.

[0091] In step 408, the seam visualization engine 224 places the predicted seam on the 3D model 220 based on the 2D image. Placing the predicted seam on the 3D model 220 based on the 2D image is performed in a manner similar to the manner disclosed above with respect to the seam visualization engine 224. In an embodiment, the seam visualization engine 224 places the predicted seam on the 3D model by projecting the predicted seam indicated in the 2D image onto the 3D model 220. For example, each vertex of the 3D model 220 may be associated with a probability value indicative of the probability that the vertex is part of a seam. For each pixel of the 2D image, the seam visualization engine 224 determines the vertex of the 3D model 220 corresponding to the pixel and updates the probability value associated with the vertex based on the probability value associated with the pixel.

[0092] The above steps 404 - 408 are repeated for each 2D image in the set of 2D images. After steps 404 - 408 are performed for the set of 2D images, the 3D model 220 includes the predicted seams corresponding to the set of 2D images.

[0093] In step 410, the post - processing engine 226 improves the predicted seams placed on the 3D model 220. Improving the predicted seams is performed in a manner similar to the manner disclosed above with respect to the post - processing engine 226. In some embodiments, the post - processing engine 226 improves the predicted seams on the 3D model 220, for example, by removing predicted seams that do not meet a threshold probability value, reducing the predicted seams to a specified thickness, connecting two or more predicted seams to fill gaps, smoothing the predicted seams, removing isolated vertices, and adjusting the predicted seams based on model symmetry.

[0094] In some embodiments, the post - processing engine 226 uses the machine - learning model 120 to adjust or improve the predicted seams. The post - processing engine 226 generates 3D model data 230 based on the predicted seams and provides the 3D model data 230 to the machine - learning model 120. In one embodiment, the post - processing engine 226 generates a graphical representation of the 3D model 220 that includes a set of predicted seams. The post - processing engine 226 provides the graphical representation to the machine - learning model 120. The machine - learning model 120 adjusts and improves the set of predicted seams in the graphical representation. For example, each edge of the graphical representation can be associated with a probability value indicating the likelihood that the edge is a seam. The machine - learning model 120 adjusts or improves the set of predicted seams by adjusting the probability values associated with the edges of the graphical representation.

[0095] Figure 5 is a flowchart of method steps for generating predicted seams using a graphical representation performed by a 3D modeling application according to various embodiments of the present disclosure. Although the method steps are described in conjunction with Figure 1 and Figure 1 and Figure 2 systems, those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of the present disclosure.

[0096] In step 502, the pre - processing engine 222 processes the 3D model 220 to generate a graphical representation of the 3D model 220. Generating the graphical representation of the 3D model 220 is performed in a manner similar to the way disclosed above with respect to the pre - processing engine 222. In some embodiments, the graphical representation includes a set of vertices corresponding to the vertices of the 3D model 220 and a set of edges corresponding to the edges of the 3D model 220. Additionally, in some embodiments, each vertex of the graphical representation includes additional information related to the corresponding vertex of the 3D model 220, such as the normal vector of the corresponding vertex, the 3D position information of the corresponding vertex, and the likelihood that the corresponding vertex is mapped to a plane while minimizing distortion.

[0097] In some embodiments, the pre - processing engine 222 divides the 3D model 220 into a plurality of groups that do not occlude and do not cross. The pre - processing engine 222 generates a corresponding graphical representation for each of the plurality of groups.

[0098] In step 504, the 3D modeling application 118 applies the machine - learning model 120 to the graphical representation. In some embodiments, applying the machine - learning model 120 to the graphical representation includes providing the graphical representation to one or more models of the machine - learning model 120. The machine - learning model 120 receives the graphical representation from the 3D modeling application 118 and generates output data indicating the predicted seams corresponding to the graphical representation. For example, the output data can indicate whether one or more vertices and / or one or more edges of the graphical representation are predicted to be part of a seam on the 3D model 220.

[0099] In step 506, the 3D modeling application 118 receives output data from the machine learning model 120 indicating which edges and / or vertices of the graph representation correspond to seams or portions of seams on the 3D model 220. For example, for each edge or vertex of the graph representation, the output data may indicate whether the edge or vertex is predicted to correspond to a seam on the 3D model 220.

[0100] In step 508, the seam visualization engine 224 places one or more predicted seams on the 3D model 220 based on the output data. Placing the predicted seams on the 3D model 220 based on the output data is performed in a manner similar to that disclosed above with respect to the seam visualization engine 224. In an embodiment, the seam visualization engine 224 places one or more predicted seams based on the edges and / or vertices of the graph representation indicated by the output data as corresponding to seams. In some embodiments, each vertex or edge of the 3D model 220 may be associated with a probability value indicating the probability that the edge or vertex is part of a seam. The seam visualization engine 224 updates the probability value of each edge or vertex based on the probability value associated with the corresponding edge or vertex in the graph representation.

[0101] In step 510, the post - processing engine 226 improves the predicted seams placed on the 3D model 220. Improving the predicted seams is performed in a manner similar to that disclosed above with respect to the post - processing engine 226. In some embodiments, the post - processing engine 226 improves the predicted seams on the 3D model 220, for example, by removing predicted seams that do not meet a threshold probability value, reducing the predicted seams to a specified thickness, connecting two or more predicted seams to fill gaps, smoothing the predicted seams, removing isolated vertices, and adjusting the predicted seams based on model symmetry.

[0102] In some embodiments, the post - processing engine 226 uses the machine learning model 120 to adjust or improve the predicted seams. The post - processing engine 226 generates 3D model data 230 based on the predicted seams and provides the 3D model data 230 to the machine learning model 120. In one embodiment, the post - processing engine 226 generates a graph representation of the 3D model 220 including a set of predicted seams. The post - processing engine 226 provides the graph representation to the machine learning model 120. The machine learning model 120 adjusts and improves the set of predicted seams in the graph representation. For example, each edge of the graph representation may be associated with a probability value indicating the likelihood that the edge is a seam. The machine learning model 120 adjusts or improves the set of predicted seams by adjusting the probability values associated with the edges of the graph representation.

[0103] Figure 6 is an illustration of a predicted seam generation process according to various embodiments of the present disclosure. In Figure 6In [the figure], the 3D model shows an example 3D model 220 generated by or provided to the 3D modeling application 118.

[0104] In step 610, the preprocessing engine 222 generates a set of 2D images based on the 3D model. 2D Image (1), 2D Image (2), and 2D Image (3) show example 2D images rendered by the preprocessing engine 222 from the 3D model. As Figure 6 shown, each 2D image depicts a different perspective of the 3D model. Although not shown in the figure, each 2D image may also be based on different information related to the 3D model. Although Figure 6 three 2D images are shown, any number of 2D images may be rendered from the 3D model. In some embodiments, the preprocessing engine 222 renders multiple 2D images such that each part of the 3D model is depicted in at least one 2D image.

[0105] In step 620, the 3D modeling application 118 provides the set of 2D images to the machine learning model 120. The machine learning model 120 receives the set of 2D images and generates one or more predicted seams for the 3D model for each 2D image in the set. Predicted Seam (1), Predicted Seam (2), and Predicted Seam (3) respectively show examples of predicted seams generated by the machine learning model 120 based on 2D Image (1), 2D Image (2), and 2D Image (3). For example, the machine learning model 120 receives 2D Image (1) and generates Predicted Seam (1) that indicates one or more predicted seams for the view of the 3D model depicted in 2D Image (1) based on 2D Image (1). Although Figure 6 a single predicted seam corresponding to each 2D image is shown, the machine learning model 120 may generate multiple predicted seams for each 2D image provided by the 3D modeling application 118.

[0106] In step 630, the machine learning model 120 provides the set of predicted seams (Predicted Seam (1), Predicted Seam (2), and Predicted Seam (3)) to the 3D modeling application 118.

[0107] In step 640, the seam visualization engine 224 places the set of predicted seams on the 3D model. The updated 3D model shows an example of the predicted seams placed on the 3D model by the seam visualization engine 224.

[0108] Figure 7 is a diagram of a predicted seam generation process according to various embodiments of the present disclosure. In Figure 7 [the figure], the 3D model shows an example 3D model 220 generated by or provided to the 3D modeling application 118.

[0109] In step 710, the preprocessing engine 222 generates a graph representation based on the 3D model.

[0110] In step 720, the 3D modeling application 118 provides the graph representation to the machine learning model 120. The machine learning model 120 receives the graph representation and generates an annotated graph representation indicating one or more predicted seams for the 3D model. The annotated graph representation includes data indicating, for each vertex and / or edge in the graph representation, whether the vertex and / or edge is predicted to be a seam for the 3D model.

[0111] In step 730, the machine learning model 120 provides the annotated graph representation to the 3D modeling application 118.

[0112] In step 740, the seam visualization engine 224 places one or more predicted seams on the 3D model based on the annotated graph representation. The updated 3D model shows an example of the predicted seams placed on the 3D model by the seam visualization engine 224.

[0113] As discussed above with respect to the graphical user interface 228, the graphical user interface displays the 3D model and the updated 3D model. In some embodiments, the graphical user interface 228 includes graphical controls that enable a user to rotate, scale, and pan the updated 3D model to view the predicted seams placed on the 3D model. In some embodiments, the graphical user interface 228 includes tools and controls that enable a user to add seams to the updated 3D model, modify the predicted seams placed on the updated 3D model, remove predicted seams, or merge two or more predicted seams.

[0114] In some embodiments, the 3D modeling application 118 evaluates the predicted seams for the 3D model and / or the UV mapping resulting from applying the predicted seams to the 3D model.

[0115] In some embodiments, validating or evaluating the predicted seams includes, for example, determining whether applying the UV mapping to the 3D model results in minimal visible distortion; whether the UV mapping is within the fewest number of shells in the UV space; whether the predicted seams are placed in hidden or less visible locations on the 3D model; or whether the UV mapping is defined within an optimized UV space.

[0116] Minimizing the amount of distortion minimizes the amount of compression or stretching that occurs when mapping a texture from a 2D image to the 3D model. In some embodiments, determining whether applying the UV mapping to the 3D model results in minimal visible distortion may include determining how much compression or stretching occurs when mapping the texture to the 3D model. Determining whether applying the UV mapping to the 3D model results in minimal visible distortion may also include determining whether the amount of compression or stretching is within a threshold amount.

[0117] Minimizing the number of shells protects the semantic boundaries of the 3D model. In some embodiments, determining whether a UV mapping is contained within a minimum number of shells in UV space includes determining how many shells are created in the UV mapping when applying seams. Determining whether a UV mapping is contained within a minimum number of shells in UV space may also include determining whether the number of shells is within a threshold amount, for example, based on the type of object in the 3D model, an amount indicated by a machine learning model based on the 3D model, or a maximum number of shells indicated by a user.

[0118] In some embodiments, determining whether a predicted seam is placed in a hidden or less visible location on the 3D model includes determining whether the predicted seam is visible from one or more camera positions. The one or more camera positions may be based on how the 3D model will be visualized. For example, if the 3D model will be viewed from an aerial perspective, the one or more camera positions include one or more aerial camera positions.

[0119] Constraining the UV mapping within an optimized UV space reduces the amount of resources required to store and process the UV mapping, such as computer storage space, computer memory usage, and network bandwidth. In some embodiments, determining whether a UV mapping is constrained within an optimized UV space includes determining the size of the UV mapping. Determining whether a UV mapping is constrained within an optimized UV space may also include determining whether the size is within a threshold dimension. In some embodiments, determining whether a UV mapping is constrained within an optimized UV space includes determining the amount of empty space in the UV mapping compared to the amount of space corresponding to the 3D model.

[0120] In some embodiments, evaluating predicted seams may include generating one or more validation values associated with the predicted seams. The validation values may indicate, for example, the confidence level that the set of predicted seams is correct, whether the set of predicted seams meets one or more of the criteria discussed above, the likelihood that the predicted seams are plausible seams for the 3D model, how closely the set of predicted seams matches a set of provided seams for the 3D model (e.g., previously created by a user). The one or more validation values may be displayed to the user in the graphical user interface 228. For example, the 3D modeling application 118 may generate and display an analysis report indicating the one or more validation values.

[0121] In some embodiments, generating one or more validation values includes generating a confidence interval associated with the predicted seams. The accuracy of the machine learning model 120 is evaluated based on a set of validation samples. The confidence interval may be generated based on the accuracy of the particular machine learning model of the machine learning model 120 used to generate the predicted seams. The confidence interval indicates the confidence of the machine learning model in a subset of the predicted seams.

[0122] In some embodiments, generating one or more validation values includes comparing the predicted seams to a set of seams provided for the 3D model. The set of provided seams can be, for example, seams previously created by a user or by other machine learning models for the creation of the 3D model. One or more validation values are generated based on the comparison (such as distortion differences, shell count differences, true position rate, false position rate, or false negative rate).

[0123] In some embodiments, generating one or more validation values includes providing the predicted seams to one or more trained machine learning models that are trained to receive an input indicative of a set of predicted seams and generate an output indicative of the accuracy of the set of predicted seams. The one or more trained machine learning models can be discriminative models for evaluating the results produced by the corresponding machine learning model that generated the predicted seams.

[0124] In some embodiments, based on the evaluation of the predicted seams, the 3D model with the predicted seams is stored or otherwise provided to the machine learning model 120 for training additional machine learning models or improving the trained machine learning models. For example, if the evaluation indicates that the predicted seams do not meet the criteria discussed above, the predicted seams can be improved or the 3D model can be provided to one or more users to manually define a set of seams. The 3D model with the improved or manually defined seams is provided as a training input to the machine learning model 120. An advantage of using the evaluation results to retrain the machine learning model is that the machine learning model can learn from 3D models that are inconsistent with the 3D models previously used to train the machine learning model (such as 3D models with uncommon shapes or depicting new object types).

[0125] In summary, the computer system generates a set of seams for a 3D model that indicate how the 3D model can be unfolded and flattened into a 2D image to apply textures. The computer system processes the 3D model to generate one or more inputs provided to a trained neural network.

[0126] The trained neural network is configured to receive the one or more inputs and generate an output indicative of one or more predicted seams for the 3D model. The computer system further improves the one or more predicted seams to, for example, increase symmetry, straighten the seams, and reduce distortion. Additionally, the computer system enables a user to view one or more predicted seams on the 3D model via a GUI, modify parameters to improve the predicted seams, and select a probability threshold indicating which predicted seams to accept.

[0127] In one method, to generate one or more inputs for a trained neural network, a computer system renders a 3D model into a set of 2D images, where each 2D image depicts the 3D model from a different perspective. The trained neural network receives each 2D image and generates an indication of predicted seams on each 2D image. Based on the positions of the predicted seams on the 2D image and the perspective depicted by the 2D image, the computer system places the predicted seams at corresponding positions on the 3D model.

[0128] In another method, to generate one or more inputs for a trained neural network, a computer system generates a graph representation of the 3D model. The graph representation is provided as an input to the trained neural network, and the trained neural network generates an output indicating whether each edge or vertex of the graph representation is predicted to be a seam or part of a seam. Based on the edges of the 3D model that are predicted to be seams or parts of seams, the computer system places the predicted seams at corresponding edges on the 3D model.

[0129] In another method, a first trained neural network is used to generate a first set of predicted seams. The computer system processes the first set of predicted seams to generate an input for a second trained neural network. The second trained neural network is configured to receive the first set of predicted seams and improve the first set of predicted seams to generate a second set of predicted seams, for example, by enhancing symmetry, straightening the seams, and reducing distortion. Any of the methods discussed above can be used to generate the inputs provided to the first trained neural network and the second trained neural network. For example, the first set of predicted seams can be generated by providing a set of 2D images of the 3D model to the first trained neural network. After the computer system places the first set of predicted seams on the 3D model, the computer system generates a graph representation of the 3D model that includes the first set of predicted seams. The graph representation is provided as an input to the second trained neural network.

[0130] Compared with previous methods, at least one advantage of the disclosed techniques is that, unlike previous methods, the computer system automatically generates seams for the 3D model that take into account semantic boundaries and seam positions while minimizing distortion and low layout efficiency. These technical advantages provide one or more technological advancements over prior art methods.

[0131] 1. In some embodiments, a method for automatically generating seams for a three-dimensional (3D) model includes: generating one or more representations of the 3D model based on the 3D model as inputs to one or more trained machine learning models; generating a set of seam predictions associated with the 3D model by applying the one or more trained machine learning models to the one or more representations of the 3D model, wherein each seam prediction included in the set of seam predictions identifies a different seam along which the 3D model can be cut; and placing one or more seams on the 3D model based on the set of seam predictions.

[0132] 2. The method of clause 1, the method further includes: dividing the 3D model into multiple groups; and for each group of the multiple groups, generating a corresponding one or more representations of the 3D model as inputs to the one or more trained machine learning models.

[0133] 3. The method of any one of clauses 1 and 2, wherein the one or more representations of the 3D model include one or more 2D images, and wherein the set of seam predictions indicates, for each 2D image of the one or more 2D images, corresponding one or more seam predictions in the 2D image.

[0134] 4. The method of any one of clauses 1-3, wherein placing the one or more seams on the 3D model includes, for each 2D image of the one or more 2D images, projecting the corresponding one or more seam predictions in the 2D image onto the 3D model.

[0135] 5. The method of any one of clauses 1-4, wherein the one or more representations of the 3D model include a graph representation of the 3D model, and wherein the set of seam predictions indicates one or more edges of a portion of the graph representation predicted to be a seam.

[0136] 6. The method of any one of clauses 1-5, wherein placing the one or more seams on the 3D model includes: for each of the one or more edges of the portion of the graph representation predicted to be a seam, determining a corresponding edge of the 3D model; and placing an edge of the seam at the corresponding edge of the 3D model.

[0137] 7. The method of any one of clauses 1-6, wherein the one or more representations of the 3D model include a graph representation of the 3D model, and wherein the set of seam predictions indicates one or more vertices of a portion of the graph representation predicted to be a seam.

[0138] 8. The method according to any one of clauses 1-7, wherein placing the one or more seams on the 3D model comprises: for each vertex of the one or more vertices of the portion of the graphical representation predicted to be a seam, determining the corresponding vertex of the 3D model; and placing the vertices of the seam at the corresponding vertices of the 3D model.

[0139] 9. The method according to any one of clauses 1-8, wherein the 3D model comprises a plurality of edges, each edge being associated with a respective seam probability value indicative of the likelihood that the edge corresponds to a seam, and wherein placing the one or more seams on the 3D model comprises: determining the one or more edges of the plurality of edges associated with the one or more seams; and for each edge of the one or more edges, updating the respective seam probability value associated with the edge.

[0140] 10. The method according to any one of clauses 1-9, wherein the 3D model comprises a plurality of vertices, each vertex being associated with a respective seam probability value indicative of the likelihood that the vertex corresponds to a seam, and wherein placing the one or more seams on the 3D model comprises: determining the one or more vertices of the plurality of vertices associated with the one or more seams; and for each vertex of the one or more vertices, updating the respective seam probability value associated with the vertex.

[0141] 11. The method according to any one of clauses 1-10, the method further comprising generating a validation value associated with the set of seam predictions by at least evaluating a subset of the seam predictions of the set of seam predictions.

[0142] 12. The method according to any one of clauses 1-11, the method further comprising improving the one or more seams, wherein improving the one or more seams comprises one or more of the following: removing one or more specific seams of the one or more seams; reducing the thickness of one or more specific seams of the one or more seams; connecting two or more specific seams of the one or more seams; smoothing one or more specific seams of the one or more seams; removing one or more seam vertices; adjusting one or more specific seams of the one or more seams based on the symmetry of the 3D model.

[0143] 13. The method as described in any one of clauses 1 - 12, the method further comprising improving the one or more seams, wherein improving the one or more seams includes: generating a graphical representation of the 3D model based on the one or more seams; generating a set of improved seam predictions associated with the 3D model by applying the one or more trained machine learning models to the graphical representation of the 3D model; and updating the one or more seams based on the set of improved seam predictions.

[0144] 14. In some embodiments, a non - transitory computer - readable medium stores program instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps: generating one or more representations of a three - dimensional (3D) model as inputs to one or more trained machine learning models; generating a set of seam predictions associated with the 3D model by applying the one or more trained machine learning models to the one or more representations of the 3D model, wherein each seam prediction included in the set of seam predictions identifies a different seam along which the 3D model can be cut; and placing one or more seams on the 3D model based on the set of seam predictions.

[0145] 15. The non - transitory computer - readable medium as described in clause 14, the non - transitory computer - readable medium further comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform the following steps: dividing the 3D model into multiple groups; and for each group of the multiple groups, generating a corresponding one or more representations of the 3D model as inputs to the one or more trained machine learning models.

[0146] 16. The non - transitory computer - readable medium as described in any one of clauses 14 and 15, wherein the one or more representations of the 3D model include one or more 2D images, wherein the set of seam predictions indicates, for each 2D image of the one or more 2D images, the corresponding one or more seam predictions in the 2D image, and wherein placing the one or more seams on the 3D model includes, for each 2D image of the one or more 2D images, projecting the corresponding one or more seam predictions in the 2D image onto the 3D model.

[0147] 17. A non-transitory computer-readable medium as described in any one of clauses 14-16, wherein the one or more representations of the 3D model include a graphical representation of the 3D model, wherein the set of seam predictions indicates one or more edges of the portion of the graphical representation predicted to be a seam, and wherein placing the one or more seams on the 3D model includes: for each of the one or more edges of the portion of the graphical representation predicted to be a seam, determining a corresponding edge of the 3D model; and placing an edge of the seam at the corresponding edge of the 3D model.

[0148] 18. A non-transitory computer-readable medium as described in any one of clauses 14-17, wherein the one or more representations of the 3D model include a graphical representation of the 3D model, wherein the set of seam predictions indicates one or more vertices of the portion of the graphical representation predicted to be a seam, and wherein placing the one or more seams on the 3D model includes: for each of the one or more vertices of the portion of the graphical representation predicted to be a seam, determining a corresponding vertex of the 3D model; and placing a vertex of the seam at the corresponding vertex of the 3D model.

[0149] 19. A non-transitory computer-readable medium as described in any one of clauses 14-18, the non-transitory computer-readable medium further comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform the following steps: generating a graphical representation of the 3D model based on the one or more seams; generating a set of improved seam predictions associated with the 3D model by applying the one or more trained machine learning models to the graphical representation of the 3D model; and updating the one or more seams based on the set of improved seam predictions.

[0150] 20. In some embodiments, a system includes: a memory that stores one or more software applications; and a processor that, when executing the one or more software applications, is configured to perform the following steps: generating one or more representations of a three-dimensional (3D) model as input to one or more trained machine learning models based on the 3D model; generating a set of seam predictions associated with the 3D model by applying the one or more trained machine learning models to the one or more representations of the 3D model, wherein each seam prediction included in the set of seam predictions identifies a different seam along which the 3D model can be cut; and placing one or more seams on the 3D model based on the set of seam predictions.

[0151] Any and all combinations of any claim elements set forth in any claim and / or any elements described in this application, in any way, fall within the scope and contemplation of the present invention and protection.

[0152] The descriptions of the various embodiments have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

[0153] Aspects of the embodiments of the present invention may be embodied in a system, method, or computer program product. Accordingly, aspects of the present disclosure may take the form of an all-hardware embodiment, an all-software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, which may all be generally referred to herein as a "module", "system", or "computer". Additionally, any hardware and / or software technologies, processes, functions, components, engines, modules, or systems described in the present disclosure may be implemented as a circuit or a set of circuits. Further, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer-readable media having computer-readable program code embodied thereon.

[0154] Any combination of one or more computer-readable media may be utilized. The computer-readable media may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0155] The foregoing has described aspects of the present disclosure in terms of flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block in the flowchart and / or block diagram, and combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine. The instructions, when executed by the processor of the computer or other programmable data processing apparatus, cause the functions / actions specified in one or more blocks of the flowchart and / or block diagram to be implemented. Such a processor can be, but is not limited to, a general purpose processor, a special purpose processor, an application specific processor, or a field programmable gate array.

[0156] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code, which includes one or more executable instructions for implementing the specified logical function. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a system based on dedicated hardware that performs the specified functions or actions, or by combinations of dedicated hardware and computer instructions.

[0157] While the foregoing is directed to embodiments of the present disclosure, other and additional embodiments of the present disclosure may be envisioned without departing from the basic scope of the present disclosure, and the scope of the present disclosure is determined by the appended claims.

Claims

1. A method for automatically generating seams for a three-dimensional (3D) model, the method comprising: generating one or more representations of the 3D model based on the 3D model for input to one or more trained machine learning models; generating a set of seam predictions associated with the 3D model by applying the one or more trained machine learning models to the one or more representations of the 3D model, wherein each seam prediction included in the set of seam predictions identifies a different seam along which the 3D model can be cut; and placing one or more seams on the 3D model based on the set of seam predictions, wherein the one or more representations of the 3D model include one or more 2D images, and wherein the set of seam predictions indicates, for each 2D image of the one or more 2D images, corresponding one or more seam predictions in the 2D image.

2. The method of claim 1, the method further comprising: dividing the 3D model into a plurality of groups; and for each of the plurality of groups, generating a corresponding one or more representations of the 3D model for input to the one or more trained machine learning models.

3. The method of claim 1, wherein placing the one or more seams on the 3D model includes, for each 2D image of the one or more 2D images, projecting the corresponding one or more seam predictions in the 2D image onto the 3D model.

4. The method of claim 1, wherein the one or more representations of the 3D model include a graphical representation of the 3D model, and wherein the set of seam predictions indicates one or more edges of a portion of the graphical representation predicted to be a seam.

5. The method of claim 4, wherein placing the one or more seams on the 3D model comprising: for each of the one or more edges of the portion of the graphical representation predicted to be a seam, determining a corresponding edge of the 3D model; and placing an edge of the seam at the corresponding edge of the 3D model.

6. The method of claim 1, wherein the one or more representations of the 3D model include a graphical representation of the 3D model, and wherein the set of seam predictions indicates one or more vertices of a portion of the graphical representation predicted to be a seam.

7. The method of claim 6, wherein placing the one or more seams on the 3D model comprising: for each of the one or more vertices of the portion of the graphical representation predicted to be a seam, determining a corresponding vertex of the 3D model; and placing a vertex of the seam at the corresponding vertex of the 3D model.

8. The method of claim 1, wherein the 3D model includes a plurality of edges, wherein each edge is associated with a corresponding seam probability value indicating the likelihood that the edge corresponds to a seam, and wherein placing the one or more seams on the 3D model comprising: determining one or more edges of the plurality of edges associated with the one or more seams; and For each of the one or more edges, update the corresponding seam probability value associated with the edge.

9. The method of claim 1, wherein the 3D model includes a plurality of vertices, each vertex being associated with a corresponding seam probability value indicative of the likelihood that the vertex corresponds to a seam, and placing the one or more seams on the 3D model comprises: determining one or more vertices among the plurality of vertices that are associated with the one or more seams; and for each of the one or more vertices, updating the corresponding seam probability value associated with the vertex.

10. The method of claim 1, the method further comprising generating a validation value associated with the set of seam predictions by at least evaluating a subset of the seam predictions of the set of seam predictions.

11. The method of claim 1, the method further comprising improving the one or more seams, wherein improving the one or more seams comprises one or more of the following: removing one or more specific seams among the one or more seams; reducing the thickness of one or more specific seams among the one or more seams; connecting two or more specific seams among the one or more seams; smoothing one or more specific seams among the one or more seams ； removing one or more seam vertices; adjusting one or more specific seams among the one or more seams based on the symmetry of the 3D model.

12. The method of claim 1, the method further comprising improving the one or more seams, wherein improving the one or more seams comprises: generating a graph representation of the 3D model based on the one or more seams; generating a set of improved seam predictions associated with the 3D model by applying the one or more trained machine learning models to the graph representation of the 3D model; and updating the one or more seams based on the set of improved seam predictions.

13. A non-transitory computer-readable medium, the non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps: generating one or more representations of the 3D model based on the three-dimensional 3D model as inputs to one or more trained machine learning models; generating a set of seam predictions associated with the 3D model by applying the one or more trained machine learning models to the one or more representations of the 3D model, wherein each seam prediction included in the set of seam predictions identifies a different seam along which the 3D model can be cut; and placing one or more seams on the 3D model based on the set of seam predictions, wherein the one or more representations of the 3D model include one or more 2D images, and the set of seam predictions indicates corresponding one or more seam predictions in the 2D image for each of the one or more 2D images.

14. The non-transitory computer-readable medium as claimed in claim 13, wherein the non-transitory computer-readable medium further comprises instructions which, when executed by the one or more processors, cause the one or more processors to perform the following steps: Dividing the 3D model into a plurality of groups; and For each of the plurality of groups, generating a corresponding one or more representations of the 3D model as an input to the one or more trained machine learning models.

15. The non-transitory computer-readable medium as claimed in claim 13, wherein placing the one or more seams on the 3D model comprises, for each of the one or more 2D images, projecting the corresponding one or more seam predictions in the 2D image onto the 3D model.

16. The non-transitory computer-readable medium as claimed in claim 13, wherein the one or more representations of the 3D model comprise a graph representation of the 3D model, wherein the set of seam predictions indicates one or more edges of a portion of the graph representation predicted to be a seam, and wherein placing the one or more seams on the 3D model Comprises: For each of the one or more edges of the portion of the graph representation predicted to be a seam, determining a corresponding edge of the 3D model; And Placing the edge of the seam at the corresponding edge of the 3D model.

17. The non-transitory computer-readable medium as claimed in claim 13, wherein the one or more representations of the 3D model comprise a graph representation of the 3D model, wherein the set of seam predictions indicates one or more vertices of a portion of the graph representation predicted to be a seam, and wherein placing the one or more seams on the 3D model Comprises: For each of the one or more vertices of the portion of the graph representation predicted to be a seam, determining a corresponding vertex of the 3D model; And Placing the vertex of the seam at the corresponding vertex of the 3D model.

18. The non-transitory computer-readable medium as claimed in claim 13, wherein the non-transitory computer-readable medium further comprises instructions which, when executed by the one or more processors, cause the one or more processors to perform the following steps: Generating a graph representation of the 3D model based on the one or more seams; Generating a set of improved seam predictions associated with the 3D model by applying the one or more trained machine learning models to the graph representation of the 3D model; and Updating the one or more seams based on the set of improved seam predictions.

19. A system, the system Comprises: A memory that stores one or more software applications; And A processor which, when executing the one or more software applications, is configured to perform the following steps: Generating one or more representations of the 3D model based on a three-dimensional 3D model as an input to one or more trained machine learning models; Generating a set of seam predictions associated with the 3D model by applying the one or more trained machine learning models to the one or more representations of the 3D model, wherein each seam prediction included in the set of seam predictions identifies a different seam along which the 3D model can be cut; and Placing one or more seams on the 3D model based on the set of seam predictions, wherein the one or more representations of the 3D model include one or more 2D images, and wherein the set of seam predictions indicates corresponding one or more seam predictions in the 2D image for each 2D image of the one or more 2D images.

Images