Determining deformations of a 3D modeled object

Abstract

To provide a computer-implemented method for determining deformation of a 3D modeled object.SOLUTION: A method provides a deformation basic function for inferring a deformation base of an input 3D modeled object, a first 3D modeled object and a deformation restriction of the first 3D modeled object. The method determines a second 3D modeled object respecting the deformation restriction, and the determination of the second 3D modeled object calculates a transient locus of the 3D modeled object between the first 3D modeled object and the second 3D modeled object. The locus deforms each of the transition 3D modeled objects according to a linear combination of a result obtained by applying the deformation basic function to the transition 3D modeled object, and reduces loss. The loss gives penalty to the degree of non-compliance of the deformation restriction by the deformed transition 3D modeled object.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to the field of computer programs and systems, and more particularly to a method, system and program for determining a deformation of a 3D modeled object. [Background technology]

[0002] Many systems and programs are offered on the market for designing, engineering and manufacturing objects. CAD stands for Computer-Aided Design and relates to software solutions for designing objects. CAE stands for Computer-Aided Engineering and relates to software solutions for simulating, for example, the physical behavior of a future product. CAM stands for Computer-Aided Manufacturing and relates to software solutions for defining, for example, manufacturing processes and operations. In such computer-aided design systems, the graphical user interface plays a key role with regard to the efficiency of the technology. These technologies may be integrated within Product Lifecycle Management (PLM) systems. PLM is a business strategy in which companies share product data, apply common processes and leverage corporate knowledge to support the development of a product from conception to the end of its life, going beyond the concept of the extended enterprise. Dassault Systèmes' PLM solutions (under its CATIA, ENOVIA and DELMIA trademarks) feature an "Engineering Hub" to organize product engineering knowledge, a "Manufacturing Hub" to manage manufacturing engineering knowledge, and an "Enterprise Hub" to enable enterprise integration and connectivity to both the Engineering and Manufacturing Hubs. The system provides an open object model that links products, processes and resources, enabling dynamic knowledge-based product creation and decision support, facilitating the optimization of product definition, manufacturing preparation, production and service.

[0003] In this and other contexts, deformation of 3D modeled objects has become of widespread importance.

[0004] The following papers are relevant to this field and are cited below: [1] W. Wang, D. Ceylan, R. Mech, and U. Neumann. 3Dn: 3D deformation network. In Conference on Computer Vision and Pattern Regognition (CVPR), 2019. [2] T. Groueix, M. Fisher, V. G. Kim, B. Russell, and M. Aubry. AtlasNet: A Papier-Mache Approach to Learning 3D Surface Generation. In Conference on Computer Vision and Pattern Regognition (CVPR), 2018. [3] Reconstructing a 3D Modeled Object. Patent US9978177B2, granted in 2018. Eloi Mehr and Vincent Guitteny. [4] D. Jack, J. K. Pontes, S. Sridharan, C. Fookes, S. Shirazi, F. Maire, and A. Eriksson. Learning free-form deformations for 3D object reconstruction. In Asian Conference on Computer Vision (ACCV), 2018. [5] I. Kokkinos, and A. Yuille. Unsupervised Learning of Object Deformation Models. In International Conference on Computer Vision (ICCV), 2007. [6] J. Mairal, F. Bach, and J. Ponce. Sparse Modeling for Image and Vision Processing. New Foundations and Trends, 2014. [7] V. Blanz and T. Vetter. A morphable model for the synthesis of 3D faces. In SIGGRAPH, 1999. [8] C. Qi, H. Su, K. Mo, and L. Guibas. Pointnet: Deep learning on point sets for 3D classification and segmentation. In Conference on Computer Vision and Pattern Regognition (CVPR), 2017. [9] R. Hanocka, A. Hertz, N. Fish, R. Giryes, S. Fleishman, and D. Cohen-Or. Meshcnn: A network with an edge. In SIGGRAPH, 2019.

[10] Eloi Mehr, Ariane Jourdan, Nicolas Thome, Matthieu Cord, and Vincent Guitteny. DiscoNet: Shapes Learning on Disconnected Manifolds for 3D Editing. In ICCV 2019.

[11] J. Liu, F. Yu, and T. Funkhouser. Interactive 3D Modeling with a Generative Adversarial Network. In International Conference on 3D Vision (3DV), 2017.

[12] B. Levy and H. Zhang. Spectral Mesh Processing. In SIGGRAPH Asia, 2009.

[0005] However, there remains a need for improved solutions for deforming 3D modeled objects. Summary of the Invention

[0006] Accordingly, a computer-implemented method for determining a deformation of a 3D modeled object is provided. The method comprises providing a deformation basis function configured to infer a deformation basis of an input 3D modeled object. The method further comprises providing a first 3D modeled object. Furthermore, the method comprises providing a deformation constraint of the first 3D modeled object. The method further comprises determining a second 3D modeled object that respects the deformation constraint. The determination of the second 3D modeled object comprises calculating a trajectory of a transitional 3D modeled object between the first 3D modeled object and the second 3D modeled object. The trajectory deforms each transitional 3D modeled object by a linear combination of results of applying the deformation basis function to the transitional 3D modeled object. The trajectory reduces a loss. The loss penalizes the extent to which the deformed transitional 3D modeled object ignores the deformation constraint.

[0007] The method may include one or more of the following. The trajectory corresponds to the following integral equation:

number

[0008] Additionally, a computer readable data storage medium having a computer program recorded thereon is provided.

[0009] Further provided is a computer including a processor coupled to the memory, the memory having a computer program recorded thereon. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of a computer according to the present invention. [Diagram 2] 1 illustrates a neural network that may be used by the method. [Diagram 3] Here's how. [Figure 4] Here's how. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] A first computer-implemented method of machine learning for inferring a deformation basis for a 3D modeled object is provided. The method comprises providing a dataset of 3D modeled objects. The method further comprises training a neural network. The neural network is configured to infer a deformation basis for an input 3D modeled object. The method may be referred to as a "training method."

[0012] This constitutes an improved solution for deforming 3D modeled objects.

[0013] It is worth noting that the neural network trained by the learning means is configured to take a 3D modeled object as input and to output a deformation basis for the 3D modeled object. The deformation basis is composed of each vector forming a deformation of the input 3D modeled object. This allows to obtain many deformations of the 3D modeled object, since the vectors of the deformation basis are linearly combined, each linear combination resulting in a different deformation of the 3D modeled object. The learning method thus allows to increase the deformation possibilities of the 3D modeled object. Furthermore, the learning method allows to calculate the deformation basis once by a single application of the trained neural network, so that only linear combinations need to be performed thereafter. This allows to perform the deformation calculation efficiently, especially in terms of computation time and resource usage. It is also worth noting that the neural network does not need a target deformation of the 3D modeled object to obtain the deformation basis: the neural network alone can obtain a basis of linearly combined deformation vectors to arrive at a target deformation of the input 3D modeled object. In this way the learning method has a large generalization ability and maximizes the expressive power of the input 3D modeled object.

[0014] The neural network may infer a local deformation basis consisting of local deformation vectors, i.e., each vector representing a local (e.g., small) deformation of an input 3D modeled object, which allows the 3D modeled object to be deformed realistically in a local neighborhood of the 3D modeled object (i.e., in the vicinity of the 3D modeled object). In other words, the neural network outputs a satisfactory basis in the sense that it allows the input object to be deformed towards nearby objects in a physically acceptable way.

[0015] Furthermore, the neural network is trained on the 3D modeled objects of the dataset, at least a majority (e.g., overall) of which are plausible 3D modeled objects, such that the neural network obtains a basis of plausible vectors of deformations of the input 3D modeled objects. As previously mentioned, these vectors may be linearly combined to obtain, for example, plausible deformations in a local neighborhood of the input 3D modeled object.

[0016] Moreover, the provided dataset does not need to be made of objects of the same category (e.g., all chairs) for the neural network to be trained to infer the basis of plausible deformation vectors of the input 3D modeled objects. In particular, if the provided dataset is made or is substantially made of a large dataset of 3D modeled objects of the same category (e.g., all chairs), the neural network is allowed to infer physically realistic deformation bases for objects of this category (e.g., chairs). However, the neural network may do so even if the 3D modeled objects of the provided dataset form different categories (e.g., chairs and benches) and even if these categories are not represented by the large number of 3D modeled objects in the provided dataset. Notably, learning does not require that the objects of the provided dataset are annotated with labels (if any) indicating their respective categories. Nor does learning require the use of template modeling objects, such as the average shape of a category of objects. Moreover, the provided dataset does not need to be clustered according to the category of objects. This further improves the generalization ability of the neural network.

[0017] Nevertheless, the generalization ability of the neural network is further improved if the provided dataset consists of two or more categories of 3D modeled objects forming a manifold (e.g., chairs and benches). Indeed, a manifold may be formed by different categories of objects having at least 2-to-2 connectivity between them, i.e., a certain amount of shared features / characteristics (as chairs and benches do, for example). Due to this, a single vector of deformation basis may be a meaningful and plausible deformation for many object categories in the manifold (e.g., the seats of chairs and benches may be deformed similarly). In other words, if the provided dataset forms a manifold of 3D modeled objects (e.g., chairs and benches), the neural network is trained in such a way that it interacts between the deformations of the different categories of objects. This allows the neural network to learn to infer physically realistic / acceptable deformation criteria even if the provided dataset contains a small number of modeled objects of different categories forming a manifold. This improves the generalization ability of the neural network.

[0018] Generally speaking, the generalization power that learning gives to the neural network is such that compared to other deep learning frameworks, the neural network requires a training dataset (i.e., the provided dataset) with a smaller number of 3D modeled objects per category without affecting its ability to infer the basis of deformations. Specifically, the number of objects per category is smaller because the neural network can match learned deformations in different categories to each other. However, if the provided dataset consists of only one category of objects from two very different categories, a larger number of objects may be required.

[0019] The deformations obtained using the deformation basis output by the neural network may be used for 3D modeling, such as for 3D editing applications. The deformation basis inferred by the neural network may also be used for other applications, such as shape matching and nonlinear deformation inference, as described below. Furthermore, the neural network provides linear deformations of the input 3D modeled object, making it efficient for use in real-time applications.

[0020] Also provided is a second computer-implemented method for determining a deformation of a 3D modeled object. The method includes providing a deformation basis function configured to infer a deformation basis of an input 3D modeled object. The method further includes providing a first 3D modeled object. The method further includes providing a deformation constraint of the first 3D modeled object. The method further includes determining a second 3D modeled object that respects the deformation constraint. The determination of the second 3D modeled object includes calculating a trajectory of a transitional 3D modeled object between the first 3D modeled object and the second 3D modeled object. The trajectory deforms each transitional 3D modeled object by a linear combination of results of applying the deformation basis function to the transitional 3D modeled object. The trajectory reduces a loss. The loss penalizes the degree of non-compliance of the deformation constraint by the deformed transitional 3D modeled object. Note that the second method may be referred to as a "determining method."

[0021] The determination method constitutes an improved solution for deforming a 3D modeled object.

[0022] Notably, the determining method determines the second 3D modeled object from the first 3D modeled object by calculating a trajectory of a transient 3D modeled object between these two objects, each transient 3D modeled object being the result of a deformation of another transient 3D modeled object. In other words, the determining method deforms the first 3D modeled object along a trajectory of the deformed transient 3D modeled object that ultimately arrives at the second 3D modeled object. Thereby, the determining method determines a large nonlinear deformation of the first 3D modeled object.

[0023] Furthermore, the determination method deforms each transitional 3D modeled object by linearly combining the results of applying the deformation basis functions to the transitional 3D modeled object. The application of the deformation basis functions results in a deformation basis for the transitional 3D modeled object, i.e. a basis for the deformation vectors of the transitional 3D modeled object. The basis is a set of vectors, each of which is a deformation direction of a transitional 3D modeled object. The basis is, or at least tends to be, a basis in the sense of linear algebra. The vectors of the basis are such that they may be linearly combined to deform the transitional 3D modeled object into another transitional 3D modeled object, being plausible local deformations of the transitional 3D modeled object. For example, the deformation basis function may be a neural network learnable by the learning means, e.g. learned by the learning means, such deformation basis function resulting in vectors as previously described. Thereby, the determination method linearly and locally deforms each transitional 3D modeled object into a neighboring plausible transitional 3D modeled object until the second 3D modeled object is reached. In other words, the determination method calculates a trajectory from the first 3D modeled object to the second 3D modeled object based only on deformation basis functions that provide a basis for realistic local deformation directions of the input 3D modeled object. In other words, the determination method achieves large deformations of the first 3D modeled object based only on deformation basis functions that result in locally plausible deformations, making the determination method efficient. Furthermore, the local deformations used along the trajectory are plausible and realistic, i.e., they realistically deform the 3D modeled object into another object that belongs to the neighborhood of the 3D modeled object, so that the large deformations to reach the second 3D modeled object from the first 3D modeled object are likewise realistic.

[0024] The efficiency and realism of the determination method are significantly improved if the deformation basis functions are configured to infer deformation bases of the input 3D modeled objects that belong to a manifold of 3D modeled objects. This is the case, for example, when the deformation basis functions are neural networks that can be trained (e.g., trained) according to a training method on a data set of 3D modeled objects forming a manifold, as previously described. Indeed, as previously described, neural networks trained in such a way have improved generalization power. In this case, the 3D modeled objects included in the determination method may belong to this manifold. In such a case, physically realistic local deformations of the transitioning 3D modeled objects are efficiently inferred by the neural network, as previously described for the training method. Finally, large deformations of the first 3D modeled object, which is the second 3D modeled object, are determined efficiently and realistically as well.

[0025] Also, the trajectory is such that the loss decreases, which penalizes the extent to which the deformed transitional 3D modeled object ignores the deformation constraints. In other words, the loss is such that the more the 3D modeled object respects the deformation constraints, the smaller it becomes. Also, the trajectory of the transitional 3D modeled object is such that the loss decreases / decreases along the trajectory. In other words, along the trajectory to the second 3D modeled object, the transitional 3D modeled object respects the deformation constraints better and better. Thus, the second 3D modeled object respects the deformation constraints best among the transitional 3D modeled objects. In other words, the second 3D modeled object tends to be the minimum of the loss, or at least close to the minimum of the loss. The determination method thereby results in an optimal large nonlinear plausible deformation of the first 3D modeled object.

[0026] That is, the determination method is able to obtain a 3D modeled object (i.e. a second 3D modeled object) with a large plausible deformation of the first 3D modeled object and respecting the deformation constraints by relying only on locally plausible deformation basis inference functions. This makes the determination method efficient and robust. This also means that the determination method allows to determine large realistic deformations of the 3D modeled object. In this way, the determination method can be efficiently used for 3D modeling and 3D editing, since the user only needs to define deformation constraints on the first 3D modeled object, for the determination method to provide the user with a large realistic deformation of this object that meets the user's intention. Thereby, the determination method also constitutes an improved ergonomic solution for 3D editing.

[0027] The training method and the determination method may be performed independently. Alternatively, the determination method may be the application of a training means, i.e. the use of a neural network trained by the training means. The modified basis functions of the determination method may in fact be a neural network. Alternatively or additionally, the modified basis functions of the determination method may be any neural network that can be trained according to the training means, for example a neural network trained directly according to the training means. For example, providing the modified basis functions consists of executing the training method, whereby a neural network forming the modified basis functions is obtained.

[0028] These methods may be integrated in the same computer-implemented process for performing the deformation of the 3D modeled object. The process comprises an offline stage followed by an online stage, the learning method forming the offline stage and the determining method forming the online stage. Thus, the offline stage comprises the provision of a dataset and training the neural network according to the learning method, and the online stage comprises the provision and determining of a second 3D modeled object according to the determining method.

[0029] The learning and use methods are computer-based methods.

[0030] This means that the steps of the method (or substantially all steps) are performed by at least one computer, or similarly any system. Thus, the steps of the method are performed by a computer, possibly fully automatically, or semi-automatically. In an example, triggering of at least some of the steps of the method may be performed by user / computer interaction. The level of user / computer interaction required depends on the level of automation expected and can be balanced against the need to implement the user's wishes. In an example embodiment, this level may be user-defined and / or predefined.

[0031] A typical example of a computerized method is performing the method using a system adapted for this purpose. The system comprises a processor coupled to a memory and a graphical user interface (GUI), the memory having recorded thereon a computer program comprising instructions for performing the method. The memory may also store a database. The memory may be any hardware adapted for such storage and may consist of several physically distinct parts (e.g. one for the program and one for the database).

[0032] The methods typically operate on modeled objects. A modeled object is an object that is defined by data stored in a database or the like. By extension, the expression "modeled object" means the data itself. Depending on the type of system, the modeled object is defined by different types of data. A system may in fact be any combination of a CAD system, a CAE system, a CAM system, a PDM system and / or a PLM system. In these different systems, the modeled object is defined by the corresponding data. Thus, one may say CAD objects, PLM objects, PDM objects, CAE objects, CAM objects, CAD data, PLM data, PDM data, CAM data, CAE data. However, these systems are not exclusive, since a modeled object may be defined by data corresponding to any combination of these systems. Thus, a system may be both a CAD and a PLM system.

[0033] A CAD system additionally means a system adapted to design a modeled object based on at least a graphical representation of the modeled object, such as CATIA. In this case, the data defining the modeled object consists of data enabling the representation of the modeled object. The CAD system may for example provide a representation of the CAD modeled object using edges or lines, in particular cases with faces or surfaces. The lines, edges or surfaces may be represented in various ways, for example with Non-Uniform Rational B-Splines (NURBS). In particular, a CAD file contains a specification, from which a geometry is generated, whereby a representation is generated. The specification of the modeled object may be stored in one CAD file or in several CAD files. A typical size of a file representing a modeled object in a CAD system is in the range of one megabyte per part. And the modeled object may typically be an assembly of several thousand parts.

[0034] In the context of CAD, a modeled object is typically a 3D modeled object, e.g., representing a product such as a part or an assembly of parts, or possibly an assembly of products. By "3D modeled object" we mean any object modeled by data that allows for a 3D representation. The 3D representation allows for viewing the part from any angle. For example, a 3D modeled object, when represented in 3D, can be manipulated and rotated around any of its axes or around any of the axes of the screen on which the representation is displayed. This notably excludes 2D icons that are not 3D modeled. The display of 3D representations facilitates design (i.e., statistically increases the speed with which designers accomplish their tasks). In industry, this translates to a speedup of the manufacturing process, since the design of the product is part of the manufacturing process.

[0035] Any 3D modeled object (in any method) may represent the shape of a product that is manufactured in the real world, for example after the virtual design is completed using a CAD software solution or CAD system, such as a (e.g. mechanical) part or assembly of parts (or equivalently an assembly of parts, which may be viewed as a part itself from the method perspective, or the method may be applied to each part of the assembly independently), or more generally any rigid body assembly (e.g. moving mechanism), etc. CAD software solutions enable the design of products in a variety of unlimited industry sectors, including: aerospace, architecture, construction, consumer goods, high tech equipment, industrial equipment, transportation, marine, and / or offshore oil / gas production or transportation. Any 3D modeled object involved in the method may therefore represent an industrial product which may be any mechanical part, for example, parts of land vehicles (including, for example, automobiles and light truck equipment, racing cars, motorcycles, trucks and motor equipment, trucks and buses, trains), parts of air vehicles (including, for example, airframe equipment, aerospace equipment, propulsion equipment, defense products, aviation equipment, space equipment), parts of naval vehicles (including, for example, naval equipment, commercial vessels, marine equipment, yachts and workboats, marine equipment), general mechanical parts (including, for example, industrial manufacturing machinery, heavy mobile machinery or equipment, installed equipment, industrial equipment products, fabricated metal products, tire manufacturing products), electric machinery or electronic parts (including, for example, consumer electronics products, security and / or control and / or instrumentation products, computing and communications equipment, semiconductors, medical equipment and devices), consumer goods (including, for example, furniture, home and garden products, leisure products, fashion products, hard goods retail products, soft goods retail products), packaging (including, for example, food and beverage and tobacco, beauty and personal care, household product packaging).

[0036] FIG. 1 shows an example of a system, where the system is a client computer system, such as a user's workstation.

[0037] The client computer of this embodiment comprises a central processing unit (CPU) 1010 connected to an internal communication BUS 1000, and a random access memory (RAM) 1070 also connected to the BUS. The client computer further comprises a graphical processing unit (GPU) 1110 associated with a video random access memory 1100 connected to the BUS. The video RAM 1100 is also known in the art as a frame buffer. A mass storage controller 1020 manages access to mass memory devices such as a hard drive 1030. Mass memory devices suitable for embodying computer program instructions and data include all forms of non-volatile memory including, illustratively, semiconductor memory devices such as EPROM, EEPROM, flash memory devices, magnetic disks such as internal hard disks and removable disks, magneto-optical disks, and CD-ROM disks 1040. Any of the foregoing may be supplemented by or incorporated in specially designed ASICs (Application Specific Integrated Circuits). A network adapter 1050 manages access to a network 1060. The client computer may also include a haptic device 1090 such as a cursor control device, a keyboard, etc. A cursor control device is used in the client computer to allow a user to selectively place a cursor at any desired location on the display 1080. Additionally, the cursor control device allows a user to select various commands and input control signals. The cursor control device includes a number of signal generating devices for inputting control signals to the system. Typically, the cursor control device may be a mouse, and the buttons of the mouse are used to generate the signals. Alternatively or additionally, the client computer system may have a sensitive pad and / or a sensitive screen.

[0038] A computer program may include instructions executable by a computer, the instructions including means for causing the system to carry out the method. The program may be recordable on any data storage medium including the memory of the system. The program may be implemented, for example, in digital electronic circuitry, or computer hardware, firmware, software, or a combination thereof. The program may be implemented as an apparatus, for example an article tangibly embodied in a machine-readable storage device for execution by a programmable processor. The steps of the method may be executed by a processor capable of executing a program of instructions to perform the functions of the method by operating on input data and generating output. Thus, the processor may be programmable and coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. The application program may be implemented in a high level procedural or object-oriented programming language, or in assembly or machine language as desired. In either case, the language may be a compiled or interpreted language. The program may be a full installation program or an update program. Application of the program on a system provides, in any case, instructions for carrying out the method.

[0039] The learning method will be explained further.

[0040] The provision of a dataset of 3D modeled objects will now be described, but prior to this description the data structures involved will be described.

[0041] Any 3D modeled object herein may form a discrete geometric representation of a 3D shape, for example representing an object from the real world, such as a machine part as previously described. A discrete geometric representation herein is a data structure consisting of a discrete set of data. Each piece of data represents a respective geometric entity located in 3D space. Each geometric entity represents a respective position of the 3D shape (in other words, a respective portion of the material that makes up the solid represented by the 3D shape). The collection (i.e., combination or juxtaposition) of the geometric entities as a whole represents the 3D shape. A discrete geometric representation here may be composed of, for example, 100, 1000, or even higher than 10000 pieces of data.

[0042] Any discrete geometric representation herein may be, for example, a 3D point cloud, with each geometric entity being a point. Any discrete geometric representation herein may alternatively be a 3D mesh, with each geometric entity being a mesh tile or face. Any 3D mesh herein may be regular or irregular (i.e., composed of faces of the same type or not). Any 3D mesh herein may be a polygonal mesh, for example a triangular mesh. Any 3D mesh herein may be obtained from a 3D point cloud, for example by triangulating (e.g. Delaunay triangulation) the 3D point cloud. Any 3D point cloud herein may be determined from physical measurements on a real object, for example within a 3D reconstruction process. The 3D reconstruction process may consist of providing a real object, providing one or more physical sensors, each configured to acquire a respective physical signal, and acquiring one or more respective physical signals by operating the one or more physical sensors on the real object (i.e., scanning the real object with the respective sensors). The 3D reconstruction may then automatically determine a 3D point cloud and / or a 3D mesh based on the measurements according to any known technique. The one or more sensors may consist of multiple (e.g., RGB, and / or image or video) cameras, and the determination may consist of a structure-motion analysis. The one or more sensors may alternatively or additionally consist of one or more depth sensors (e.g., RGB depth cameras), and the determination may consist of a 3D reconstruction from the depth data. The one or more depth sensors may consist of, for example, a laser (such as a lidar) or an ultrasonic emitter / receiver.

[0043] Any 3D point cloud or 3D mesh described herein may alternatively be obtained from a 3D modeled object representing the skin (i.e., outer surface) of a solid (e.g., corresponding to a B-Rep model representing the skin, i.e., exact surface), for example, by ray-casting on the 3D modeled object or tessellation of the 3D modeled object. The tessellation may be performed according to a rendering process of any 3D modeled object. Such a rendering process may be coded on any CAD system to display a graphical representation of the 3D modeled object. The 3D modeled object may be or has been designed by a user with a CAD system.

[0044] Providing the dataset may be forming the dataset, for example by creating 3D modeled objects. Alternatively, providing the dataset may consist of retrieving the dataset from a (e.g., remote) memory in which the dataset is stored after it is created. The 3D modeled objects of the dataset may all be 3D point clouds. Alternatively, they may all be 3D meshes. In such a case, the learning method may include pre-processing to sample these meshes into a 3D point cloud. The pre-processing may then center each 3D mesh. The pre-processing may then include independently rescaling the dimensions of each 3D mesh so that the vertices of the meshes fit exactly into a unit cube. The pre-processing may then extract a dense point cloud from each normalized shape in the dataset, for example by ray-casting each of the six orthographic views. The pre-processing may then consist of uniformly subsampling each point cloud (e.g., downsampling each point cloud to the same size). Subsampling may start at a random point in the point cloud and iteratively select the furthest point in the point cloud from the already selected point to arrive at the desired number of points.

[0045] The 3D modeled objects of the dataset may be plausible (e.g., realistic) 3D modeled objects. A plausible 3D modeled object may specify a 3D modeled object that represents a real-world object, e.g., a plausible mechanical part. A plausible mechanical part may specify a mechanical part that can be realistically manufactured in a real-world industrial manufacturing process. A plausible mechanical part may refer to a mechanical part that respects all constraints that must be respected to realistically manufacture the mechanical part in a real-world industrial manufacturing process. The constraints may consist of one or more of mechanical constraints (e.g., constraints arising from the laws of classical mechanics), functional constraints (e.g., constraints related to one or more mechanical functions performed by the mechanical part once manufactured), manufacturing constraints (e.g., constraints on the ability to apply one or more manufacturing tools to the mechanical part during one or more manufacturing steps to manufacture the mechanical part), structural constraints (e.g., constraints on the strength and / or resistance of the mechanical part), and / or assembly constraints (e.g., constraints defining how the mechanical part can be assembled with one or more other mechanical parts).

[0046] As previously mentioned, the 3D modeled objects of the provided dataset may all or substantially all belong to a single category of 3D modeled objects, or may form different categories of 3D modeled objects. In any case, the 3D modeled objects of the provided dataset represent respective objects from the real world, such as machine parts as previously described.

[0047] Explain neural network training.

[0048] As known in the field of machine learning, processing an input by a neural network includes applying an operation to the input, the operation being defined by data including the values ​​of the weights. Thus, training a neural network includes determining the values ​​of the weights based on a dataset configured for learning, such a dataset may be referred to as a learning dataset or training dataset. To that end, the dataset includes pieces of data each forming a respective learning sample. The training samples represent the diversity of situations in which the neural network will be used after being learned. Any dataset referred to herein may include a number of training samples higher than 10000, 100000, or 1000000. Here, the neural network is trained on the provided dataset, meaning that the provided dataset is the learning / training dataset for the neural network. The learning may be performed by any suitable known method.

[0049] The neural network is configured to infer a deformation basis for an input 3D modeled object. That is, the neural network receives a 3D modeled object as an input and outputs a deformation basis for the 3D modeled object. To do so, the input 3D modeled object needs to be of the same data type as the 3D modeled object of the dataset. For example, if the 3D modeled object of the dataset is a 3D mesh, the input 3D modeled object is also a 3D mesh. Alternatively, if the 3D modeled object of the dataset is a 3D point cloud, the input 3D modeled object is also a 3D point cloud. Nevertheless, the 3D modeled object may be a 3D point cloud obtained from sampling a mesh. The deformation basis is a collection of vectors, each of which is a deformation direction. The vectors may be linearly combined to deform the input 3D modeled object, and the linear combination, with its coefficients (also called amplitudes), results in the deformation. The vectors may be linearly combined in such a way that the deformed input 3D modeled object approaches the input 3D modeled object if the linear combination is small enough. In such a case, the deformation is realistic. Mathematically, the deformation is realistic as long as the deformation is sufficiently close to the deformation on the tangent space defined by the deformation basis. The deformation basis is a basis in the sense of linear algebra, a set of linearly independent vectors, where the basis may be normalized, e.g., orthogonal. Specifically, the learning aims for the neural network to infer, or at least tend to infer, a basis of deformation vectors that are linearly independent (e.g., uncorrelated and / or orthogonal vectors, as further explained below). The basis of deformation vectors may have a fixed size (e.g., 2 to 10 vectors, or more than 10 vectors). In other words, the neural network may always (i.e., for each input 3D modeled object) output a deformation basis that has the same fixed number of vectors.If the dataset consists of plausible 3D modeled objects, the neural network infers a deformation basis that has plausible deformation vectors of the 3D modeled objects, since it has been trained to do so.

[0050] The neural network has an architecture configured to take a 3D modeled object as input and output a deformation basis for it. The neural network may be composed of an encoder and a deep feedforward neural network. The encoder is configured to take a 3D modeled object as input and output a latent vector representing the input 3D modeled object. Thus, the encoder may be configured to take a 3D mesh or a 3D point cloud (e.g., sampled from a 3D mesh) as input. The deep feedforward neural network is configured to take a latent vector output by the encoder as input and output a deformation basis for the 3D modeled object represented by the latent vector. The encoder architecture may be based on PointNet (e.g., as described in previously cited reference [8], which is incorporated herein by reference), or any extension thereof. Alternatively, the encoder architecture may use a mesh topology (i.e., where the 3D modeled objects of the dataset are 3D meshes), as is done, for example, in Mesh CNN (see, for example, previously cited reference [9]).

[0051] The training may minimize a loss for at least a portion of the dataset, where for each 3D modeled object in the at least a portion of the dataset and for each candidate deformation basis having vectors, the loss penalizes a distance between a deformation of the 3D modeled object with a linear combination of the vectors and another 3D modeled object.

[0052] Explain loss minimization.

[0053] At least a portion of the dataset is a sample of the 3D modeled objects of the dataset. For example, at least a portion of the dataset may be mini-batches. The learning is then performed mini-batch-by-mini-batch and consists of minimizing a loss for each mini-batch. The concept of learning mini-batch-by-mini-batch is known in the field of machine learning. For example, the learning may implement a known mini-batch stochastic optimization method such as mini-batch stochastic gradient descent. Learning mini-batch-by-mini-batch improves the efficiency of the learning, as known per se from the field of machine learning.

[0054] For each 3D modeled object of at least a portion of the dataset, the training evaluates the deformation basis of the modeled object output by the neural network with the current weight value. This calculated deformation basis forms a candidate deformation basis of the 3D modeled object. The loss penalizes the distance between the deformation of the 3D modeled object by a linear combination of the vector of this candidate deformation basis and another 3D modeled object. That is, the loss tends to be large when the distance between the deformation of the 3D modeled object by the linear combination of the vector of this candidate basis and another 3D modeled object is large. In this case, the training to minimize the loss modifies the weights of the neural network to reduce the value of the loss to obtain a new candidate. The training does so (e.g., for each mini-batch) until the loss reaches its minimum value, or at least a sufficiently small value (e.g., with respect to a convergence criterion).

[0055] Said another 3D modeled object may be a target object (i.e., a linear combination of vectors is for deforming the 3D modeled object). This allows training the neural network to infer the basis of the deformation. For example, the another 3D modeled object may be a 3D modeled object among at least some 3D modeled objects of the dataset that is close to at least some 3D modeled objects of the dataset. In other words, the learning may include, among at least some 3D modeled objects of the dataset, selecting another 3D modeled object based on a distance from at least some 3D modeled objects of the dataset. The selection may include calculating a distance between at least some 3D modeled objects of the dataset and at least some 3D modeled objects of the dataset, and evaluating which 3D modeled object is closer, i.e., for which 3D modeled object the distance is smaller. Then, the selection may select the 3D modeled object with the smaller distance. In an example, the selection may evaluate which 3D modeled object is closest, i.e., for which 3D modeled object the distance is the smallest. In other words, the selection is performed by calculating a minimum value of the distance and evaluating which 3D modeled object results in this minimum value. In these examples, the other 3D modeled object is a 3D modeled object that is closest to the at least some 3D modeled object of the dataset among the at least some 3D modeled objects of the dataset. In the examples, the learning is performed per mini-batch, and for each mini-batch, it consists of minimizing the loss and selecting the closest 3D modeled object among the 3D modeled objects of the mini-batch. In other words, the selection of the other 3D modeled object is performed among the objects of the mini-batch, where the at least some of the dataset is the mini-batch.

[0056] The distance from at least some of the 3D modeled objects of the dataset may be any distance, such as a distance in 3D between 3D modeled objects, e.g., a distance between point clouds or 3D meshes. In an example, the distance may be a distance in a latent space, as currently described.

[0057] Specifically, in these examples, the learning method may consist of providing a separate encoder configured to take as input a 3D modeled object of the provided dataset, e.g., prior to learning, e.g., in an initial stage of the learning method, and to output a latent vector encoding the 3D modeled object. The separate encoder may be trained in an autoencoder framework, a classification task, or other machine learning methods, as known in the field of machine learning, to learn a meaningful latent space of the 3D modeled object. Providing the separate encoder consists of training the separate encoder. Alternatively, the separate encoder may be pre-trained. In these examples, the distance between the first 3D modeled object and the second 3D modeled object may be a distance in the latent space of the separate encoder between a result of applying the separate encoder to the first 3D modeled object and a result of applying the separate encoder to the second 3D modeled object. This latent space distance allows training a neural network to more accurately infer the basis of the deformation, since the latent vector output by the separate encoder captures the semantics of the encoded 3D modeled object. In other words, the encoder implicitly clusters the dataset with respect to semantics. This allows another 3D modeled object to be determined as the 3D modeled object that is most similar to at least some of the 3D modeled objects in the dataset. The use of this latent space distance can thus improve the accuracy of learning, as well as improve efficiency, as compared to, for example, distances in the 3D modeled object space, which are computationally more expensive.

[0058] The selection of the alternative 3D modeled object may be performed prior to the minimization, for example, the results of applying the alternative encoder to the 3D modeled objects of the dataset may be pre-computed, i.e., calculated prior to the minimization.

[0059] A linear combination has coefficients, each corresponding to the contribution of each vector in the combination. The coefficients are calculated by the neural network during training, or they can be the result of an optimization performed during training.

[0060] The loss may penalize the minimum of the distance between the deformation of the 3D modeled object by the linear combination of vectors and another 3D modeled object. In this way, the optimal deformation by the linear combination is selected when minimizing the loss, i.e., the deformation that minimizes the distance. The minimum may be, for example, the minimum of the distance between the deformation of the 3D modeled object by the linear combination of vectors and another 3D modeled object, or may be the minimum of all possible values ​​of the coefficients of the linear combination. The loss penalizes the minimum in that the larger the minimum, the larger the loss tends to be. In this case, the learning to minimize the loss modifies the weights of the neural network to lower the value of the loss. The learning does so (e.g., for each mini-batch) until the loss reaches the minimum, or at least a sufficiently small value (e.g., with respect to a convergence criterion).

[0061] The losses may be of the following types:

number

number

[0062] e1,...,e N may already be a 3D point cloud, in which case e1=p1,…,e N =p N Or, e1, ,e N may be a 3D mesh, in which case p1,…,p N is e1,…,e N This may be due to sampling. CH d may be any 3D distance between 3D point clouds, such as earthmover distance or chamfer distance.CH Examples include the following types:

number

[0063] q1 is calculated as follows: Let f' be another encoder as described above. Then, for each i, its training method is

number

[0064] The loss may further penalize sparsity, including a (e.g., minimum) function that takes as input the coefficients of the linear combination for each 3D modeled object and each candidate deformation basis. The sparsity function may, for example, be the L1 norm of the coefficients of the linear combination, as previously described (see, for example, reference [6], which is incorporated herein by reference). In other words, if the sparsity-inducing function (e.g., minimum) is large, the loss tends to be large. In this case, training to minimize the loss modifies the weights of the neural network to lower the value of the loss. Training does so (e.g., for each mini-batch) until the loss reaches a minimum, or at least a sufficiently small value (e.g., with respect to a convergence criterion). In this case, the loss may be of the following type:

number

[0065] where λ is a trade-off parameter (a well-known concept in the field of machine learning), φ is a sparsity-inducing function such as the L1 norm (see, e.g., previously cited reference [6], which is incorporated herein by reference), and (α1, ..., α p) is the vector of coefficients, also called the vector of magnitudes.

[0066] This allows the coefficients / amplitudes to be sparse during training. Ultimately, this allows the neural network to be trained to output uncorrelated deformation basis vectors, which further improves the generalization power of the training method. It is noteworthy that a neural network trained in this way can output a satisfactory basis, since its sparseness allows a small number of vectors to be used to calculate the deformation of the input 3D modeled object with a linear combination of basis vectors. Also, the output basis is, or at least tends to be, a basis in the linear algebra sense.

[0067] The loss may further include, for each 3D modeled object and each candidate deformation basis, rewarding the orthogonality of the deformation basis of the candidate deformation basis. In other words, the loss may further include a term that captures the orthogonality of the deformation basis of the 3D modeled object estimated by the neural network with the current weight value, and the term tends to be small if the deformation basis tends to be orthogonal. The loss in this case may be of the following type:

number

[0068] δ is a trade-off parameter (a well-known concept in the field of machine learning) that further allows the neural network to learn to output transformed basis vectors that tend to be orthogonal matrices and therefore uncorrelated, such that such bases are, or at least tend to be, bases in the linear algebraic sense.

[0069] The loss may further comprise any suitable regularization function, as known per se from the field of machine learning.

[0070] An implementation of the learning method is described.

[0071] The present embodiment brings improvements to the fields of 3D modeling and 3D machine learning. The results of the present embodiment can be used in fields such as virtual and augmented reality (more generally, any kind of immersive experience), video games, manufacturing and 3D printing, or 3D modeling. The present embodiment provides a solution to compute realistic sparse deformation linear bases for any 3D modeled object, which can be used to perform shape synthesis, shape reconstruction from images, or shape matching.

[0072] In this embodiment, any modeled 3D object can be provided with a plausible linear deformation basis such that any linear combination of the deformation vectors that make up the basis can deform the model realistically in a local neighborhood. Such deformations can be used in 3D modeling software, particularly in 3D editing applications, for inference of large nonlinear deformations and shape matching.

[0073] In this embodiment, a neural network (training-while-training neural network) is trained to learn a linear 3D deformation field with sparse constraints to obtain plausible deformations that are independent of each other. In this embodiment, a neural network (training-while-training neural network) is trained to infer a deformation basis at each point of an input 3D modeled object from a provided dataset, a dataset of unlabeled 3D modeled objects. To train this network, the training proceeds mini-batch by mini-batch, as described above.

[0074] (1) Match each 3D modeled object of each batch with its closest 3D modeled object in the same batch, i.e., select from among the 3D modeled objects of the batch another 3D modeled object that is closest to the 3D modeled object of the batch. (2) Compute a linear combination of the predicted deformation vectors of the 3D modeled objects that minimizes the distance between the deformation vector inferred by the neural network with the current weights and the nearest 3D modeled object in the batch, with an additional sparseness regularization term. (3) Minimize the same loss (the distance between the deformed 3D modeled object and its nearest 3D modeled object) across the neural network weights, optimizing the basis for the predicted deformation.

[0075] In this embodiment, it is not necessary to cluster different types of 3D modeled objects and train them separately. Instead, it is suitable to train the entire dataset directly. In this embodiment, we perform uncorrelated deformation-based learning where each deformation is a plausible deformation in itself. The estimated deformations are linear, allowing real-time use. The deformations are estimated as a 3D deformation field, maximizing the expressiveness of the input 3D modeled object.

[0076] Upon providing the data set, in this embodiment, if the provided data set consists of 3D meshes, pre-processing may be performed as described above. Pre-processing consists of centering each mesh and independently rescaling the dimensions of each mesh so that the vertices of the mesh fit exactly into a unit cube. Next, pre-processing begins by extracting a dense point cloud from each normalized shape in the data set by ray-casting into six orthographic views. Next, pre-processing uniformly sub-samples each point cloud (e.g., down-samples each point cloud to the same size). To do this, pre-processing may start with a random point in the point cloud and iteratively selects the furthest point in the point cloud from the already selected points until a predefined (e.g., desired) number of points is reached.

[0077] The architecture of the neural network according to the presently described embodiment will now be described.

[0078] In this embodiment, the neural network infers the same fixed number of deformation basis vectors (e.g., for any input 3D modeled object). Let n be this fixed and same size of the inferred deformation basis. In this embodiment, the neural network takes a 3D point cloud or a 3D mesh as input and generates latent vectors

number

[0079] f w In addition, the present embodiment uses a deep feedforward neural network g w g w takes as input the 3D points x, as well as the latent vectors h of a 3D point cloud or a 3D mesh. w is the deformation basis at x

number

number

number

number

number

number

number

[0080] This embodiment may include providing another encoder architecture f' similar to fw. As mentioned above, f' is trained in an autoencoder framework, or classification task, or any other machine learning method, to learn a meaningful latent space of 3D objects. In this embodiment, another encoder is used to efficiently match 3D modeled objects (3D meshes or 3D point clouds) with their nearest 3D modeled objects in batches during the training of fw.

[0081] In this embodiment, unsupervised learning is performed when training the neural network. This will be described.

[0082] In training, both networks fw and gw are trained on a set of N points p1, ..., p N It learns by minimizing the loss E(w) (also called "energy") for e1, ..., e N Let f be the corresponding 3D modeled objects in the input space of fw. That is, these inputs are the point cloud e i =p i If it is a mesh, then e i HAp i This is the source mesh used to sample the

[0083] For each i,

number

[0084] Then the loss is given by:

number

[0085] d CH is here the chamfered 3D distance, but can be replaced by other 3D losses such as the Earth motion distance.

number

[0086] φ is L 1 is a sparsity-inducing function such as the α1,…,α p ) is called the vector of amplitudes. We seek to make the amplitudes sparse in order to make the deformations as uncorrelated as possible, so that any deformation vector is itself plausible. For the same reason, we enforce that the deformation basis is orthonormal with respect to the δ-penalty, which precedes a term that captures the regularity of the basis as explained before.

[0087] σ(e i ,e i +v) may be any regularization function, including but not limited to combinations of the following: e i and e i + v gives a penalty for the difference in edge length between i If x is a point cloud, then this embodiment involves generating edges on the point cloud using a k nearest neighbor graph). ei and e i + v (i.e., L(e i ) and L(e i +v), where L is e i is the Laplace operator of λ, δ, and γ are trade-off parameters.

[0088] To minimize E(w), training involves computing the amplitudes (α1,…,α p )

[0089] For each input i,

number

[0090] But the minimum

number

number

[0091] Where:

number

number

number

[0092] The envelope theorem states that

number

number

number

number

number

[0093] FIG. 2 is a diagram illustrating a deep model architecture of a neural network trained according to the presently described embodiment.

[0094] Next, the determination method will be further explained.

[0095] The determination method is for determining a deformation of a 3D modeled object. In other words, the determination method takes a 3D modeled object as input and outputs a deformation of the 3D modeled object, the deformation of the 3D modeled object being a large non-linear deformation. The determination method comprises, in particular, providing a first 3D modeled object (e.g., by a user's manipulation, e.g., during a 3D editing process), which is the 3D modeled object to be deformed by the determination method. The first 3D modeled object may be a 3D point cloud or a 3D mesh. For example, if the deformation basis function takes a 3D point cloud as input, the first modeled object is a 3D point cloud. Also, for example, if the deformation basis function takes a 3D mesh as input, the first modeled object is a 3D mesh.

[0096] The determining method further comprises providing a deformation constraint of the first 3D modeled object. The concept of deformation constraint of a 3D modeled object is known per se. However, here, for completeness, an example of the concepts involved is described. Nevertheless, the determining method may implement any known method for providing a deformation constraint.

[0097] The provision of the deformation constraints may be performed by a user, i.e., as a result of a user action. For example, a user may define the deformation constraints. The deformation constraints may specify a set of (e.g., geometric) specifications that determine the transformation of a geometric shape of a part of the first 3D modeled object into another geometric shape (hereinafter may be referred to as a "new geometric shape"). In other words, the deformation may not be directly defined by the user, but rather the user's action may result in a specification that enforces the deformation. Illustratively, this means that the user provides (e.g., declares, e.g., specifies) a set of specifications, for example using a keyboard and / or a touch and / or haptic device. Alternatively, the set of specifications may be obtained (e.g., automatically) from a user action of (e.g., part of) the first 3D modeled object. The set of specifications may also be obtained (e.g., automatically) from a user drawing on a screen that constitutes the first 3D modeled object. Thus, the transformation defined by the transformation constraint may define a transformation to a deformed version of the first 3D modeled object by transforming the geometric shape of a part of the first 3D modeled object into a new geometric shape.

[0098] The definition of the deformation constraints is performed by a user performing a graphical interaction with a portion of the first 3D modeled object. The graphical interaction can be achieved by clicking on the first 3D modeled object using, for example, a haptic device such as a mouse, or by touching the first 3D modeled object using an accessory such as a finger. The graphical interaction may further consist of moving the clicked or touched 3D modeled object, for example by performing a drag-and-drop operation. For example, the user selects one or more points on the portion of the 3D modeled object and moves them by a drag-and-drop operation (e.g., moving them one by one or via a one-shot drag-and-drop operation), as known in the art, for example by clicking on one or more points with a haptic device such as a mouse. The one or more positions of each of the one or more moved points determine (e.g., form, e.g., result in) a new geometric shape of the portion of the first 3D modeled object. In other words, the first 3D modeled object has a predefined geometric shape when provided, and the user may define a new geometric shape by moving points. The transformation of the given geometric shape into the new geometric shape is thus constrained by one or more positions of each of the one or more points moved by the user. In other words, by moving one or more points a new geometric shape of the part is defined, and the new geometric shape of the part defines the transformation constraints.

[0099] Alternatively, the user may define the deformation constraints by drawing (e.g., using a haptic device) a 3D or 2D drawing / sketch that represents a new geometric shape of a part of a first 3D modeled object that, when provided, has a given geometric shape. The deformation of the given geometric shape to the new geometric shape is thus constrained by the user's drawing, e.g., a drawing captured by a set of pixels. In other words, the user's drawing defines the new geometric shape of the part, and the new geometric shape of the part defines the deformation constraints.

[0100] The method further comprises providing a deformation basis function configured to infer a deformation basis for the input 3D modeled object. In other words, the deformation basis function takes the input 3D modeled object as input and outputs a basis. The basis is, or at least tends to be, a basis in the linear algebra sense. The basis is a set of vectors, each vector being a direction of a locally plausible deformation of the input 3D modeled object. The vectors of the basis may be linearly combined, the resulting linear combination forming a locally plausible deformation of the input 3D modeled object. The vectors of the basis are, or at least tend to be, uncorrelated. For example, the basis may be orthogonal, e.g., a square.

[0101] To that end, the modified basis functions may be (e.g., may be) constructed from a neural network obtainable according to a learning means, for example a neural network trained according to a learning means, in which case the basis consists (e.g., is constructed from) a vector of modified bases output by this neural network.

[0102] Additionally or alternatively, the deformation basis functions may consist of (e.g., may be functions that take as input a 3D modeled object and output geometric and / or physical spectral modes. The modes are vectors computed by spectral decomposition of quantities such as the Laplacian and stiffness matrix of the input 3D modeled object, e.g., eventually adding additional forces such as damping forces. In such a case, the basis includes (e.g., consists of) these modes. These modes may be those of the previously cited reference

[12] (in the case of the Laplacian) or those of the previously cited reference [3] (in the case of the stiffness matrix), both of which are incorporated herein by reference.

[0103] Additionally, the transformation basis functions may consist of (e.g., may be) functions that take as input the input 3D modeled object and output the principal components of a PCA applied to a dataset of the input 3D modeled object. In such a case, the basis consists of (e.g., consists of) these vectors. These vectors may be those of the previously cited reference [7], which is incorporated herein by reference.

[0104] Additionally or alternatively, the deformation basis functions may consist of (e.g., may be functions that take as input an input 3D modeled object and output hand-crafted geometric modes. In such a case, the deformation basis includes (e.g., consists of) these modes. These modes may be those of the previously cited reference [3], which is incorporated herein by reference.

[0105] In an embodiment, the modified basis functions are

number

number

number

number

[0106] These functions are linearly combined to infer a locally plausible deformation of the input 3D modeled object X. In other words,

number

number

[0107] As mentioned before, the deformation vector V i The deformation basis of may be any combination of the following: A vector of linear transformation bases. The vector of linear transformation bases is output by a neural network that can be trained according to a training method, for example, a neural network trained according to a training method. In such a case,

number

[12] ) or the stiffness matrix (such as those considered in the previously cited reference [3]), finally with additional forces (e.g. damping forces) added. Principal Components PCA. Principal components PCA is applied to a dataset of input shapes that share correspondences, resulting in uncorrelated deformation vectors (such as those considered in the previously cited reference [7]). Geometric modes of handcraft, such as those discussed in the previously cited reference [3]:

number

[0108] Next, the determination of the second 3D modeled object will be described.

[0109] The determination of the second 3D modeled object comprises calculating a trajectory. The trajectory is a transformation between the first 3D modeled object and the second 3D modeled object that respects the deformation constraints, i.e., the first 3D modeled object that satisfies the (e.g., user-defined) deformation constraints. The trajectory is a trajectory of the transitional 3D modeled objects. Each transitional 3D modeled object is a transitional transformation of the first 3D modeled object, i.e., respects the deformation constraints less than the second 3D modeled object, but respects the deformation constraints more than the previous transitional 3D modeled object. Each transitional 3D modeled object is transformed into another object by linearly combining the results of applying the deformation basis functions to the transitional 3D modeled object. That is, along the trajectory, each transitional 3D modeled object is transformed by being transformed into a linear combination of vectors of the deformation basis functions applied to the transitional 3D modeled object. This linear combination is a locally plausible linear transformation of the transitional 3D modeled object. That is, a trajectory of the successive deformation of the first 3D modeled object by successive linear combinations of vectors resulting from successive applications of the deformation basis functions. In this regard, in the present disclosure, the trajectory is considered to start from the first 3D modeled object and deform each transitional 3D modeled object by a linear combination of vectors of the deformation basis functions applied to the transitional 3D modeled object, which deformation ultimately results in the second 3D modeled object.

[0110] The trajectory reduces the loss. The loss may be user-defined, i.e., defined by the user, for example, for the purpose of capturing non-respect of user-provided / defined deformation constraints. The loss penalizes the degree of non-respect of the deformation constraints by the deformed transitional 3D modeled object. In other words, the loss is a function that tends to have a low value if the transitional 3D modeled object tends to respect the deformation constraints and a high value if the transitional 3D modeled object tends not to respect the deformation constraints. For example, the deformation constraints may be a 3D shape / sketch that represents the deformation of the first 3D modeled object, and the loss may penalize the distance between the sketch and the transitional 3D modeled object. The trajectory reduces the loss, i.e., along the trajectory, the value of the loss applied to the transitional 3D modeled object decreases (not necessarily in strict terms). In other words, along the trajectory, the transitional 3D modeled object respects more and more of the deformation constraints until it reaches the second 3D modeled object that respects it the most. For example, 3D modeled objects tend to be close to the loss minimum.

[0111] The loss is an arbitrary 3D loss L

number

number

[0112] where X is the input 3D modeled object of the loss, d is an arbitrary distance between the 3D modeled objects (such as chamfer distance or earth movement distance), and ρ is an additional regularization term (optional, λ is zero) to enforce some constraints. For example, ρ(X) can enforce symmetry as follows:

number

[0113] The trajectory may correspond to an integral equation of the following type:

number

number

[0114] By "the trajectory corresponds to an integral equation" we mean that the calculated trajectory tends to be captured by X(t). The determination method may calculate this trajectory exactly, or alternatively, the determination method may calculate the trajectory by approximating X(t) (e.g., by discretizing it by any known discretization method).

[0115] In fact, the calculation may consist in discretizing the trajectory by calculating a sequence of intermediate 3D modeled objects that converge towards the second 3D modeled object and reduce the loss. This means that the sequence converges and at the same time reduces the loss, i.e. the loss decreases (not necessarily strictly) along the sequence. The discretization may use any known discretization method (for example any method suitable for discretizing integral trajectories). The discretization is performed in such a way that the sequence converges and its limit forms the second 3D modeled object and the loss decreases.

[0116] Notably, the computation of the sequence may be repeated starting from a first 3D modeled object as follows: Applying the deformation basis functions to the intermediate 3D modeled objects calculated in the previous iteration and their applied yield vectors (i.e. forming a basis of vectors each of which is a deformation direction of the intermediate 3D modeled objects). Among the possible deformations (i.e. possible linear deformations) of the intermediate 3D modeled object calculated in the previous iteration, determine the one that minimizes the loss by a respective linear combination of the obtained vectors. · Select the determined transformation candidate applied to the current intermediate 3D modeled object as the next intermediate modeled object.

[0117] In other words, the computation of the sequence as described above iteratively computes each intermediate 3D modeled object, which minimizes a loss among a set of possible linear combinations of deformation vectors output by the deformation basis functions applied to the previously computed intermediate 3D modeled object. The minimization of the loss may be performed by any known minimization method, for example any known constrained or unconstrained minimization method.

[0118] The determination of the candidate deformation that minimizes the loss (i.e., among the candidate deformations) may involve searching a candidate set of coefficients of each linear combination and operating a selection among the searched candidate sets. In this case, the set of selected candidate coefficients corresponds to each of the determined candidate deformations. In other words, each loss minimization performed at a given iteration in the calculation determines, among the possible sets of coefficients, a set of linear combination coefficients such that a linear combination with these coefficients of the deformation vector output by the deformation basis functions applied to the previously calculated intermediate 3D modeled object minimizes the loss. This linear combination forms the selected next intermediate 3D modeled object. This is an efficient way of performing the calculation iteratively.

[0119] The determination of the candidate transformations that minimize the loss may be performed under the constraint that the norm (e.g., L1 norm or L2 norm) of each candidate set of coefficients is smaller than a predefined value. That is, in each minimization of the loss, the minimization is performed under the constraint that the norm of the set of coefficients determined must be smaller than a predefined value. The predefined value may be a trade-off parameter between the convergence speed and the validity of the results. Thereby, the coefficients of the selected linear combination may remain sufficiently small in the following way: Discretization results in plausible nonlinear deformations of the first 3D modeled object. The minimization is forced to select only those deformation vectors that, when linearly combined, bring the intermediate 3D modeled object closer to the 3D modeled object that tends to minimize the loss the most. This allows the optimization to focus only on the most meaningful deformations, instead of globally deforming the entire intermediate 3D modeled object (which may result in the optimized model reaching too far from the initial 3D modeled object).

[0120] The norm may be, for example, the L1 norm, which results in a sparse selected set of linear combination coefficients at each iteration. Each minimization may be performed by any known method, for example any known constrained optimization method or any known unconstrained optimization method, for example by turning the constraints into additional penalty terms used in such unconstrained methods.

[0121] In the example, the determination of the second 3D modeled object may be performed by iteratively determining a trajectory (also called a "path") X(t) for a sequence

number

number

number

number

number

number

[0122] γ is a predefined value, here a trade-off parameter between convergence speed and validity of the results (e.g., in terms of deviation from the 3D shape manifold that constitutes the first 3D modeled object). For example, γ may be smaller than 1 / 100 of the shape diagonal if the deformation basis is regularized.

[0123] An embodiment of the determination method will now be described.

[0124] The present embodiment relates to the field of 3D modeling and 3D editing. The present embodiment may be useful in the fields of virtual and augmented reality (more generally, any kind of immersive experience), video games, manufacturing and 3D printing, modeling, etc. The present embodiment provides an improved solution for inferring optimal non-linear deformation of a 3D modeled object with respect to user constraints, utilizing linear deformation basis functions (also called linear deformation fields). Such deformation basis functions may be obtained in any manner, for example, using geometric analysis or using machine learning techniques. For example, the deformation basis functions may be a neural network that can be trained according to a learning method, for example, a neural network trained by a learning method (for example, according to the previously described embodiment of the learning method). Given this deformation basis function, the present embodiment deforms the input 3D modeled object (i.e., a first 3D modeled object) in a large and plausible way in order to arrive at a second 3D modeled object (e.g., a 3D sketch) that meets the user constraints. The 3D modeled object in this embodiment is referred to as a "3D mesh" for purposes of explanation, but may also be a 3D point cloud, and the embodiment may be similarly implemented using a 3D point cloud instead of a 3D mesh.

[0125] In this embodiment, the local deformation basis is considered as a tangent space on a manifold of plausible 3D modeled objects. On this manifold, we find an optimal 3D shape that meets the requirements defined by the user's constraints. Thus, we search for an optimal velocity vector, i.e., the directional derivative of an optimal path on the manifold from the original mesh to an optimized edited mesh that meets the user's constraints. This velocity can be "integrated" to obtain a corresponding path. To do so, we iteratively calculate this path. At each time step, we take a step along that vector as a linear combination of the vectors of the deformation basis at the current position on the manifold to find the deformation and thereby the path closer to the goal.

[0126] This embodiment can leverage handcrafted deformation foundations. This embodiment generates large plausible deformations. This embodiment optimizes small linear deformations into large nonlinear deformations.

[0127]

number

number

number

number

number

[0128] These functions are linearly combined to infer a locally plausible deformation of the input 3D modeled object X. In other words,

number

number

[0129] As mentioned before, the deformation vector V i The deformation basis of may be any combination of the following: A vector of linear transformation bases. The vector of linear transformation bases is output by a neural network that can be trained according to a training method, for example, a neural network trained according to a training method. In such a case,

number

[0130] Geometric and / or physical spectral modes. These are eigenvectors computed by spectral decomposition of quantities such as the Laplacian of the input geometry (such as those considered in the previously cited reference

[12] ) or the stiffness matrix (such as those considered in the previously cited reference [3]), finally with additional forces (e.g. damping forces) added.

[0131] Principal Components PCA. Principal components PCA is applied to a dataset of input shapes that share correspondences, resulting in uncorrelated deformation vectors (such as those considered in the previously cited reference [7]). Geometric modes of handcraft, such as those discussed in the previously cited reference [3]:

number

[0132] The loss is an arbitrary 3D loss L

number

number

[0133] where X is the input 3D modeled object of the loss, d is an arbitrary distance between the 3D modeled objects (such as chamfer distance or earth movement distance), and ρ is an additional regularization term (optional, λ is zero) to enforce some constraints. For example, ρ(X) can enforce symmetry as follows:

number

[0134] Linear local deformation vector

number

number

[0135] Generalizing to B deformation vectors, in this embodiment, trajectories may be computed from a set of plausible trajectories of the following types:

number

[0136] Figure 3 shows a plausible shape manifold 30, a tangent space with bases at three different locations 31, 32, 33, and a directional derivative of a path 39 from an original mesh / first 3D modeled object 34 to an optimized mesh / second 3D modeled object 35, non-linearly deforming the original mesh 34. For comparison, 36 shows a mesh resulting from a linear trajectory method for deforming the original mesh 34. Figure 3 further shows deformation constraints 37.

[0137] In this embodiment,

number

number

number

number

number

[0138] In this embodiment, to determine the second 3D modeled object, a trajectory discretization algorithm is executed to iteratively discretize the trajectory X(t).

number

number

[0139] At each time step n, the algorithm computes the amplitude

number

number

number

number

[0140] γ is a predefined value, here a trade-off parameter between convergence speed and validity of the results (e.g., in terms of deviation from the 3D shape manifold that constitutes the first 3D modeled object). For example, γ may be smaller than 1 / 100 of the shape diagonal if the deformation basis is regularized.

[0141] The gradient of this objective can be easily derived as follows:

number

[0142] Optimization

number

[0143] FIG. 4 shows an example of the results obtained using this embodiment. In this embodiment, the user selects a part of a first 3D modeled object / 3D mesh 40 by drawing a sketch 42 and defines a deformation constraint by drawing another sketch 44: the deformation constraint of the first 3D modeled object / 3D mesh 40 is defined as follows: In this example, the loss is the sum of the squared distances between the mesh vertices closest to the sketch 42 and their corresponding points in the sketch 44. The corresponding points are calculated as the curvilinear abscissa of the sketch closest to the curvilinear abscissa of the mesh vertices of the sketch 42. By minimizing this loss, the part of the mesh selected by the sketch 42 is effectively mapped to the other sketch 44. For comparison, FIG. 4 shows a second 3D modeled object / 3D mesh 46 determined by the method and another 3D modeled object 48 representing the deformation of the first 3D modeled object 40 obtained by linear regression.

[0144] The determination method may be used in a number of applications. By "application" we mean a computer-implemented method that includes the steps of the determination method, which may include further steps.

[0145] A first example of application of the decision method is 3D editing, which may for example comprise (or consist of the repetition of) the following steps: · Providing a first 3D modeled object according to the determining method. · Provide the deformation basis functions according to a determination method (which may be the same for all iterations). Providing a deformation constraint for the provided first 3D modeled object according to the determining method. determining a second 3D modeled object which respects the deformation constraints according to the determination method, the determined second 3D modeled object forming for example a next first 3D modeled object, i.e. the first 3D modeled object of the next iteration;

[0146] A second example of the application of the learning method is shape matching. Shape matching involves providing two 3D meshes e1 and e2. The mesh e1 forms a provided first 3D modeled object. The other mesh e2 forms a provided deformation constraint. The two meshes may be close to each other or far from each other. Shape matching then involves determining a second 3D modeled object according to a determination method. The second 3D modeled object is a 3D mesh that is a deformation of the first 3D mesh e1, which matches or tends to match the second 3D mesh e2.

Claims

1. 1. A computer-implemented method for determining a deformation of a 3D modeled object, comprising: providing - Deformation basis functions constructed to infer a deformation basis for an input 3D modeled object; a first 3D modeled object, and - transformation constraints of the first 3D modeled object determining a second 3D modeled object, the step including a step of calculating a trajectory of a transient 3D modeled object between the first 3D modeled object and the second 3D modeled object, the trajectory being determined by deforming each transient 3D modeled object by a linear combination of results of applying a deformation basis function to the transient 3D modeled object, the trajectory being determined by a reducing loss that penalizes the degree of non-compliance of a deformation constraint by the deformed transient 3D modeled object; method.

2. The trajectory corresponds to the integral equation [0010] The method of claim 1.

3. The computing includes discretizing a trajectory and reducing losses by computing a sequence of intermediate 3D modeled objects that converge towards the second 3D modeled object. The method according to claim 1 or 2.

4. said sequence of calculations including, starting from a first 3D modeled object, the following iterations: Applying the deformation basis functions to the intermediate 3D modeled object calculated in the previous iteration and applying the applied yield vector; determining, among the possible deformations of the intermediate 3D modeled object calculated in said iterative process, the one that minimizes the loss by a respective linear combination of the respective obtained vectors; Selecting the determined transformation candidate that has been applied to the current intermediate 3D modeled object as the next intermediate modeled object. The method according to claim 3.

5. The determination of the candidate transformations that minimize the loss includes searching a candidate set of coefficients of each linear combination and selecting from the searched candidate sets, the selected candidate sets of coefficients corresponding to the determined candidate transformations. The method according to claim 4.

6. The candidate transformations that minimize the loss are determined under the constraint that the norm of each candidate set of coefficients is less than a predetermined value. The method according to claim 5.

7. The norm is an L1 norm or an L2 norm. The method according to claim 6.

8. Providing deformation constraints for the first 3D modeled object includes a user defining the deformation constraints.

8. The method according to any one of claims 1 to 7.

9. A computer program comprising instructions which, when executed on a computer, cause the computer to carry out a method according to any one of claims 1 to 8.

10. 10. A computer readable data storage medium having recorded thereon the computer program of claim 9.

11. 10. A computer comprising a processor coupled to a memory, the memory having the computer program of claim 9 recorded thereon.

Images