Forming a data set for inferring solid CAD features

Abstract

The present invention relates in particular to a computer-implemented method for forming a data set configured for learning a neural network. The neural network is configured to infer solid CAD features representing 3D shapes from hand drawings representing the 3D shapes. The method comprises: generating one or more solid CAD features, the solid CAD features comprising each solid CAD feature representing a respective 3D shape. The method further comprises: for each solid CAD feature, determining one or more corresponding hand drawings respectively representing the corresponding 3D shapes; and inserting one or more training samples into the data set. Each training sample comprises the solid CAD feature and the corresponding hand drawing. The method forms an improved solution for inferring 3D modeled objects representing 3D shapes from hand drawings representing the 3D shapes.

Description

Technical Field

[0001] The present disclosure relates to the field of computer programs and systems, and more particularly, to solutions related to learning a neural network configured to infer a 3D modeled object representing a 3D shape from a hand drawing representing the 3D shape. Background Art

[0002] A large number of systems and programs for the design, engineering and manufacturing of objects are available on the market. CAD is an abbreviation for computer-aided design, for example, and it refers to software solutions for designing objects. CAE is an abbreviation for computer-aided engineering, for example, and it refers to software solutions for simulating the physical behavior of future products. CAM is an abbreviation for computer-aided manufacturing, for example, and it refers to software solutions for defining manufacturing processes and operations. In these computer-aided design systems, graphical user interfaces play an important role in the efficiency of the technology. These technologies can be embedded in product lifecycle management (PLM) systems. PLM refers to a business strategy that helps companies share product data, apply common processes, and exploit corporate knowledge to develop products from concept to end of life throughout the concept of an extended enterprise. The PLM solution provided by Dassault Systèmes (trademarks CATIA, ENOVIA and DELMIA) provides an engineering center (which organizes product engineering knowledge), a manufacturing center (which manages manufacturing engineering knowledge), and an enterprise center (which allows the integration and connection of enterprises into engineering and manufacturing centers). Together, the system delivers an open object model that links products, processes, resources together to enable dynamic, knowledge-based product creation and decision support that drives optimized product definition, manufacturing preparation, production, and service.

[0003] In this context, researchers have been looking for solutions to infer 3D modeled objects representing 3D shapes from hand drawings representing 3D shapes. It is worth noting that the following references are related to this problem:

[0004] -[1] T. Igarashi, S. Matsuoka and H. Tanaka, “Teddy: a sketching interface for 3D freeform design”, in Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, 1999, pp. 409-416.

[0005] -[2] Z. Lun, M. Gadelha, E. Kalogerakis, S. Maji and R. Wang, “3D shape reconstruction from sketches via multi-view convolutional networks”, in 2017 International Conference on 3D Vision (3DV), 2017, pp. 67-77.

[0006] -[3] J. Delanoy, M. Aubry, P. Isola, A. Efros, and A. Bousseau, “3D Sketching using Multi-View Deep Volumetric Prediction”, Proceedings of the ACM on Computer Graphics and Interactive Techniques, vol. 1, no. 21, May 2018.

[0007] -[4] B. Xu, W. Chang, A. Sheffer, A. Bousseau, J. McCrae, and K. Singh, “True2Form: 3D curve networks from 2D sketches via selective regularization”, ACM Transactions on Graphics, vol. 33, no. 4, 2014.

[0008] -[5] C. Li, H. Pan, Y. Liu, X. Tong, A. Sheffer, and W. Wang, “BendSketch: modeling freeform surfaces through 2D sketching,” ACM Transactions on Graphics, vol. 36, no. 4, pp. 1-14, July 2017.

[0009] -[6] Y. Zheng, H. Liu, J. Dorsey and N. J. Mitra, “Smart Canvas: Context-inferred Interpretation of Sketches for Preparatory Design Studies”, Computer Graphics Forum, 2016, vol. 35, pp. 37-48.

[0010] -[7] Y. Li, X. Luo, Y. Zheng, P. Xu, and H. Fu, “SweepCanvas: Sketch-based 3D Prototyping on an RGB-D Image,” in Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, 2017, pp. 387–399.

[0011] Papers [1] to [3] provide methods for generating 3D meshes from sketches.

[0012] The main disadvantage of meshes is that they are barely editable and provide no semantics. A 3D mesh discretizes a surface into a collection of primitive polygons (usually triangles). If the surface is watertight and manifold, it can represent a solid. This representation is not suitable for 3D modeling, as triangles should be edited one by one.

[0013] In addition, the method in [1] generates circular meshes. In addition, the method in [1] requires the sketch strokes to be annotated with associated actions (create, stretch, bend).

[0014] Papers [2] and [3] avoid this circle-like prior and avoid the need for annotated strokes due to recent advances in deep learning. However, both methods still output meshes. In addition, paper [2] requires one or more sketches of the object (i.e., front, side, top) from a canonical perspective, which is a significant limitation for users.

[0015] Papers [4] to [7] provide methods for generating parameterized surfaces from sketches.

[0016] The paper [4] generates curves by imposing some regularity (orthogonality, parallelism, coplanarity) on the sketch strokes. Then, the surface can be fitted to the curve. This approach requires the user to be very accurate when drawing the sketch: the strokes must represent the geometry of the object (following its surface, but not necessarily its outline), and closed strokes may be required. In addition, the user must specify the actual 3D intersections between the strokes.

[0017] The paper [5] fits surfaces directly to sketches. However, the sketch strokes must be labeled. The labels provide some information about the surface on which the sketch strokes should lie: the limits of the surface, concavity / convexity, sharp edges. Due to the nature of these labels, the user must be an expert in geometry, as a simple perspective view of the object will not be sufficient.

[0018] Therefore, the solutions provided in papers [4] and [5] fit parametric surfaces to the input drawing, but the generated surfaces are not necessarily closed and, therefore, they may not represent solid bodies. Even if they do represent solid bodies, they cannot be edited into meaningful solid CAD features. Furthermore, both approaches require additional user input (intersection points or stroke markers).

[0019] The paper [6] generates 3D curves from sketch strokes. The method assumes that each stroke represents a planar 3D curve. Once the plane is determined, the 3D curve can be obtained by back-projecting the strokes from the 2D sketch onto the plane where the strokes should be located. Strokes located on the same plane can intersect each other and define the boundaries of a planar patch. The obtained planar patch can then be stretched in the perpendicular direction. The main disadvantage of this method is that the plane directions are limited to three canonical directions of an orthogonal reference system. In order to define this reference system, the user is required to explicitly display two sets of parallel lines, the first set of lines being orthogonal to the second set of lines. The reference system is then inferred from the vanishing point.

[0020] A very similar approach is proposed in [7]. However, in addition to the classic RGB image, the proposed method requires a depth image. The brush stroke plane is estimated directly from the 3D representation of the scene provided by the depth map.

[0021] Therefore, papers [6] and [7] provide techniques for producing results that include surfaces that may be stretched. However, both methods assume that the underlying 2D sketch lies on a set of constrained planes. In addition, both methods require additional input: images, user definitions of parallel lines, and depth images.

[0022] In this context, there remains a need for improved solutions for inferring a 3D modeled object representing a 3D shape from a hand-drawn drawing representing the 3D shape. Summary of the invention

[0023] Thus, a computer-implemented method for forming a data set configured for learning a neural network is provided. The neural network is configured to infer solid CAD features representing a 3D shape from a hand-drawn drawing representing the 3D shape. The method includes: generating one or more solid CAD features, the solid CAD features including each solid CAD feature representing a respective 3D shape. The method also includes: for each solid CAD feature, determining one or more corresponding hand-drawn drawings respectively representing the corresponding 3D shape; and inserting one or more training samples into the data set. Each training sample includes a solid CAD feature and a corresponding hand-drawn drawing.

[0024] The method may include one or more of the following:

[0025] - said determining each hand-drawn image comprises: providing a viewpoint from which a corresponding 3D shape is visible; and rendering one or more edges in an image relative to a solid representation of the corresponding 3D shape based on the solid CAD feature, each edge being a contour edge or a visible edge of the solid representation from the viewpoint;

[0026] - Each visible edge is the entity represented by C k discontinuity, k is an integer greater than or equal to 1;

[0027] - said determining each hand-drawn image further comprises: perturbing at least a portion of one or more rendered edges;

[0028] - the rendering of each edge is performed in a vector manner, each edge is represented by one or more corresponding parameterized curves, and the perturbation includes, for at least one corresponding parameterized curve, adding noise to one or more parameters of the at least one corresponding parameterized curve;

[0029] - Each solid CAD feature includes a corresponding curve sequence, and the generating includes: providing a curve set and sampling the curve set;

[0030] - the curve set comprises a set product between: a discrete set of curve types and, for each curve type, a corresponding discrete set of one or more parameter domains for each corresponding parameter, each parameter domain having a corresponding parameter value for the corresponding parameter, the sampling comprising providing samples, each sample comprising the corresponding curve type and the corresponding parameter value for each parameter domain in the corresponding discrete set of one or more parameter domains;

[0031] - the discrete curve type set includes: line type, arc type, and / or cubic Hermite interpolation segment type;

[0032] - each solid CAD feature comprises a scan operation, said generating comprising: providing a set of scan operations and sampling said set of scan operations; and / or

[0033] - the set of scanning operations comprises a set product between: a discrete set of scanning curve types, and for each scanning curve type, a corresponding discrete set of one or more parameter domains for each corresponding parameter, each parameter domain having a corresponding parameter value for the corresponding parameter, and the sampling comprises providing samples, each sample comprising the corresponding scanning curve type and the corresponding parameter value for each parameter domain in the corresponding discrete set of the one or more parameter domains.

[0034] A data structure representing a data set that can be formed according to the data set forming method is also provided.

[0035] A computer-implemented method of using a data set is also provided, the method comprising learning a neural network based on the data set.

[0036] A computer program is also provided, comprising instructions for executing the data set forming method and / or the using method.

[0037] A device is also provided, the device comprising a data storage medium having the data structure and / or the program recorded thereon. The device can form or be used as a non-transitory computer-readable medium, for example on a SaaS (Software as a Service) or other server, or on a cloud-based platform, etc. The device may alternatively include a processor coupled to the data storage medium. Thus, the device may form a computer system in whole or in part (for example, the device is a subsystem of the entire system). The system may also include a graphical user interface coupled to the processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Embodiments will now be described by way of non-limiting examples and with reference to the accompanying drawings, in which:

[0039] - Figure 1 An example of a graphical user interface for the system is shown;

[0040] - Figure 2 An example of a system is shown; and

[0041] - Figure 3-Figure 12 The method is shown. DETAILED DESCRIPTION

[0042] Therefore, a computer-implemented method is proposed in connection with learning of a neural network. In this context, a neural network is a function configured to take as input any one of a plurality of hand-drawn images, each representing a corresponding 3D shape, process the input, and output, as a result of the processing, corresponding data, the data being configured to infer a solid CAD feature representing the 3D shape corresponding to the input. Therefore, such a neural network can be used to infer a solid CAD feature corresponding to a hand-drawn image representing a 3D shape.

[0043] Therefore, a computer-implemented method for performing such speculation is proposed, which method may be referred to as a "speculation method".

[0044] The inference method may include providing a hand-drawn drawing representing the 3D shape. The inference method may also include applying a neural network to the hand-drawn drawing. The inference method may also include inferring (e.g., substantially in real time when applied) at least one solid CAD feature representing the 3D shape based on a result of the application (i.e., data output by the neural network when input with the hand-drawn drawing). The output data may (directly) include the solid CAD feature, and / or be processed to (indirectly) determine the solid CAD feature, e.g., automatically and / or according to a predetermined algorithm.

[0045] In the case of indirect determination, the output data may, for example, include one or more probability distributions for selecting values ​​defining at least one solid CAD feature, and the predetermined algorithm may, for example, include determining at least one solid CAD feature based on the one or more probability distributions. The determination may, for example, disadvantage the selection of values ​​with low probability, for example, by selecting one or more most likely values ​​based on the one or more probability distributions. The determination may then derive one or more elements (e.g., geometries) of the solid CAD feature, each element corresponding to a respective one or more selected values.

[0046] In an example, determining may also include modifying one or more of the elements. For example, the inferred solid CAD feature may include a sequence of corresponding geometric shapes (e.g., curves). In such an example, the modification may include modifying the geometry of one or more geometric shapes (e.g., curves) in the corresponding sequence. In a specific example, the inference may be limited to closed (e.g., curves forming a loop) corresponding sequences of geometric shapes (e.g., curves). In such a specific example, modifying the geometry of one or more geometric shapes (e.g., curves) in the sequence may include closing the corresponding sequence.

[0047] As is known from the field of machine learning, processing of an input by a neural network comprises applying an operation to the input, the operation being defined by data including weight values. Thus, learning of the neural network comprises determining the values ​​of the weights based on a data set configured for such learning. To this end, the data set comprises data segments, each data segment forming a respective training sample. The training samples represent the diversity of situations in which the neural network is to be used after learning. Any data set cited herein may comprise a number of training samples higher than 1,000, 10,000, 100,000 or 1,000,000.

[0048] Therefore, in particular, a computer-implemented method for forming such a data set is proposed, which method may be referred to as a “data forming method”. Also proposed is a computer-implemented method for performing such learning, which method may be referred to as a “learning method”.

[0049] The machine learning process may include a dataset formation method and / or a learning method, and optionally an inference method.

[0050] The machine learning process may in particular include both a data set forming method and a learning method. The learning method may, for example, be performed at least in part based on a data set formed by the data set forming method, for example after the data set forming method. Such a machine learning process is particularly effective and provides improved accuracy.

[0051] Alternatively, the machine learning process may include a data set formation method, and performing any other computer-implemented method for learning a neural network based on the data set (other than the proposed learning method). Alternatively, the machine learning process may include performing a learning method on a data set provided by any other computer-implemented method (other than the proposed data set formation method), such as another method for forming a data set or retrieving such a data set.

[0052] Before presenting the data set formation method and the learning method, the data structures involved therein are now discussed. It will be appreciated that the data structure definitions and examples provided herein can be applied to at least a portion (e.g., all) of a data set formed by the data set formation method and / or at least a portion (e.g., all) of a data set provided in a learning method, and / or applied to at least one input and / or at least one output of an inference method.

[0053] A solid CAD feature is a 3D modeled object that represents the 3D shape of a solid. Any 3D shape herein represents the 3D spatial distribution of the material that makes up the solid. A 3D modeled object may represent the skin (i.e., the outer surface) of a solid. A 3D modeled object may be provided to a CAD system for display to a user and / or for the user to perform one or more CAD design operations on the 3D modeled object.

[0054] A modeling object is any object defined by stored (e.g., in a database) data. By extension, the expression "modeling object" indicates the data itself. Depending on the type of system used to design the modeling object, the modeling object may be defined by different kinds of data. The system may indeed be any combination of a CAD system, a CAE system, a CAM system, a PDM system, and / or a PLM system. In those different systems, the modeling object is defined by corresponding data. Thus, one may mention CAD objects, PLM objects, PDM objects, CAE objects, CAM objects, CAD data, PLM data, PDM data, CAM data, CAE data. However, these systems are not mutually exclusive, since a modeling object may be defined by data corresponding to any combination of these systems. Thus, a system may very well be both a CAD system and a PLM system.

[0055] By CAD system, it is also meant any system, such as CATIA, which is at least suitable for designing a modeled object based on a graphical representation of the modeled object. In this case, the data defining the modeled object includes data allowing the modeled object to be represented. The CAD system can, for example, use edges or lines (in some cases using faces or surfaces) to provide a representation of the CAD modeled object. Lines, edges or surfaces can be represented in various ways, such as non-uniform rational B-splines (NURBS). Specifically, a CAD file contains specifications from which geometric figures can be generated, thereby allowing a representation to be generated. The specifications of the modeled object can be stored in a single CAD file or in multiple CAD files. The typical size of a file representing a modeled object in a CAD system is in the range of 1 megabyte per component. A modeled object can typically be a combination of thousands of components.

[0056] In the context of CAD, a modeled object may typically be a 3D modeled object, for example, representing a product such as a part or a combination of parts, or possibly a combination of products. "3D modeled object" means any object modeled by data that allows its 3D representation. The 3D representation allows the part to be viewed from various angles. For example, when represented in 3D, a 3D modeled object can be manipulated and rotated about any of its axes or about any axis in the screen on which the representation is displayed. In particular, this excludes 2D icons that are not modeled in 3D. The display of a 3D representation facilitates design (i.e., increases the speed at which a designer can statistically complete a task). Since the design of a product is part of the manufacturing process, this can speed up the industrial manufacturing process.

[0057] Any 3D shape herein may represent the geometry of a product to be manufactured in the real world, such as a (e.g., mechanical) component or assembly of components (or equivalently, a combination of components, since from the method's perspective a combination of components may be considered a component in itself, or the method may be applied independently to each component in the assembly), or more generally any rigid assembly (e.g., a movable machine). The product may be one of a variety of products in a wide and unlimited number of industrial sectors, including: aerospace, architecture, construction, consumer products, high-tech equipment, industrial equipment, transportation, marine, and / or offshore oil / gas production or transportation. Thus, the 3D shape can represent an industrial product, which can be any mechanical part, such as parts of land vehicles (including, for example, automobile and light truck equipment, racing cars, motorcycles, vans and motor equipment, trucks and buses, trains), parts of aircraft (including fuselage equipment, aerospace equipment, propulsion equipment, defense products, aviation equipment, aerospace equipment), parts of marine transportation vehicles (including naval equipment, commercial vessels, offshore equipment, yachts and work vessels, ship equipment), general mechanical parts (including, for example, industrial manufacturing machinery, heavy mobile machinery or equipment, installed equipment, industrial equipment products, metal products, tire manufacturing products), electromechanical or electronic components (including, for example, consumer electronics products, safety and / or control and / or instrumentation products, computing and communications equipment, semiconductor products, medical devices and equipment), consumer products (including, for example, furniture, home and garden products, leisure goods, fashion products, products of hard goods retailers, products of soft goods retailers), packaging (including, for example, food, beverage and tobacco, beauty and personal care, household product packaging).

[0058] As is known from the CAD field, a solid CAD feature representing a 3D shape is herein a list of one or more instructions that allow a CAD system to generate a 3D geometry of a solid, and, for example, render and display a 3D graphical representation of the solid. The 3D graphical representation may include a 3D subdivision of the outer surface of the solid, such as a mesh subdivision (e.g., polygons, optionally triangles). In an example, the mesh subdivision may include a number of faces (e.g., polygons, optionally triangles) greater than 100 (e.g., greater than 1000).

[0059] As is known from the CAD field, solid CAD features can be selected and / or edited by a user of the CAD system. The list of one or more instructions can relate to one or more elements, each of which is selectable and / or modifiable. The user can perform editing of the solid CAD feature in any manner known from the CAD field, for example including selecting at least one element thereof and / or deleting or modifying the element and / or adding at least one new element. Such editing presents ergonomic features compared to the editing of discrete geometric representations such as meshes.

[0060] Specifically, the instruction list may represent one or more constraints involving one or more parameterized geometric shapes to create a solid. Each parameterized geometric shape includes one or more corresponding parameters, for example, including a type parameter and / or one or more shape parameters. One or more parameters may be optional and / or editable (for example, including at least one (for example, each) shape parameter). Editing of a parameter may include modifying the value of a parameter. In such a case, the number of geometric shapes that may be involved in the inferred solid CAD feature is lower than the number of faces of its subdivided mesh by a factor of 5 or more (for example, more than 10), and / or the number of geometric shapes that may be involved is lower than 50 (for example, lower than 20). Each geometric shape may be defined by multiple parameters less than 20 (for example, less than 10). Therefore, compared with a mesh, a solid CAD feature represents a 3D shape in a more compact and organized manner. Moreover, as is known from the CAD field, acting on a single element of a solid CAD feature may have a global impact, while in the case of a mesh, acting on a single geometric entity will only cause a local impact. For example, as opposed to moving / adding a large number of mesh triangles, any solid CAD feature herein can be configured to enlarge the 3D shape it represents (e.g., curved, such as a cylinder) by modifying a parameter value of a quantity below 5 (e.g., below 3). Furthermore, solid CAD functionality is more semantically meaningful to the user than discrete geometric shape representations.

[0061] For any entity CAD feature in this article, the instruction list can represent one or more real-time / declarative constraints and / or one or more process constraints. For any entity CAD feature in this article, the geometric shape can include a geometric 3D shape, that is, a geometric shape defined in a 3D space. The 3D shape can, for example, include a 3D curve and / or any other 3D shape (e.g., a 3D point and / or a 3D entity). Any 3D curve in this article can be planar (i.e., defined in a 3D space, but contained in a plane of the 3D space) or non-planar (i.e., not contained in any plane). In an example, for any entity CAD feature in this article, the geometric shape can include one or more geometric shape sequences, such as one or more (e.g., closed) sequences of (e.g., planar) 3D curves (e.g., sequence curves). A sequence of planar 3D curves that results in a closed contour (must be in a plane containing each of these curves) is referred to as a "closed 2D plane sketch". In an example, for any solid CAD feature herein, the geometry may alternatively or additionally include one or more scanning operations, each scanning operation taking one or more geometric objects (e.g., a single closed 2D planar sketch) as input and / or based on another geometric object (e.g., a 3D curve). Any scanning operation herein may, for example, be defined by a 3D curve (e.g., a scanning curve), optionally non-planar.

[0062] Any 3D curve herein may belong to a category defined by one or more fields (referring to object-oriented programming), each field corresponding to a corresponding parameter defining the shape of the 3D curve in the context of the category. Any 3D curve herein may be defined in particular by one or more shape parameters, for example including one or more positioning parameters (e.g., 3D coordinates) for one or more boundary (e.g., start and / or end) points. Any 3D curve herein may be defined alternatively or additionally by a type parameter (e.g., among curve types including line type, arc type, and / or cubic Hermite interpolation fragment type). Depending on its type, any 3D curve herein may be defined alternatively or additionally by additional shape parameters, optionally including: one or more positioning parameters (e.g., 3D coordinates) for each of one or more intermediate points of the curve and / or one or more tangent vector parameters (e.g., 3D vector definitions) for each of one or more points of the curve (e.g., including the start and / or end points). This parameterization of 3D curves is particularly ergonomic for users to edit.

[0063] In terms of accuracy and robustness of learning and the diversity of situations represented by the intended applications for hand drawings, in a particularly effective example, any solid CAD feature in this context can be defined specifically (and only) by the following, relative to a given 3D space:

[0064] - a closed 2D planar sketch consisting of a series of planar curves (e.g., a sequence of curves),

[0065] This forms a closed contour.

[0066] - A sweep operation (eg, defined by a sweep curve) is then performed to generate a solid by sweeping a closed surface defined by the 2D planar sketch in 3D space, ie, the solid is defined by the volume traversed by the closed surface.

[0067] In such particularly effective examples, solid CAD features have meaningful parameters and can be easily edited. For example, the underlying 2D planar sketch can be modified and the resulting geometry can be automatically updated by applying a sweep operation to the new base. Editing can include selecting and editing parameters of individual 3D curves (e.g., 3D curves that make up a closed 2D planar sketch or 3D curves that define a sweep operation). Editing may be limited by planes and / or requirements to not modify order and / or maintain closure. This type of solid CAD feature is particularly ergonomic for the user to edit.

[0068] In an example, the inference method may include: displaying a 3D graphical representation of a 3D shape represented by a solid CAD feature (i.e., a 3D shape obtained by executing a list of instructions for the solid CAD feature), such as a 3D subdivision thereof. The inference method may further include displaying a graphical representation of the structure of the solid CAD feature, such as a tree arrangement of graphical items representing each corresponding geometric shape or geometric operation, or highlighting a 3D graphical representation of the geometric shapes involved in the solid CAD feature. The editing of the solid CAD feature may include, for example, selecting a graphical item through graphical interaction. Graphic interaction herein refers to operating a tactile device to activate pixels at a position corresponding to the position of a graphical item on the display. For example, a mouse cursor may be placed above the displayed graphical item, or the displayed graphical item may be touched directly via a touch screen or indirectly via a touchpad. The selection may trigger the opening of a dialog box, etc., and the user may enter one or more parameter values, for example, via a keyboard. In a particularly effective example of solid CAD features, graphical interaction may include drag-and-drop actions on graphical representations of points or vectors linked to a 3D curve to move the position of the point and / or modify the direction and / or magnitude of the vector, such as when the 3D curve belongs to a closed 2D planar sketch constrained in the plane. This type of ergonomic editing is well known in the CAD field.

[0069] In an example, as a substitute or supplement to the solid CAD features inferred by the user editing, the inference method may include: for example, via user interaction, combining the solid CAD function to one or more other solid CAD features. For example, the inference method may include: adding the solid CAD feature to an existing feature tree including one or more other solid CAD features, for example, by connecting the solid CAD feature to any node of the feature tree. Optionally, the inference method may also include editing one or more solid CAD features of the feature tree. In a specific example, the inference method may be repeated, and the inference method includes: providing several hand drawings, and inferring several solid CAD features respectively by applying a neural network to each hand drawing. In such a specific example, the inference method may then include combining the inferred solid CAD features in a single feature tree. These specific examples allow the creation of feature trees based on hand drawings in accordance with ergonomic principles, thereby performing complex CAD designs. Any feature tree herein may be a constructed solid geometry (CSG) tree.

[0070] CAD systems can be history-based. In this case, modeled objects can be further defined by data including the history of geometric features. Modeled objects can indeed be designed by a natural person (i.e., a designer / user) using standard modeling features (e.g., extrusion, rotation, cutting and / or wrapping) and / or standard surface design features (e.g., sweeping, blending, lofting, filling, deformation and / or smoothing). Many CAD systems that support such modeling functions are history-based systems. This means that the creation history of design features is usually saved through a non-cyclic data flow that links the geometric features together through input and output links. The history-based modeling paradigm has been well known since the early 1980s. Modeled objects can be described by two persistent data representations: history and B-rep (i.e., boundary representation). B-rep is the result of the calculations defined in the history. When representing a modeled object, the shape of the component displayed on the computer screen is B-rep (e.g., a subdivision of B-rep). The history of a component is the design intent. Basically, history collects information about the operations that the modeled object has undergone. B-rep can be saved with history to make it easier to display complex components. The history can be saved with the B-rep to allow design changes to be made to the part according to the design intent. Due to the hand-drawing method, the speculative method of the specific example allows the history of such geometric features to be retrieved via the feature tree in an ergonomic manner.

[0071] Mechanical simulation can be performed based on solid CAD features and / or feature trees. Indeed, since solid CAD features and / or feature trees form a compact and parameterized representation of the 3D shape in a compact form (relative to a discrete geometric representation), mechanical calculations can be performed more accurately. Specifically, it is easier and more direct to calculate finite element meshes from CSG (as taught by Edwin Boender, Willem F.Bronsvoort, Frits H.Post, "Finite-elementmesh generation from constructive-solid-geometry models", Computer-Aided Design, Vol. 26, No. 5, May 1994, pp. 379-392, or K Ho-Le, "Finite element mesh generation methods: a review and classification", Computer-Aided Design 38, Vol. 20, No. 1, January / February 1988). In addition, CSG also ensures the waterproofness of the surface. Thus, a mechanical simulation may include computing one or more finite element meshes from a solid CAD feature and / or a feature tree (e.g., CSG) and / or one or more modifications thereof, and then performing a finite element analysis based on each finite element mesh. Furthermore, since the solid CAD features are parameterized, the solid CAD features may be easily optimized based on the simulation results. Thus, one or more modifications may be used to optimize the solid CAD features based on each previous finite element analysis result.

[0072] Any hand drawing representing a 3D shape herein is a 2D image that a (human) user can design by freehand for representing the 3D shape. "Freehand designable" means that the drawing represents the characteristics of a 2D image drawn by a user. Thus, the hand drawing may have been created by the user, but alternatively, it may have been synthesized, i.e., synthesized manually and automatically by a computer system. Specifically, the hand drawing in the data set formation method can be synthesized. On the other hand, it is desired that the hand drawing in the inference method is effectively drawn by freehand by the user, although a synthesized hand drawing can also be provided to the inference method.

[0073] Thus, a hand drawing is a 2D image data structure that includes one or more (2D plane) strokes that represent a 3D shape from the corresponding viewing angles from which the 3D shape is visible. One or more strokes are defined on the same plane and represent perspective views of the 3D shape from various viewing angles. Each stroke is a continuous or substantially continuous curve defined in a plane.

[0074] In an example, for any hand-drawn image herein, the one or more strokes may include a number of strokes less than 20 (e.g., less than 10) strokes, optionally greater than or equal to 2. In an example, for any hand-drawn image herein, the one or more strokes may each represent a corresponding feature edge of a 3D shape as viewed from a viewpoint, such as a contour edge or a visible edge of an entity represented by the 3D shape. "Visible edge" refers to an edge of an entity that is noticeable to a user through the effect of light reflection. The visible edge of any entity herein may include (e.g., only include) one or more edges, each of which forms a C of the entity representation. k Discontinuity, k is an integer greater than or equal to 1 (eg, and optionally less than 10 or 4).

[0075] In an example, for any hand-drawn drawing herein, one or more (e.g., all) pen strokes may each have an end point at a certain distance from another pen stroke (e.g., from an end point of another pen stroke), the distance being, for example, below a predetermined threshold, such as 10 mm or 5 mm. This represents the fact that when drawing a hand-drawn drawing, the user may intend to connect the end point to the other pen stroke (e.g., the end point of the other pen stroke) in such a way as to represent 3D in 2D. In an example, for any hand-drawn drawing herein (e.g., one or more hand-drawn drawings and / or one or more hand-drawn drawings of a data set input to the inference method), one or more (e.g., all) pen strokes may each have an end point at a non-zero distance (i.e., a positive gap) from another pen stroke (e.g., from an end point of another pen stroke). This represents the fact that when drawing a hand-drawn drawing, the user may often fail to perfectly connect the end point to the other pen stroke (e.g., the end point of the other pen stroke), thereby causing a gap. This is because the user is sketching casually, and therefore quickly / ergonomically. Despite this inaccuracy, the inference method can still accurately infer solid CAD features. And the dataset can represent this imprecision to make the learning accurate.

[0076] In the example, for any hand-drawn drawing herein, the strokes freely represent 3D shapes. This means that they can be defined in any order, and there are no predefined constraints (e.g., must have parallel and / or orthogonal strokes), and they are not organized into clusters (e.g., based on distance retrieval), wherein each cluster represents a plane of an entity. In the example, for any hand-drawn drawing herein, alternatively or additionally, one or more strokes can each be irregular. This means that, without approximating the strokes and thus introducing errors, the strokes cannot be represented by parameterized 2D curves (at least one of any type described herein). Since the strokes are expected to be hastily drawn by the user, they generally do not form a standardized geometric figure. In the example, irregular strokes are allowed to be particularly / obviously irregular. In the example, any hand-drawn drawing herein can be provided alternatively or additionally in an original format, for example, without any tags associated with the strokes (e.g., action annotations, such as creation, stretching, bending) and / or without any additional data (e.g., pictures, user-defined parallel lines, depth images). The inference method is thus ergonomic because the user is free from any constraints and can draw strokes freely (eg, the inference method makes no assumptions on a 2D sketch plane) and no additional input is required beyond the freehand drawing.

[0077] According to speculation, each stroke is unparameterized, which means that the data structure representing the stroke includes data fragments, which represent the parts of the stroke (e.g., pixels) respectively, and the data structure represents the original aggregation of these parts. The speculation method can include drawing each stroke of the input hand drawing by the user. Sketching can be performed by graphical interaction with the screen of the computer, by sequentially (e.g., substantially continuously) activating a series of pixels of the screen that define the strokes. The activation can be performed via a tactile device (e.g., a mouse) acting on the pixels (e.g., a cursor movement that defines the strokes) or via user interaction with a touch screen or touch pad (e.g., the interaction includes drawing the strokes on the touch screen or touch pad by physical touch). This is called sketch design or stroke design in the CAD field. In an alternative, a scan of a paper sketch or any other image can be input to the speculation method. Alternatively, the hand drawing (e.g., already generated by other users and / or any of the above-mentioned technologies) can be retrieved from a memory or received from a remote computer system.

[0078] In an example, the inference method may include displaying a graphical representation of a hand-drawn drawing as the user draws the strokes (e.g., in real time). The inference method may trigger the application of the neural network. For example, when the user is finished, the user may trigger the application of the neural network and infer the solid CAD features. As a result, the method may display a 3D graphical representation of the solid CAD features, optionally while still displaying the graphical representation of the hand-drawn drawing. The user may then hide the graphical representation of the hand-drawn drawing to process (e.g., edit, integrate into a tree, simulate) the solid CAD features. The inference method may be performed while displaying a complex 3D modeled object designed by the user, optionally including repeating the inference method described above. Thus, the inference method provides sketch-based design capabilities to a history-based CAD system.

[0079] Thus, any method herein may be part of a process for designing a 3D modeled object comprising solid CAD features, in particular a speculative method. "Designing a 3D modeled object" specifies any action or series of actions that is at least part of a process of making a 3D modeled object. Thus, the method may comprise creating a 3D modeled object from scratch. Alternatively, the method may comprise providing a previously created 3D modeled object and then modifying the 3D modeled object.

[0080] Any of the methods herein may be included in a manufacturing process, which may include, after executing the method, producing a physical product corresponding to the modeled object. In any case, the modeled object designed by the method may represent a production object. Thus, the modeled object may be a modeled entity (i.e., a modeled object representing an entity). The production object may be a product, such as a component or a combination of components. Because the method improves the design of the modeled object, the method also improves the production of the product, thereby increasing the productivity of the manufacturing process.

[0081] Now we briefly introduce the dataset formation method and learning method.

[0082] The method for forming a data set includes generating (e.g., synthesizing) one or more solid CAD features, the one or more solid CAD features including each solid CAD feature representing a respective 3D shape. The method for forming a data set also includes, for each solid CAD feature, determining (e.g., synthesizing) one or more corresponding hand drawings, each hand drawing representing a corresponding 3D shape (e.g., based on the generated solid CAD feature), and inserting one or more training samples into the data set, each sample including a (e.g., synthesized) solid CAD feature and a corresponding (e.g., synthesized) hand drawing.

[0083] “Providing / generating / determining / synthesizing a corresponding data segment” means that, at least at some point in time, the data segment is recorded on a memory of the computer system and is accessible to a processor.

[0084] A "data set" refers to any data structure including training samples and configured for machine learning, which is recorded on a memory of a computer system and accessible to a processor. A method for forming a data set may include: recording the data set on a non-volatile memory.

[0085] "Inserting" or "not inserting" a data segment as a training sample in a data set means that at least at a certain moment (e.g., at the end) of the data set formation method, the data set is recorded on a (e.g., non-volatile) memory of a computer system and is in the following state: the data set includes all "inserted" data segments (i.e., the data segment is therefore recorded on the memory in a data structure representing the data set), while the data set excludes (i.e., does not include) all "uninserted" data segments (i.e., the data segment is therefore not recorded on the memory, at least not in a data structure representing the data set). Then, in an example after further processing the data set, learning can be based on the data set. Therefore, the learned neural network will be consistent with the data segments inserted into the data set.

[0086] The dataset formation method provides an improved solution for CAD design. The formed dataset includes training samples, each training sample includes a hand drawing of a corresponding 3D shape and a corresponding solid CAD feature representing the 3D shape. Therefore, the dataset is configured to learn a neural network that can be used to convert the hand drawing into a solid CAD feature, and the proposed learning method is adopted in the example. In addition, the dataset is labeled for expected learning. In fact, the potential input of the neural network (hand drawing) is respectively labeled in the dataset by the output of the ground truth (corresponding solid CAD feature). Therefore, learning can be supervised and is therefore particularly effective.

[0087] The learning method includes providing a data set including hand drawings, each hand drawing representing a corresponding 3D shape, and learning a neural network based on the data set.

[0088] The learning method provides an improved solution for CAD design. Notably, the learned neural network can be used to convert hand drawings into solid CAD features. As already discussed, this is particularly useful in the context of CAD. In addition, using a learned neural network to perform this conversion can achieve the benefits provided by the field of machine learning, such as fast or real-time execution, no need for cumbersome previous manual programming / writing code, and accuracy of the results (for example, accuracy here refers to the value of the 3D similarity evaluated between the result of executing the solid CAD feature and the 3D shape desired by the user when drawing the hand drawing, for example, optionally measured as the value of the 2D similarity evaluated between the 2D projection of the result of executing the solid CAD feature (e.g., feature edge) and the hand drawing). Therefore, the learning method improves ergonomics in the context of 3D design.

[0089] The learning may include supervised training based on at least a portion of the data set. In this case, the at least a portion of the data set includes, for each hand-drawn image, a corresponding solid CAD feature that is associated with the hand-drawn image. Figure 1 The at least a part of the data set may be provided according to any method, for example, formed according to a data set forming method.

[0090] Alternatively or additionally, learning may comprise unsupervised training, which is based on at least another part of the data set. In this case, the hand drawings of the at least another part of the data set respectively form corresponding (unlabeled) training samples. The at least another part of the data set may be provided according to any method, for example as a commercial database of hand drawings to retrieve.

[0091] In a particularly effective example, the learning method may include: supervised training based on a data set formed by a data set formation method. In a reliable example of such an example, learning may also include: unsupervised training based on another data set, such as after supervised training. It is known from the field of machine learning that each training may include iteratively processing the corresponding data set, such as processing in small batches one by one, and modifying the weight values ​​of the neural network along the iterative processing. This can be performed according to stochastic gradient descent. For each training, the weight values ​​can be initialized in any way. In the example, the weight values ​​obtained at the end of the supervised training can be used to initialize the weight values ​​of the unsupervised training. The weight values ​​can be initialized in any way for supervised training, such as randomly or respectively initialized to zero values. Therefore, the data set formation method and supervised training form a method for initializing the weight values ​​for further training, and the further training is unsupervised training, which can be based on another data set. Another data set can be, for example, an unlabeled data set, which is more representative of reality than the labeled data set formed by the data set formation method. Therefore, unsupervised training can bring more accurate results than stopping training after supervised training. However, as is known from the field of machine learning, unsupervised training can be very long and provide inaccurate results if initialized arbitrarily. In the proposed method, supervised learning provides a relatively good initialization.

[0092] These methods are computer implemented.

[0093] This means that the steps (or substantially all steps) of the method are performed by at least one computer or any similar system. Thus, the steps of the method may be performed fully automatically or semi-automatically by a computer. In an example, the triggering of at least some steps of the method may be performed by user-computer interaction. The required level of user-computer interaction may depend on the foreseen level of automation and be balanced with the need to implement the user's wishes. In an example, the level may be user-defined and / or predefined.

[0094] This also means that the neural network, the data set, one or more (e.g., all) inputs and / or one or more (e.g., all) outputs can be represented by corresponding data structures, respectively. One or more (e.g., all) data structures can be non-temporarily recorded on corresponding data storage media, respectively.

[0095] A typical example of a computer implementation of the method is to perform the method using a system suitable for this purpose. The system may include a processor coupled to a memory and a graphical user interface (GUI), the memory having recorded thereon a computer program including instructions for performing the method. The memory may also store a database. The memory is any hardware suitable for such storage, and may include several physically distinct parts (e.g., one for the program and possibly one for the database). The system may be a CAD system.

[0096] Figure 1 An example of a GUI for any system herein is shown, where the system is a CAD system.

[0097] GUI 2100 can be a typical CAD-like interface, with standard menu bars 2110, 2120 and bottom and side toolbars 2140, 2150. As known in the art, such menus and toolbars contain a user-selectable set of icons, each associated with one or more operations or functions. Some of these icons are associated with software tools, suitable for editing and / or operating the 3D modeling object 2000 displayed in GUI 2100. Software tools can be grouped into workbenches. Each workbench includes a subset of software tools. Specifically, one of the workbenches is an editing workbench, which is suitable for editing the geometric features of the modeling product 2000. In operation, the designer can, for example, pre-select a part of the object 2000, and then initiate an operation (e.g., change size, color, etc.) or edit geometric constraints by selecting an appropriate icon. For example, a typical CAD operation is to model the punching or folding of the 3D modeling object displayed on the screen. The GUI can, for example, display data 2500 related to the displayed product 2000. In the example of the figure, data 2500 displayed as a "feature tree" and its 3D representation 2000 relate to a brake assembly including a brake caliper and a disc. The GUI may also show various types of graphical tools 2130, 2070, 2080, such as for facilitating 3D orientation of an object, for triggering simulation of an operation of the edited product, or for rendering various properties of the displayed product 2000. The cursor 2060 may be controlled by a haptic device to allow the user to interact with the graphical tools.

[0098] The 3D modeled object 2000 may be a graphical representation of a discrete geometric representation of a 3D shape, and the inference method may allow determining any solid CAD features of the editable feature tree 2500. The inference method thus allows editing the editable feature tree 2500 (e.g. adding solid CAD features) based on the 2D sketching functionality of the CAD system and simultaneously updating the discrete geometric representation 2000 of the 3D shape. This represents a high degree of ergonomics.

[0099] Figure 2An example of any system herein is shown where the system is a client computer system, such as a user's workstation.

[0100] The client computer of this example includes: a central processing unit (CPU) 1010 connected to an internal communication bus 1000, a random access memory (RAM) 1070 also connected to the bus. The client computer is also provided with a graphics processing unit (GPU) 1110, which is associated with a video random access memory 1100 connected to the bus. The video RAM 1100 is also known in the art as a frame buffer. A mass storage device controller 1020 manages access to a mass storage device, such as a hard disk drive 1030. Mass storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including, by way of example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM disks 1040. Any of the foregoing may be supplemented or incorporated by a specially designed ASIC (application specific integrated circuit). A network adapter 1050 manages access to a network 1060. The client computer may also include a tactile device 1090, such as a cursor control device, a keyboard, etc. A cursor control device is used in the client computer to allow the user to selectively position the cursor at any desired location on the display 1080. In addition, the cursor control device allows the user to select various commands and input control signals. The cursor control device includes a plurality of signal generating devices for inputting control signals to the system. Typically, the cursor control device may be a mouse, the buttons of which are used to generate signals. Alternatively or additionally, the client computer system may include a sensor pad and / or a sensor screen.

[0101] Any computer program herein may include instructions executable by a computer, including units for causing the above-mentioned system to perform a method. The program may be recorded on any data storage medium, including the memory of the system. The program may be implemented, for example, in digital electronic circuits or computer hardware, firmware, software, or a combination thereof. The program may be implemented as a device, for example, a product tangibly embodied in a machine-readable storage device for execution by a programmable processor. The method steps may be performed by a programmable processor executing an instruction program to perform the functions of the method by operating on input data and generating output. Therefore, the processor may be programmable and coupled to receive data and instructions from a data storage system, at least one input device, and at least one output device, and to send data and instructions to the data storage system, at least one input device, and at least one output device. If necessary, the application may be implemented in a high-level procedural or object-oriented programming language or in an assembly or machine language. In any case, the language may be a compiled language or an interpreted language. The program may be a complete installer or updater. In any case, the application of the program on the system results in the execution of the instructions of the method.

[0102] The examples given above of solid CAD features including corresponding (eg, closed) curve sequences and / or sweep operations (eg, defined by sweep curves) are now discussed further.

[0103] One or more (e.g. all) entity CAD features generated in the dataset formation method, one or more (e.g. all) (e.g. candidate) entity CAD features provided in the learning method and / or one or more (e.g. all) entity CAD features provided by the inference method may be consistent with these examples, respectively.

[0104] In these examples, the learning method may include providing a candidate set of sequence curves and / or a candidate set of scanning operations, such as a candidate set of scanning curves (e.g., a common candidate set of curves), and the learning is also based on the candidate set of curves and / or the candidate set of scanning operations. By "providing a candidate set", it is meant that the learning method, specifically the learning step, is able to treat each element in the candidate set as a candidate for speculation (at least at some times). As is known from the field of machine learning, the learning step may include freely setting the weight values ​​of the neural network so that the speculation method tends to perform speculation consistent with the data set fed to the learning method. And, "providing a candidate set of curves / scanning operations" means that the neural network architecture is predetermined in some way, and the learning step is performed in some way so that each element of the candidate set can be used to potentially speculate in the speculation process. In the example, each element in the candidate set is a candidate for each inferred solid CAD feature. Specifically, each element in the candidate set of curves can be a candidate for each inferred curve of the sequence of curves.

[0105] The candidate set of curves and / or the candidate set of scanning operations may respectively comprise at least one continuous subset of curves (respectively scanning operations). By "continuous" is meant that each element in the subset can be obtained by a specific continuous deformation of another element of the subset (e.g. parameterized with a fixed number of parameters), so that each intermediate state of the continuous deformation is itself a candidate element. In the example, each candidate set comprises several continuous subsets (e.g. one for each curve type). This improves the accuracy of the learning. In fact, since at least one continuum of each element is accessible to become an element of the inferred solid CAD feature, the inference can be performed in a good way in such a continuum.

[0106] Examples of curves in a curve sequence (eg, planar 2D curves, ie, sequence curves) and / or curves defining a scanning operation (eg, scanning curves) are now discussed. For each sequence curve and / or scanning curve, the data structures involved may optionally be the same.

[0107] Given a discrete set of curve types, and for each curve type, given a corresponding discrete set of one or more parameter domains, each discrete set having a corresponding (e.g., continuous) parameter, a curve can be formed by the corresponding curve type given that each parameter domain has a corresponding parameter value for the corresponding continuous parameter, each of the corresponding discrete sets of one or more parameter domains having a corresponding parameter value.

[0108] Thus, the dataset formation method and / or the learning method may be based on such a discrete set of curve types and, for each curve type, on such a corresponding discrete set of one or more parameter domains. These sets may be the same or different for the dataset formation method and the learning method. In the case where the sets are the same, the machine learning process is particularly accurate because the curves of the training samples are consistent with the curves that are candidates for inference.

[0109] Now, the candidate curve set in the learning method may include a set product between a discrete set of curve types and a corresponding discrete set of one or more parameter domains for each curve type. Each element of the set product represents a corresponding candidate sequence / scan curve.

[0110] In the case of a data set formation method, each sequence / scan curve of a training sample may belong to such a set product. In an example, the data set formation method may include providing initial data, the initial data including a discrete set of curve types (data representing them), and for each curve type, a corresponding discrete set of one or more parameter domains (data representing them). In such an example, sequence curves and / or scan curves may be generated (e.g., synthesized) based on such initial data.

[0111] A "curve type" refers to a class of curves (e.g., a sequence curve is a 2D plane, a scan curve is a 3D one), where each curve presents a common general form (e.g., recognizable to the user). Curve types are associated with parameters. Each parameter takes values ​​in a set called a "parameter domain" (which can be an interval, i.e., a continuous range of real numbers), and providing specific parameter values ​​for each parameter domain defines an instance of the curve type. A curve type can be defined by a corresponding set of one or more parameters and a predetermined algorithm that is configured to generate such curve instances of the curve type when fed with a corresponding set of parameter values. The instances form a sequence curve or a scan curve that can intervene in one or more solid CAD features.

[0112] Each curve type can be associated with one or more continuous parameters. A parameter is "continuous" if the function of the output curve is at least piecewise continuous with respect to the parameter. Alternatively, the parameter domain is said to be at least piecewise continuous. This allows a wide variety of shapes to be represented compactly.

[0113] In an example, for each of the one or more curve types, the one or more corresponding continuous parameters include one or more shape parameters as described above, for example, including a positioning parameter for the start point of the curve and a positioning parameter for the end point of the curve, and optionally including one or more additional parameters depending on the curve type (for example, including one or more midpoint positioning parameters and / or one or more tangent vectors, each constraining the curve at a corresponding (e.g., boundary) point of the curve). The positioning parameters can be the same for all curve types, such as (x, y, z) coordinates defined relative to a (0, X, Y, Z) reference.

[0114] In an example that is particularly efficient in terms of computational time and / or convergence for learning, compared to being representative of the diversity of real situations, the discrete set of curve types may include line, arc, and / or cubic Hermitian interpolation segment types, and optionally include line, arc, and cubic Hermitian interpolation segment types. The discrete set of curve types may include any other curve type.

[0115] Any curve provided herein (e.g., on a memory) can be recorded (e.g., in a provided data segment, training sample, and / or inferred output) as a data structure, the data structure comprising: data representing the type of the curve (e.g., text information), and for each parameter, corresponding data representing the parameter value (e.g., a value, such as a floating point number). Any curve provided herein can be post-processed, for example, represented, rendered, and / or displayed in a vector manner (e.g., by dividing the curve into sub-curves, and / or using one or more Bezier curves and / or spline curves to approximate the curve, for example, for a corresponding portion of the curve, for example, in svg format for a 2D curve).

[0116] Examples of dataset formation methods are now discussed.

[0117] In an example, providing the corresponding training sample may include synthesizing the data segment. "Synthesizing" the solid CAD features and / or hand drawings (e.g., training samples) refers to an automatic process that includes generating the solid CAD features and / or determining at least one corresponding hand drawing (e.g., representing the 3D drawing represented by the solid CAD features).

[0118] In an example, a data set formation method may include providing initial data, the initial data including a discrete set of curve types, and for each curve type, a corresponding discrete set of one or more parameter domains. This means that for each curve type, the initial data includes a corresponding data structure corresponding to an object class (referring to object-oriented programming), the data structure having fields corresponding to the parameters of the curve type. In this case, a solid CAD feature can be synthesized based on the initial data. For example, the generation of at least one corresponding solid CAD feature may include synthesizing the solid CAD feature based on the initial data. The synthesis may include: generating an integer representing the number of curves in a sequence; and based on the initial data, generating the number of sequence curves corresponding to the integer, for example, the generation is constrained by the requirements of the solid CAD feature definition, for example, each curve is planar and / or on the same plane, the curves are sequential (i.e., the end point of a curve is the starting point of the next curve), and / or the sequence is closed (i.e., the sequence is cyclic). The generation may also include generating a scan curve based on the initial data.

[0119] In particular, the generation of each sequence / scan curve may include selecting a curve type, and selecting parameter values ​​for each parameter of the curve type. This corresponds to creating an instance of a given class.

[0120] Any selection performed when synthesizing solid CAD features can be performed via (e.g. random) sampling. This allows the generation of data that represents the diversity of real situations relatively well. Therefore, this leads to accurate learning in the end. Any (e.g. random) sampling can include uniform (e.g. random) sampling within the set, where the variable to be selected takes values ​​(e.g., random uniform sampling is random sampling according to a uniform probability distribution).

[0121] For example:

[0122] - The lengths of the curve sequences may be sampled according to a uniform distribution over the set of all integers greater than N1 and less than N2, where N1 is a "minimum depth" integer greater than 0 and / or less than 5, for example equal to 1; and N2 is a "maximum depth" integer greater than N1 and / or less than 100, 50 or 20, for example equal to 10.

[0123] - Sequence / scan curve types may be sampled according to a uniform distribution over a discrete set of curve types.

[0124] - Parameter values ​​for each selected curve type may be sampled in any manner, for example by providing a parameter domain directly or indirectly (e.g. in the initial data), each parameter domain having one or more intervals (e.g. each interval having an upper limit (e.g. floating point) and a lower limit (e.g. floating point)), for example to comply with the requirements of a sequential curve.

[0125] The synthesis may also include determining a solution for each hand-drawn image to determine one or more plausible hand-drawn images.

[0126] The scheme includes providing viewpoints (i.e., 3D locations in 3D space defining solid CAD features) from which corresponding 3D shapes (represented by the synthesized solid CAD features) can be seen, and rendering one or more edges in the image based on these viewpoints. The scheme may optionally include providing a perspective view, or the perspective view may be predetermined (e.g., a default conical perspective view).

[0127] The viewpoints may be provided in any manner, such as sampling (eg, randomly and / or uniformly) among a set of candidate viewpoints (eg, the set of all points surrounding a sphere of the 3D shape).

[0128] The edges define the location of reasonable strokes for a freehand drawing of a 3D shape. For example, relative to a solid representation of a corresponding 3D shape based on a solid CAD feature (i.e., corresponding to an executing solid CAD feature), each edge may be a contour edge or a visible edge of the solid representation from a viewpoint. The contours and / or visible edges may be determined in any manner based on the viewpoint.

[0129] In an example, a visible edge may be considered to be one or more (e.g., all) edges that are accessible from the viewpoint (not occluded by the 3D shape from the viewpoint), lie on the 3D shape (thus potentially including non-planar edges), and are associated with a C having a solid representation on the solid representation. k k is an integer greater than or equal to 1. k may optionally be less than 10 or 4. In an example, the scheme may include providing or based on the maximum discontinuity value k max (For example, k max = 2), and calculate all discontinuities C that form the entity representation k The edge of the discontinuity, where k is greater than or equal to 1 and less than or equal to k max This allows extracting visible tangent discontinuities (i.e. sharp edges) and / or visible curvature discontinuities that are naturally represented by hand-drawn sketches. This therefore allows populating the dataset with realistic data.

[0130] In an example, determining each hand-drawn drawing further comprises: perturbing at least a portion of one or more rendered edges. In other words, after rendering from a viewpoint, the rendered edges may be modified. This allows the irregularities of a hand-drawn sketch to be reproduced, whereas rendering starting from a solid CAD feature would produce overly regular geometry. This therefore makes learning more accurate and reliable.

[0131] As mentioned before, the rendering of each edge can be performed in a vector manner. In other words, each edge can be represented by one or more corresponding parametric curves, for example with less than 50, 20 or 10 control points. Bezier curves, represented in a 2D image in svg format. Other formats can be used. In this case, for at least one corresponding parametric curve, the perturbation can include adding noise to one or more parameters of the at least one corresponding parametric curve. For example, the noise can move the control points. This makes the learning particularly robust.

[0132] Now let's discuss examples of learning methods.

[0133] In an example, the neural network may include recurrent neural network (RNN) units (i.e., units of the same RNN). Such an example allows learning an accurate neural network that exhibits a sequential structure (via the RNN) corresponding to sequential features of a sequence of geometric shapes (e.g., curves) of a solid CAD feature to be inferred, thereby performing accurate inference.

[0134] Specifically, each RNN unit can output corresponding data for inferring the corresponding geometric shape (e.g., sequence curve) of the solid CAD feature at a corresponding time step. Therefore, the output data allows the corresponding curve to be inferred. The length of the curve sequence can be inferred in any predetermined manner (e.g., the inferred curve sequence can be limited to a curve sequence with a given length). Alternatively, each RNN unit can also output corresponding data at a corresponding time step to infer the end or continuation decision (i.e., end flag) of the curve sequence.

[0135] We now discuss particularly effective examples of RNN outputs.

[0136] The corresponding data for inferring the corresponding curve of the solid CAD feature may include: corresponding first data for inferring the corresponding curve type, and corresponding second data for inferring the corresponding parameter value for each parameter domain in the corresponding discrete set of one or more parameter domains.

[0137] In an example, the corresponding first data may include a corresponding discrete probability distribution, each probability being attributed to a corresponding one of a discrete set of curve types. In other words, each RNN unit outputs a probability distribution on the curve type, so that a possible curve type can be inferred for the sequence curve corresponding to the corresponding time step.

[0138] Alternatively or additionally, in an example, the corresponding second data may include (for each curve type) corresponding parameter values ​​for each parameter domain in a corresponding discrete set of one or more parameter domains. In other words, the RNN unit directly outputs the inferred parameter values ​​of the curve type. This simplifies learning and improves convergence and accuracy, especially when the parameters are continuous and it is difficult to learn a neural network with an output density distribution.

[0139] Similarly, the neural network may include, in an example, a standard neural network (NN) component that outputs data for inferring a scanning operation of a solid CAD feature. In an example, the data for inferring a scanning operation of a solid CAD feature may include: first data for inferring a type of a scanning curve, and, for each parameter domain in a corresponding discrete set of one or more parameter domains, second data for inferring a parameter value.

[0140] In an example, the corresponding first data may include a corresponding discrete probability distribution, each probability being attributed to a corresponding one of a discrete set of curve types. In other words, the NN component outputs a probability distribution of the curve type, so that a possible curve type can be inferred for the scanned curve.

[0141] Alternatively or additionally, in an example, the corresponding second data may include (for each curve type) corresponding parameter values ​​for each parameter domain in a corresponding discrete set of one or more parameter domains. In other words, the NN component directly outputs the parameter values ​​of the inferred curve type. This simplifies learning and improves convergence and accuracy, especially when the parameters are continuous and it is difficult to learn a neural network with an output density distribution.

[0142] We now discuss the architecture of the neural network that leads to particularly accurate results.

[0143] In an alternative, the neural network may include an initial encoder that takes a hand-drawn image (which is a 2D image) as input. The encoder may be a convolutional neural network (CNN). The CNN may be a 2D CNN that takes 2D image data as input.

[0144] We now discuss example features of the loss for supervised training that lead to accurate results.

[0145] In an example, the loss may be invariant to cyclic permutations of the sequence and / or to reference-based sketches. In other words, for a candidate solid CAD feature for inference, the loss may not vary with cyclic permutations of its curve sequence, and / or the loss may be invariant to replacing a closed 2D planar sketch with its image and reversing the scanning operation at the end of a scanning operation. Such a loss handles the case of hand-drawn drawings that potentially and correctly represent different solid CAD features if the solid CAD features produce the same 3D shape. The accuracy of learning is improved by focusing the learning on important parameters, thereby sparing the learning method from having to learn unnecessary information.

[0146] The loss may penalize one or more quantities, for example, by penalizing each quantity via a corresponding loss term. For each time step of the sequence curve and the step size relative to the scan curve (each hand-drawn image is here associated with a corresponding ground truth solid CAD feature), the penalty quantity may include any one of the following quantities or any combination thereof:

[0147] - the probability of the corresponding data of the respective curve type being attributed to the corresponding ground truth curve is low,

[0148] - the difference between one or more corresponding parameter values ​​of the ground truth curve and one or more corresponding parameter values ​​of its corresponding respective data, and / or

[0149] - (for RNN cell timesteps only) attributed to the corresponding ground truth ending having a lower probability relative to the corresponding probability of the ongoing state (output by the RNN cell).

[0150] In other words, supervised training, by minimizing such losses, can act on the weights of the neural network, thereby tending to make the corresponding probabilities of each RNN unit and NN part output for the corresponding ground truth curve type close to 1 (i.e. not low), and the corresponding probabilities of the output for other curve types close to 0 (i.e. low), and / or make each continuous parameter value output by each RNN unit and the component close to its ground truth value. Such losses accurately handle the discrete / continuous mixture of problems. Moreover, the loss penalizes the difference between the length of the inferred curve sequence and its ground truth value.

[0151] An example of unsupervised training leading to accurate results is now discussed, wherein respective first data comprises a discrete distribution of respective probabilities, each being attributed to a respective one of a discrete set of curve types.

[0152] It is well known that unsupervised training can minimize losses. Minimization includes exploring the distribution of each discrete probability of the candidate. In other words, minimization iteratively modifies the neural network weights to explore each discrete probability distribution of the output and make them consistent with the unlabeled data set (where each hand-drawn drawing is not associated with any ground entity CAD feature). In a sense, the exploration of each discrete probability distribution of the candidate is indirect and is the result of directly exploring the weights. In the selection, minimization also includes: exploring each discrete probability distribution of the candidate of the end mark. In the example, minimization also includes exploring the corresponding parameter values ​​of the candidates for each parameter domain in each discrete set of one or more parameter domains.

[0153] Now, for each hand drawing of the dataset and for each candidate, the loss penalizes the difference from the corresponding hand drawing. The corresponding hand drawing is derived from the corresponding solid CAD features, which themselves can be inferred based on the respective discrete probability distributions of the explored candidates. In other words, given the weight values ​​of the neural network (in any iteration of unsupervised training), the hand drawing of the unlabeled dataset can be input into the neural network to perform inference on the solid CAD features. A hand drawing can be derived based on each inferred solid CAD feature itself (by building a tree and determining the resulting hand drawing). The hand drawing thus derived can be compared with the initial hand drawing of the corresponding dataset.

[0154] In an example, unsupervised training may include a sampling step for at least one (e.g., each) hand drawing. Training may include providing mini-batches, and sampling may include: for each mini-batch, sampling solid CAD features for each hand drawing in the mini-batch by following data output by the neural network (including probability distributions and continuous parameter values ​​output by the RNN). Sampling may then calculate a loss that compares the input hand drawing (or a representation thereof, such as a rendering) to the hand drawing corresponding to the sampled solid CAD features (i.e., penalizes the difference). Since the sampling is random and discrete, the loss is not differentiable with respect to the sampling. Therefore, since there is no true gradient, classical backpropagation with respect to the true gradient of the variable cannot be performed.

[0155] The proposed unsupervised training example proposes using an enhancement method to provide pseudo gradients of the loss with respect to variables representing candidate discrete distributions of the corresponding first data, and optionally provide pseudo gradients of the loss with respect to variables representing candidate discrete distributions on geometric operations, and / or pseudo gradients of the loss with respect to variables representing end flag distributions. Minimization can then include backpropagation (e.g., stochastic descent) of such pseudo gradients of the loss to learn the neural network. Since the pseudo gradient is provided by the enhancement method, the learning is accurate even if the backpropagation is not one of the true gradients. Minimization can also include backpropagation of the (true) gradient of the loss with respect to the respective parameter values ​​of each parameter domain in each discrete set of one or more parameter domains (because the loss is differentiable with respect to these variables). Therefore, the enhancement method solves the differentiability problem caused by the introduction of discrete probability distributions, and only solves this problem.

[0156] An example machine learning process is now discussed, including examples of the method.

[0157] Recurrent Neural Networks

[0158] The example machine learning process is based on recurrent neural networks (RNNs). Recurrent neural networks are deep neural networks with internal memory states. Therefore, unlike feedforward neural networks, recurrent neural networks can exhibit temporal behavior. They implicitly take into account past entries to predict the next outcome. They can be written as Where y is the output, x is the input, and h is the internal hidden state, also known as the memory. w is the weight of the RNN. The memory h is not a hard memory that stacks all past inputs, it is a fixed-size vector that can be obtained by Learning is performed so that the RNN learns to retain only useful information from past steps.

[0159] Goals of the example machine learning process

[0160] Given a hand drawing of a solid, the example machine learning process aims to obtain reasonable solid CAD features representing the hand drawing. The example machine learning process focuses on the solid CAD features defined for the 3D space by (and only by) the following:

[0161] - A closed 2D planar sketch, consisting of a series of planar curves that form a closed profile,

[0162] -Then comes the scanning operation, which generates a solid by scanning a closed surface defined by the 2D planar sketch in 3D space, i.e. the solid is defined by the volume passed by the closed surface.

[0163] Example of how machine learning works

[0164] The example machine learning process takes a hand-drawn perspective view of a solid as input and generates solid CAD features through an end-to-end deep learning model.

[0165] 1. First, a compact latent representation of the drawing is generated through a convolutional neural network (CNN).

[0166] 2. The solid CAD features are then inferred from this latent representation in two steps:

[0167] a. First, a closed 2D plane sketch is inferred through a recurrent neural network (RNN). The sketch is defined by a sequence of curves. Each time step of the RNN corresponds to a curve in the sequence. At each time step, the corresponding RNN unit estimates the type of curve (straight line, spline, arc), its parameters, and the end / continue flag (whether it is the last element of the sequence).

[0168] b. Second, a neural network (NN) predicts the scan.

[0169] When ground truth solid CAD features are available, the entire model (CNN encoder, RNN, NN) is trained end-to-end in a supervised manner. Otherwise, for unsupervised training, rendered images of the inferred geometry are generated and the entire model is optimized to fit the rendered images to the input hand drawing.

[0170] Implementation options for the example machine learning process

[0171] The example machine learning process can be generalized to any type of 2D planar sketch consisting of a series of parametric curves and parametric sweep operations.

[0172] However, this discussion only considers samples indexed by i, where the 2D plane sketch consists of a series of length T i The parameterized curve (c x (u), With parameter x), each curve consists of Separately, the example machine learning process only considers scanning operations, including scanning along the curve (c x (u), 2D planar sketch with parameter x). For both 2D planar sketch and sweep operations, the example machine learning process considers the following curve types:

[0173] · Straight line: with parameter x=(p start ,p end ) of the straight line segment, where p start and p end are the two boundary points of the segment (i.e. the starting point and the end point):

[0174] cx (u) = p start +(p end -p start )u,

[0175] Arc: with parameter x=(p start ,p inter ,p end ) that passes through p start , p inter and p end . p start and p end are the two boundary points of the segment, and p inter is the middle point. For completeness, the appendix provides the arc formula from three points.

[0176] · Cubic Hermitian interpolation fragments A fragment defined by cubic Hermitian interpolation has parameters x = (p start ,t start ,p end ,t end ), where p start and p end are the two boundary points of the segment, and t start and t end Their tangents are:

[0177]

[0178] Different reasons for choosing these curve types include:

[0179] - The parameters of these curves are meaningful to any user. Therefore, the obtained solid CAD features can be easily edited. In particular, cubic Hermite interpolation is widely used in design (e.g. Catmull-Rom splines).

[0180] - They can represent a wide variety of shapes. Moreover, if a shape cannot be represented by a sequence of these curves, such a sequence can easily approximate the shape (each curve in the sequence locally approximates a region of the shape).

[0181] - This representation also allows easy implementation of the most common scanning operations: stretching and rotation.

[0182] Pipeline options for example machine learning process

[0183] The main steps of the example machine learning pipeline are as follows:

[0184] 1. Generate synthetic labeled dataset (if needed)

[0185] 2. Train the model end-to-end (supervised, using labeled data)

[0186] 3. Train the model end-to-end (unsupervised, using unlabeled data)

[0187] 4. Speculation

[0188] The example machine learning process considers two different ways to train the model. When labeled data is available (or synthetically generated in step 1), the model can be trained in a supervised manner. In an example of this case, the model can be trained in a supervised manner only and / or step 3 can be skipped. In an alternative example of this case, the model can be trained in an unsupervised manner after pre-training with synthetic data. As an alternative to this case, the model can be trained in an unsupervised manner only.

[0189] Description of the example machine learning process

[0190] 1. Generate synthetic labeled datasets (if necessary)

[0191] The example machine learning process considers a labeled dataset as a collection of hand drawings of entities associated with their CAD features, from which the actual geometry of the entity can be generated. Such datasets are rarely available. If missing, they can be generated synthetically using a random generator.

[0192] a. Solid CAD feature generation

[0193] First, solid CAD features are sampled from a given distribution. The distribution can be designed by the user with any later application in mind. The example machine learning process will be specifically designed to learn to retrieve solid CAD features from this distribution.

[0194] Now, the following describes an example of a general distribution that processes a 2D planar sketch consisting of a sequence of curves of length at most N=10 (N can take any strictly positive integer value), and a sweep operation that involves sweeping the 2D planar sketch along the curves. Each sampled solid CAD feature is indexed by i.

[0195] Sample a 2D planar sketch as follows:

[0196] ·exist The sequence T i The length of is uniformly sampled.

[0197] · Two real-valued α t and r t are uniformly sampled in [0,2π[and]0,1], respectively, and the curve type is uniformly selected among the possible curve types. Then, α is adjusted by increasing values. t Sort by value Figure 3 An example of such random sampling is shown, where T i =3, expressed in one yuan.

[0198] For each Use convention T i +1=0, for parameters Types Parameters are selected as follows:

[0199] ο Regardless of the curve type How to set the parameter p start and p end So that p start (Respectively, p end ) is of length r t (Respectively, r t+1 ) with respect to α t (Respectively, α t+1 ) radian axis , and lies on the "ground plane" (through the origin and perpendicular to ).

[0200] In (arc) case, in the fragment Center point μ inter Perform uniform sampling. From a Gaussian distribution For the additional parameter p inter Sampling is performed, where Σ is a diagonal matrix whose non-zero elements are equal to ‖p end -p start ‖2, p inter The z coordinate of is set to 0 so that it also lies at the "ground plane".

[0201] In In the case of fragment For two points μ start and μ end Perform uniform sampling. From Gaussian distribution and Point p start ′ and p end ′ is sampled, where Σ is a diagonal matrix whose non-zero elements are equal to ‖p end -p start ‖2. start ′ and p end ′ is set to 0 so that it is also in the “ground plane”.start and t end Set to t start =p start ′-p start , and t end =p end ′-p end .

[0202] If the instantiated curve intersects itself or one of the previously instantiated curves, the process is repeated until there is no more intersection. If after many trials (e.g. 10 times), there are still some intersections, the entire sample will be regenerated from the beginning.

[0203] Figure 4 Shows Figure 3 An example of and

[0204] The scan operation is sampled as follows:

[0205] · Select the curve type s uniformly from the possible curve types.

[0206] ·For parameters Types i Parameters are selected as follows:

[0207] Regardless of the curve type i How to set the parameter p start Set to the origin. In [-0.5, 0.5] × [-0.5, 0.5] × [0.1, 1.0], p end The parameters are uniformly sampled.

[0208] οDepends on curve type i , the remaining parameters are sampled the same way as curves in a 2D planar sketch, but are not forced to lie in the "ground plane".

[0209] Figure 5 Shown with Figure 4 The same example, where s i =Cubic Hermitian interpolation fragment.

[0210] This general distribution can be specialized to generate specific types of solid CAD features. The following provides some restrictions that can be applied to generate different kinds of solids:

[0211] ·Right prism

[0212] οThe curve type is limited to Line curve type ( andi =Line type).

[0213] In in rather than in The sequence T i The length of is uniformly sampled.

[0214] The boundary points of the linear segment are regularly distributed in a one-dimensional circle ( We set r t =1 and ).

[0215] ο Set the scan line to be vertical (in {0}×{0}×[0.1,1.0] for the scan curve p end parameters are uniformly sampled).

[0216] Cylinder

[0217] ο2D planar sketches consist of arcs that form a univariate circle. Limit the type of curve to 2D planar sketches The arc curve type is chosen and its parameters are selected so that its control points (defined by the arc parameters) lie in this univariate circle.

[0218] Limit the type of scanning operation curve to linear curve type (s i =Line type).

[0219] ο Set the scan line to be vertical (in {0}×{0}×[0.1,1.0] for the scan curve p end parameters are uniformly sampled).

[0220] Solid CAD Feature SCF i The example machine learning process can be completely defined by different curve types and their parameters:

[0221] In order to make all solid CAD features of similar size (which makes the learning process easier), they are sized so that the largest side of their bounding box is equal to 1 and centered (the center of the bounding box is at the origin).

[0222] b. Training sample generation

[0223] Then, for each solid CAD feature, one or more viewpoints are uniformly sampled in a sphere centered at the origin. The sphere radius can be selected so that the entire solid CAD feature is visible in an image rendered from that position and looking toward the origin.

[0224] We define an edge as the C of entities generated kDiscontinuities. For each sample, we randomly choose k = 1 (tangent discontinuities, sharp edges) or k = 2 (curvature discontinuities, such as the edge between a planar and cylindrical surface).

[0225] Then, the outlines and visible edges of the solid CAD features can be obtained from the sampled and viewed image I i Point-of-view rendering of the origin in .

[0226] Finally, by converting the rendered image I i Its corresponding solid CAD feature SCF i The association is performed to generate the training sample i, whose parameters have been previously expressed in the viewpoint coordinate system.

[0227] Figure 6 Shows the use of C 1 Rendered image of a solid CAD feature with discontinuities as edges. Figure 7 Shows the use of C 2 Rendered image of the same solid CAD feature with discontinuities as edges. Figure 8 Shows Figure 5 Rendered image of a solid CAD feature represented in . In this case, each C 2 Discontinuity is also C 1 Discontinuity, i.e., using C 1 or C 2 Rendering the discontinuities as edges will provide the same results.

[0228] c. Sample Perturbation

[0229] Finally, the edges and contours of the rendering may be disturbed to simulate hand-painted strokes.

[0230] The example machine learning process follows the following optional strategy:

[0231] • Contours and edges are previously rendered in vector form, such as in svg format (ie, each rendered fragment corresponds to a parameterized curve).

[0232] Divide each curve into any number of curves (e.g. Bezier curves).

[0233] Add Gaussian noise to each parameter of the curve.

[0234] Fig. 9 Shows Figure 8 Rendered perturbed image shown. Fig.10 Another example of a solid CAD feature rendering is shown (representing a regular prism). Fig.11 Shows Fig.10 Rendered perturbed image of .

[0235] 2. Train the model end-to-end (supervised, using labeled data)

[0236] To train the model, the example machine learning process considers a dataset of size N. Each sample of the dataset indexed by i consists of a hand-drawn image 2D basic sketch curve sequence, scanning operation and its associated parameters Assume that the length of the i-th 2D basic sketch curve sequence is Write each curve sequence as a sequence of curve types For each curve Let its parameters be

[0237] Model definition

[0238] The model takes as input a single-channel image representing a hand drawing (synthetic or non-synthetic) CNN encodes the image into a feature vector Where w is the weight of the model.

[0239] RNN estimates a 2D basis sketch. Each RNN unit outputs a discrete distribution p over the possible types of curves at each time step t. t , and a Bernoulli distribution q on the end / continue flag indicating whether the sketch is finished t Each unit also generates a time step t for each curve type c and state vector h t Output predicted continuous parameters

[0240] Each RNN cell takes the following elements as input:

[0241] Eigenvector

[0242] The state vector h output by the previous unit t-1 , where h0=O.

[0243] Optionally, from the estimated distribution predicted by the previous RNN unit Sample one or several curve types and / or their estimated parameters

[0244] More specifically, each RNN unit formula can be written as:

[0245]

[0246]

[0247] We remember Input is optional.

[0248] The standard NN then outputs the possible scan operation types and the continuous parameters y for each scan operation type s s The discrete distribution r over . It takes the following as input:

[0249] Eigenvector

[0250] The state vector output by the last unit

[0251]

[0252] Fig.12 An example pipeline of a model presented in an example machine learning process is shown. As described above, the RNN unit and / or NN can take the different inputs described above.

[0253] Definition of loss

[0254] For simplicity, we will from now on take the estimated distribution and parameters as the input image , although they may depend on other variables as set out above.

[0255] We define the loss, its penalty curve, and the prediction error for the sweep operation type:

[0256]

[0257] Similarly, we define the following loss, which enforces the curve sequence to have the correct length:

[0258]

[0259] We also define the following loss, which penalizes the squared error on the prediction curve and the sweep operation parameters:

[0260]

[0261] Finally, we minimize the loss as follows to train the weights of the network (using standard deep learning techniques such as the ADAM solver on mini-batches):

[0262] L sup (w) = L type (w)+L length (w)+λL parameters (w) (Formula 4)

[0263] Where λ is a weighting parameter, which can be initialized as:

[0264] Here, w0 is the randomly initialized weight of the neural network.

[0265] Any low penalty function other than the logarithm (cross entropy term) can be used for the term L type (w) and L length Any of (w), and / or any distance d other than the L2 distance, and / or any increasing and / or positive incr function other than the square function may be used for L parameters (w).

[0266] Loss modification: making it invariant to sequence cyclic permutations

[0267] Two curve sequences are equivalent (generate the same 2D basis sketch) if one curve sequence is a cyclic permutation of another, i.e. the same sequence but starting at different points. As a result, during training, the model of the example machine learning process may predict a curve sequence distribution that generates an exact 2D basis sketch, but incur a high loss because the predictions are cyclic permutations of the ground truth. To avoid this problem, we replace the formula and The loss in is modified to be invariant to cyclic permutations:

[0268]

[0269]

[0270] Loss correction: make it invariant to reference-based sketches

[0271] Equivalently, two solid CAD features can be created on the same geometry. In effect, a second 2D plane will be formed at the end of the scanning operation. As a result, a solid CAD feature defined by the 2D plane sketch of this second surface and a scanning operation performed after the original scanning operation (but in the opposite direction) will produce the same solid. Given a solid CAD feature defined, we indicate its equivalent solid CAD feature We modify the loss defined by formula (Equation 6) to be invariant to the feature representation of a single entity:

[0272]

[0273]

[0274] 3. Train the model end-to-end (unsupervised, using unlabeled data)

[0275] The model can be fine-tuned or trained from scratch using real data (i.e. skipping step 2).

[0276] The model architecture remains almost unchanged (see step 2).

[0277] At each iteration, solid CAD features are sampled from the estimated distribution by inputting the plot , and instantiate it with the parameters predicted by the model. Some of the predicted parameters were modified to ensure a closed 2D base sketch (i.e. the end point of a curve in the sequence must coincide with the start point of the next curve). In the example machine learning process, we enforce equal In the special case of the start point of the first curve of the sequence and the end point of the last curve, in which case we force equal

[0278] We denote T i is a sequence of sampled sketches (the smallest t, where the end mark is from q t The length of the sample, is the sampled curve, and s i This is a sampling scan operation.

[0279] Using a fixed arbitrary camera pose in image I i The visible edges of the geometry obtained by rendering in . The rendering process can be seen as the transition between the solid CAD feature and the generated rendered view. i The mapping between . The differential of this mapping is computed using finite differences.

[0280] Instead of using the loss defined in step 2, we define a loss for comparing the input drawing to the obtained rendered image. An example of such a loss could be:

[0281]

[0282] Where d is I i and For example, we can use I i The rendered edges in The Chamfer distance of non-background pixels (or vice versa).

[0283] Minimizing this loss allows us to optimize the weights of the network, except for the discrete distributions p,q,r, which are only used for sampling. The example machine learning process can backpropagate the gradients of the loss with respect to these distributions. The example machine learning process can use a boosting algorithm to obtain this gradient for the discrete distributions p,q,r. The boosting algorithm is limited to these discrete distributions so that the unsupervised learning stage can obtain the true gradients for other variables. This limits the approximation and thus affects the accuracy.

[0284] The enhancement algorithm can be the enhancement algorithm described in the following paper:

[0285] - R. J. Williams. A simple statistical gradient following algorithm for connectionist reinforcement learning. Machine Learning, 229–256, 1992.

[0286] This provides the following gradient formula:

[0287]

[0288]

[0289]

[0290] where ρ is an additional sparsity bonus to encourage sketch sequences to be maximally expressive at minimal size, and b is a baseline for reducing the variance of the estimated gradients and is computed as a moving average of past losses.

[0291] 4. Speculation

[0292] Once the model has been learned, inference is straightforward. The user can create new freehand drawings and feed them to the model. The most likely (according to the inferred distribution) solid CAD features can then be generated. Otherwise, several plausible solutions can be computed by sampling from the inferred distribution. Some of the predicted parameters are modified to ensure a closed 2D base sketch (i.e., the end point of a curve in the sequence must coincide with the start point of the next curve). In one implementation, we enforce equal In the special case of the start point of the first curve of the sequence and the end point of the last curve, in which case the procedure can force equal

[0293] The example machine learning process discussed may allow for the proposal of several plausible solid CAD features, or geometry resulting from a scanned 2D sketch (CAD features may be hidden). The predictions of the example machine learning process are robust to noise.

[0294] Appendix: Arc equation from 3 points

[0295] Let p start 、p inter and p end are three points in space. We define:

[0296] ο plane Π passes through point p start 、p inter and p end ,

[0297] ο Application Π : Convert the 3D points belonging to Π to 2D points in the local 2D coordinate system of plane Π,

[0298] οApplication Convert a 2D point in the local coordinate system of plane Π to the corresponding 3D point,

[0299] ο

[0300] ο and

[0301] ο

[0302] Pass through start 、p inter and p end The arc segments are then defined by:

[0303]

[0304] in,

[0305] and

[0306]

[0307]

[0308] Finally, we note that θ start ,θ inter and θ end They are vectors and Relative to axis Angle:

[0309] If start <θ inter <θ end Then α=θ start And β=θ end

[0310] If start <θend <θ inter Or θ inter <θ star <θ end

[0311] Then α=θ end And β=θ start +2π

[0312] If inter <θ end <θ star Or θ end <θ star <θ inter

[0313] Then α=θ start And β=θ end +2π

[0314] If end <θ inter <θ start Then α=θ end And β=θ start .

Claims

1. A computer-implemented method for forming a data set configured for learning a neural network configured to infer solid CAD features representing a 3D shape from a hand-drawn drawing representing the 3D shape, the method comprising: - generating one or more solid CAD features, including each solid CAD feature representing a corresponding 3D shape; -For each solid CAD feature: o determining one or more corresponding hand-drawn images respectively representing the corresponding 3D shapes; as well as o inserting one or more training samples into the data set, each training sample comprising the solid CAD feature and a corresponding hand drawing, Wherein, determining each hand-drawn picture includes: - providing a viewpoint from which the corresponding 3D shape is visible; - rendering one or more edges in an image relative to a solid representation of said corresponding 3D shape based on said solid CAD feature, each edge being a silhouette edge or a visible edge of said solid representation from said viewpoint; and - perturb at least a portion of one or more rendered edges; - wherein the rendering of each edge is performed in a vector manner, each edge is represented by one or more corresponding parameterized curves, and the perturbation includes: for at least one corresponding parameterized curve, adding noise to one or more parameters of the at least one corresponding parameterized curve.

2. The method according to claim 1, wherein: Each visible edge is the entity represented by C k Discontinuous, k is an integer greater than or equal to 1.

3. The method according to any one of claims 1 to 2, wherein: Each solid CAD feature includes a corresponding curve sequence, and the generating includes: providing a curve set and sampling the curve set.

4. The method according to claim 3, wherein: The curve set includes the set product between: - a discrete set of curve types, and - for each curve type, a respective discrete set of one or more parameter domains for each respective parameter, each parameter domain having a respective parameter value for said respective parameter, The sampling includes providing samples, each sample including a respective curve type and a respective parameter value for each parameter domain in a respective discrete set of the one or more parameter domains.

5. The method according to claim 4, wherein: The discrete curve type set includes: - Line type, - curved, and / or - Cubic Hermitian interpolation fragment type.

6. The method according to any one of claims 1 to 2, wherein: Each solid CAD feature includes a scan operation, and the generating includes: providing a scan operation set and sampling the scan operation set.

7. The method according to claim 6, wherein: The set of scan operations includes the set product between: - a discrete set of sweep curve types, and - for each scanning curve type, a respective discrete set of one or more parameter domains for each respective parameter, each parameter domain having a respective parameter value for said respective parameter, The sampling includes providing samples, each sample including a respective scan curve type and a respective parameter value for each parameter domain in a respective discrete set of the one or more parameter domains.

8. A method for providing a data structure representing a data set formable by a method as claimed in any one of claims 1 to 7.

9. A computer-implemented method using a data set according to claim 8, the method comprising: A neural network is learned based on the data set.

10. A computer program product comprising instructions for executing the method according to any one of claims 1 to 7 and / or the method according to claim 9.

11. A device comprising a data storage medium having recorded thereon a data structure according to claim 8 and / or instructions of a computer program product according to claim 10.

12. The device according to claim 11, wherein The device also includes a processor coupled to the data storage medium.

Images