Methods and systems for providing three-dimensional computer-aided design (CAD) models in a CAD environment

Abstract

A method and system for providing a three-dimensional computer-aided design (CAD) model of an object in a CAD environment are disclosed. One method includes receiving a request for a three-dimensional CAD model of a physical object. The request includes a two-dimensional image of the object. An image vector is generated from the two-dimensional image using a first trained machine learning algorithm. The method includes generating a three-dimensional point cloud model of the object based on the generated image vector using a second trained machine learning algorithm, and generating a three-dimensional CAD model of the object using the three-dimensional point cloud model. The method also includes outputting the three-dimensional CAD model of the object on a graphical user interface.

Description

Technical Field

[0001] This disclosure relates to the field of computer-aided design (CAD), and more particularly to methods and systems for providing three-dimensional computer-aided design models in a CAD environment. Background Technology

[0002] Computer-aided design applications enable users to create 3D CAD models of "real-world" objects via a graphical user interface (GUI). Users can manually perform operations to generate 3D CAD models of objects by interacting with the GUI. For example, to create a hole in a rectangular block, a user might have to specify the hole's diameter, location, and length via the GUI. If the user wants holes at several locations within the rectangular block, they must select the locations where the holes will be created. If the same operations need to be performed multiple times on similar entities, the user must repeat the same activities (e.g., translation, scaling, rotation, selection, etc.). Repeating the same operations multiple times can become time-consuming and monotonous.

[0003] Furthermore, some of these operations are based on the user's experience and expertise. Therefore, beginners or less experienced users may find it difficult to perform these operations without significant exposure to their job role, domain, and industry. Consequently, beginners or less experienced users may make mistakes when performing operations on geometric components. Typically, these errors are identified during the design verification process after the geometric component has been designed. However, correcting these errors can be a cumbersome and time-consuming activity, and may also increase the time to market of the object.

[0004] Furthermore, it's possible that such 3D CAD models are pre-created by the same user or another user and stored in a geometry model database. Currently known CAD applications may not be able to efficiently search for similar 3D CAD models in the geometry model database, leading to the redesign of the 3D CAD model. This could potentially increase the time to market for objects. Summary of the Invention

[0005] The scope of this disclosure is defined only by the appended claims and is not affected in any way by the statements within this description. This embodiment can eliminate one or more disadvantages or limitations in the related art. A method and system for providing three-dimensional computer-aided design (CAD) models in a CAD environment are disclosed.

[0006] In one aspect, a method for providing a three-dimensional computer-aided design (CAD) model of an object in a CAD environment includes: receiving a request for a three-dimensional CAD model of the object. The request includes a two-dimensional image of the object. The method includes generating image vectors from the two-dimensional image using a first trained machine learning algorithm. The method further includes generating a three-dimensional point cloud model of the object based on the generated image vectors using a second trained machine learning algorithm, and generating a three-dimensional CAD model of the object using the three-dimensional point cloud model of the object. The method includes outputting the three-dimensional CAD model of the object on a graphical user interface. The method may include storing the three-dimensional point cloud model of the object and the generated image vectors from the two-dimensional image in a geometric model database.

[0007] The method may include receiving a request for a 3D CAD model of the object. The request includes a 2D image of the object. The method may include generating an image vector from the 2D image using a first trained machine learning algorithm, and performing a search for a 3D CAD model of the object in a geometric model database comprising multiple 3D CAD models based on the generated image vector. The method may include determining whether a 3D CAD model of the object is successfully found in the geometric model database, and outputting the 3D CAD model of the object on a graphical user interface.

[0008] In the action of performing a search for a 3D CAD model of the object in a geometric model database using a third trained machine learning algorithm, the method may include comparing the generated image vector of the 2D image with each image vector associated with the corresponding 3D CAD model in the geometric model database using the third machine learning algorithm, and identifying the 3D CAD model from the geometric model database based on the best match between the generated image vector and the image vector of the 3D CAD model.

[0009] In another aspect, a method for providing a three-dimensional computer-aided design (CAD) model of an object in a CAD environment includes: receiving a request for a three-dimensional CAD model of the object. The request includes a two-dimensional image of the object. The method includes generating an image vector from the two-dimensional image using a first trained machine learning algorithm, and performing a search for a three-dimensional CAD model of the object in a geometric model database comprising a plurality of three-dimensional CAD models based on the generated image vector. The method includes determining whether the requested three-dimensional CAD model of the object is successfully found in the geometric model database, and if the requested three-dimensional CAD model of the object is successfully found in the geometric model database, outputting the requested three-dimensional CAD model of the object on a graphical user interface.

[0010] The method may include: if the requested 3D CAD model of the object is not found in a geometric model database, generating a 3D CAD model of the object based on the generated image vectors using a second trained machine learning algorithm, and outputting the generated 3D CAD model of the object on a graphical user interface.

[0011] In the action of generating a 3D CAD model of the object based on the generated image vectors using a second trained machine learning model, the method may include generating a 3D point cloud model of the object based on the generated image vectors using the second trained machine learning algorithm, and generating a 3D CAD model of the object using the 3D point cloud model of the object. The method may include storing the generated 3D CAD model of the object and the generated image vectors of the corresponding 2D image of the object in a geometric model database.

[0012] In the action of performing a search for a 3D CAD model of the object in a geometric model database, the method may include performing a search for a 3D CAD model of the object in the geometric database using a third trained machine learning algorithm.

[0013] In the action of performing a search for a 3D CAD model of the object in a geometric model database using a third trained machine learning algorithm, the method may include comparing an image vector generated from the 2D image with each image vector associated with a corresponding geometric model in the geometric model database using the third machine learning algorithm, and identifying the one or more 3D CAD models from the geometric model database based on the matching between the generated image vector and the image vectors of one or more 3D CAD models.

[0014] The method may include sorting one or more 3D CAD models based on matching with a requested 3D CAD model of the object, and determining at least one 3D CAD model having an image vector that best matches a generated image vector of the 2D image based on the sorting of the one or more 3D CAD models. The method may include modifying the determined 3D CAD model based on the generated image vector of the 2D model.

[0015] In another aspect, a data processing system includes a processing unit and a memory unit coupled to the processing unit. The memory unit includes a CAD model configured to receive a request for a three-dimensional computer-aided design (CAD) model of an object. The request includes a two-dimensional image of the object. The CAD model is configured to generate image vectors from the two-dimensional image using a first trained machine learning algorithm, and is configured to perform a search for a three-dimensional CAD model of the object in a geometric database comprising multiple three-dimensional CAD models based on the generated image vectors. The CAD module is configured to determine whether the requested three-dimensional CAD model of the object is successfully found in the geometric model database, and is configured to output the requested three-dimensional CAD model of the object on a graphical user interface if the requested three-dimensional CAD model of the object is successfully found in the geometric model database.

[0016] The CAD module can be configured to: if the requested 3D CAD model of the object is not found in the geometric model database, generate a 3D CAD model of the object based on the generated image vectors using a second trained machine learning algorithm, and be configured to output the generated 3D CAD model of the object on a graphical user interface.

[0017] In the action of generating a 3D CAD model of the object using a second trained machine learning model based on the generated image vectors, the CAD module can be configured to generate a 3D point cloud model of the object using the second trained machine learning algorithm based on the generated image vectors, and to generate a 3D CAD model of the object using the 3D point cloud model of the object. The CAD module can be configured to store the generated 3D CAD model of the object and the generated image vectors of the corresponding 2D image of the object in a geometric model database.

[0018] In the action of performing a search for a 3D CAD model of the object in the geometric model database, the CAD module can be configured to perform the search for a 3D CAD model of the object in the geometric database using a third trained machine learning algorithm.

[0019] In the action of performing a search for a 3D CAD model of the object in a geometric model database using a third trained machine learning algorithm, the CAD module may be configured to compare the generated image vector of the 2D image with each image vector associated with a corresponding geometric model in the geometric model database using the third machine learning algorithm, and to identify the one or more 3D CAD models from the geometric model database based on the matching between the generated image vector and the image vector of one or more 3D CAD models.

[0020] The CAD module can be configured to sort identified 3D CAD models based on matching with a requested 3D CAD model of the object, and to determine at least one 3D CAD model having an image vector that best matches the generated image vector of the 2D image based on the sorting of the one or more 3D CAD models. The CAD module can be configured to modify the determined 3D CAD model based on the generated image vector of the 2D model.

[0021] In another aspect, a non-transitory computer-readable medium is provided, which stores machine-readable instructions that, when executed by a data processing system, cause the data processing system to perform the methods described above.

[0022] The summary is provided to present, in a simplified form, the selection of concepts further described below. The summary is not intended to identify features or essential features of the claimed subject matter. The claimed subject matter is not limited to implementations that address any or all the shortcomings pointed out in any part of this disclosure. Attached Figure Description

[0023] Figure 1 This is a block diagram of an exemplary data processing system according to one embodiment for providing a three-dimensional computer-aided design (CAD) model of an object using one or more trained machine learning algorithms.

[0024] Figure 2 This is a block diagram of a CAD module for providing a 3D CAD model of an object based on a 2D image of the object, according to one embodiment.

[0025] Figure 3 This is a flowchart illustrating an exemplary method for generating a 3D CAD model of an object in a CAD environment according to one embodiment.

[0026] Figure 4 This is a process flowchart depicting an exemplary method for generating a three-dimensional CAD model of an object in a CAD environment according to another embodiment.

[0027] Figure 5This is a flowchart depicting a method for providing a three-dimensional CAD model of an object in a CAD environment according to yet another embodiment.

[0028] Figure 6 This is a flowchart depicting a method for providing a three-dimensional CAD model of an object in a CAD environment according to yet another embodiment.

[0029] Figure 7 This is a schematic representation of a data processing system for providing a three-dimensional CAD model of an object, according to another embodiment.

[0030] Figure 8 The diagram illustrates a block diagram of a data processing system for providing a 3D CAD model of an object using a trained machine learning algorithm, according to yet another embodiment.

[0031] Figure 9 The illustration depicts an example of, according to one embodiment, such as Figure 2 The image vector generation module shown is a schematic representation.

[0032] Figure 10 The illustration depicts an example of, according to one embodiment, such as Figure 2 The diagram shows a schematic representation of the model search module.

[0033] Figure 11 The illustration depicts an example of, according to one embodiment, such as Figure 2 The diagram shows a schematic representation of the model generation module. Detailed Implementation

[0034] A method and system for providing three-dimensional computer-aided design (CAD) models in a CAD environment are disclosed. Various embodiments are described with reference to the accompanying drawings, in which similar reference numerals are used. Similar reference numerals are always used to refer to similar elements. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments. These specific details are not required to be adopted in practicing the embodiments. In other instances, well-known materials or methods have not been described in detail to avoid unnecessarily obscuring the embodiments. While this disclosure allows for various modifications and alternatives, specific embodiments thereof are shown by way of example in the accompanying drawings and will be described in detail herein. There is no intention to limit this disclosure to the specific forms disclosed. Instead, this disclosure is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of this disclosure.

[0035] Figure 1 This is a block diagram of an exemplary data processing system 100 for providing a 3D CAD model of an object using one or more trained machine learning algorithms, according to one embodiment. The data processing system 100 may be a personal computer, workstation, laptop computer, tablet computer, etc. Figure 1 In this system, the data processing system 100 includes a processing unit 102, a memory unit 104, a storage unit 106, a bus 108, an input unit 110, and a display unit 112. The data processing system 100 is a dedicated computer configured to provide 3D CAD models using one or more trained machine learning algorithms.

[0036] The processing unit 102 used herein can be any type of computing circuit, such as, but not limited to, a microprocessor, microcontroller, complex instruction set computing microprocessor, reduced instruction set computing microprocessor, very long instruction word microprocessor, explicit parallel instruction computing microprocessor, graphics processor, digital signal processor, or any other type of processing circuit. The processing unit 102 may also include an embedded controller, such as a general-purpose or programmable logic device or array, application-specific integrated circuit, single-chip computer, etc.

[0037] Memory cell 104 may be a non-transitory volatile memory or a non-volatile memory. Memory cell 104 may be coupled for communication with processing unit 102, such as a computer-readable storage medium. Processing unit 102 may execute instructions and / or code stored in memory cell 104. Various computer-readable instructions may be stored in and accessed from memory cell 104. Memory cell 104 may include any suitable element for storing data and machine-readable instructions, such as read-only memory, random access memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, hard disk drive, removable media drive for handling compressed disks, digital video disks, magnetic disks, tape cartridges, memory cards, etc.

[0038] In this embodiment, the memory unit 104 includes a CAD module 114 stored in the form of machine-readable instructions on any of the aforementioned storage media, and can communicate with and be executed by the processing unit 102. When the machine-readable instructions are executed by the processing unit 102, the CAD module 114 causes the processing unit 102 to generate an image vector from a two-dimensional image of the object using a first trained machine learning algorithm. The two-dimensional (2-D) image can be a photograph of the physical object, a hand-drawn sketch, a single-view preview of a 3D CAD model, etc. When the machine-readable instructions are executed by the processing unit 102, the CAD module 114 causes the processing unit 102 to perform a search for a 3D CAD model of the object in a geometric database 116 containing multiple 3D CAD models based on the generated image vector, determine whether the requested 3D CAD model of the object is successfully found in the geometric model database 116, and if the requested 3D CAD model of the object is successfully found in the geometric model database 116, output the requested 3D CAD model of the object on the display unit 112. Furthermore, when machine-readable instructions are executed by processing unit 102, if the requested 3D CAD model of the object is not found in geometric model database 116, CAD module 114 causes processing unit 102 to generate a 3D CAD model of the object based on the generated image vectors using a second trained machine learning algorithm, and outputs the generated 3D CAD model of the object on display unit 112. Figures 3 to 6 The method steps performed by the processing unit 102 to achieve the above-mentioned functions are described in more detail below.

[0039] Storage unit 106 may be a non-transitory storage medium for storing a geometric model database 116. The geometric model database 116 stores image vectors of 3D CAD models along with 2D images of objects represented by the 3D CAD models. Input unit 110 may include an input device capable of receiving input signals (e.g., a request for a 3D CAD model of an object), such as a keypad, a touch-sensitive display, a camera (e.g., a camera that receives gesture-based input), etc. Display unit 112 may be a device having a graphical user interface for displaying a 3D CAD model of an object. The graphical user interface may also allow a user to select CAD commands for providing the 3D CAD model. Bus 108 serves as an interconnect between processing unit 102, memory unit 104, storage unit 106, input unit 110, and display unit 112.

[0040] Those skilled in the art will understand that Figure 1The hardware components depicted may vary for a particular implementation. For example, other peripheral devices, such as optical disc drives, local area network (LAN) / wide area network (WAN) / wireless (e.g., Wi-Fi) adapters, graphics adapters, disk controllers, and input / output (I / O) adapters, may be used in addition to or in place of the depicted hardware. The examples depicted are provided for illustrative purposes only and are not intended to imply any architectural limitations with respect to this disclosure.

[0041] A data processing system 100 according to an embodiment of this disclosure includes an operating system employing a graphical user interface (GUI). The operating system allows multiple display windows to be presented simultaneously in the GUI, each providing an interface to different applications or different instances of the same application. A user can manipulate a cursor in the GUI using a pointing device. The cursor position can be changed, and / or events such as clicking a mouse button can be generated to trigger a desired response.

[0042] With proper modifications, it can be adopted from one of various commercial operating systems, such as Microsoft Windows. TM This is a version of a product of Microsoft Corporation, located in Redmond, Washington. As described, this disclosure pertains to modifying or creating an operating system.

[0043] Figure 2 This is a block diagram of a CAD module 114 for providing a 3D CAD model of an object based on a 2D image of the object, according to one embodiment. The CAD module 114 includes a vector generation module 202, a model search module 204, a model sorting module 206, a model modification module 208, a model generation module 210, and a model output module 212.

[0044] Vector generation module 202 is configured to generate image vectors of a two-dimensional image of an object. The two-dimensional image is input by a user of data processing system 100, enabling data processing system 100 to provide a three-dimensional CAD model of the object. In one embodiment, vector generation module 202 uses a trained convolutional neural network to generate a high-dimensional image vector of size 4096 from the two-dimensional image. For example, vector generation module 202 preprocesses the two-dimensional image to generate a three-dimensional image matrix and transforms the three-dimensional matrix into a high-dimensional image vector using a trained VGG convolutional neural network. In the preprocessing of the image, vector generation module 202 resizes the two-dimensional image to [224, 224, 3] and normalizes the resized image to generate a three-dimensional image matrix of size [224, 224, 3]. In some embodiments, the trained VGG convolutional neural network has a stack of convolutional layers followed by two fully connected (FC) layers. The first FC layer accepts a three-dimensional image matrix of size [224, 224, 3]. The 3D image matrix is ​​processed through each layer and passed to the second FC layer in the desired shape. The second FC layer has 4096 channels. The second FC layer transforms the preprocessed 3D image matrix into a one-dimensional image vector of size 4096.

[0045] The model search module 204 is configured to use a trained machine learning algorithm (e.g., Figure 10 The K-Nearest Neighbor algorithm 1002 performs a search for a requested 3D CAD model of the object in a geometric model database 116 based on the generated image vectors. The geometric model database 116 includes multiple 3D CAD models of the object and corresponding image vectors of 2D images of the object. In one embodiment, the model search module 204 is configured to compare the image vectors of the 2D images with the image vectors corresponding to the multiple 3D CAD models stored in the geometric model database 116 using the K-Nearest Neighbor algorithm. In an exemplary implementation, the K-Nearest Neighbor algorithm indicates the probability that each image vector in the geometric model database 116 matches a generated image vector corresponding to the requested 3D CAD model. For example, the K-Nearest Neighbor algorithm uses a distance metric such as Euclidean distance to calculate the distance between the generated image vector and each image vector in the geometric model database 116. The model search module 204 outputs the image vector with the minimum distance to the generated image vector. The image vector with the minimum distance is considered the best match for the generated image vector. Alternatively, the model search module 204 outputs one or more image vectors with distances relative to the generated image vectors that fall within a predefined range.

[0046] The model search module 204 is configured to identify one or more 3D CAD models from a plurality of 3D CAD models that have an image vector that best matches the image vector of the requested 3D CAD model corresponding to the object. In an exemplary implementation, the model search module 204 identifies one or more 3D CAD models from a plurality of 3D CAD models based on probability values ​​associated with the image vectors corresponding to the one or more 3D CAD models. For example, if the image vector corresponding to the 3D CAD model with the probability value falls within a predefined range (e.g., 0.7 to 1.0), the model search module 204 can select the 3D CAD model.

[0047] The model ranking module 206 is configured to rank each of the identified 3D CAD models based on its match with the requested 3D CAD model. In one embodiment, the model ranking module 206 ranks the identified 3D CAD models based on the probability value of the corresponding image vector. For example, if the probability of a corresponding image vector matching the image vector of the 2D image is the highest, the model ranking module 206 assigns the highest ranking to the identified 3D CAD model. This is because the highest probability indicates the best match between the identified 3D CAD model and the requested 3D CAD model. Therefore, the model ranking module 206 can select one of the identified 3D CAD models with the highest ranking as the result of a search performed in the geometric model database 116.

[0048] The model modification module 208 is configured to modify the selected 3D CAD model if there is no exact match between the selected 3D CAD model and the requested 3D CAD model. In one embodiment, if the probability value of the image vector corresponding to the selected 3D CAD model is less than 1.0, the model modification module 208 determines that there is no exact match between the selected 3D CAD model and the requested 3D CAD model. The model modification module 208 compares the image vector corresponding to the selected 3D CAD model with the image vector corresponding to the requested 3D CAD model. The model modification module 208 determines the two-dimensional points between the mismatched image vectors. The model modification module 208 uses another trained machine learning algorithm (e.g., Figure 11 The multilayer perceptron 1102A-N generates 3D points corresponding to the 2D points based on the image vectors of the requested 3D CAD model. The model modification module 208 uses these 3D points to modify the 3D point cloud model of the selected 3D CAD model. For example, the model modification module 208 modifies the 3D point cloud model by replacing the 3D points with the generated 3D points. Therefore, the model modification module 208 generates a modified 3D CAD model based on the modified 3D point cloud model of the selected 3D CAD model.

[0049] The model generation module 210 is configured to use yet another trained machine learning algorithm (e.g., Figure 11 A multilayer perceptron 1102A-N generates a 3D CAD model of the object from image vectors of the 2D image. In one embodiment, the model generation module 210 is configured to generate a 3D CAD model if a search for the requested 3D CAD model in the geometric model database 116 is unsuccessful. The search for the requested 3D CAD model is unsuccessful if the model search module 204 does not find any best-matching 3D CAD model(s) in the geometric model database 116. In an alternative embodiment, the model generation module 210 is configured to generate a 3D CAD model from image vectors without performing a search for similar 3D CAD models in the geometric model database 116.

[0050] According to the foregoing embodiment, the model generation module 210 uses another trained machine learning algorithm to generate three-dimensional points for each two-dimensional point in the image vector of the two-dimensional image. The model generation module 210 then generates a three-dimensional point cloud model based on these three-dimensional points. Therefore, the model generation module 210 generates the requested three-dimensional CAD model based on the three-dimensional point cloud model.

[0051] The model output module 212 is configured to output the requested 3D CAD model on the display unit 112 of the data processing system 100. Alternatively, the model output module 212 is configured to generate a CAD file including the requested 3D CAD model for manufacturing the object using an additive manufacturing process. Furthermore, the model output module 212 is configured to store the requested 3D CAD model along with the image vectors of the 2D image in the CAD file. Alternatively, the model output module 212 is configured to store the 3D point cloud model in Standard Template Library (STL) format, allowing the data processing system 100 to reproduce the 3D CAD model based on the STL-formatted 3D point cloud model.

[0052] Figure 3 This is a process flowchart 300 depicting an exemplary method for generating a 3D CAD model of an object in a CAD environment according to one embodiment. At action 302, a request for a 3D CAD model of a physical object is received from a user of the data processing system 100. The request includes a 2D image of the object. At action 304, an image vector is generated from the 2D image using a VGG network.

[0053] At action 306, a three-dimensional point cloud model of the object is generated based on the generated image vectors using a multilayer perceptron. At action 308, a three-dimensional CAD model of the object is generated using the three-dimensional point cloud model of the object. At action 310, the three-dimensional CAD model of the object is output on the graphical user interface of the data processing system 100. At action 312, the three-dimensional point cloud model of the object and the generated image vectors of the two-dimensional image are stored in the geometric model database 116 in a standard template library format.

[0054] Figure 4 This is a process flowchart 400 depicting an exemplary method for generating a 3D CAD model of an object in a CAD environment according to another embodiment. At action 402, a request for a 3D CAD model of an object is received from a user of the data processing system 100. The request includes a 2D image of the object. At action 404, an image vector is generated from the 2D image using a VGG network.

[0055] At action 406, a search for the requested 3D CAD model of the object is performed in a geometric model database 116, which includes multiple 3D CAD models, based on the generated image vectors. In some embodiments, the generated image vectors of the 2D image are compared with each image vector associated with a corresponding 3D CAD model in the geometric model database 116 using a K-nearest neighbor algorithm. In these embodiments, the 3D CAD model is identified from the geometric model database based on the best match between the generated image vectors and the image vectors of the 3D CAD models. At action 408, it is determined whether the 3D CAD model of the object has been successfully found in the geometric model database 116. If the 3D CAD model has been successfully found in the geometric model database 116, then at action 410, the 3D CAD model of the object is output on the graphical user interface. Otherwise, process 400 ends at step 412.

[0056] Figure 5This is a flowchart 500 depicting a method for providing a 3D CAD model of an object in a CAD environment according to yet another embodiment. At action 502, a request for a 3D CAD model of an object is received from a user of a data processing system 100. The request includes a 2D image of the object. At action 504, an image vector is generated from the 2D image using a VGG network. At action 506, a search for the requested 3D CAD model of the object is performed in a geometry model database 116, which includes multiple 3D CAD models, based on the generated image vector. In some embodiments, the generated image vector of the 2D image is compared with each image vector associated with a corresponding geometry model in the geometry model database 116 using a K-nearest neighbor algorithm. In these embodiments, the one or more 3D CAD models are identified from the geometry model database based on the matching between the generated image vector and the image vectors of one or more 3D CAD models.

[0057] At action 508, it is determined whether the requested 3D CAD model of the object has been successfully found in the geometric model database 116. If the requested 3D CAD model of the object has been successfully found in the geometric model database 116, action 18 is executed. At action 514, the requested 3D CAD model of the object is output on the graphical user interface of the data processing system 100. If one or more 3D CAD models are found, the one or more 3D CAD models are sorted based on their matching with the requested 3D CAD model of the object. Therefore, based on the sorting of the one or more 3D CAD models, at least one 3D CAD model having an image vector that best matches the generated image vector of the 2D image is determined and output. In an alternative embodiment, the one or more 3D CAD models are output along with their sorting.

[0058] If the requested 3D CAD model of the object is not found in the geometric model database 116, then at action 510, a 3D point cloud model of the object is generated using a multilayer perceptron based on the generated image vectors. At action 512, a 3D CAD model of the object is generated using the 3D point cloud model of the object. At action 514, the 3D CAD model of the object is output on the graphical user interface of the data processing system 100. Additionally, the generated 3D CAD model of the object and the generated image vectors of the corresponding 2D image of the object are stored in the geometric model database 116 in a standard template library format.

[0059] Figure 6This is a flowchart 600 depicting a method for providing a 3D CAD model of an object in a CAD environment according to another embodiment. At action 602, a request for a 3D CAD model of an object is received from a user of a data processing system 100. The request includes a 2D image of the object. At action 604, an image vector is generated from the 2D image using a VGG network. At action 606, a search for a 3D CAD model of the object is performed in a geometry model database 116 comprising multiple 3D CAD models based on the generated image vector. In some embodiments, the generated image vector of the 2D image is compared with each image vector associated with a corresponding geometry model in the geometry model database 116 using a K-nearest neighbor algorithm. In these embodiments, the one or more 3D CAD models are identified from the geometry model database based on the matching between the generated image vector and the image vectors of one or more 3D CAD models.

[0060] At action 608, it is determined whether the requested 3D CAD model of the object has been successfully found in the geometric model database 116. If the requested 3D CAD model of the object has been successfully found in the geometric model database, at action 610, the identified 3D CAD model is modified based on the image vector generated from the 2D image of the object to match the requested 3D CAD model of the object. If one or more 3D CAD models are found, the one or more 3D CAD models are sorted based on their matching with the requested 3D CAD model of the object. Therefore, based on the sorting of the one or more 3D CAD models, at least one 3D CAD model with an image vector that best matches the generated image vector of the 2D image is determined. Therefore, the determined 3D CAD model is modified based on the image vector of the 2D image of the object to match the requested 3D CAD model. At step 616, the requested 3D CAD model of the object is output on the graphical user interface of the data processing system 100.

[0061] If the requested 3D CAD model of the object is not found in the geometric model database 116, then at action 612, a 3D point cloud model of the object is generated using a multilayer perceptron based on the generated image vectors. At action 614, the requested 3D CAD model of the object is generated using the 3D point cloud model of the object. At action 616, the requested 3D CAD model of the object is output on the graphical user interface of the data processing system 100. Furthermore, the generated 3D CAD model of the object and the generated image vectors of the corresponding 2D image of the object are stored in the geometric model database 116.

[0062] Figure 7This is a schematic representation of a data processing system 700 for providing a three-dimensional CAD model of an object according to another embodiment. Specifically, the data processing system 700 includes a cloud computing system 702 configured to provide cloud services for designing three-dimensional CAD models of objects.

[0063] The cloud computing system 702 includes a cloud communication interface 706, cloud computing hardware and an operating system 708, a cloud computing platform 710, a CAD module 114, and a geometric model database 116. The cloud communication interface 706 enables communication between the cloud computing platform 710 and user devices 712A-N, such as smartphones, tablets, and computers, via a network 304.

[0064] The cloud computing hardware and OS 708 may include one or more servers on which an operating system (OS) is installed, and include one or more processing units, one or more storage devices for storing data, and other peripheral devices required to provide cloud computing functionality. The cloud computing platform 710 is a platform that implements functions such as data storage, data analysis, data visualization, and data communication on the cloud hardware and OS 708 via APIs and algorithms, and delivers the aforementioned cloud services using cloud-based applications (e.g., computer-aided design applications). The cloud computing platform 710 employs a CAD module 114 to provide a three-dimensional CAD model of the object based on a two-dimensional image of the object, such as... Figures 3 to 6 As described in [the document]. The cloud computing platform 710 also includes a geometric model database 116 for storing image vectors of three-dimensional CAD models of objects along with two-dimensional images of the objects.

[0065] According to the foregoing embodiments, the cloud computing system 702 enables users to design objects using trained machine learning algorithms. Specifically, the CAD module 114 can use a trained machine learning algorithm to search for a 3D CAD model of the object in the geometric model database 116 based on the image vectors of the object's 2D image. The CAD module 114 can output the best-matching 3D CAD model of the object on a graphical user interface. If the geometric model database 116 does not have the requested 3D CAD model, the CAD module 114 uses another trained machine learning algorithm to generate the requested 3D CAD model of the object based on the image vectors of the object's 2D image. The cloud computing system 702 enables users to remotely access the 3D CAD model of the object using a 2D image of the object.

[0066] User equipment 712A-N includes graphical user interfaces 714A-N for receiving requests for 3D CAD models and displaying 3D CAD models of objects. Each of user equipment 712A-N may be provided with a communication interface for interfacing with cloud computing system 702. Users of user equipment 712A-N can access cloud computing system 702 via graphical user interfaces 714A-N. For example, a user can send a request to cloud computing system 702 to perform geometric operations on geometric components using a machine learning model. Graphical user interfaces 714A-N may be specifically designed to access component generation module 114 in cloud computing system 702.

[0067] Figure 8 A block diagram of a data processing system 800 for providing a 3D CAD model of an object using machine learning algorithms, according to yet another embodiment, is illustrated. Specifically, the data processing system 800 includes a server 802 and a plurality of user devices 806A-N. Each of the user devices 806A-N is connected to the server 802 via a network 804 (e.g., a local area network (LAN), a wide area network (WAN), Wi-Fi, etc.). The data processing system 800 is... Figure 1 Another implementation of the data processing system 100 is that the component generation module 114 resides in the server 802 and is accessed by the user equipment 806A-N via the network 804.

[0068] Server 802 includes a component generation module 114 and a geometric component database 116. Server 802 may also include a processor, memory, and storage units. CAD module 114 may be stored in memory as machine-readable instructions and is executable by the processor. Geometric component database 116 may be stored in storage units. Server 802 may also include a communication interface for enabling communication with client devices 806A-N via network 804.

[0069] When machine-readable instructions are executed, component generation module 114 causes server 802 to use a trained machine learning algorithm to search for and output a 3D CAD model of the object from the geometric model database 116 based on a 2D image of the object, and if the requested 3D CAD model is not found in the geometric model database 116, another trained machine learning algorithm is used to generate a 3D CAD model of the object. Figures 3 to 6 The method steps performed by server 402 to achieve the above functions are described in more detail below.

[0070] Client devices 812A-N include graphical user interfaces 814A-N for receiving requests for 3D CAD models and displaying 3D CAD models of objects. Each of the client devices 812A-N may be provided with a communication interface for interfacing with the cloud computing system 802. Users of the client devices 812A-N can access the cloud computing system 802 via the graphical user interface 814A-N. For example, a user can send a request to the cloud computing system 802 to perform geometric operations on geometric components using a machine learning model. The graphical user interface 814A-N may be specifically designed to access the component generation module 114 in the cloud computing system 802.

[0071] Figure 9 The illustration depicts an example of, according to one embodiment, such as Figure 2 A schematic representation of the image vector generation module 202 shown. (See diagram below.) Figure 9 As shown, the vector generation module 202 includes a preprocessing module 902 and a VGG network 902. The preprocessing module 902 is configured to preprocess the 2-D image 906 of the object by resizing and normalizing it. For example, the preprocessing module 902 resizes the 2-D image 906 to a size of 224×224 pixels with 3 channels and normalizes the resized 2-D image using the mean and standard deviation of the VGG network 904 (e.g., mean = [0.485, 0.456, 0.406], standard deviation = [0.229, 0.224, 0.225]). The VGG network 904 is configured to transform the preprocessed 2-D image into a high-dimensional latent image vector 908. VGG Network 904 is a convolutional neural network trained to transform a normalized 2-D image of size 224×224 pixels with 3 channels into a high-dimensional latent image vector 908 of size 4096. The high-dimensional latent image vector 908 represents relevant features from the 2-D image, such as edges, corners, colors, and textures.

[0072] Figure 10 The illustration depicts an example of, according to one embodiment, such as Figure 2 The schematic representation of the model search module 204 shown. Figure 10 As shown, the model search module 204 employs the K-nearest neighbor algorithm to perform a search within the geometric model database 116 for 3D CAD models of objects requested by the user of the data processing system 100. The K-nearest neighbor algorithm 1002 can be an unsupervised machine learning algorithm, such as a nearest neighbor algorithm with a Euclidean distance metric. The K-nearest neighbor algorithm 1002 is based on... Figure 9The high-dimensional image vector 908 generated by the VGG network 904 is used to perform a search for the requested 3D CAD model in the geometric model database 116. The geometric model database 116 stores various 3D CAD models along with their corresponding high-dimensional image vectors 908. In an exemplary implementation, the K-nearest neighbor algorithm 1002 compares the high-dimensional image vector 908 with the high-dimensional image vectors in the geometric model database 116. The K-nearest neighbor algorithm 1002 identifies (multiple) best-matching high-dimensional image vectors from the geometric model database 116. The model search module 204 retrieves and outputs (multiple) 3D CAD models 1004 corresponding to (multiple) best-matching high-dimensional image vectors from the geometric model database 116.

[0073] Figure 11 The illustration depicts an example of, according to one embodiment, such as Figure 2 A schematic representation of the model generation module 210 shown. (See diagram.) Figure 11 As shown, the model generation module 210 uses a multilayer perceptron 1102A-N to generate a new 3D CAD model of the object based on a high-dimensional image vector 908 of the object's 2D image. In some embodiments, when the model search module 204 cannot find any best-matching 3D CAD model in the geometric model database 116, the model generation module 210 generates a new 3D CAD model of the object.

[0074] In an exemplary implementation, the multilayer perceptron 1102A-N generates three-dimensional points 1106A-N corresponding to two-dimensional points 1104A-N in the high-dimensional image vector 908. This indicates that the two-dimensional points representing the object are uniformly sampled in a unit square space. The high-dimensional image vector 908 is concatenated with the sampled two-dimensional points to form the two-dimensional points 1104A-N.

[0075] The model generation module 210 generates a 3D point cloud model by converting two-dimensional points 1104A-N in the high-dimensional image vector 908 into three-dimensional points 1106A-N. The model generation module 210 then generates a new 3D CAD model of the object based on the 3D point cloud model.

[0076] The multilayer perceptron 1102A-N comprises five fully connected layers of sizes 4096, 1024, 516, 256, and 128, with Rectified Linear Units (ReLUs) on the first four layers but not on the last fifth layer (e.g., the output layer). The multilayer perceptron 1102A-N is trained to generate N number of 3D surface patch points from input data (e.g., image vectors concatenated with sampled 2D points). The trained multilayer perceptron 1102A-N is evaluated using a Chamfer distance loss by measuring the difference between the generated 3D surface patch points and the nearest ground truth 3D surface patch point. The multilayer perceptron 1102A-N is considered trained when the difference between the generated 3D surface patch points and the nearest ground truth 3D surface patch point is within acceptable limits or negligible. The trained multilayer perceptron 1102A-N can accurately generate 3D surface patch points corresponding to 2D points in the image vector of a 2D image of an object.

[0077] It should be understood that the systems and methods described herein can be implemented in various forms of hardware, software, firmware, dedicated processing units, or combinations thereof. One or more of the embodiments may take the form of a computer program product, including program modules accessible from a computer-usable or computer-readable medium storing program code for use by or in connection with one or more computers, processing units, or instruction execution systems. For the purposes of this description, a computer-usable or computer-readable medium can be any means that can contain, store, transmit, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device), or a propagation medium therein and its own propagation medium, because signal carriers are not included in the definition of a physical computer-readable medium, which includes semiconductor or solid-state memory, magnetic tape, removable computer disk, random access memory (RAM), read-only memory (ROM), rigid disk, optical disk, such as a compact disc read-only memory (CD-ROM), compact disc read / write, digital versatile disc (DVD), or any combination thereof. As is known to those skilled in the art, the processing units and program code used to implement each aspect of the technology can both be centralized or distributed (or a combination thereof).

[0078] While this disclosure has been described in detail with reference to certain embodiments, it is not limited to those embodiments. In view of this disclosure, many modifications and variations will appear to those skilled in the art without departing from the scope of the various embodiments of this disclosure as described herein. Therefore, the scope of this disclosure is indicated by the following claims rather than by the foregoing description. All changes, modifications, and variations within the equivalent meaning and scope of the claims are to be considered within that scope.

[0079] It should be understood that the elements and features recited in the appended claims can be combined in different ways to produce new claims that also fall within the scope of this disclosure. Therefore, although the dependent claims appended below are subordinate to only a single independent or dependent claim, it should be understood that these dependent claims can alternatively be made to be subordinate to any preceding or following claim, whether independent or dependent, and such new combinations should be understood to form part of this specification.

Claims

1. A method for providing a three-dimensional computer-aided design CAD model from a two-dimensional image of an object in a CAD environment, the method comprising: The data processing system receives a request for a three-dimensional CAD model of a physical object, wherein the request includes the two-dimensional image of the object; An image vector is generated from the two-dimensional image using a first trained machine learning algorithm, wherein generating the image vector includes preprocessing the two-dimensional image to generate a three-dimensional image matrix, and transforming the three-dimensional image matrix into a high-dimensional image vector using a trained VGG convolutional neural network having a stack of convolutional layers followed by two fully connected FC layers, and the stack of convolutional layers processes the three-dimensional image matrix, wherein the three-dimensional image matrix is ​​generated by resizing and normalizing the two-dimensional image; A second trained machine learning algorithm is used to generate three-dimensional points for each two-dimensional point in the generated image vector; A 3D point cloud model of the object is generated based on the generated 3D points; Use the 3D point cloud model of the object to generate a 3D CAD model of the object; and Output the three-dimensional CAD model of the object on the graphical user interface.

2. The method according to claim 1, further comprising storing the generated image vectors of the three-dimensional point cloud model and the two-dimensional image of the object in a geometric model database.

3. The method according to claim 2, further comprising: Receive a request for a 3D CAD model of the object, wherein the request includes a 2D image of the object; Image vectors are generated from the two-dimensional image using a first trained machine learning algorithm; Based on the generated image vectors, a search for a 3D CAD model of the object is performed in a geometric model database that includes multiple 3D CAD models; Determine whether the 3D CAD model of the object has been successfully found in the geometric model database; as well as Output the three-dimensional CAD model of the object on the graphical user interface.

4. The method of claim 3, wherein performing a search for a 3D CAD model of the object in a geometric model database using a third trained machine learning algorithm comprises: A third machine learning algorithm is used to compare the generated image vector of the two-dimensional image with each image vector associated with the corresponding three-dimensional CAD model in the geometric model database; as well as The 3D CAD model is identified from the geometric model database based on the best match between the generated image vector and the image vector of the 3D CAD model.

5. A method for providing a three-dimensional computer-aided design CAD model from a two-dimensional image of an object in a CAD environment, the method comprising: The data processing system receives a request for a three-dimensional CAD model of an object, wherein the request includes the two-dimensional image of the object; An image vector is generated from the two-dimensional image using a first trained machine learning algorithm, wherein generating the image vector includes preprocessing the two-dimensional image to generate a three-dimensional image matrix, and transforming the three-dimensional image matrix into a high-dimensional image vector using a trained VGG convolutional neural network having a stack of convolutional layers followed by two fully connected FC layers, and the stack of convolutional layers processes the three-dimensional image matrix, wherein the three-dimensional image matrix is ​​generated by resizing and normalizing the two-dimensional image; Based on the generated image vectors, a search for a 3D CAD model of the object is performed in a geometric model database that includes multiple 3D CAD models; Determine whether the requested 3D CAD model of the object has been successfully found in the geometric model database; as well as When the requested 3D CAD model of the object is successfully found in the geometric model database, the requested 3D CAD model of the object is output on the graphical user interface.

6. The method of claim 5, further comprising: When the requested 3D CAD model of the object is not found in the geometric model database, a second trained machine learning algorithm is used to generate a 3D CAD model of the object based on the generated image vectors; and The generated 3D CAD model of the object is output on the graphical user interface.

7. The method of claim 6, wherein generating a 3D CAD model of the object using a second trained machine learning model based on the generated image vectors comprises: A second trained machine learning algorithm is used to generate a 3D point cloud model of the object based on the generated image vectors; as well as A 3D CAD model of the object is generated using the object's 3D point cloud model.

8. The method according to claim 7, further comprising storing the generated three-dimensional CAD model of the object and the generated image vector of the corresponding two-dimensional image of the object in a geometric model database.

9. The method of claim 5, wherein performing a search for a 3D CAD model of the object in a geometric model database comprises performing a search for a 3D CAD model of the object in the geometric database using a third trained machine learning algorithm.

10. The method of claim 9, wherein performing a search for a 3D CAD model of the object in a geometric model database using a third trained machine learning algorithm comprises: The generated image vector of the two-dimensional image is compared with each image vector associated with the corresponding geometric model in the geometric model database using a third machine learning algorithm. as well as Based on the matching between the generated image vector and the image vectors of one or more 3D CAD models, the one or more 3D CAD models are identified from the geometric model database.

11. The method of claim 10, further comprising: The one or more 3D CAD models are sorted based on their matching with the requested 3D CAD model of the object. as well as Based on the sorting of the one or more 3D CAD models, at least one 3D CAD model is determined to have an image vector that best matches the generated image vector of the 2D image.

12. The method of claim 11, further comprising modifying the determined three-dimensional CAD model based on the generated image vector of the two-dimensional image.

13. A data processing system for providing a three-dimensional computer-aided design CAD model from a two-dimensional image of an object in a CAD environment, comprising: Processing unit; as well as A memory unit coupled to the processing unit, wherein the memory unit includes a CAD module, the CAD module being configured to: Receive a request for a three-dimensional computer-aided design CAD model of an object, wherein the request includes the two-dimensional image of the object; An image vector is generated from the two-dimensional image using a first trained machine learning algorithm, wherein generating the image vector includes preprocessing the two-dimensional image to generate a three-dimensional image matrix, and transforming the three-dimensional image matrix into a high-dimensional image vector using a trained VGG convolutional neural network having a stack of convolutional layers followed by two fully connected FC layers, and the stack of convolutional layers processes the three-dimensional image matrix, wherein the three-dimensional image matrix is ​​generated by resizing and normalizing the two-dimensional image; Based on the generated image vectors, a search for a 3D CAD model of the object is performed in a geometric database that includes multiple 3D CAD models; Determine whether the requested 3D CAD model of the object has been successfully found in the geometric model database; as well as When the requested 3D CAD model of the object is successfully found in the geometric model database, the requested 3D CAD model of the object is output on the graphical user interface.

14. The data processing system according to claim 13, wherein the CAD module is configured to: When the requested 3D CAD model of the object is not found in the geometric model database, a second trained machine learning algorithm is used to generate a 3D CAD model of the object based on the generated image vectors; and The generated 3D CAD model of the object is output on the graphical user interface.

15. The data processing system of claim 14, wherein in generating a 3D CAD model of the object based on the generated image vectors using a second trained machine learning model, the CAD module is configured to: A second trained machine learning algorithm is used to generate a 3D point cloud model of the object based on the generated image vectors; and A 3D CAD model of the object is generated using the object's 3D point cloud model.

16. The data processing system of claim 15, wherein the CAD module is configured to store the generated three-dimensional CAD model of the object and the generated image vector of the corresponding two-dimensional image of the object in a geometric model database.

17. The data processing system of claim 13, wherein in performing a search for a 3D CAD model of the object in a geometric model database, the CAD module is configured to perform the search for a 3D CAD model of the object in the geometric database using a third trained machine learning algorithm.

18. The data processing system of claim 17, wherein in performing a search for a 3D CAD model of the object in a geometric model database using a third trained machine learning algorithm, the CAD module is configured to: A third machine learning algorithm is used to compare the generated image vector of the two-dimensional image with each image vector associated with a corresponding geometric model in a geometric model database; and Based on the matching between the generated image vector and the image vectors of one or more 3D CAD models, the one or more 3D CAD models are identified from the geometric model database.

19. The data processing system of claim 18, wherein the CAD module is configured to: Based on the matching with the requested 3D CAD model of the object, the identified one or more 3D CAD models are sorted; and Based on the sorting of the one or more 3D CAD models, at least one 3D CAD model is determined to have an image vector that best matches the generated image vector of the 2D image.

20. The data processing system of claim 19, wherein the CAD module is configured to modify the determined three-dimensional CAD model based on the generated image vectors of the two-dimensional image.

Images