Multi-modal encoding, multi-modal fusion, and multi-modal decoding for CAD data

Abstract

This patent application discloses a computing system to identify multi-modal input data that describes features of a mechanical design for a product, and to separately encode the input data on a per-modality basis into modality-specific tokens configured for consumption by a machine-learning algorithm implemented by the computing system. The computing system implementing the machine-learning algorithm can analyze the modality-specific tokens together to generate output tokens embedded with a set of domain-specific operations. The computing system can decode the domain-specific operations embedded in the output tokens to determine one or more tasks with corresponding instructions configured for implementation by a mechanical design tool developing the mechanical design for the product. The mechanical design tool can utilize the tasks having the corresponding instructions to predict a domain-specific feature for the mechanical design.

Description

202421173 FOUNDATION MODEL FOR MECHANICAL DESIGNRELATED APPLICATION

[0001] This application claims priority and benefit of U.S. Provisional Patent Application No. 63 / 728,262, filed, December 5, 2024, which is incorporated by reference herein in its entirety.TECHNICAL FIELD

[0002] This application is generally related to mechanical design automation and, more specifically, to utilizing a foundation model to perform multiple interdependent tasks in mechanical design development.BACKGROUND

[0003] Mechanical design of products typical involves performing multiple interdependent tasks, such as sketching, three dimensional (3D) modeling, drafting, assembly, reverse engineering, design refactoring, simulation, or the like. These mechanical design tasks usually rely on a variety of data inputs, in differing representations and / or modalities. For example, the mechanical design task of sketching typically involves operating on two dimensional (2D) profiles associated with a product, 3D modeling tasks operate with 3D geometries associated with the product, assembly design tasks work with multiple different geometries, or the like. Performance of any individual mechanical design task can be complicated, including working with multiple input design representations and needing to202421173 understand how to manipulate those input to create an output of the product. For example, when a mechanical design task reverse-engineers a 3D geometry from multiple 2D design input, a designer may analyze the 2D design inputs, such as a technical drawing, an associated image of the product, or the like, and accurately create the 3D geometry of the product from those inputs. Some designers utilize computing systems, for example, including some machine learning (ML) and / or artificial intelligence (AI) algorithms, to help perform at least a portion of individual mechanical design tasks, for example, feature prediction using sequential models, assembly mate prediction, or the like. While the utilization of computing systems with AI / ML algorithms has aided designers in performing those individual tasks, the siloed approach to task performance that ignores the interdependence of these mechanical design tasks, can lead to task outputs for the product that conflict, are incongruent, or correspond to sub-optimal product designs.SUMMARY

[0004] This application discloses a computing system to identify multi-modal input data that describes features of a mechanical design for a product, and to separately encode the input data on a per-modality basis into modality-specific tokens configured for consumption by a machine -learning algorithm implemented by the computing system. The computing system implementing the machine-learning algorithm can analyze the modality-specific tokens together to generate output tokens embedded with a set of domain- specific operations. The computing system can decode the domain- specific operations embedded in the output tokens to determine one or more tasks with corresponding instructions configured for implementation by a mechanical design tool developing the mechanical202421173 design for the product. The mechanical design tool can utilize the tasks having the corresponding instructions to predict a domain- specific feature for the mechanical design. Embodiments will be described in greater detail below.DESCRIPTION OF THE DRAWINGS

[0005] Figures 1 and 2 illustrate an example of a computer system of the type that may be used to implement various embodiments.

[0006] Figure 3 illustrates an example of a mechanical design system having a foundation model for mechanical design prediction using multi-modal input according to various embodiments.

[0007] Figure 4 illustrates an example of mechanical design prediction using multi-modal input according to various embodiments.

[0008] Figure 5 illustrates an example of a multi-modal encoding system of a mechanical design system according to various embodiments.

[0009] Figure 6 illustrates an example of a multi-domain decoding system of a mechanical design system according to various embodiments.

[0010] Figure 7 illustrates an example flowchart for multi-domain decoding system of output tokens from a foundation model according to various embodiments.

[0011] Figure 8 illustrates an example of a shape retrieval augmented generation system in the multi-domain decoding system according to various embodiments.202421173DETAILED DESCRIPTIONIllustrative Operating Environment

[0012] Various embodiments may be implemented through the execution of software instructions by a computing device 101, such as a programmable computer. Accordingly, Figure 1 shows an illustrative example of a computing device 101. As seen in this figure, the computing device 101 includes a computing unit 103 with a processing unit 105 and a system memory 107. The processing unit 105 may be any type of programmable electronic device for executing software instructions, but will conventionally be a microprocessor. The system memory 107 may include both a read-only memory (ROM) 109 and a random access memory (RAM) 111. As will be appreciated by those of ordinary skill in the art, both the read-only memory (ROM) 109 and the random access memory (RAM) 111 may store software instructions for execution by the processing unit 105.

[0013] The processing unit 105 and the system memory 107 are connected, either directly or indirectly, through a bus 113 or alternate communication structure, to one or more peripheral devices 117-123. For example, the processing unit 105 or the system memory 107 may be directly or indirectly connected to one or more additional memory storage devices, such as a hard disk drive 117, which can be magnetic and / or removable, a removable optical disk drive 119, and / or a flash memory card. The processing unit 105 and the system memory 107 also may be directly or indirectly connected to one or more input devices 121 and one or more output devices 123. The input devices 121 may include, for example, a keyboard, a pointing device (such as a mouse, touchpad, stylus, trackball, or joystick), a202421173 scanner, a camera, and a microphone. The output devices 123 may include, for example, a monitor display, a printer and speakers. With various examples of the computing device 101, one or more of the peripheral devices 117-123 may be internally housed with the computing unit 103. Alternately, one or more of the peripheral devices 117-123 may be external to the housing for the computing unit 103 and connected to the bus 113 through, for example, a Universal Serial Bus (USB) connection.

[0014] With some implementations, the computing unit 103 may be directly or indirectly connected to a network interface 115 for communicating with other devices making up a network. The network interface 115 can translate data and control signals from the computing unit 103 into network messages according to one or more communication protocols, such as the transmission control protocol (TCP) and the Internet protocol (IP). Also, the network interface 115 may employ any suitable connection agent (or combination of agents) for connecting to a network, including, for example, a wireless transceiver, a modem, or an Ethernet connection. Such network interfaces and protocols are well known in the art, and thus will not be discussed here in more detail.

[0015] It should be appreciated that the computing device 101 is illustrated as an example only, and it is not intended to be limiting. Various embodiments may be implemented using one or more computing devices that include the components of the computing device 101 illustrated in Figure 1, which include only a subset of the components illustrated in Figure 1, or which include an alternate combination of components, including components that are not shown in Figure 1. For example, various embodiments may be implemented using a202421173 multi-processor computer, a plurality of single and / or multiprocessor computers arranged into a network, or some combination of both.

[0016] With some implementations, the processor unit 105 can have more than one processor core. Accordingly, Figure 2 illustrates an example of a multi-core processor unit 105 that may be employed with various embodiments. As seen in this figure, the processor unit 105 includes a plurality of processor cores 201A and 201B. Each processor core 201A and 201B includes a computing engine 203A and 203B, respectively, and a memory cache 205A and 205B, respectively. As known to those of ordinary skill in the art, a computing engine 203A and 203B can include logic devices for performing various computing functions, such as fetching software instructions and then performing the actions specified in the fetched instructions. These actions may include, for example, adding, subtracting, multiplying, and comparing numbers, performing logical operations such as AND, OR, NOR and XOR, and retrieving data. Each computing engine 203A and 203B may then use its corresponding memory cache 205A and 205B, respectively, to quickly store and retrieve data and / or instructions for execution.

[0017] Each processor core 201A and 201B is connected to an interconnect 207. The particular construction of the interconnect 207 may vary depending upon the architecture of the processor unit 105. With some processor cores 201A and 201B, such as the Cell microprocessor created by Sony Corporation, Toshiba Corporation and IBM Corporation, the interconnect 207 may be implemented as an interconnect bus. With other processor units 201A and 201B, however, such as the Opteron™ and Athlon™ dual-core processors available from Advanced Micro Devices of Sunnyvale, California, the interconnect 207 may202421173 be implemented as a system request interface device. In any case, the processor cores 201A and 201B communicate through the interconnect 207 with an input / output interface 209 and a memory controller 210. The input / output interface 209 provides a communication interface to the bus 113. Similarly, the memory controller 210 controls the exchange of information to the system memory 107. With some implementations, the processor unit 105 may include additional components, such as a high-level cache memory accessible shared by the processor cores 201A and 201B. It also should be appreciated that the description of the computer network illustrated in Figure 1 and Figure 2 is provided as an example only, and it is not intended to suggest any limitation as to the scope of use or functionality of alternate embodiments.Foundation Model for Mechanical Design

[0018] Figure 3 illustrates an example of a mechanical design system 300 having a foundation model for mechanical design prediction using multi-modal input according to various embodiments. Figure 4 illustrates an example of mechanical design prediction using multi-modal input according to various embodiments. Referring to Figures 3 and 4, the mechanical design system 300 can include a multi-modal encoding system 310 that, in a block 401 of Figure 4, can identify multi-modal input data 301 describing features of a mechanical design for a product. The input data 301 can include 3D geometric representations of a part or mechanical device, for example, boundary representations of a 3D shape (B-Reps), point clouds, voxels, or the like. The input data 301 also can include computer-aided design (CAD) features, such as feature trees, to provide a structured representation of steps taken to create a part or mechanical device. The input data 301 can202421173 include manufacturing information, such as material indicators, product manufacturing information, or the like, and simulation information, such as 3D fields, or the like.

[0019] The multi-modal encoding system 310, in a block 402 of Figure 4, can separately encode the input data 301 on a per-modality basis into modality- specific input tokens 311 configured for consumption by a foundation model system 320 that includes a machinelearning algorithm. The multi-modal encoding system 310 can include a plurality of modality- specific encoders, which can each process input data 301 having a particular modality or representation to create modality-specific encodings for the input tokens 311 and project the input tokens 311 into a data space corresponding to the machine-learning algorithm of the foundation model system 320. The modalities of the input data 310, which may be used interchangeably with representations of the input data 310, can correspond to perceivable input, such as sounds, images, or the like. Embodiments of the multi-modal encoding system 310 will be described below in greater detail with reference to Figure 5.

[0020] Figure 5 illustrates an example of a multi-modal encoding system 500 of a mechanical design system according to various embodiments. Referring to Figure 5, the multi-modal encoding system 500 can receive multi-modal input data 501 describing features of a mechanical design for a product, for example, as 3D geometric representations of a part or mechanical device, CAD features, manufacturing information, simulation information, or the like.

[0021] The multi-modal encoding system 500 can include sets of encoders, such as a geometry encoder 510 and a text encoder 520. The geometry encoder 510 can process202421173 portions of the multi-modal input data 501 corresponding to 3D geometry of the mechanical design to generate corresponding tokens. The geometry encoder 510 can include a plurality of modality-specific encoders 511-1 to 511-M, each to process input data 301 having a particular modality to create modality- specific encodings for the tokens. For example, the modality- specific encoders 511-1 to 511-M can include encoders that process boundary representation data, part material data, point clouds associated with the mechanical design, voxel data, 3D fields, model-based definition (MBD) notes, product manufacturing information, drawings, CAM program data, feature graphs, design context data, or the like. In some embodiments, the modality-specific encoders 511-1 to 511-M can include transformation functions that operate on the particular modalities of the multi-modal input data 501 to create encodings for tokens.

[0022] One or more of the modality- specific encoders 511-1 to 511-M, in some embodiments, can process multi-modal input data 501 for use in a retrieval-augmented generation system incorporated into the mechanical design system. The modality-specific encoders 511-1 to 511-M can utilize a native or neutral representation of 3D shapes within the multi-modal input data 501 and possibly utilize an application context or design requirements included in the multi-modal input data 501 to create tokens having shape and / or context embeddings. As will be described below in greater detail, the tokens created by the modality- specific encoders 511-1 to 511-M can be converted into vector representations configured for insertion into the vector database during queries of the retrieval-augmented generation system.202421173

[0023] The text encoder 520 can process portions of the multi-modal input data 501 corresponding to text or language-based input from users and then generate corresponding tokens from the text. In some embodiments, the text or language-based input can correspond to a natural language description of the overall project to be performed by a mechanical design system that includes the multi-modal encoding system 500, and include an indication of a portion of a mechanical design for the mechanical design system to predict.

[0024] The multi-modal encoding system 500 can include a space projection system 540 to convert the tokens from the geometry encoder 510 and the text encoder 520 into input tokens 502, which projects the tokens into a data space corresponding to the machinelearning algorithm of the foundation model system. In some embodiments, the tokens from the geometry encoder 510 and the text encoder 520 can have different data sizes and / or correspond to different data spaces, the space projection system 540 can transform the tokens into a common data space to generate the input tokens 502.

[0025] The space projection system 540, in some embodiments, can include a machine learning model trained in a manner such that the projected tokens belong to the same dimensionality, for example, that the projected tokens lie in the same data space. The projection system 540 can be trained to have projection layers, one for each of the modalityspecific encoders 511-1 to 511-M in the multi-modal encoding system 500. To train the projection system 540, weights of the foundation model 320, conventional encoders, such as the text encoder 520, or the like, and the modality- specific encoders 511-1 to 511-M can be frozen, while weights of the projection layer associated with a specific modality of the multi-202421173 modal input data 501 can be altered according to data being used to the train the projection system 540. The training of the space projection system 540 can be implemented to allow the foundation model system 320 to correctly generate output tokens 321 in response to the input tokens 502 from the space projection system 540. In some embodiments, the space projection system 540 can be a latent diffusion-based projection system, which can convert the tokens from the geometry encoder 510 and the text encoder 520 into the input tokens 502, which projects the tokens into a data space corresponding to the machine-learning algorithm of the foundation model system.

[0026] Referring back to Figures 3 and 4, the mechanical design system 300 can include a foundation model system 320 including a machine-learning algorithm that, in a block 403 of Figure 4, can analyze the modality- specific tokens together to generate output tokens 321 embedded with a set of domain- specific operations. The input tokens 311 can be modalityspecific representations of the multi-modal input data 301 that have been projected into a common data space for consumption by the machine-learning algorithm in the foundation model system 320. The output tokens 321 can correspond to a series of tokens in a unified representation, abstracted over the tasks involved in mechanical design.

[0027] In some embodiments, the foundation model system 320 also can generate shape vectors 322 from at least some of the input tokens 311. The shape vectors 322 can correspond to vector representations of shape and / or design context information in the multi-modal input data 301 that has been configured for insertion into the vector database during queries of a retrieval-augmented generation system incorporated in the mechanical design system 300.202421173

[0028] The mechanical design system 300 can include a multi-domain decoding system 330 that, in a block 404 of Figure 4, can decode the domain-specific operations embedded in the output tokens 321 to determine one or more tasks 332 with corresponding instructions configured for implementation by the mechanical CAD system 340.

[0029] The multi-domain decoding system 330 also can decode the shape vectors 322, for example, by implementing a retrieval augmented generation system, which can generate predicted shape characteristics 336 for the mechanical CAD system 340. In some embodiments, the multi-domain decoding system 330 can utilize the shape vectors 322 to query a vector database to identify characteristics of mechanical designs similar to the characteristics associated with the design shapes and / or context represented the shape vectors 322. The multi-domain decoding system 330 can utilize the results of the vector database query, such as the geometric data and / or context data associated with a shape along with its characteristics, to generate the predicted shape characteristics 336 for the shape vectors 322. Embodiments of the multi-domain decoding system 330 will be described below in greater detail with reference to Figures 6-8.

[0030] Figure 6 illustrates an example of a multi-domain decoding system 600 of a mechanical design system according to various embodiments. Figure 7 illustrates an example flowchart for multi-domain decoding system of output tokens from a foundation model according to various embodiments. Referring to Figures 6 and 7, the multi-domain decoding system 600, in a block 701 of Figure 7, can receive output tokens 601 embedded with domain- specific operations generated by a machine-learning algorithm. In some embodiments, the output tokens 601 can include multiple series of tokens that each can202421173 correspond to different instructions, defined based on the semantics and syntax of the series of the tokens. Each of the different series of tokens can include a domain identification token, which can annunciate a domain associated with the instruction in the respective series of tokens. The remaining tokens in each series can correspond to a tokenized representation of those instructions.

[0031] The multi-domain decoding system 600 can include a geometry decoder 610 that, in a block 702 of Figure 7, can parse the output tokens 601 to locate tokens identifying different types of decoders to process the corresponding domain- specific operations. In some embodiments, after the geometry decoder 610 locates the domain identification token within the output tokens 601 and select the type of decoding to utilize to decode a portion of the output tokens 601 based on the domain identification token.

[0032] The geometry decoder 610, in a block 703 of Figure 7, can separately decode the domain-specific operations from the output tokens 601 using the decoding specified in the domain-specific operations, which generates instructions. In some embodiments, the geometry decoder 610 can include a plurality of domain-specific decoders 611-1 to 611-N, each to implementing a different type of decoding, which can transform the embedding from the output tokens 601 into instructions. For example, the domain-specific decoders 611-1 to 611-N can process different portions of the output tokens 601 to generate instructions corresponding to boundary representation data, part material data, point clouds associated with the mechanical design, voxel data, 3D fields, model-based definition (MBD) notes, product manufacturing information, drawings, CAM program data, feature graphs, design context data, or the like.202421173

[0033] The geometry decoder 610, in a block 704 of Figure 7, can group sets of the instructions to form tasks 603 based on a syntax and schematics of the output tokens 601. In some embodiments, the tasks 603 can correspond to part design tasks, such as part selection tasks, feature tasks to identify feature, entity tasks, or the like, which can provide various predictions for part, assembly, manufacture characteristic to a mechanical design tool developing the mechanical design for a product.

[0034] The multi-domain decoding system 600 can include a translation system 620 that, in a block 705 of Figure 7, can convert the tasks 603 into a configuration capable of implementation by the mechanical design tool developing the mechanical design for a product. In some embodiments, the tasks 603 generated by the geometry decoder 610 can be represented at a tool-agnostic level of abstraction, and the domain translation system 620 can convert the tasks into a format capable of execution or consumption by the mechanical design tool.

[0035] The multi-domain decoding system 600 can include a shape retrieval augmented generation system 630 to receive shape vectors 602 corresponding to vector representations of shape and / or design context information for a mechanical part or design. The shape retrieval augmented generation system 630 can utilize the shape vectors 602 to perform a query of a vector database to identify characteristics of mechanical designs similar to the characteristics associated with the design shapes and / or context represented the shape vectors 602. The shape retrieval augmented generation system 630 can utilize the results of the vector database query, such as the geometric data and / or context data associated with a shape along with its characteristics, to generate the predicted shape characteristics202421173 604 for the shape vectors 602. Embodiments of the shape retrieval assisted generation system 630 will be described below in greater detail with reference to Figure 8.

[0036] Figure 8 illustrates an example of a shape retrieval augmented generation system 800 in the multi-domain decoding system according to various embodiments. Referring to Figure 8, the shape retrieval augmented generation system 800 can receive shape vectors 801 that can correspond to vector representations of shape data and / or context data for a mechanical design. In some embodiments, the shape data can correspond to measurements or descriptions of facets, faces, edges, or the like, of a part or design shape in the mechanical design. The context data can correspond to information, such as a deployment of a product associated with the mechanical design, assembly annotations including reference origins, paste assembly points, or the like, part annotations including dimensions, tolerances, or the like. As discussed above, the shape data and the context data can be encoded into input tokens for consumption by a foundation model implemented in a machine-learning algorithm. The foundation model can generate the shape vectors 801 by processing the input tokens associated with the shape data and the context data of the mechanical design.

[0037] The shape retrieval augmented generation system 800 can include a shape retrieval system 810 to utilize the shape vectors 801 to query a vector database 830 to identify characteristics of mechanical designs similar to the characteristics associated with the shape data and / or context data represented the shape vectors 801. The vector database 830 can be populated with data records including sets of vectorized geometric data, vectorized context data, and characteristic data for different parts or devices. In some embodiments,202421173 the vector database 830 can receive geometry vectors 831, context vectors 832, and characteristics labels 833, which can generate record entries of the characteristics labels 833, indexable by the geometry vectors 831 and context vectors 832 in the vector database 830.

[0038] The shape retrieval system 810 can send an embedding query 811 to the vector database 830, which can include the shape vectors 801 corresponding to vectorized representations of the shape data and possibly the context data. The vector database 830, in response to the embedding query 811, can utilize the shape vectors 801 to access one or more of its record entries, for example, by indexing the shape vectors 801 with the record entries. The vector database 830 can output a query response 812 to the embedding query 811, which can include the record entries that the vector database 830 accessed in response to the shape vectors 801 in the embedding query 811.

[0039] The shape retrieval augmented generation system 800 can include a shape retrieval system 820 to utilize the shape vectors 801, and the contents of the query response 812, namely, one or more geometric vectors 831, context vectors 832, and characteristics labels 833 to generate a predicted shape characteristics 802 that correspond to the shape vectors 801. In some embodiments, the shape decoder 820 can implement a generative artificial intelligence (Al) algorithm to analyze a relationship between the geometric vectors 831, context vectors 832, and characteristics labels 833 to identify one or more characteristics to associate with the shape vectors 801, and then output the identified characteristics as the predicted shape characteristic 802. In some embodiments, a foundation model system can implement the generative Al algorithm in the shape decoder 820.202421173

[0040] Referring back to Figures 3 and 4, the mechanical CAD system 340 that, in a block 405 of Figure 4, can utilize the tasks 332 having the corresponding instructions to predict a domain-specific feature for the mechanical design. In some embodiments, the mechanical CAD system 340 can utilize the tasks 332 for predictions of part, assembly, manufacture characteristics that can be implemented in the development of the mechanical design for a product.

[0041] The system and apparatus described above may use dedicated processor systems, micro controllers, programmable logic devices, microprocessors, or any combination thereof, to perform some or all of the operations described herein. Some of the operations described above may be implemented in software and other operations may be implemented in hardware. Any of the operations, processes, and / or methods described herein may be performed by an apparatus, a device, and / or a system substantially similar to those as described herein and with reference to the illustrated figures.

[0042] The processing device may execute instructions or "code" stored in memory. The memory may store data as well. The processing device may include, but may not be limited to, an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, or the like. The processing device may be part of an integrated control system or system manager, or may be provided as a portable electronic device configured to interface with a networked system either locally or remotely via wireless transmission.202421173

[0043] The processor memory may be integrated together with the processing device, for example RAM or FLASH memory disposed within an integrated circuit microprocessor or the like. In other examples, the memory may comprise an independent device, such as an external disk drive, a storage array, a portable FLASH key fob, or the like. The memory and processing device may be operatively coupled together, or in communication with each other, for example by an I / O port, a network connection, or the like, and the processing device may read a file stored on the memory. Associated memory may be "read only" by design (ROM) by virtue of permission settings, or not. Other examples of memory may include, but may not be limited to, WORM, EPROM, EEPROM, FLASH, or the like, which may be implemented in solid state semiconductor devices. Other memories may comprise moving parts, such as a known rotating disk drive. All such memories may be "machine-readable" and may be readable by a processing device.

[0044] Operating instructions or commands may be implemented or embodied in tangible forms of stored computer software (also known as "computer program" or "code"). Programs, or code, may be stored in a digital memory and may be read by the processing device. “Computer-readable storage medium" (or alternatively, "machine-readable storage medium") may include all of the foregoing types of memory, as well as new technologies of the future, as long as the memory may be capable of storing digital information in the nature of a computer program or other data, at least temporarily, and as long at the stored information may be "read" by an appropriate processing device. The term "computer-readable" may not be limited to the historical usage of "computer" to imply a complete mainframe, mini- computer, desktop or even laptop computer. Rather, "computer-readable"202421173 may comprise storage medium that may be readable by a processor, a processing device, or any computing system. Such media may be any available media that may be locally and / or remotely accessible by a computer or a processor, and may include volatile and non-volatile media, and removable and non- removable media, or any combination thereof.

[0045] A program stored in a computer-readable storage medium may comprise a computer program product. For example, a storage medium may be used as a convenient means to store or transport a computer program. For the sake of convenience, the operations may be described as various interconnected or coupled functional blocks or diagrams. However, there may be cases where these functional blocks or diagrams may be equivalently aggregated into a single logic device, program or operation with unclear boundaries.Conclusion

[0046] While the application describes specific examples of carrying out embodiments, those skilled in the art will appreciate that there are numerous variations and permutations of the above described systems and techniques that fall within the spirit and scope of the invention as set forth in the appended claims. For example, while some of the specific terminology has been employed above to refer to electronic design automation processes, it should be appreciated that various examples may be implemented using any electronic system.

[0047] One of skill in the art will also recognize that the concepts taught herein can be tailored to a particular application in many other ways. In particular, those skilled in the202421173 art will recognize that the illustrated examples are but one of many alternative implementations that will become apparent upon reading this disclosure.

[0048] Although the specification may refer to “an”, “one”, “another”, or “some” example(s) in several locations, this does not necessarily mean that each such reference is to the same example(s), or that the feature only applies to a single example.

Claims

202421173CLAIMS1. A method comprising:identifying, by a computing system, input data describing features of a mechanical design for a product, wherein the input data has multiple different modalities;separately encoding, by the computing system, the input data on a per- modality basis into modality-specific tokens configured for consumption by a machine-learning algorithm implemented by the computing system;analyzing, by the machine-learning algorithm implemented by the computing system, the modality- specific tokens together to generate output tokens embedded with a set of domain- specific operations; anddecoding, by the computing system, the domain- specific operations embedded in the output tokens to determine one or more tasks with corresponding instructions configured for implementation by a mechanical design tool developing the mechanical design for the product.

2. The method of claim 1, wherein the mechanical design tool can utilize the tasks having the corresponding instructions to predict a domain- specific feature for the mechanical design.

3. The method of claim 1, wherein decoding the domain-specific operations in the output tokens further comprises:parsing the output tokens to identify the domain- specific operations;202421173 separately decoding the domain- specific operations to identify the instructions; and grouping the instructions to form the tasks configured for implementation by the mechanical design tool developing the mechanical design for the product.

4. The method of claim 3, wherein each of the domain-specific operations include at least one token corresponding to a type of decoder to process the corresponding domainspecific operations, and wherein separately decoding the domain-specific operations is performed by using the types of decoders specified in the domain- specific operations.

5. The method of claim 1, wherein decoding the output tokens further comprises: utilizing shape embeddings within the output tokens to query a vector database populated with geometry information, context information, and mechanical design characteristics corresponding to multiple different shapes;in response to the query, receiving one or more shape data sets including the geometry information, the context information, and the mechanical design characteristics that the vector database associated with the shape embeddings; andpredicting characteristics corresponding to the shape embeddings based, at least in part, on the shape data sets.

6. The method of claim 5, wherein the vector database is populated with the mechanical design characteristics, indexable by a geometry vector corresponding to the geometry information and a context vector corresponding to the context information, and202421173 wherein querying the vector database includes matching the shape embeddings against at least one of the geometry vector and context vector to identify the corresponding mechanical design characteristics.

7. The method of claim 5, wherein predicting characteristics corresponding to the shape embeddings based, at least in part, on the shape data sets is performed by a generative artificial intelligence algorithm implemented by the computing system.

8. An apparatus comprising at least one computer-readable memory device storing instructions configured to cause one or more processing devices to perform operations comprising:identifying input data describing features of a mechanical design for a product, wherein the input data has multiple different modalities;separately encoding the input data on a per-modality basis into modality- specific tokens configured for consumption by a machine-learning algorithm implemented by the processing devices;analyzing, by the machine-learning algorithm implemented by the processing devices, the modality- specific tokens together to generate output tokens embedded with a set of domain- specific operations; anddecoding the domain- specific operations embedded in the output tokens to determine one or more tasks with corresponding instructions configured for implementation by a mechanical design tool developing the mechanical design for the product.2024211739. The apparatus of claim 8, wherein the mechanical design tool can utilize the tasks having the corresponding instructions to predict a domain- specific feature for the mechanical design.

10. The apparatus of claim 8, wherein the instructions are configured to cause one or more processing devices to perform operations further comprises decoding the domainspecific operations in the output tokens by:parsing the output tokens to identify the domain- specific operations;separately decoding the domain- specific operations to identify the instructions; and grouping the instructions to form the tasks configured for implementation by the mechanical design tool developing the mechanical design for the product.

11. The apparatus of claim 10, wherein each of the domain- specific operations include at least one token corresponding to a type of decoder to process the corresponding domain-specific operations, and wherein separately decoding the domain- specific operations is performed by using the types of decoders specified in the domain- specific operations.

12. The apparatus of claim 8, wherein the instructions are configured to cause one or more processing devices to perform operations further comprises decoding the output tokens by:202421173 utilizing shape embeddings within the output tokens to query a vector database populated with geometry information, context information, and mechanical design characteristics corresponding to multiple different shapes;in response to the query, receiving one or more shape data sets including the geometry information, the context information, and the mechanical design characteristics that the vector database associated with the shape embeddings; andpredicting characteristics corresponding to the shape embeddings based, at least in part, on the shape data sets.

13. The apparatus of claim 12, wherein the vector database is populated with the mechanical design characteristics, indexable by a geometry vector corresponding to the geometry information and a context vector corresponding to the context information, and wherein querying the vector database includes matching the shape embeddings against at least one of the geometry vector and context vector to identify the corresponding mechanical design characteristics.

14. The apparatus of claim 12, wherein predicting characteristics corresponding to the shape embeddings based, at least in part, on the shape data sets is performed by a generative artificial intelligence algorithm implemented by the computing system.

15. A system comprising:a memory system configured to store computer-executable instructions; and202421173 a computing system, in response to execution of the computer-executable instructions, is configured to:identify input data describing features of a mechanical design for a product, wherein the input data has multiple different modalities;separately encode the input data on a per-modality basis into modalityspecific tokens configured for consumption by a machine-learning algorithm implemented by the computing system;analyze, by the machine-learning algorithm implemented by the computing system, the modality- specific tokens together to generate output tokens embedded with a set of domain-specific operations; anddecode the domain- specific operations embedded in the output tokens to determine one or more tasks with corresponding instructions configured for implementation by a mechanical design tool developing the mechanical design for the product.

16. The system of claim 15, wherein the mechanical design tool can utilize the tasks having the corresponding instructions to predict a domain- specific feature for the mechanical design.

17. The system of claim 15, wherein the computing system, in response to execution of the computer-executable instructions, is further configured to decode the domain- specific operations in the output tokens by:202421173 parsing the output tokens to identify the domain- specific operations; separately decoding the domain- specific operations to identify the instructions; and grouping the instructions to form the tasks configured for implementation by the mechanical design tool developing the mechanical design for the product.

18. The system of claim 17, wherein each of the domain-specific operations include at least one token corresponding to a type of decoder to process the corresponding domainspecific operations, and wherein separately decoding the domain-specific operations is performed by using the types of decoders specified in the domain- specific operations.

19. The system of claim 15, wherein the computing system, in response to execution of the computer-executable instructions, is further configured to decode the output tokens further by:utilizing shape embeddings within the output tokens to query a vector database populated with geometry information, context information, and mechanical design characteristics corresponding to multiple different shapes;in response to the query, receiving one or more shape data sets including the geometry information, the context information, and the mechanical design characteristics that the vector database associated with the shape embeddings; andpredicting characteristics corresponding to the shape embeddings based, at least in part, on the shape data sets.202421173 20. The system of claim 19, wherein the vector database is populated with the mechanical design characteristics, indexable by a geometry vector corresponding to the geometry information and a context vector corresponding to the context information, and wherein querying the vector database includes matching the shape embeddings against at least one of the geometry vector and context vector to identify the corresponding mechanical design characteristics.

Images