Semi-generative artificial intelligence

Abstract

The invention relates to semi-generative artificial intelligence. The invention provides inter-domain semi-generative artificial intelligence modeling. The method includes receiving a source image of an object in a first domain and diffusing the source image by a source diffusion model to generate a first Gaussian distribution in the first domain. An insert is generated from metadata that provides constraints for image reconstruction. The insert is fed into the double diffusion implicit bridge. Through a double diffused implicit bridge, a first Gaussian distribution is sampled and mapped from the first Gaussian distribution to a second Gaussian distribution in a second domain. Then, back diffusion is performed on the second Gaussian distribution through a target diffusion model to generate a target image of the object in the second domain according to the metadata.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to artificial intelligence systems, and more particularly to controlling random noise in generative AI models. BACKGROUND

[0002] Object detection systems implementing artificial intelligence are trained to accurately detect target objects using large realistic visual datasets with high accuracy, specificity, and diversity. High accuracy is achieved by reducing the bias of human labelers. High specificity is achieved by capturing different images of target objects under various environmental conditions. High diversity is achieved by including various images of target objects from various perspectives and angles. However, developing large realistic visual datasets with high accuracy, specificity, and diversity is a challenge, especially using conventional methods, i.e., manually taking photos of target objects using a camera and having an operator label each image with a ground truth target object class. These limitations slow down the development and deployment of large-scale object detection systems. SUMMARY

[0003] An illustrative embodiment provides a method for inter-domain semi- generative artificial intelligence modeling. The method includes receiving a source image of an object in a first domain and diffusing the source image through a source diffusion model to generate a first Gaussian distribution in the first domain. An embedding is generated from metadata that provides constraints for image reconstruction. The embedding is fed into a dual diffusion implicit bridge. The first Gaussian distribution is sampled and mapped from the first Gaussian distribution to a second Gaussian distribution in a second domain through the dual diffusion implicit bridge. The second Gaussian distribution is then inverse diffused through a target diffusion model to generate a target image of the object in the second domain according to the metadata.

[0004] Another illustrative embodiment provides a system for inter-domain semi- generative artificial intelligence modeling. The system includes a storage device storing program instructions and one or more processors operably connected to the storage device and configured to execute the program instructions to cause the system to receive a source image of an object in a first domain; diffuse the source image through a source diffusion model to generate a first Gaussian distribution in the first domain; generate an embedding from metadata, where the metadata provides constraints for image reconstruction; feed the embedding into a dual diffusion implicit bridge; sample from the first Gaussian distribution; map the sample from the first Gaussian distribution to a second Gaussian distribution in a second domain through the dual diffusion implicit bridge; and inverse diffuse the second Gaussian distribution through a target diffusion model to generate a target image of the object in the second domain according to the metadata.

[0005] Another illustrative embodiment provides a computer program product for inter-domain semi-generated artificial intelligence modeling. The computer program product includes a computer readable storage medium having program instructions embodied therewith, to perform the following steps: receiving a source image of an object in a first domain; diffusing the source image by a source diffusion model to generate a first Gaussian distribution in the first domain; generating an embedding from metadata, wherein the metadata provides constraints for image reconstruction; feeding the embedding to a dual diffusion implicit bridge; sampling from the first Gaussian distribution; mapping, by the dual diffusion implicit bridge, the sample from the first Gaussian distribution to a second Gaussian distribution in a second domain; and reverse diffusing the second Gaussian distribution by a target diffusion model to generate a target image of the object in the second domain according to the metadata.

[0006] The features and functionalities can be implemented independently in various embodiments of the present disclosure, or can be implemented in combination in other embodiments, in which additional details can be appreciated with reference to the following detailed description and the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0007] The novel features described herein are set forth with particularity in the claims that follow. These illustrative embodiments, together with further objectives, features and advantages thereof, will be best understood by reference to the following detailed description taken in connection with the accompanying drawings, in which:

[0008] Figure 1 A block diagram depicting a semi-generated system in accordance with illustrative embodiments is depicted;

[0009] Figure 2 A schematic diagram illustrating diffusion models that can be implemented by illustrative embodiments is depicted;

[0010] Figure 3 A schematic diagram illustrating an example of inter-domain image translation in accordance with illustrative embodiments is depicted;

[0011] Figure 4 A schematic diagram illustrating semi-generated AI image generation using metadata constraints in accordance with illustrative embodiments is depicted;

[0012] Figure 5 A schematic diagram illustrating an example of imaging decoding from a 3D CAD model to a photorealistic model in accordance with illustrative embodiments is depicted;

[0013] Figure 6 A flowchart depicting a process of inter-domain semi-generated AI modeling in accordance with illustrative embodiments is depicted; and

[0014] Figure 7 A diagram is a depiction of a block diagram of a data processing system in accordance with illustrative embodiments. DETAILED DESCRIPTION

[0015] Illustrative embodiments recognize and account for the lack of accuracy and validation layers for generative artificial intelligence (AI) models to validate the integrity of the generated content.

[0016] Illustrative embodiments provide a semi-generative AI modeling approach in which the content generated for generative AI models is augmented with additional layers of precision and fidelity. By leveraging metadata from design, production, and inspection data in manufacturing and production environments, illustrative embodiments control the output of generative AI models through a dual fusion technique.

[0017] Figure 1 is a block diagram of a semi-generative system depicted in accordance with illustrative embodiments. The semi-generative system 100 leverages data from various sources, including real images and videos of artifacts 102, three-dimensional (3D) computer-aided design (CAD) data 104, two-dimensional (2D) CAD drawings 106, and alternative signal modalities 108.

[0018] The real images and videos of artifacts 102 serve as ground truth data for real domain dataset generation. The 3D CAD data 104 provides another domain for image data. The 3D CAD data 104 can be exported from engineering product data management (PDM) and used for synthetic domain dataset generation.

[0019] The 2D CAD drawings 106 provide production and design engineering metadata (i.e., any data that is not a 3D object). The alternative signal modalities 108 provide additional metadata, such as textual data, audio data, or graphical representations that describe the real images and videos 102. The metadata provides constraints to ensure that the artificially intelligent generated images conform to desired characteristics. The constraints impose control on the output domain when converting image data from one domain to another (e.g., a particular type of object viewed from a particular angle). Such constraints help to ensure that the output images are as close as possible in all respects to the original input images.

[0020] The data and metadata from the real images and videos of artifacts 102, three-dimensional (3D) computer-aided design (CAD) data 104, two-dimensional (2D) CAD drawings 106, and alternative signal modalities 108 are fed to an unsupervised / semi-supervised model generator 110, which includes an end-to-end object detection system that uses structured and unstructured data to identify objects. The images are cropped and converted to latent space, and then clustered according to visual similarity. The clusters are then matched to features of provided examples, and labeled.

[0021] The data of different modalities are then fed by the unsupervised / semi-supervised model generator 110 to a dual fusion pipeline 116.

[0022] An artificial intelligence (AI) design parser 112 reads and interprets information about the constraints of the target object to be generated from production references 114 (e.g., engineering drawings) to match with real images and videos 102, providing additional constraints. The AI design parser 112 is able to provide information that can be missing in the alternative signal modalities 108. The AI design parser 112 can cross-reference text, geometry, engineering specifications, and mathematics against the real images and videos 102.

[0023] The dual fusion pipeline 116 includes a source diffusion model 118 trained on a first data domain 120 (e.g., 3D CAD). The source diffusion model 118 generates a first Gaussian distribution 126 in the first domain 120 from image data using a source Schrodinger bridge 136. (See Figure 2 and Figure 3 ). The diffusion Schrodinger bridge attempts to find a stochastic process that connects two probability distributions in the most likely way given the constraints.

[0024] The dual fusion pipeline 116 also includes a target diffusion model 122 trained on a second data domain 124 (e.g., real world). The target diffusion model 122 performs reverse diffusion using a target Schrodinger bridge 138 to generate image data in the second domain 124 from a second Gaussian distribution 128.

[0025] The dual diffusion implicit bridge 130 provides a translation between the first Gaussian distribution 126 and the second Gaussian distribution 128 by mapping from the first domain 120 to the second domain 124. (See Figure 4 ). The dual diffusion implicit bridge 130 is a back-to-back concatenation of the source to latent space and the latent space to target Schrodinger bridge 138. The dual diffusion implicit bridge 130 can incorporate embeddings 132 generated from metadata provided by the 2D CAD 106, the alternative signal modalities 108, and the AI design parser 112. The metadata represented by the embeddings 132 provide constraints when translating between Gaussian distributions of different domains.

[0026] The dual fusion pipeline 116 has an advantage of images having a higher FID (Frechet Inception Distance) score compared to other models. FID is a measure of quality of images created by generative models. FID compares the distribution of generated synthetic images to the distribution of a set of real images (the function of the real images is to serve as a benchmark of reality). A lower FID score indicates that there is greater similarity between the generated images and the real images. Diffusion models can generate lower FID scores than other types of models, such as generative adversarial networks (GANs).

[0027] The output of the dual fusion pipeline 116 is fed to a generative AI real synthetic data generator 136 that generates synthetic data for use by an object detection model.

[0028] Semi-generative system 100 can be implemented in software, hardware, firmware, or a combination thereof. When using software, operations performed by semi-generative system 100 can be implemented in program code configured to run on hardware, such as a processor unit. When using firmware, operations performed by semi-generative system 100 can be implemented in program code and data, and stored in persistent memory for running on a processor unit. When hardware is employed, the hardware can include circuitry for performing operations in semi-generative system 100.

[0029] In an illustrative example, the hardware can take the form of at least one selected from circuitry, an integrated circuit, an application-specific integrated circuit (ASIC), a programmable logic device, or other suitable type of hardware configured to perform a number of operations. With a programmable logic device, the device can be configured to perform a number of operations. The device can be reconfigured at a later time, or can be permanently configured to perform a number of operations. Programmable logic devices include, for example, programmable logic arrays, programmable array logic, field-programmable logic arrays, field-programmable gate arrays, and other suitable hardware devices. Moreover, these processes can be implemented in organic components that integrate with inorganic components and can consist entirely of organic components such as an organic semiconductor. For example, the processes can be implemented in an organic semiconductor as a circuit.

[0030] Computer system 150 is a physical hardware system and includes one or more data processing systems. When there is more than one data processing system in computer system 150, the data processing systems communicate with each other using a communication medium. The communication medium can be a network. The data processing systems can be selected from at least one of a computer, a server computer, a mobile device such as a tablet computer, or some other suitable data processing system.

[0031] As depicted, computer system 150 includes several processor units 152 that are capable of executing program code 154 that implements the processes in the illustrative examples. As used herein, a processor unit of several processor units 152 is a hardware device and is comprised of hardware circuitry on, for example, an integrated circuit that responds to and processes instructions and program code of an operating computer. When several processor units 152 execute program code 154 for a process, several processor units 152 are one or more processor units that can be on the same computer or on different computers. In other words, the process can be distributed among processor units on the same or different computers in the computer system. Moreover, several processor units 152 can be processor units of the same type or different types. For example, several processor units can be selected from at least one of a single-core processor, a dual-core processor, a multi-core processor, a general-purpose central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or some other type of processor unit.

[0032] Figure 2 A schematic diagram illustrating a diffusion model that can implement illustrative embodiments is depicted. Diffusion model 200 is an example of a first diffusion model 118 and a second diffusion model 122 in FIG. 1. Figure 1

[0033] Diffusion model 200 is trained by progressively adding noise to an image in stages (i.e., X1, X2, etc.) and then reconstructing the image from each stage (reverse diffusion). As diffusion model 200 learns, it is able to add more and more noise to the original image and successfully reconstruct the original image. In the last stage (Z) of training, diffusion model 200 generates a random Gaussian distribution of noise from which it can reconstruct the original image.

[0034] For a diffusion process in generative modeling, the goal is to design an algorithm that maps a given distribution P prior to a given data distribution P data :

[0035]

[0036] X t is interpreted as a noise process that perturbs the data distribution p data . p prior is the invariant distribution of the noise process. The process progressively increases noise, and if it is done in small steps for a sufficiently long time step T > 0, then X t approximates where the mean is 0 and the standard deviation is

[0037] ​For reverse diffusion, in principle, the process can be (approximately) first from p. prior Sample and invert X t The dynamics from P data Sampling. This time reversal operation causes another diffusion process with explicit drift and diffusion matrices:

[0038]

[0039] Where P t It is X t The density is then required. The process is then required to follow a Markov chain and a discretized diffusion process.

[0040] The main limitation of generative modeling is that it requires a large step size to ensure that the initial positive dynamic approximation distribution P is close. prior Furthermore, the step size is small enough to make the neural network approximate.

[0041] Adding a Schrödinger bridge can significantly reduce the number of steps required to define fraction-based generative modeling.

[0042] Assume reference diffusion X t Distributions that describe the process of adding noise to data The goal is to find π * , making and And minimize π * and Kullback-Leibler divergence between:

[0043]

[0044] Figure 3 An example of inter-domain image translation according to an illustrative embodiment is depicted. This image-to-image translation relies on a source diffusion model 302 and a target diffusion model 304, which are independently trained on separate domains A and B, respectively. In this example, domain A includes CAD image data, while domain B includes real-world images. Training the source diffusion model 302 and the target diffusion model 304 independently on each domain allows for the identification of the Pprior for each domain.

[0045] The diffusion process performed by source diffusion model 302 produces a Gaussian distribution 306, which represents the latent encoding of domain A. These latent encodings from the source image using source diffusion model 302 are then input into target diffusion model 304 to construct the target image. In this way, the Gaussian distribution 306 of domain A is translated into a Gaussian distribution 308 of domain B, which contains the latent encoding of domain B. Target diffusion model 304 then performs backdiffusion on Gaussian distribution 308 to construct the target image. Therefore, in this example, a CAD image in domain A is transformed into a realistic image in domain B.

[0046] The process of encoding the source image and decoding it to generate the target image is defined via ordinary differential equations (ODEs). Therefore, the process is cyclically consistent until the discretization error of the ODS solver is reached.

[0047] Figure 4 A schematic diagram illustrating semi-generative AI image generation using metadata constraints according to an illustrative embodiment is depicted. In this example, a source diffusion model 402 generates a Gaussian distribution 408 from a source 3D CAD image 406 in the CAD domain. This first Gaussian distribution 408 is translated by a double-diffusion implicit bridge 414 into a second Gaussian distribution 412 in the real-world domain.

[0048] For the latter part of the process (from the latent space to the second domain), more metadata 416 is added to aid backdiffusion into the real-world domain. Examples of added metadata may include data clustering, cue embeddings (e.g., black), materials (e.g., steel), and 2D drawing information. Metadata 416 acts as a Schrödinger's bridge constraint on the target diffusion model 404, which generates a realistic image 410 in the real-world domain from a Gaussian distribution 412. These constraints provide control that ensures not only that the resulting target image is of the same type as the source image, but also that it is shown facing the same direction, at the same angle, etc. This kind of accuracy and specificity is important for applications such as generating synthetic training data for training object detection models. Therefore, metadata helps ensure that the output is as close as possible to the original input.

[0049] Figure 5 An example of imaging decoding from a 3D CAD model to a realistic model is illustrated according to an illustrative embodiment.

[0050] Figure 6 A flowchart illustrating the process of inter-domain semi-generative AI modeling according to an illustrative embodiment is provided. Process 600 can be performed... Figure 1 Implemented in the semi-generative system 100.

[0051] Process 600 begins by receiving a source image of an object in a first domain (operation 602). The first domain may include 3D CAD (computer-aided drafting) image data, and the source image may include a CAD image. Process 600 diffuses the source image using a source diffusion model to generate a first Gaussian distribution in the first domain (operation 604).

[0052] Then, process 600 generates embeddings from the metadata, where the metadata provides constraints for image reconstruction (operation 606). The metadata includes data clustering, cue embeddings, segmentation masks, 2D drawing information, textual descriptions of target objects, audio descriptions of target objects, graphical representations of target objects, and materials or background. The metadata may also include information provided by an AI design parser that cross-references text and specifications with the images and video.

[0053] The insert is fed to the double-diffused implicit bridge (operation 608).

[0054] Process 600 samples from a first Gaussian distribution (operation 610) and maps the samples from the first Gaussian distribution to a second Gaussian distribution in a second domain via a double-diffusion implicit bridge (operation 612). The second domain may include real-world image data.

[0055] Process 600 then back-diffusions the second Gaussian distribution using a target diffusion model to generate a target image of the object in the second domain based on the metadata (operation 612). In the case of the real-world image data domain, the target image generated from the back-diffusion of the second Gaussian distribution includes a realistic image.

[0056] The process then ended at 600.

[0057] Turn now Figure 7 A block diagram of a data processing system is depicted according to an illustrative embodiment. The data processing system 700 can be implemented. Figure 1 The computer system 150 is shown in this illustrative example. The data processing system 700 includes a communication framework 702 that provides communication between a processor unit 704, a memory 706, persistent storage 708, a communication unit 710, an input / output (I / O) unit 712, and a display 714. In this example, the communication framework 702 takes the form of a bus system.

[0058] Processor unit 704 is used to execute instructions for software that can be loaded into memory 706. Depending on the specific implementation, processor unit 704 may be a plurality of processors, a multiprocessor core, or some other type of processor. In one embodiment, processor unit 704 includes one or more conventional general-purpose central processing units (CPUs). In an alternative embodiment, processor unit 704 includes one or more graphics processing units (GPUs).

[0059] Memory 706 and persistent storage 708 are examples of storage device 716. A storage device is any piece of hardware capable of storing information temporarily, permanently, or both temporarily and permanently, the information being at least one of, for example, but not limited to, data, program code in a functional form, or other suitable information. In these illustrative examples, storage device 716 may also be referred to as a computer-readable storage device. In these examples, memory 706 may be, for example, random access memory or any other suitable volatile or non-volatile storage device. Persistent storage 708 may take various forms depending on the specific implementation.

[0060] For example, persistent storage 708 may include one or more components or devices. For example, persistent storage 708 may be a hard disk drive, flash memory, rewritable optical disk, rewritable magnetic tape, or some combination thereof. The media used in persistent storage 708 may also be removable. For example, a removable hard disk drive may be used in persistent storage 708. In these illustrative examples, communication unit 710 provides communication with other data processing systems or devices. In these illustrative examples, communication unit 710 is a network interface card.

[0061] The input / output unit 712 allows data input and output with other devices that can be connected to the data processing system 700. For example, the input / output unit 712 can provide connectivity for user input via at least one of a keyboard, mouse, or other suitable input device. Furthermore, the input / output unit 712 can send output to a printer. The display 714 provides a mechanism for displaying information to the user.

[0062] Instructions for at least one of the following: an operating system, an application program, or a program, may be located in storage device 716, which communicates with processor unit 704 via communication frame 702. Processes in different embodiments may be executed by processor unit 704 using computer-implemented instructions, which may reside in memory, such as memory 706.

[0063] These instructions are referred to as program code, computer-usable program code, or computer-readable program code, which can be read and executed by a processor in processor unit 704. The program code in different embodiments may be embodied on different physical or computer-readable storage media, such as memory 706 or persistent storage 708.

[0064] Program code 718 is functionally located on a selectively removable computer-readable medium 720 and can be loaded to or transferred to a data processing system 700 for execution by a processor unit 704. In these illustrative examples, program code 718 and computer-readable medium 720 form a computer program product 722. In one example, computer-readable medium 720 may be a computer-readable storage medium 724 or a computer-readable signal medium 726.

[0065] In these illustrative examples, computer-readable storage medium 724 is a physical or tangible storage device for storing program code 718, and not a medium for propagating or transmitting program code 718. As used herein, computer-readable storage medium 724 should not be construed as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0066] Alternatively, program code 718 can be transferred to data processing system 700 using computer-readable signal medium 726. Computer-readable signal medium 726 may be, for example, a propagated data signal containing program code 718. For example, computer-readable signal medium 726 may be at least one of electromagnetic signals, optical signals, or any other suitable type of signal. These signals can be transmitted via at least one communication link, such as a wireless communication link, fiber optic cable, coaxial cable, wire, or any other suitable type of communication link.

[0067] The different components illustrated for the data processing system 700 are not intended to be architecturally limiting of the implementation of different embodiments. Different illustrative embodiments may be implemented in the data processing system, which includes components other than those illustrated for the data processing system 700 or components that replace those illustrated for the data processing system 700. Figure 7 Other components shown may differ from the illustrative example shown. Different embodiments may be implemented using any hardware device or system capable of running program code 718.

[0068] As used in this article, the phrase "at least one of" when used with a list of items means that different combinations of one or more of the listed items may be used, and only one of each item in the list is required. In other words, "at least one of" means that any combination of the items and the number of items in the list may be used, but not all items in the list are required. Items can be specific objects, things, or categories.

[0069] For example, but not limited to, "at least one of Item A, Item B, or Item C" may include Item A, Item A and Item B, or Item B. This example may also include Item A, Item B, and Item C, or Item B and Item C. Of course, any combination of these items may exist. In some illustrative examples, "at least one of..." may be, for example, but not limited to, two of Item A; one of Item B; and ten of Item C; four of Item B and seven of Item C; or other suitable combinations.

[0070] As used in this article, “a number of” when referring to projects means one or more projects. For example, “a number of different types of networks” means one or more different types of networks. In illustrative examples, “a set” used with reference projects means one or more projects. For example, a set of metrics means one or more metrics.

[0071] For purposes of illustration and description, various illustrative embodiments have been described, but are not intended to be exhaustive or limiting of the embodiments of the disclosed forms. The various illustrative examples describe components that perform actions or operations. In the illustrative embodiments, components may be configured to perform the described actions or operations. For example, a component may have a configuration or design that provides the component with the ability to perform actions or operations described in the illustrative examples as being performed by the component. Furthermore, with regard to the terms “comprising,” “having,” “including,” and variations thereof as used herein, these terms are intended to be included in a manner similar to the term “comprises” as an open transition word, without excluding any additional or other elements.

[0072] This application also includes embodiments pursuant to the following provisions:

[0073] Clause 1. A system for inter-domain semi-generative artificial intelligence modeling, the system comprising:

[0074] Storage device (716), the storage device stores program instructions;

[0075] One or more processors (704), operatively connected to the storage device and configured to execute the program instructions to cause the system to:

[0076] Receive (602) the source image of the object in the first field (120);

[0077] The source image is diffused (604) by a source diffusion model (118) to generate a first Gaussian distribution (126) in the first domain;

[0078] An embedding (132) is generated (606) from metadata, wherein the metadata provides constraints for image reconstruction;

[0079] The insert is fed (608) into the double-diffused implicit bridge (130);

[0080] Sample from the first Gaussian distribution (610);

[0081] The samples are mapped from the first Gaussian distribution (612) to the second Gaussian distribution (128) in the second domain (124) via the double-diffusion implicit bridge; and

[0082] The second Gaussian distribution is back-diffused (614) using a target diffusion model (122) to generate a target image of the object in the second domain based on the metadata.

[0083] Clause 2. The system according to Clause 1, wherein the first domain includes three-dimensional CAD image data, i.e., computer-aided drawing image data (104).

[0084] Clause 3. The system as described in Clause 1, wherein the second domain includes real-world image data (102).

[0085] Clause 4. The system according to Clause 3, wherein the target image includes a realistic image.

[0086] Clause 5. The system pursuant to Clause 1, wherein the metadata includes at least one of the following:

[0087] Data clustering;

[0088] Prompt for embedded components;

[0089] Segmentation mask;

[0090] Two-dimensional drawing information;

[0091] The text description of the target object;

[0092] The audio description of the target object;

[0093] A graphical representation of the target object;

[0094] Materials; or

[0095] background.

[0096] Clause 6. The system pursuant to Clause 1, wherein the metadata includes information provided by an artificial intelligence design parser (112) that cross-references text and specifications with images and videos.

[0097] Clause 7. The system according to Clause 1, wherein the source diffusion model and the target diffusion model include Schrödinger bridges (136, 138).

[0098] Clause 8. A computer program product for inter-domain semi-generative artificial intelligence modeling, said computer program product comprising:

[0099] A computer-readable storage medium (724) having program instructions embodied thereon to perform the following steps:

[0100] Receive (602) the source image of the object in the first field (120);

[0101] The source image is diffused (604) by a source diffusion model (118) to generate a first Gaussian distribution (126) in the first domain;

[0102] An embedding (132) is generated (606) from metadata, wherein the metadata provides constraints for image reconstruction;

[0103] The insert is fed (608) into the double-diffused implicit bridge (130);

[0104] Sample from the first Gaussian distribution (610);

[0105] The samples are mapped from the first Gaussian distribution (612) to the second Gaussian distribution (128) in the second domain (124) via the double-diffusion implicit bridge; and

[0106] The second Gaussian distribution is back-diffused (614) using a target diffusion model (122) to generate a target image of the object in the second domain based on the metadata.

[0107] Clause 9. The computer program product pursuant to Clause 8, wherein the first field includes three-dimensional CAD image data, i.e., computer-aided drawing image data (104).

[0108] Clause 10. The computer program product pursuant to Clause 8, wherein the second field includes real-world image data (102).

[0109] Clause 11. A computer program product pursuant to Clause 8, wherein the metadata includes at least one of the following:

[0110] Data clustering;

[0111] Prompt for embedded components;

[0112] Segmentation mask;

[0113] Two-dimensional drawing information;

[0114] The text description of the target object;

[0115] The audio description of the target object;

[0116] A graphical representation of the target object;

[0117] Materials; or

[0118] background.

[0119] Clause 12. The computer program product pursuant to Clause 8, wherein the metadata includes information provided by an artificial intelligence design parser (112) that cross-references text and specifications with images and videos.

[0120] Clause 13. The computer program product pursuant to Clause 8, wherein the source diffusion model and the target diffusion model include Schrödinger bridges (136, 138).

[0121] Many modifications and variations will be apparent to those skilled in the art. Furthermore, different illustrative embodiments may provide different features compared to other desired embodiments. The selection and description of one or more embodiments are intended to best explain the principles of the embodiments, their practical application, and to enable others skilled in the art to understand the disclosure of various embodiments with various modifications suitable for the intended particular purpose.

Claims

1. A computer-implemented method for inter-domain semi-generative artificial intelligence modeling, the method comprising: Execute using multiple processors: Receive (602) the source image of the object in the first field (120); The source image is diffused (604) by a source diffusion model (118) to generate a first Gaussian distribution (126) in the first domain; An embedding (132) is generated (606) from metadata, wherein the metadata provides constraints for image reconstruction; The insert is fed (608) into the double-diffused implicit bridge (130); Sample from the first Gaussian distribution (610); The sample is mapped from the first Gaussian distribution (612) to the second Gaussian distribution (128) in the second domain (124) via the double-diffusion implicit bridge. as well as The second Gaussian distribution is back-diffused (614) using a target diffusion model (122) to generate a target image of the object in the second domain based on the metadata.

2. The method according to claim 1, wherein the first domain includes three-dimensional CAD image data, i.e., computer-aided drawing image data (104).

3. The method according to claim 1, wherein the second domain comprises real-world image data (102).

4. The method according to claim 3, wherein the target image includes a realistic image.

5. The method of claim 1, wherein the metadata comprises at least one of the following: Data clustering; Prompt for embedded components; Segmentation mask; Two-dimensional drawing information; The text description of the target object; The audio description of the target object; A graphical representation of the target object; Materials; or background.

6. The method of claim 1, wherein the metadata includes information provided by an artificial intelligence design parser (112) that cross-references text and specifications with reference to images and videos.

7. The method according to claim 1, wherein the source diffusion model and the target diffusion model include Schrödinger bridges (136, 138).

8. A system for inter-domain semi-generative artificial intelligence modeling, the system comprising: Storage device (716), the storage device stores program instructions; One or more processors (704), operatively connected to the storage device and configured to execute the program instructions to cause the system to: Receive (602) the source image of the object in the first field (120); The source image is diffused (604) by a source diffusion model (118) to generate a first Gaussian distribution (126) in the first domain; An embedding (132) is generated (606) from metadata, wherein the metadata provides constraints for image reconstruction; The insert is fed (608) into the double-diffused implicit bridge (130); Sample from the first Gaussian distribution (610); The sample is mapped from the first Gaussian distribution (612) to the second Gaussian distribution (128) in the second domain (124) via the double-diffusion implicit bridge. as well as The second Gaussian distribution is back-diffused (614) using a target diffusion model (122) to generate a target image of the object in the second domain based on the metadata.

9. The system according to claim 8, wherein the first domain includes three-dimensional CAD image data, i.e., computer-aided drawing image data (104).

10. The system of claim 8, wherein the second domain comprises real-world image data (102).

Images