Generative artificial intelligence configuration system for aircraft internal design

Abstract

To provide a generative artificial intelligence configuration system for the internal design of aircraft. [Solution] The design system comprises a computer system, a machine learning model system, and a design generator. The design generator is configured to perform an operation. The operation includes: identifying a reference image of the interior design of an aircraft; identifying engineering data for the design; receiving element selections of elements in the reference image of the interior design for modification; generating an enhanced image of the design using the elements, element selections, and the machine learning model system, wherein the elements in the enhanced image are enhanced elements; receiving element modifications for modifying the enhanced elements; modifying the enhanced elements using the machine learning model system with element modifications that take the engineering data into account to form a customized image of the design having modified elements instead of the enhanced elements; and displaying the customized image on a display system.

Description

[[Technical Field]]

[0001] The present disclosure relates generally to aircraft, and in particular to interior configurations of aircraft. [[Background Art]]

[0002] Designing an interior such as an aircraft cabin for presentation and for customer feedback is a collaborative and time-consuming process. Design engineers and three-dimensional artists collaborate to generate an interior design. Design engineers and three-dimensional artists spend a significant amount of time designing and rendering aircraft interiors using configuration files from configuration engineering teams. These configuration files provide details such as structures, systems, and constraints. This type of file is highly technical and difficult to understand. Furthermore, not all elements in these files are required for creating the design and visual representation of an aircraft interior.

[0003] After a design is created, multiple iterations of design modifications may occur based on feedback from customers. This feedback can result in refinement of layouts, materials, lighting, seating configurations, and other aspects of aircraft interior design. [[Summary of the Invention]] [[Means for Solving the Problem]]

[0004] One embodiment of the present disclosure provides an internal design system comprising a computer system, a machine learning model system within the computer system, and a design generator within the computer system. The design generator is configured to perform operations including: identifying a reference image of the internal design of the interior of an aircraft; identifying engineering data for the internal design; receiving element selections of several elements in the reference image of the internal design for modification; generating an enhanced image of the internal design using several elements in the reference image, element selections, and the machine learning model system, wherein several elements in the enhanced image are several enhanced elements; receiving element modifications for modifying several enhanced elements in the enhanced image; modifying several enhanced elements using element modifications that take engineering data into account using the machine learning model system to form a customized image of the internal design having several modified elements instead of some enhanced elements; and displaying the customized image on a display system.

[0005] Another embodiment of this disclosure provides a method for generating changes to the internal design of an aircraft's interior. A reference image of the aircraft's interior design is identified. Engineering data for the interior design is identified. Element selections of several elements in the reference image of the interior design for modification are received. A highlight image of the interior design is generated using several elements in the reference image, the element selections, and a machine learning model system, where several elements in the highlight image are several highlight elements. Element modifications are received to modify some of the highlight elements in the highlight image. Some of the highlight elements are modified using element modifications that take the engineering data into account using the machine learning model system to form a customized image of the interior design having several modification elements instead of some of the highlight elements. The customized image is displayed on a display system.

[0006] A further embodiment of the present disclosure provides a computer program product for generating changes to the internal design of an aircraft interior. The computer program product includes a set of one or more computer-readable storage media and program instructions stored in the set of one or more storage media. The program instructions are for performing operations including: identifying a reference image of the internal design of an aircraft interior; identifying engineering data for the interior design; receiving element selections of several elements in the reference image of the interior design for modification; generating an enhanced image of the interior design using several elements in the reference image, element selections, and a machine learning model system, wherein some elements in the enhanced image are several enhanced elements; receiving element modifications for modifying some of the enhanced elements in the enhanced image; modifying some of the enhanced elements using element modifications that take engineering data into account using a machine learning model system to form a customized image of the interior design having some modified elements instead of some of the enhanced elements; and displaying the customized image on a display system.

[0007] A further embodiment of the present disclosure provides an internal design system comprising a computer system, a generative artificial intelligence model system within the computer system, and a design generator within the computer system. The design generator is configured to perform operations including: identifying a reference image of an internal design of the interior of a vehicle; identifying engineering data for the internal design; receiving element selections of several elements in the reference image of the internal design for modification; generating an enhanced image of the internal design using several elements in the reference image, element selections, and the machine learning model system, wherein some elements in the enhanced image are several enhanced elements; receiving element modifications for modifying some of the enhanced elements in the enhanced image; modifying some of the enhanced elements using element modifications that take engineering data into account using the machine learning model system to form a customized image of the internal design having some modified elements instead of some of the enhanced elements; and displaying the customized image on a display system.

[0008] The features and functions can be achieved individually in various embodiments of this disclosure and can also be combined in other embodiments, which can be understood in more detail by referring to the following description and drawings.

[0009] Novel features that may be considered characteristics of the exemplary embodiments are described in the appended claims. However, the exemplary embodiments and their preferred modes of use, further purposes, and features are best understood by referring to the following detailed description of the exemplary embodiments of this disclosure in conjunction with the appended drawings. [Brief explanation of the drawing]

[0010] [Figure 1] This is a conceptual diagram of a network for a data processing system in which exemplary embodiments may be implemented. [Figure 2]This is an example of a block diagram of a design environment according to an exemplary embodiment. [Figure 3] This is an example of a process flow diagram for modifying the visualization of the internal design according to an exemplary embodiment. [Figure 4] This is an example of a process flow diagram for modifying the visualization of the internal design according to an exemplary embodiment. [Figure 5] This is an example of a process flow for customizing the interior of a guest room according to an exemplary embodiment. [Figure 6] This is an example of a process flow for customizing the interior of a guest room according to an exemplary embodiment. [Figure 7] This is an example of a process flow for customizing an aircraft structure according to an exemplary embodiment. [Figure 8] This is an illustrative flowchart of the process for generating changes to the internal design of an aircraft according to an exemplary embodiment. [Figure 9] This is an illustrative flowchart of the process for displaying a highlight image, according to an exemplary embodiment. [Figure 10] This is an illustrative flowchart of a process for generating enhanced images according to an exemplary embodiment. [Figure 11] This is an example flowchart of the process for modifying several emphasis elements according to an exemplary embodiment. [Figure 12] This is an illustrative flowchart of the process for generating changes to the vehicle design, according to an exemplary embodiment. [Figure 13] This is an example block diagram of a data processing system according to an exemplary embodiment. [Figure 14] This is an illustrative block diagram of an aircraft manufacturing and maintenance inspection method according to one exemplary embodiment. [Figure 15] This is an illustrative block diagram of an aircraft in which one exemplary embodiment may be implemented. [Modes for carrying out the invention]

[0011] In exemplary embodiments, one or more diverse considerations are recognized and taken into account, as described below. For example, collaboration between design engineers and 3D artists is time-consuming because it requires balancing technical accuracy with visual appeal and taking customer preferences into account. The design engineer interprets configuration files and extracts relevant structural and system data. The 3D artist uses this data to develop detailed models and realistic renderings of cabin interiors. This process involves multiple iterative cycles where customer feedback leads to design revisions, requiring adjustments to both technical and visual elements. Each iteration adds to the workload. For example, even minor changes based on customer requests or ideas can impact the design, requiring updates, re-rendering, and technical verification to meet customer expectations. Furthermore, the involvement of third-party design firms may increase the complexity of the collaboration and the time required.

[0012] Therefore, the exemplary examples provide methods, apparatus, systems, and computer program products for generating aircraft interior designs. The exemplary examples allow the use of machine learning models, such as generative artificial intelligence models, in the design process. These models enable the generation of visualizations of interior details much faster than current processes. For example, the way generative artificial intelligence models are used in the exemplary examples provides much faster rendering of interior details, such as seat color, material changes, and placement, compared to current technologies. Currently, design engineers collaborate with 3D design artists to create and modify models using computer-aided design systems.

[0013] Furthermore, it is possible to perform simulations of the aircraft's internal construction. These simulations can determine whether conflicts may arise between the design and the aircraft's internal structure and systems. As a result, this type of simulation can reduce the amount of time required to handle potential changes that may occur during program development, thereby lowering overall costs.

[0014] In one exemplary example, the internal design system comprises a computer system, a generative artificial intelligence model system within the computer system, and a design generator within the computer system. The design generator is configured to identify a reference image of the internal design of the vehicle's interior, identify engineering data for the internal design, and receive element selections of several elements within the reference image of the internal design for modification. The design generator is configured to generate an enhanced image of the internal design using several elements in the reference image, element selections, and the machine learning model system. Some elements of the enhanced image are several enhanced elements. Element modifications are received to modify some of the enhanced elements in the enhanced image. The design generator is configured to take the engineering data into account and modify some of the enhanced elements using element modifications that utilize the machine learning model system to form a customized image of the internal design having some modified elements instead of some of the enhanced elements. The design generator is configured to display the customized image on a display system.

[0015] In a different illustrative example, the generation of a customized image with modifications to selected elements within an image for design is based on inputs that select elements and identify modifications. Furthermore, these modifications take into account engineering data that provides information about the elements. This information may also include what changes can be made to the elements. As a result, the changes made based on the identified modifications are realistic changes to the design that can be used for manufacturing or reconfiguring platforms such as aircraft.

[0016] Referring now to the figures, and particularly to Figure 1, there is shown a diagram of a data processing system network in which an exemplary embodiment may be implemented. Network data processing system 100 is a computer network in which an exemplary embodiment may be implemented. Network data processing system 100 includes network 102, which is the medium used to provide communication links between various devices and computers connected together within network data processing system 100. Network 102 may include connections such as wired, wireless communication links, or fiber optic cables.

[0017] In the depicted example, server computer 104 and server computer 106 connect to network 102 along with storage unit 108. Client device 110 also connects to network 102. Client device 110 may be, for example, a computer, a workstation, a network computer, a vehicle, a machine, an apparatus, or another device capable of processing data. As depicted, client device 110 includes client computer 112, client computer 114, client computer 116, mobile phone 118, tablet computer 120, and smart glasses 122. Client device 110 may be, for example, a computer, a workstation, or a network computer. In the depicted example, server computer 104 provides information such as boot files, operating system images, and applications to client device 110.

[0018] In the illustrated example, the server computer 104 provides information such as boot files, operating system images, and applications to the client device 110. Further, in this illustrative example, the server computer 104, the server computer 106, the storage unit 108, and the client device 110 are network devices connected to the network 102, and the network 102 is a communication medium for these network devices. Some or all of the client devices 110 may form the Internet of Things (IoT), in which these physical devices connect to the network 102 and can exchange information with each other via the network 102.

[0019] The client device 110 is a client to the server computer 104 in this example. The network data processing system 100 may include additional server computers, client computers, and other devices not shown in the figure. The client device 110 connects to the network 102 using at least one of a wired connection, an optical fiber connection, or a wireless connection.

[0020] Program instructions located in the network data processing system 100 are stored on a computer-recordable storage medium and can be downloaded to a data processing system or other device for use. For example, the program instructions may be stored on a computer-recordable storage medium on the server computer 104 and downloaded to the client device 110 via the network 102 for use on the client device 110.

[0021] In this illustrative example, the design generator 130 is located within the server computer 104. this component can operate to at least one of generating or modifying an interior design of an aircraft.

[0022] Furthermore, the phrase "at least one of..." when used with a list of items means that one or more different combinations of the enumerated items can be used, and only one of each item in the list may be required. In other words, "at least one of..." means that any combination of items, and some items from the list, can be used, but not all items on the list are necessarily required. An item can be a specific object, thing, or category.

[0023] For example, “at least one of item A, item B, or item C” could, but is not limited to, item A, item A and item B, or item B. Examples of this could include item A, item B, and item C, or item B and item C. Naturally, any combination of these items is possible. In some exemplary examples, “at least one of ~” could, but is not limited to, two item A, one item B, and ten item C, or four item B and seven item C, or other preferred combinations.

[0024] As shown in the diagram, a human operator 131 can operate a client computer 112 and interact with a design generator 130 to make changes to the internal design 132 of a civilian aircraft 133. In this example, the design generator 130 can identify a reference image 134 for the internal design 132 of the civilian aircraft 133. The internal design 132 can be a model such as a computer-aided design model of the aircraft cabin within the civilian aircraft 133. The reference image 134 can be identified from a database of images within the computer-aided design model of the cabin. The design generator 130 can display the reference image 134 to the human operator 131 on the client computer 114.

[0025] This reference image may be for a part of the interior design, such as a portion of the passenger area within the aircraft cabin. In another example, reference image 134 may be for a different part of the design, such as a galley within the aircraft cabin.

[0026] A human operator 131 can select a portion of the internal design 132 from a reference image by input such as text, voice, or drawing a bounding box around that portion of the design. For example, this portion could be the overhead bins inside the cabin of a commercial aircraft 133.

[0027] In this example, the design generator 130 generates a highlight image 135. This highlight image can also be displayed to a human operator 131 on a client computer 114. This highlight image has several highlight elements based on the aircraft cabin section selected for modification. In this example, some highlight elements may be bins in the aircraft cabin. The highlights may be graphical indicators that draw attention to some highlight elements, such as highlighting, color, or other graphical indicators that draw attention to the bins.

[0028] As used herein, "several" when used in relation to an item means one or more items. For example, "several emphasis elements" refers to one or more emphasis elements.

[0029] In this exemplary example, a human operator 131 can create an element modification. In this exemplary example, the human operator 131 could be a reviewer for a customer, a design engineer, or another person. The element modification could also be based on input from the human operator 131. This input could be text or audio providing a description of the element modification.

[0030] Furthermore, element modifications instructed by input from a human operator 131 are also made taking into account engineering data 137 for the internal design 132. This engineering data provides tolerances for the internal design 132. Tolerances can be constraints on values ​​or ranges of values ​​for modifications made by a human operator. For example, when modifying bins in the internal design 132, the maximum size of these bins can be defined by the tolerances in the engineering data 137. Thus, modifications made by a human operator 131 are realistic changes that can actually be implemented in a civilian aircraft 133.

[0031] In response, the design generator 130 modifies the generated elements to form a customized image 136 that can be displayed to the human operator 131 on the client computer 114. In response to viewing the customized image 136, the human operator 131 can approve the internal design 132, make additional modifications to the internal design 132, or perform other actions.

[0032] Furthermore, modifications to the customized image 136 are made to the internal design 132. For example, if the internal design 132 is a computer-aided design model, these modifications to the customized image 136 are made to the corresponding elements within the computer-aided design model.

[0033] These modifications may be made to the internal design 132 in response to approval of the modifications by a human operator 131. These updates to the internal design 132 can be verified by a design engineer before being implemented for the civilian aircraft 133. In other examples, these modifications to the internal design 132 may also be made in response to a generated customized image 136.

[0034] In this example, an internal design 132 with modifications can be sent to a client computer 116 of facility 160 for use in at least one of the manufacturing, reconfiguration, or upgrading of a civilian aircraft 133 at facility 160. Facility 160 could be, for example, a manufacturing plant, a maintenance facility, a hangar, or any other suitable location for manufacturing or modifying a civilian aircraft 133. As a result, modifications to the internal design 132 can be made by a human operator 131, taking into account engineering data 137.

[0035] In these exemplary examples, image generation and modifications to the internal design can be performed using the machine learning model system 150.

[0036] In the illustrated example, the network data processing system 100 is the Internet, which has a network 102 representing a global collection of networks and gateways that communicate with each other using a set of Transmission Control Protocol / Internet Protocol (TCP / IP) protocols or other networking protocols. At the heart of the Internet is a backbone of high-speed data communication lines between major nodes or host computers, consisting of thousands of commercial, government, educational, and other computer systems that route data and messages. Naturally, the network data processing system 100 may also be implemented using several different types of networks. For example, network 102 could consist of at least one of the following: the Internet, an intranet, a local area network (LAN), a metropolitan area network (MAN), or a wide area network (WAN). Figure 1 is intended as an example and is not intended to impose architectural limitations on various exemplary embodiments.

[0037] As another example, the design generator 130 may be located within the client computer 112. In yet another exemplary example, the design generator 130 may be distributed between the server computer 104 and different client devices within the client device 110. For example, processing may be performed on the server computer 104, while the graphical user interface is located within the client device 110.

[0038] Referring now to Figure 2, an example block diagram of a design environment is shown according to an exemplary embodiment. In this exemplary example, the design system 202 within the design environment 200 includes components that can be implemented in hardware, such as the hardware shown in the network data processing system 100 in Figure 1. This internal design system operates to create or modify at least one of the following: design 201 of platform 290. For example, design 201 may be for the interior 206 of platform 290 in the form of a vehicle 203.

[0039] Design 201 is a model of the internal 206 of the electronic form. This model can be, for example, a computer-aided design model, a point cloud model, a finite element analysis model, a two-dimensional model, or another suitable model.

[0040] The internal design 132 can be associated with engineering data 221, which provides tolerances for the design 201. In this example, the engineering data 221 may include at least one of several tolerances for at least one of the following: physics-based parameters, volume, material, dimensions, density, elasticity, stiffness, surface texture, temperature-based material behavior, size, position, orientation, weight, or other types of engineering data.

[0041] The tolerances in engineering data 221 may be constraints on one or more values ​​or ranges of values ​​for modifications that can be made to design 201. In addition to tolerances, engineering data 221 may also include other information such as descriptions of various elements in design 201, vendor identification, and other information.

[0042] For example, the dimensions of an overhead bin may have a range of values ​​for width, length, and other dimensions, based on the specific aircraft in which the overhead bin is located. In some cases, the dimensions may be specific values ​​rather than a range of tolerances for the overhead bin.

[0043] As another example, temperature-based material behavior can be a change in color based on temperature, which can be attributed as a variable, and this material can be used by the design generator 214 to provide visualizations based on different ambient temperatures.

[0044] Vehicle 203 can take several forms. For example, Vehicle 203 can be selected from a group that includes aircraft 204, civilian aircraft, cargo aircraft, rotary-wing aircraft, tiltrotor aircraft, inclined-wing aircraft, vertical take-off and landing aircraft, unmanned aerial vehicles, artificial intelligence-controlled vehicles, electric vertical take-off and landing vehicles, personal air vehicles, surface ships, cruise ships, tanks, personnel carriers, trains, spacecraft, manned spacecraft, spacecraft, submarines, buses, automobiles, and other vehicles that have an interior inside.

[0045] Interior 206 can be any interior area within the vehicle 203. For example, if the vehicle 203 is an aircraft 204, then interior 206 could be the aircraft's cabin 207. The aircraft's cabin 207 could be, for example, a passenger seating area, a flight attendant seating area, a crew rest area, a galley, a toilet, or other areas. In yet another exemplary example, interior 206 could be the cockpit of the aircraft 204.

[0046] In this exemplary example, the design system 202 comprises a computer system 212 and a design generator 214. The design generator 214 is located within the computer system 212.

[0047] The design generator 214 can be implemented in software, hardware, firmware, or a combination thereof. When software is used, the operations performed by the design generator 214 can be implemented as program instructions configured to run on hardware such as a processor unit. When firmware is used, the operations performed by the design generator 214 are implemented as program instructions and data, which can be stored in persistent memory for execution on a processor unit. When hardware is employed, the hardware may include circuitry that operates to perform the operations in the design generator 214.

[0048] In exemplary examples, hardware can take the form of at least one selected from circuit systems, integrated circuits, application-specific integrated circuits (ASICs), programmable logic devices, or other suitable types of hardware configured to perform several operations. Programmable logic devices can be configured to perform several operations. The device can be reconfigured later or permanently configured to perform several operations. Examples of programmable logic devices include programmable logic arrays, programmable array logic, field-programmable logic arrays, field-programmable gate arrays, and other suitable hardware devices. Furthermore, processes can be implemented as organic components integrated with inorganic components, or as organic components only, excluding humans. For example, a process can be implemented as a circuit of organic semiconductors.

[0049] The computer system 212 is a physical hardware system and includes one or more data processing systems. If the computer system 212 has multiple data processing systems, these systems communicate with each other using a communication medium. The communication medium can be a network. The data processing systems can be selected from at least one of the following: a computer, a server computer, a tablet computer, or other suitable data processing systems.

[0050] As shown in the figure, the computer system 212 includes several processor units 216 that can execute program instructions 218 that carry out the process in the exemplary example. In other words, the program instructions 218 are computer-readable program instructions.

[0051] As used herein, a processor unit in some processor units 216 is a hardware device and consists of hardware circuits, such as those on an integrated circuit, that process instructions and program code that operate the computer.

[0052] If several processor units 216 execute program instructions 218 for a process, then several processor units 216 may be one or more processor units located in the same computer or in different computers. In other words, a process can be distributed among processor units 216 on the same computer or in different computers within the computer system 212.

[0053] Furthermore, some of the processor units 216 may be of the same type or different types. For example, some of the processor units 216 may be at least one of the following: a single-core processor, a dual-core processor, a multi-processor core, a general-purpose central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or some other type of processor unit.

[0054] In this exemplary example, a human operator 209 can interact with a design generator 214 via a human-machine interface 211 within a computer system 212. In this exemplary example, the human-machine interface (HMI) 211 is an interface system that can be used by the human operator 209 to interact with different components within the computer system 212. As shown in the figure, the human-machine interface 211 comprises a display system 231 and an input system 219.

[0055] The display system 231 is a physical hardware system and includes one or more display devices capable of displaying the graphical user interface 213. The display devices may include at least one of the following: a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, a computer monitor, a projector, a flat panel display, a head-up display (HUD), a head-mounted display (HMD), smart glasses, augmented reality glasses, or any other suitable device capable of outputting information for the visual presentation of information.

[0056] The human operator 209 is a person who can interact with the graphical user interface 213 via user input generated by the input system 219 of the design generator 214. The input system 219 is a physical hardware system and can be selected from at least one of the following: a mouse, keyboard, touchpad, trackball, touchscreen, stylus, motion-sensing input device, gesture detection device, data glove, cyber glove, haptic feedback device, or any other suitable type of input device.

[0057] In one exemplary example, the design generator 214 performs several different actions to make changes to the design 201 of the interior 206 of the vehicle 203. For example, the design generator 214 identifies a reference image 220 of the design 201 of the interior 206 of the vehicle 203. If the vehicle 203 is an aircraft 204, the interior 206 could be the aircraft's passenger cabin 207 or some other interior part of the aircraft 204.

[0058] The reference image 220 can be identified from an image database, design 201, or other source. For example, an image within design 201 or internal 206 can be selected from an image database for use. In another example, the reference image 220 can be generated from design 201 in the form of a computer-aided design model.

[0059] This reference image could be for a part of the interior design, such as a portion of the passenger area 206 inside aircraft 204. In another example, reference image 220 could be for a different part of the design, such as the galley 207 in the aircraft's cabin.

[0060] The design generator 214 identifies the engineering data 221 of the internal design. The engineering data 221 may reside within the design 201, be referenced by the design 201, or be associated with the design 201 in some other way.

[0061] Furthermore, the design generator 214 receives an element selection 222 of several elements 223 within the reference image 220 of the design 201 for modification.

[0062] Some elements 223 can take several different forms. For example, if the vehicle is an aircraft 204, some elements 223 can be selected from at least one of the following: passenger seats, overhead bins, several aisles, seat cushions, doors, lights, lighting systems, in-flight entertainment systems, several rows of passenger seats, seat configurations, or other elements within the interior 206 of the aircraft 204.

[0063] In this example, the element selection 222 can be received from the human-machine interface 211 based on input generated by a human operator 209. In this exemplary example, the element selection 222 can be selected from at least one of the following types of input: text, voice, touch gesture, or other input generated by the human operator 209 using the input system 219. For example, the human operator 209 can input text using a keyboard. In another example, the human operator 209 can speak to the input system 219 to cause it to generate voice, which is the element selection 222. In yet another exemplary example, the human operator 209 can use touch gestures on a touchscreen to generate the element selection 222, such as a bounding box around several elements 223. This input and other types of input can be generated by the human operator 209.

[0064] In this illustrative example, the design generator 214 uses several elements 223 in the reference image 220, element selection 222, and a machine learning model system 225 to generate a reinforced image 224 of the design 201. Some of the elements 223 in the reinforced image 224 are some reinforced elements 227. This reinforced image may also be referred to as a mask image.

[0065] In this example, the machine learning model system 225 consists of several machine learning models 226. Various operations of the design generator 214 can be performed using one or more of the machine learning models 226. One operation can be performed by one machine learning model, and two or more operations can be performed by different machine learning models.

[0066] For example, the design generator 214 can generate a reinforced image 224 by performing calculations using one or more machine learning models 226 in the machine learning model system 225. The design generator 214 uses the machine learning model system 225 to identify several elements 223 in the reference image 220 using element selection 222 and the machine learning model system 225. The design generator 214 uses the machine learning model system 225 to create a mask 241 that identifies pixels 242 representing the several elements 223 identified in the reference image 220 using element selection 222.

[0067] Furthermore, the design generator 214 modifies pixels 242 representing some elements 223 using a machine learning model system 225, creating some modified regions for some elements 223 to form a reinforced image 224 having some reinforced elements 227. Some reinforced elements 227 within some modified regions 243 are visually distinguishable from other elements outside of some modified regions 243 in the reinforced image 224.

[0068] In this example, the mask 241 consists of pixels that identify which pixels are part of some elements 223 and which pixels are not part of some elements 223. The mask 241 can be generated using a computer vision algorithm within the machine learning model system 225 or a machine learning model such as U-Net, Mark R-CNN, or a zero-shot partitioning model.

[0069] The design generator 214 can perform actions such as graphically highlighting the emphasis elements 227 on the display system 231 to display the emphasis image 224. In this exemplary example, the emphasis elements 227 may be graphically highlighted using several different types of graphical indicators to draw attention to some of the emphasis elements 227. These emphasis elements can be identified using at least one of the following graphical indicators: color, highlighting, brightness, border, pattern change, animation, or other graphical indicators.

[0070] The design generator 214 can receive element modifications 228 to modify several highlighting elements 227 within the highlighting image 224. In this example, the selection can be performed by a human operator 209 using the human-machine interface 211. This input can take a similar form to that used to generate the element selection 222.

[0071] The design generator 214 modifies some emphasis elements 227 using element modification 228 that takes engineering data 221 into account using a machine learning model system 225 to form a customized image 229 of the design 201, which has some modification elements 230 instead of some emphasis elements 227.

[0072] When modifying some emphasis elements 227, the design generator 214 uses a machine learning model system 225 to identify a set of changes 250 to some emphasis elements 227 using element modifications 228 that take engineering data 221 into account. The design generator 214 uses the machine learning model system 225 to spread the identified set of changes 250 from noise to a customized image 229, generating a customized image 229 with the set of changes 250 to some emphasis elements 227 and forming some modification elements 230 within the customized image 229.

[0073] As used herein, “set of ~” means one or more items when used in relation to an item. For example, set of changes 250 is one or more of the changes 250.

[0074] In this example, the set of 250 changes to the set of 227 emphasis elements can take several different forms. For example, the set of changes to some elements could be selected from at least one of the following: color, material, dimensions, shape, position, location, orientation, surface finish, or coating.

[0075] To perform diffusion, the machine learning model system 225 includes several machine learning models 226 of the form of diffusion models that can perform diffusion. The diffusion model generates images by iteratively transforming random noise into a coherent image through a denoising process. The model can also learn to stepwise add Gaussian noise to the image and reverse the diffusion process until it consists only of noise. During training, the model learns to predict and remove this noise at each step, effectively reconstructing the original image from the noisy version. In image generation, the process is reversed so that training starts with pure noise. The diffusion model iteratively applies the learned denoising steps to refine the noise into an image.

[0076] In these examples, we can use dual diffusion, which involves training two diffusion models simultaneously to learn image-to-noise diffusion and noise-to-image diffusion, and then applying this to domain style adaptation.

[0077] Some diffusion models can take several forms. For example, some diffusion models can be selected from at least one of the following: denoising diffusion stochastic models (DDPM), score-based generative models (SDE), forward diffusion models, backward diffusion models, or other suitable models.

[0078] Furthermore, in this example, the design generator 214 displays the customized image 229 on the display system 231. In addition, a human operator 209 can decide whether to accept some of the modification elements 230 shown in the customized image 229. In response to accepting or approving these modifications, the design generator 214 can propagate or modify the design 201 to include the modification elements 230 shown in the customized image 229.

[0079] The modifications to design 201 mean that the actual application of the results produced by the design generator 214 includes manufacturing a new aircraft using design 201 for a customized image 229. In another example, the actual application may involve reconfiguring an existing aircraft using design 201 for a customized image 229. In these examples, design 201 for a customized image 229 means that design 201 includes several modification elements 230 shown in the customized image 229. For example, if the modification element is changing the size of the in-cabin displays, design 201 is also modified to reflect the size change. Thus, design 201 is modified to accurately reflect several modification elements and the customized image 229.

[0080] In one exemplary example, there exists one or more technical solutions to overcome technical problems related to revising designs, such as cabin and aircraft designs. As a result, one or more technical solutions may provide a technical effect that enables the automatic generation of customized images that provide visualization of the modifications. These modifications are technically accurate because they take into account engineering data for the design. Consequently, approval of the customized image modifications can be implemented for actual production.

[0081] In an exemplary example, the use of the design generator 214 in computer system 212 integrates the process into a practical application for a method of generating design modifications that can be used to manufacture or reconfigure platforms such as aircraft. For example, the design generator 214 in computer system 212 provides a practical application for manufacturing a vehicle using the generated changes to the vehicle's internal design, or for reconfiguring a vehicle using the internal design.

[0082] In one exemplary example, a method, apparatus, system, or computer program product can generate a modification to the design of an aircraft cabin. This modification can be used in the manufacture of an aircraft cabin for an aircraft. Furthermore, such a modification in the design can also be used to perform reconfiguration, upgrade, or other maintenance on an existing aircraft. The modification can be made for other vehicles in addition to aircraft 204. Thus, the exemplary example can be used to perform operations for manufacturing or reconfiguring the interior of a vehicle.

[0083] The illustrative design environment 200 in Figure 2 does not imply any physical or structural limitations on how the exemplary embodiment may be carried out. Other components may be used in addition to or instead of those shown. Some components may be unnecessary. Also, blocks are presented to illustrate some functional components. One or more of these blocks may be combined, separated, or combined and separated into different blocks when implemented in the exemplary embodiment.

[0084] As another example, if the vehicle 203 takes the form of a surface vessel such as a cruise ship, the interior 206 can be any interior part of the cruise ship. For example, the interior could be a dining area, cabins, a training room, a kitchen, corridors, a theater, or any other interior part of the cruise ship. In yet another exemplary example, the design 201 could be for other parts of the vehicle 203 in addition to the interior 206. For example, the design 201 could be for the physical structure of the vehicle 203 in addition to that of the interior 206, such as the exterior, systems, internal wall structures, wiring harness locations within the hull, control surfaces, and other structural designs. Thus, the design 201 could be for the interior, physical structure, exterior, or other design of the platform 290.

[0085] In another exemplary example, element modification 228 can be performed by a different human operator in addition to the human operator 209 operating a different human-machine interface from the human-machine interface 211.

[0086] In further illustrative examples, platform 290 can take other forms in addition to vehicle 203. For example, in addition to vehicle 203, platform 290 may also be a mobile platform, a fixed platform, a land structure, a water structure, and a space structure, a power plant, a bridge, a dam, a house, a manufacturing facility, and a building.

[0087] Next, referring to Figure 3, an example of a process flow for modifying the visualization of the internal design is shown according to an exemplary embodiment. The process flow in this exemplary example can be implemented in the design system 202 using the design generator 214 in Figure 2.

[0088] In this example, reference image 300 is an image of the interior design of the cabin to be modified. Reference image 300 can be located in an image database for interior designs and can be generated from the interior design or from some other source. For example, the interior design could be a computer-aided design (CAD) model of the aircraft's interior. Images can be generated from this computer-aided design model.

[0089] This image can be viewed by a human operator on a display system. The human operator can then generate input 301 to specify which parts of the reference image 300 should be modified. These parts can be several elements within the reference image 300. These elements can be selected from a group including, for example, overhead bins, passenger seats, in-flight entertainment centers, aisles, and other elements.

[0090] These elements within reference image 300 may be of the same type or of different types. For example, some elements may include overhead bins and passenger seats, while in other examples, some elements may be overhead bins. In yet another exemplary example, some elements may consist of just one overhead bin within an overhead bin.

[0091] In this exemplary example, input 301 can be generated by a human operator via a human-machine interface to select one or more elements to modify within the reference image 300. Input 301 can take several different forms. For example, input 301 may be at least one of bounding box information 302, text 303, or speech 304.

[0092] The bounding box information 302 can be generated in several different ways. For example, the bounding box information 302 can be generated using a touch gesture to draw a bounding box around the element to be modified in the reference image 300. For example, the bounding box information 302 can be information about where a particular element is located within the reference image 300. This bounding box information may include the center of the bounding box, as well as the width and height of the bounding box surrounding one or more elements within the reference image 300.

[0093] In this example, text 303 can be a text description of an element to be changed within the reference image 300. For example, text 303 could describe several elements to be changed. For example, text 303 could be at least one of the following: overhead bins, passenger seats, aisles, or some other text to identify the element to be changed.

[0094] Audio 304 is audio information describing several elements that need to be changed. Audio 304 is the same description as text 303, but may also be in audio format.

[0095] These inputs form a description 305 of several target elements used to identify those elements within the reference image 300. The description 305 of several target elements can be an example of element selection 222 in Figure 2. In this example, the target element is the overhead bin 315 in the reference image 300.

[0096] In an exemplary example, a reference image 300 and descriptions 305 of several elements of the subject are inputs to a machine learning model 361. This machine learning model can be a fully generative artificial intelligence model. An image encoder 310 within the machine learning model 361 receives the reference image 300 and outputs a numerical representation of the reference image 300 that can be used by the machine learning model. A text encoder 311 within the machine learning model 361 receives a description 305 of several elements of the subject and outputs a numerical representation of this description that can be used by the machine learning model.

[0097] The mask decoder 312 within the machine learning model 361 receives a description 305 of several elements of interest, and the machine learning model outputs a numerical representation of this description. The outputs of these two encoders are aligned in a common latent space using the cross-attention 313 of the machine learning model 361.

[0098] These outputs are received by a mask decoder 312 within the machine learning model 361. The mask decoder 312 identifies and highlights the overhead bins 315 to generate a reinforced image 314. This mechanism for identifying the overhead bins 315 can be a segmented mask, bounding box prediction, or heatmap mechanism. The mask decoder 312 outputs a reinforced image 314 in which the overhead bins 315 are highlighted or otherwise graphically identified.

[0099] Next, the highlighted image 314 and text 330 are inputs for the spread 331 in the generative artificial intelligence model 332. In this example, the spread 331 may include the steps of generating a noisy image from the highlighted image 314 and forming noise to generate a customized image 370. When generating a noisy image, noise can be gradually added to the highlighted image 314 over multiple steps to transform this image into a noisy version. During training, the generative artificial intelligence model 332 learns how the data behaves as it becomes increasingly noisy.

[0100] When performing denoising, the generative artificial intelligence model 332 is trained to reverse the denoising process. In this example, the spread 331 within the generative artificial intelligence model 332 starts with a noisy input. The spread 331 iteratively removes the noise and reconstructs the data stepwise to form an image. This denoising in the spread 331 within the generative artificial intelligence model 332 can be performed through a series of steps that generate a customized image 370 using text 330 and engineering data 380.

[0101] In this example, text 330 identifies an elemental modification of the overhead bin 315. For example, text 330 could be "Aircraft cabin with integrated large screen" used to modify the overhead bin 315 from the highlighted image 314. In this example, diffusion 331 is performed using the highlighted image 314 and text 330 to generate a customized image 370 having an integrated screen 371 instead of the overhead bin 315.

[0102] The diffusion 331 performed on the enhanced image 314 to generate a customized image 370 having an integrated screen 371 is performed according to engineering data 380, which is also input to a generative artificial intelligence model 332 for performing the diffusion 331. The engineering data is used to provide constraints for modifications to the overhead bin 315. For example, the engineering data 380 can define the maximum size of the integrated screen 371.

[0103] In these examples, the spread 331 of the generative artificial intelligence model 332 can be implemented with several different types of models. For example, but not limited to, the generative artificial intelligence model 332 can be selected from a group that includes, but is not limited to, spread models, latent spread models, DALL-E 2 models, denoising spread stochastic models, style domain adaptive models, style transfer models, generative adversarial networks, autoencoders, Gaussian splatting models, NeRF models, or trilinear point splatting models.

[0104] Furthermore, generative artificial intelligence models can be fully generative artificial intelligence models. These types of AI models can autonomously generate new data, such as images, without requiring detailed inputs or conditions that cover all aspects of image generation. In these examples, image generation may include elemental modifications to the current image.

[0105] Next, referring to Figure 4, an example of a process flow for modifying the visualization of the internal design is shown according to an exemplary embodiment. The process flow in this exemplary example can be implemented in the design system 202 using the design generator 214 in Figure 2.

[0106] In this example, text 400 is input to a complete generative artificial intelligence model 401. In this example, the model generates an image of the internal design for modification. For example, the complete generative artificial intelligence model 401 can generate a reference image 300 of Figure 3 for modification. This image can be generated using an internal design in the form of computer-aided design. For example, the computer-aided design model is an aircraft. Text 400 is "Modify the passenger seats of aircraft order number xxx to have a carbon fiber appearance." In this example, using text 400, the complete generative artificial intelligence model 401 generates a reference image 420 of passenger seats 421 using a computer-aided design model for an aircraft to be manufactured with aircraft order number xxx.

[0107] The mask generation unit 402 generates enhanced images 423 of the passenger seats 421 in which these passenger seats are highlighted. In this example, the machine learning model 361 shown in Figure 3 can be used to form a mass generation. In this example, the text 400 also includes element selection to select passenger seats for correction.

[0108] Next, diffusion 422 is performed to correct the passenger seats from the highlighted image 423. In this example, the correction is also identified from the text 400. This correction is influenced by engineering data in the form of a two-dimensional layout 424. In this example, the highlighted image 423, the two-dimensional layout 424, and the text 400 are inputs to diffusion 422.

[0109] Diffusion 422 generates a customized image 425 having passenger seats 421 having the appearance of carbon fiber. A two-dimensional layout 424 is used by a denoising process in diffusion 422 to ensure that the passenger seats 421 in the customized image 425 follow the layout of passenger seats in the two-dimensional layout 424.

[0110] Therefore, in this example, text 400 is a single input that selects the reference image, identifies the elements to be modified, and identifies the modifications to be made.

[0111] The process flow examples in Figures 3 and 4 are illustrative implementations of process flows that can be implemented by the design generator 130 in Figure 1 and the design generator 214 in Figure 2 to modify the internal design, and are not intended to limit the ways in which other examples can be implemented. For example, other illustrative examples could modify the internals of other types of vehicles other than aircraft.

[0112] Figures 5-7 illustrate a process flow for generating customized images that can be produced for vehicle and other platform designs. The modifications shown in these images can be applied to the aircraft's internal design, subject to approval of the generation and modification of the customized images.

[0113] Referring to Figure 5, an example of a process flow for customizing the interior of a guest room is shown according to an exemplary embodiment. The process flow in this example can be implemented using the design system 202 in Figure 2.

[0114] In this example, reference image 500 is the reference image of guest room 501. In this example, input 502 is both element selection, which selects elements for modification, and element modification, which describes the design of guest room 501.

[0115] In this example, input 502 is "Large in-flight display for middle-row business class" and can take several different forms. For example, input 502 could be at least one of text or audio in this example. This input selects both the elements for correction and the corrections to be made.

[0116] A customized image 504 of cabin 501 is generated. In this example, a large in-flight display 505 is added to cabin 501 in the customized image 504.

[0117] Referring to Figure 6, an example of a process flow for customizing the interior of a guest room is shown according to an exemplary embodiment. The process flow in this example can be implemented using the design system 202 in Figure 2.

[0118] In this example, reference image 600 is the reference image of guest room 601. In this example, input 602 is both an element selection, which selects elements for modification, and an element modification, which describes the design of guest room 601.

[0119] In this example, input 602 is both an element selection, which selects an element for modification, and an element modification, which describes a change to the design of the cabin 601. As shown in the figure, input 602 is "red bottom cushion for middle row first class," and this input can take several different forms. For example, input 602 could be at least one of text or audio in this example. The element selected for modification by input 602 is a bottom cushion 608 for the passenger seat 607.

[0120] A customized image 604 of cabin 601 is generated in response to the input 602. In this example, a red bottom cushion 605 has been added to the passenger seat 607 in cabin 601 in the customized image 604.

[0121] Next, Figure 7 shows an example of a process flow for customizing an aircraft structure according to an exemplary embodiment. This example process flow can be implemented using the design system 202 in Figure 2.

[0122] As shown in the figure, the reference image 700 is an image of the aircraft structure 701. In this example, input 702 selects the elements for modification and the modifications to the design of the aircraft structure 701. In this example, input 702 is a "touch gesture to move the pipe" in the direction of arrow 710.

[0123] In response to input 702, a customized image 704 for the aircraft structure 701 is generated. As shown in the customized image 704, pipe 703 is moved in the direction of arrow 710 within the customized image 704. In this way, the design engineer can see how the aircraft structure 701 will look with the pipe 703 moved.

[0124] The example process flows in Figures 6 and 7, provided as examples, are not intended to limit the ways in which other exemplary examples can be implemented. For example, these process flows can be applied to the interiors of other vehicles such as trains and buses. Furthermore, in addition to vehicles, the process flows can also be applied to different platforms such as bridges, manufacturing facilities, auditoriums, or other platforms.

[0125] The different customized images generated in Figures 5-7 have modifications from a reference image that can be applied to the design of a specific platform. These modifications can be applied in response to the generation of the customized images or in response to the approval of the modifications to the customized images. In this way, designs such as computer-aided design files can be modified using this process flow. Furthermore, the modified designs can then be implemented in the manufacturing or reconfiguration of existing platforms.

[0126] These different examples generate customized images based on the design and take engineering data into consideration. For example, modifications made to the input, although not shown, are made taking the design's engineering data into account.

[0127] Engineering data can provide constraints on modifications that may be made. For example, the color of the bottom seat may be affected by the material or the acceptable color for a particular customer. Another example is that the size of in-flight displays may be limited by the amount of space specified in the design specifications for a particular class of cabin.

[0128] Referring now to Figure 8, an illustrative flowchart of a process for generating changes to the internal design of an aircraft is shown according to an exemplary embodiment. The process in Figure 8 can be implemented in hardware, software, or both. When implemented in software, the process can take the form of program instructions executed by one or more processor units located in one or more hardware devices within one or more computer systems. For example, the process can be implemented in the design generator 130 of the server computer 104 in Figure 1 and in the design generator 214 of the computer system 212 in Figure 2.

[0129] The process identifies reference images for the aircraft's interior design (operation 800). The process identifies engineering data for the design (operation 802).

[0130] The process receives element selection of several elements in a reference image of the design for modification (operation 804). The process generates a reinforced image of the design using several elements in the reference image, element selection, and a machine learning model system, and several elements in the reinforced image are several reinforced elements (operation 806).

[0131] The process receives element corrections to modify some of the highlight elements in the highlight image (operation 808). The process modifies some of the highlight elements using element corrections that take engineering data into account using a machine learning model system to form a customized image of the design that has some correction elements instead of some of the highlight elements (operation 810).

[0132] The process displays the customized image on the display system (operation 812). The process then terminates.

[0133] In this flowchart, the machine learning model system comprises a first generative artificial intelligence model trained to generate enhanced images. This machine learning model system also comprises a second generative artificial intelligence model trained to generate customized images. The second generative artificial intelligence model can be selected from a group that includes diffusion models, latent diffusion models, DALL-E 2 models, denoising diffusion stochastic models, and other suitable types of machine learning models capable of performing diffusion.

[0134] Next, Figure 9 shows an example flowchart of the process for displaying a highlight image according to an exemplary embodiment. The actions in this flowchart are examples of additional actions that can be performed with the operation shown in Figure 8.

[0135] The process displays the highlighted image with several highlighting elements graphically emphasized on the display system (action 900). The process then terminates.

[0136] Referring now to Figure 10, an example flowchart of the process for generating enhanced images is shown according to an exemplary embodiment. The process in this flowchart is an example of an implementation of operation 806 in Figure 8. In this flowchart, different operations can be performed using the machine learning model system 150 in Figure 1 and the machine learning model system 225 in Figure 2.

[0137] The process begins with a machine learning model system identifying several elements in a reference image using element selection (Action 1000). The process then creates a mask by the machine learning model system that identifies pixels representing the identified elements in the reference image (Action 1002).

[0138] The process involves a machine learning model system modifying pixels representing certain elements to create several modified regions for those elements, thereby forming a enhanced image with several enhanced elements. The enhanced elements within the modified regions are visually distinguishable from other elements outside the modified regions in the enhanced image (operation 1004). The process then terminates.

[0139] Referring to Figure 11, an example flowchart of the process for modifying several emphasis elements is shown according to an exemplary embodiment. The operation of this flowchart is an example of an implementation of operation 810 in Figure 8. These operations can be performed using the machine learning model system 150 in Figure 1 and the machine learning model system 225 in Figure 2.

[0140] The process uses a machine learning model system to identify a set of changes to highlight elements using elemental modifications that take engineering data into account (operation 1100). The process then uses the identified set of changes to spread from noise to the customized image, generating a customized image with changes to several highlight elements to form the modified elements within the customized image (operation 1102). The process then terminates.

[0141] Referring now to Figure 12, an illustrative flowchart of a process for generating changes to a vehicle design is shown according to an exemplary embodiment. The process in Figure 12 can be carried out in hardware, software, or both. When carried out in software, the process can take the form of program instructions executed by one or more processor units located in one or more hardware devices within one or more computer systems. For example, the process can be carried out in the design generator 130 of the server computer 104 in Figure 1 and in the design generator 214 of the computer system 212 in Figure 2.

[0142] The process begins by identifying a reference image for the vehicle design (operation 1200). The process then identifies engineering data for the design (operation 1202).

[0143] The process receives element selection of several elements in a reference image of the design for modification (operation 1204). The process generates a reinforced image of the design using several elements in the reference image, element selection, and a machine learning model system, and several elements in the reinforced image are several reinforced elements (operation 1206).

[0144] The process receives element corrections to modify some of the highlight elements in the highlight image (operation 1208). The process modifies some of the highlight elements using element corrections that take engineering data into account using a machine learning model system to form a customized image of the design that has some correction elements instead of some of the highlight elements (operation 1210).

[0145] The process displays the customized image on the display system (operation 1212). Afterward, the process terminates.

[0146] The flowcharts and block diagrams in various illustrated embodiments illustrate the architecture, functionality, and operation of several possible implementations of the apparatus and method in the exemplary embodiments. In this regard, each block in the flowchart or block diagram may represent at least one of a module, segment, function, or part of an operation or step. For example, one or more blocks may be implemented as program instructions, hardware, or a combination of program instructions and hardware. When implemented in hardware, the hardware may take the form of an integrated circuit manufactured or configured to perform one or more operations of the flowchart or block diagram, for example. When implemented as a combination of program instructions and hardware, this implementation may take the form of firmware. Each block in the flowchart or block diagram may be implemented using a dedicated hardware system that performs a different operation, or using a combination of dedicated hardware and program instructions executed by the dedicated hardware.

[0147] In some alternative implementations of the exemplary embodiments, one or more functions described in a block may occur in an order other than that shown in the figure. For example, in some cases, two consecutively shown blocks may be executed substantially simultaneously, or blocks may sometimes be executed in reverse order depending on the associated functionality. In addition, other blocks may be added in addition to those shown in the flowchart or block diagram.

[0148] Referring now to Figure 13, an illustrative block diagram of a data processing system is shown according to an exemplary embodiment. The data processing system 1300 can be used to implement the server computers 104 and 106 and client device 110 in Figure 1. The data processing system 1300 can also be used to implement the computer system 212 in Figure 2. In this exemplary example, the data processing system 1300 includes a communication framework 1302 that provides communication between a processor unit 1304, memory 1306, persistent storage 1308, communication unit 1310, input / output (I / O) unit 1312, and display 1314. In this example, the communication framework 1302 takes the form of a bus system.

[0149] The processor unit 1304 is responsible for executing instructions for software that can be loaded into memory 1306. The processor unit 1304 includes one or more processors. For example, the processor unit 1304 can be selected from at least one of the following: a multicore processor, a central processing unit (CPU), a graphics processing unit (GPU), a physics processing unit (PPU), a digital signal processor (DSP), a network processor, or any other suitable type of processor. Furthermore, the processor unit 1304 can be implemented using one or more heterogeneous processor systems in which the primary processor resides together with secondary processors on a single chip. As another exemplary example, the processor unit 1304 can be a symmetrical multiprocessor system containing multiple processors of the same type on a single chip.

[0150] Memory 1306 and persistent storage device 1308 are examples of storage device 1316. A storage device is any hardware capable of storing at least one of the following information, such as data, program instructions in a functional form, or other suitable information, temporarily, persistently, or both temporarily and persistently. In these exemplary examples, storage device 1316 may also be called a computer-readable storage device. In these examples, memory 1306 can be, for example, random-access memory, or any other suitable volatile or non-volatile storage device. Persistent storage device 1308 can take various forms depending on the specific implementation.

[0151] For example, the persistent storage device 1308 may include one or more components or devices. For example, the persistent storage device 1308 may be a hard drive, a solid-state drive (SSD), flash memory, a rewritable optical disk, a rewritable magnetic tape, or any combination of the above. The medium used by the persistent storage device 1308 may also be removable. For example, a removable hard drive may be used for the persistent storage device 1308.

[0152] In these exemplary examples, the communication unit 1310 provides communication with other data processing systems or devices. In these exemplary examples, the communication unit 1310 is a network interface card.

[0153] The input / output unit 1312 enables data input and output with other devices that can be connected to the data processing system 1300. For example, the input / output unit 1312 may provide a connection for user input via at least one of a keyboard, mouse, or any other suitable input device. Furthermore, the input / output unit 1312 may also send output to a printer. The display 1314 provides a mechanism for displaying information to the user.

[0154] Instructions for at least one of an operating system, an application, or a program may reside in a storage device 1316 that communicates with the processor unit 1304 via a communication framework 1302. Processes of different embodiments may be executed by the processor unit 1304 using computer implementation instructions that may reside in memory, such as memory 1306.

[0155] These instructions are referred to as program instructions, computer-readable program instructions, or computer-readable program instructions, which can be read and executed by the processor in the processor unit 1304. In various embodiments, program instructions can be implemented on various physical media or computer-readable storage media, such as memory 1306 or persistent storage device 1308.

[0156] The program instruction 1318 is functionally located on a selectively removable computer-readable medium 1320 and can be loaded or transferred to a data processing system 1300 for execution by a processor unit 1304. The program instruction 1318 and the computer-readable medium 1320 form a computer program product 1322 in these exemplary examples. In the exemplary examples, the computer-readable medium 1320 is a computer-readable storage medium 1324.

[0157] The computer-readable storage medium 1324 is a physical or tangible storage device used to store program instructions 1318, rather than a medium for propagating or transmitting program instructions 1318. The computer-readable storage medium 1324 may be at least one of the following: electronic storage medium, magnetic storage medium, optical storage medium, electromagnetic storage medium, semiconductor storage medium, mechanical storage medium, or other physical storage medium. Some known types of storage devices, including these mediums, include diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital multipurpose disk (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or pits / lands formed on the main surface of a disk, or any suitable combination thereof.

[0158] Computer-readable storage medium 1324 should not be interpreted, when used in this disclosure, as a storage device for the form of a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides, optical pulses passing through optical fiber cables, electrical signals communicated through wiring, or at least one of other transmission media.

[0159] Furthermore, data may be moved at some point during the normal operation of the storage device. These normal operations include access, defragmentation, or garbage collection. However, these operations do not make the storage device temporary, as the data is not temporary while it is stored in the storage device.

[0160] Alternatively, the program instruction 1318 may be transmitted to the data processing system 1300 using a computer-readable signal medium. The computer-readable signal medium is a signal, and can be, for example, a propagated data signal containing the program instruction 1318. For example, the computer-readable signal medium can be at least one of an electromagnetic signal, an optical signal, or any other suitable type of signal. These signals can be transmitted via connections such as wireless connections, fiber optic cables, coaxial cables, wiring, or any other suitable type of connection.

[0161] Furthermore, as used herein, “computer-readable medium 1320” can be singular or plural. For example, program instructions 1318 may reside in computer-readable medium 1320 in the form of a single storage device or system. In another example, program instructions 1318 may reside in computer-readable medium 1320 distributed across multiple data processing systems. In other words, some instructions within program instructions 1318 may reside in one data processing system, while other instructions within program instructions 1318 may reside in one data processing system. For example, part of program instructions 1318 may reside in computer-readable medium 1320 in a server computer, while other parts of program instructions 1318 may reside in computer-readable medium 1320 located in a set of client computers.

[0162] The various components illustrated with respect to the data processing system 1300 are not intended to provide architectural limitations on how various embodiments can be carried out. In some exemplary examples, one or more components may be incorporated into another component or form part of another component. For example, memory 1306 or part thereof may be incorporated into processor unit 1304 in some exemplary examples. Various exemplary embodiments can be carried out in a data processing system that includes components in addition to, or alternative to, those illustrated with respect to the data processing system 1300. Other components shown in Figure 13 may differ from the illustrated exemplary examples. Various embodiments can be carried out using any hardware device or system capable of executing program instructions 1318.

[0163] Exemplary embodiments of this disclosure may be described in relation to the aircraft manufacturing and maintenance method 1400 shown in Figure 14 and the aircraft 1500 shown in Figure 15. Referring first to Figure 14, an exemplary block diagram of the aircraft manufacturing and maintenance method is shown according to an exemplary embodiment. In the interim until production begins, the aircraft manufacturing and maintenance method 1400 may include the specifications and design 1402 and material procurement 1404 of the aircraft 1500 shown in Figure 15.

[0164] During production, the components and subassemblies of the aircraft 1500 in Figure 15 are manufactured 1406 and system integration 1408 is carried out. Subsequently, the aircraft 1500 in Figure 15 may undergo certification and transport 1410 to enter service 1412. Once the aircraft 1500 in service 1412 is in service with the customer, routine maintenance and inspections 1414 are planned for the aircraft 1500 in Figure 15, and routine maintenance and inspections 1414 may include corrections, reconfigurations, modifications, and other maintenance or inspections.

[0165] Each of the processes in the Aircraft Manufacturing and Maintenance Method 1400 may be performed or implemented by a system integrator, a third party, an operator, or a combination thereof. In these examples, the operator may be a customer. For the purposes of this explanation, the system integrator may include, but is not limited to, any number of aircraft manufacturers and subcontractors of large systems; the third party may include, but is not limited to, any number of vendors, subcontractors, and suppliers; and the operator may be an airline, leasing company, military, and service organization, etc.

[0166] Referring now to Figure 15, an illustrative block diagram of an aircraft capable of carrying out one exemplary embodiment is shown. In this example, the aircraft 1500 may include a fuselage 1502 having several systems 1504 and internal 1506, produced by the aircraft manufacturing and maintenance inspection method 1400 of Figure 14. Examples of systems 1504 include one or more of the propulsion system 1508, electrical system 1510, hydraulic system 1512, and environmental system 1514. Any number of other systems may be included. Although an aerospace example is shown, various exemplary embodiments may be applied to other industries such as the automotive industry.

[0167] The apparatus and methods embodied herein may be used during at least one of the steps of the aircraft manufacturing and maintenance inspection method 1400 shown in Figure 14.

[0168] In one exemplary example, components or subassemblies produced in the production of components and subassemblies 1406 of Figure 14 can be manufactured or produced in a similar manner to components or subassemblies produced while the aircraft 1500 is in service 1412 of Figure 14. In yet another example, one or more device embodiments, method embodiments, or combinations thereof can be used in production stages such as the production of components and subassemblies 1406 of Figure 14, and system integration 1408. One or more device embodiments, method embodiments, or combinations thereof may be used while the aircraft 1500 is in service 1412, during maintenance and inspection 1414 of Figure 14, or both. The use of several different exemplary embodiments may substantially accelerate the assembly of the aircraft 1500, reduce the cost of the aircraft 1500, or accelerate the assembly of the aircraft 1500 and reduce the cost of the aircraft 1500.

[0169] The design generator in different exemplary examples can be used in at least one of the specifications and designs 1402 and the maintenance and service inspection 1414. In the specifications and designs 1402, the design generator can be used to reduce the amount of time required to make design changes or design updates to the aircraft 1500 being manufactured for the customer. Various changes to elements in the design of the aircraft 1500 can be made in a manner that reduces or eliminates the need for design engineers and designers to make the requested changes and verify that they can be made. Furthermore, the design generator can be used to make changes to the aircraft after the aircraft 1500 has been manufactured. These changes can be made to the maintenance and service inspection 1414, including modifications, reconfigurations, refits, and other maintenance or services. For example, passenger seat reconfigurations can be made more quickly. As another example, alternative seats with different designs, including different colors, sizes, dimensions, and materials, can be produced.

[0170] Therefore, the exemplary examples provide methods, apparatus, systems, and computer program products that enable modification of the aircraft interior design presented to the customer based on customer feedback. In these different exemplary examples, the time requirements for collaboration between the customer, design engineers, and 3D artists to design and render the interior are generated using a design system such as the design system 202 in Figure 2.

[0171] In one exemplary example, the internal design system comprises a computer system, a generative artificial intelligence model system within the computer system, and a design generator within the computer system. The design generator is configured to identify a reference image of the internal design of the vehicle's interior, identify engineering data for the internal design, and receive element selections of several elements within the reference image of the internal design for modification. The design generator is configured to generate an enhanced image of the internal design using several elements in the reference image, element selections, and the machine learning model system. Some elements of the enhanced image are several enhanced elements. Element modifications are received to modify some of the enhanced elements in the enhanced image. The design generator is configured to modify some of the enhanced elements using element modifications that take engineering data into account and use the machine learning model system to form a customized image of the internal design having some modified elements instead of some of the enhanced elements. The design generator is configured to display the customized image on a display system.

[0172] In a different illustrative example, the generation of a customized image with modifications to selected elements within an image for design is based on inputs that select elements and identify modifications. Furthermore, these modifications take into account engineering data that provides information about the elements. This information may also include what changes can be made to the elements. As a result, the changes made based on the identified modifications are changes that could actually be made to the design, because the engineering data for the design is taken into account.

[0173] As a result, the number of iterations between customers, design engineers, 3D artists, and other personnel can be reduced. The process flow implemented in the example design system uses machine learning models in a way that reduces the need for multiple iterations of the process, including design engineers, 3D artists, and customers.

[0174] In one example, a different illustrative design system can be used to generate a first pass for designing the interior of an aircraft, such as the aircraft cabin. The design systems in these examples allow for multiple iterations based on customer feedback, while reducing the time and cost required for design revisions or changes.

[0175] Furthermore, these modifications to the customized image can be implemented within the design to enable more efficient execution of at least one of the manufacturing or reconfiguration platforms, such as aircraft or other vehicles.

[0176] The descriptions of various exemplary embodiments are presented for illustrative and explanatory purposes only and are not intended to be exhaustive or to be limited to embodiments of the form disclosed. The various exemplary examples describe components that perform an action or operation. In the exemplary embodiments, components can be configured to perform the described action or operation. For example, a component may have a structural configuration or design that gives the component the ability to perform the action or operation described in the exemplary example as being performed by the component. Furthermore, the terms “includes,” “includes,” “has,” “contains,” and their variations thereof, as used herein, are intended to be as comprehensive as the term “comprises” as an open transitional term that does not exclude any additional or other elements.

[0177] Many modifications and variations will be apparent to those skilled in the art. Furthermore, various exemplary embodiments may offer different features compared to other preferred embodiments. One or more selected embodiments have been chosen and described to best illustrate the principles and practical applications of the embodiments and to enable those skilled in the art to understand the disclosure for various embodiments with various modifications suitable for specific intended uses. [Explanation of Symbols]

[0178] 100 Network data processing system, 102 Network, 104 Server computer, 106 Server computer, 108 Memory unit, 110 Client device, 112 Client computer, 114 Client computer, 116 Client computer, 118 Mobile phone, 120 Tablet computer, 122 Smart glasses, 130 Design generator, 131 Human operator, 132 Internal design, 133 Commercial aircraft, 134 Reference image, 135 Enhanced image, 136 Customized image, 137 Engineering data, 150 Machine learning model system, 160 Facility, 200 Design environment, 201 Design, 202 Design system, 203 Vehicle, 204 Aircraft, 206 Interior, 207 Aircraft cabin, 209 Human operator, 211 Human-machine interface, 212 Computer system, 213 Graphical user interface, 214 Design generator, 216 Processor unit, 218 Program instructions, 219 Input system, 220 Reference image, 221 Engineering data, 222 Element selection, 223 Element, 224 Enhanced image, 225 Machine learning model system, 226 Machine learning model, 227 Enhanced element, 228 Element modification, 229 Customized image, 230 Modified element, 231 Display system, 241 Mask, 242 Pixel, 243 Modified area, 250 Change, 290 Platform, 300 Reference image, 301 Input, 302 Bounding box information, 303 Text, 304 Audio, 305 Description of the number of target elements, 310 Image encoder, 311 Text encoder, 312 Mask decoder, 313 Cross attention, 315 Overhead bin, 330 Text, 331 Diffusion, 332 Generative artificial intelligence model, 361 Machine learning model, 370 Customized image, 371 Integrated screen, 380 engineering data, 400 text, 401 fully generative artificial intelligence model, 402 mask generation unit, 420 reference image, 421 passenger seat, 422 diffusion, 423 enhanced image, 424 2D layout, 425 customized image, 500 reference image, 501 cabin, 502 input, 504Customized images, 505 Large in-flight displays, 600 Reference images, 601 Cabin, 602 Inputs, 604 Customized images, 607 Passenger seats, 608 Bottom cushions, 700 Reference images, 701 Aircraft structure, 702 Inputs, 703 Pipes, 704 Customized images, 710 Arrows, 1300 Data processing systems, 1302 Communication framework, 1304 Processor units, 1306 Memory, 1308 Persistent storage devices, 1310 Communication units, 1312 Input / Output units, 1314 Displays, 1316 Storage devices, 1318 Program instructions, 1320 Computer-readable media, 1322 Computer program products, 1324 Computer-readable storage media, 1400 Aircraft manufacturing and maintenance inspection methods, 1402 Specifications and designs, 1404 Material procurement, 1406 Manufacturing of components and subassemblies, 1408 System integration, 1410 Certification and transport, 1412 In service, 1414 Maintenance and inspection, 1500 Aircraft, 1502 Airframe, 1504 Systems, 1506 Interior, 1508 Propulsion systems, 1510 Electrical systems, 1512 Hydraulic systems, 1514 Environmental systems

Claims

1. Computer systems and, The machine learning model system within the aforementioned computer system, The design generator in the aforementioned computer system, wherein the design generator is Steps include identifying a reference image of the aircraft's interior design, A step of identifying engineering data for the aforementioned design, A step of receiving element selections of several elements in the reference image of the design for modification, The process involves generating a reinforced image of the design using the aforementioned elements in the reference image, the element selection, and the machine learning model system, wherein the aforementioned elements in the reinforced image are several reinforced elements, The step of receiving element corrections to modify some of the emphasis elements in the emphasis image, The steps include modifying the aforementioned emphasis elements using the element modification that takes the engineering data into account using the machine learning model system in order to form a customized image of the design having several modification elements instead of the aforementioned emphasis elements, The steps include displaying the customized image on the display system and A design generator configured to perform operations including and A design system equipped with the following features.

2. The aforementioned design generator is The step of displaying the enhanced image in a state in which the aforementioned enhancement elements are graphically enhanced on the display system. The design system according to claim 1, further configured to perform operations including the following:

3. The design system according to claim 1, wherein the machine learning model system comprises a first generative artificial intelligence model trained to generate the enhanced image and a second generative artificial intelligence model trained to generate the customized image.

4. The design system according to claim 3, wherein the second generative artificial intelligence model is selected from the group including a diffusion model, a latent diffusion model, a DALL-E 2 model, a denoising diffusion stochastic model, a style domain adaptive model, a style transfer model, a generative adversarial network, an autoencoder, a Gaussian splatting model, a NeRF model, or a trilinear point splatting model.

5. When generating the enhanced image, the design generator, The machine learning model system includes the steps of identifying several elements in the reference image using the element selection, The machine learning model system creates a mask that identifies pixels representing some of the elements identified in the reference image, A step of modifying the pixels representing the elements by the machine learning model system in order to create several modification regions for the elements and form the enhanced image having the elements, wherein the elements within the modification regions are visually distinguishable from other elements outside the modification regions in the enhanced image. The design system according to claim 1, configured to perform operations including the following:

6. When modifying the aforementioned emphasis elements, the design generator, The machine learning model system includes the steps of identifying a set of changes to the emphasis element using the element modification that takes the engineering data into account, To form the modification elements within the customized image, the machine learning model system performs a step of spreading the identified set of modifications from noise to the customized image in order to generate the customized image with modifications to some of the enhancement elements. The design system according to claim 1, configured to perform operations including the following:

7. The design system according to claim 6, wherein the set of modifications to the aforementioned emphasis elements is selected from at least one of color, material, dimensions, shape, position, location, orientation, surface finish, or coating.

8. The design system according to claim 1, wherein the aforementioned elements are selected from at least one of passenger seats, overhead bins, several aisles, seat cushions, doors, lights, lighting systems, in-flight entertainment systems, several rows of passenger seats, or seat configurations.

9. The design system according to claim 1, wherein the element selection is selected from at least one of text, voice, or touch gesture.

10. The design system according to claim 1, wherein the several emphasis elements are identified in the emphasis image using at least one of color, highlight, brightness, border, pattern change, or animation.

11. The design system according to claim 1, wherein the engineering data defines several tolerances for at least one of the following: physics-based parameters, volume, material, dimensions, density, elasticity, stiffness, surface texture, temperature-based material behavior, size, position, orientation, or weight.

12. The design system according to claim 1, wherein a new aircraft is manufactured using the design for the customized image.

13. The design system according to claim 1, wherein an existing aircraft is reconfigured using the design for the customized image.

14. A method for generating changes to the internal design of an aircraft, the method being A step of identifying a reference image of the design inside the aircraft, A step of identifying engineering data for the aforementioned design, A step of receiving element selections of several elements in the reference image of the design for modification, A step of generating a reinforced image of the design using several elements in the reference image, the element selection, and a machine learning model system, wherein the several elements in the reinforced image are several reinforced elements, The step of receiving element corrections to modify some of the emphasis elements in the emphasis image, The steps include modifying the aforementioned emphasis elements using the element modification that takes the engineering data into account using the machine learning model system in order to form a customized image of the internal design having several modification elements instead of the aforementioned emphasis elements, The steps include displaying the customized image on the display system and A method that includes this.

15. The method according to claim 14, further comprising the step of displaying the enhanced image in which the several enhanced elements are graphically enhanced on the display system.

16. The method according to claim 14, wherein the machine learning model system comprises a first generative artificial intelligence model trained to generate the enhanced image and a second generative artificial intelligence model trained to generate the customized image.

17. The method according to claim 16, wherein the second generative artificial intelligence model is selected from the group including diffusion models, latent diffusion models, DALL-E 2 models, and denoising diffusion stochastic models.

18. The step of generating the enhanced image is: The machine learning model system includes the steps of identifying several elements in the reference image using the element selection, The machine learning model system creates a mask that identifies pixels representing some of the elements identified in the reference image, A step of modifying the pixels representing the elements by the machine learning model system in order to create several modification regions for the elements and form the enhanced image having the elements, wherein the elements within the modification regions are visually distinguishable from other elements outside the modification regions in the enhanced image. The method according to claim 14, including the method described in claim 14.

19. The step of modifying some of the aforementioned emphasis elements is: The machine learning model system includes the steps of identifying a set of changes to the emphasis element using the element modification that takes the engineering data into account, To form the modification elements within the customized image, the machine learning model system performs a step of spreading the identified set of modifications from noise to the customized image in order to generate the customized image with modifications to some of the enhancement elements. The method according to claim 14, including the method described in claim 14.

20. The method according to claim 19, wherein the set of modifications to the aforementioned emphasis elements is selected from at least one of color, material, dimensions, shape, position, location, orientation, surface finish, or coating.

21. The method according to claim 14, wherein the aforementioned elements are selected from at least one of passenger seats, overhead bins, several aisles, seat cushions, doors, lights, lighting systems, in-flight entertainment systems, several rows of passenger seats, or seat configurations.

22. The method according to claim 14, wherein the element selection is selected from at least one of text, voice, or touch gesture.

23. The method according to claim 14, wherein the aforementioned emphasis elements are identified in the emphasis image using at least one of color, highlight, brightness, border, pattern change, or animation.

24. The method according to claim 14, wherein the engineering data defines several tolerances for at least one of the following: physics-based parameters, volume, material, dimensions, density, elasticity, stiffness, surface texture, temperature-based material behavior, size, position, orientation, or weight.

25. The method according to claim 14, further comprising the step of manufacturing a new aircraft using the design for the customized image.

26. The method according to claim 14, further comprising the step of reconfiguring an existing aircraft using the design for the customized image.

27. A computer program product for generating changes to the internal design of an aircraft, wherein the computer program product is A set of one or more computer-readable storage media, A program instruction stored in the set of one or more storage media, A step of identifying a reference image of the design inside the aircraft, A step of identifying engineering data for the aforementioned design, A step of receiving element selections of several elements in the reference image of the design for modification, A step of generating a reinforced image of the design using several elements in the reference image, the element selection, and a machine learning model system, wherein the several elements in the reinforced image are several reinforced elements, The step of receiving element corrections to modify some of the emphasis elements in the emphasis image, The steps include modifying the aforementioned emphasis elements using the element modification that takes the engineering data into account using the machine learning model system in order to form a customized image of the internal design having several modification elements instead of the aforementioned emphasis elements, The steps include displaying the customized image on the display system and Program instructions that perform actions including A computer program product that includes the following features.

28. Computer systems and, The generative artificial intelligence model system within the aforementioned computer system, The design generator in the aforementioned computer system, wherein the design generator is Steps include identifying a reference image for the platform design, A step of identifying engineering data for the aforementioned design, A step of receiving element selections of several elements in the reference image of the design for modification, A step of generating a reinforced image of the design using several elements in the reference image, the element selection, and a machine learning model system, wherein the several elements in the reinforced image are several reinforced elements, The step of receiving element corrections to modify some of the emphasis elements in the emphasis image, The steps include modifying the aforementioned emphasis elements using the element modification that takes the engineering data into account using the machine learning model system in order to form a customized image of the design having several modification elements instead of the aforementioned emphasis elements, The steps include displaying the customized image on the display system and A design generator configured to perform operations including and An internal design system equipped with the following features.

29. The design system according to claim 28, wherein the design is for at least one of the internal, physical structure, or external aspects of the platform.

30. The design system according to claim 28, wherein the platform is selected from a group including aircraft, civilian aircraft, cargo aircraft, rotary-wing aircraft, tiltrotor aircraft, inclined-wing aircraft, vertical take-off and landing aircraft, unmanned aerial vehicles, artificial intelligence-controlled vehicles, electric vertical take-off and landing vehicles, personal air vehicles, surface ships, cruise ships, tanks, personnel carriers, trains, spacecraft, manned spacecraft, spacecraft, submarines, buses, automobiles, power plants, bridges, dams, houses, manufacturing facilities and buildings.

Images