Intelligent platform for design concept generation

Abstract

A system for generating design concepts includes a generative adversarial network (GAN) with a generator, a plurality of inspectors evaluating generated samples based on respective criteria, and a processor executing instructions. The processor generates design samples using the GAN generator, evaluates the generated samples using the plurality of inspectors, and adjusts parameters of the GAN based on feedback from the plurality of inspectors to produce diverse and novel design concepts. The plurality of inspectors may include a novelty inspector, a diversity inspector, and a desirability inspector. The system enables the generation of design concepts that are both innovative and aligned with user preferences.

Description

INTELLIGENT PLATFORM FOR DESIGN CONCEPT GENERATIONCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Application No. 63 / 613,104, titled "Intelligent Platform for Identifying, Embedding, and Optimizing the Integration of User Data into the Product Design and Development Process," fded December 21, 2023, the contents of which are hereby incorporated by reference in its entirety.FIELD OF INVENTION

[0002] The present disclosure relates to design concept generation systems, and more particularly to an intelligent platform for generating diverse and novel design concepts using generative adversarial networks and multi -criteria evaluation.STATEMENT OF GOVERNMENT SUPPORT

[0003] This invention was made with government support under Grant Number 2050052 awarded by the National Science Foundation. The Government has certain rights in the invention.BACKGROUND

[0004] In the realm of product design and development, the integration of customer feedback and sentiment analysis into the design process has become increasingly crucial. Traditionally, understanding customer preferences and needs has involved direct surveys and market research, which can be time-consuming and may not always capture the real-time and evolving desires of the market. Moreover, the rapid pace of technological advancements and changing consumer behaviors demand more agile and responsive design strategies that can adapt quickly to new information and insights.

[0005] Furthermore, the process of generating design concepts traditionally relies heavily on human creativity and iterative refinement, which can be resource-intensive and subjective. The challenge lies in efficiently generating a wide array of innovative and diverse design concepts that are not only technically feasible but also resonate with consumer expectations and preferences. Existing methods often struggle with balancing creativity with practical constraints, leading to a gap in achieving optimal design solutions that are both innovative and aligned with market demands.SUMMARY

[0006] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0007] The present invention provides a novel approach to generating diverse and novel design concepts using a generative adversarial network (GAN) architecture augmented with multiple specialized inspectors. This system significantly enhances the creativity and quality of generated designs while maintaining desirability and adherence to geometric constraints.

[0008] In a first aspect, the invention provides a system for generating design concepts, comprising a generative adversarial network (GAN) including a generator, a plurality of inspectors, each evaluating generated samples based on a respective criteria, and a processor executing instructions to generate design samples using the generator of the GAN, evaluate the generated samples using the plurality of inspectors, and adjust parameters of the GAN based on feedback from the plurality of inspectors to produce diverse and novel design concepts.

[0009] In one embodiment of the first aspect, the plurality of inspectors includes a novelty inspector evaluating uniqueness of the generated samples.

[0010] In another embodiment of the first aspect, the novelty inspector uses a Local Outlier Factor (LOF) algorithm to assess novelty of the generated samples.

[0011] In a further embodiment of the first aspect, the plurality of inspectors includes a diversity inspector evaluating variety among the generated samples.

[0012] In yet another embodiment of the first aspect, the diversity inspector uses a Covering Radius Upper Bound (CRUB) method to assess diversity of the generated samples.

[0013] In an additional embodiment of the first aspect, the plurality of inspectors includes a desirability inspector evaluating user satisfaction with the generated samples.

[0014] In a further embodiment of the first aspect, the desirability inspector uses a Deep Multimodal Design Evaluation (DMDE) model to assess desirability of the generated samples.

[0015] In a second aspect, the invention provides a method for generating design concepts, comprising generating design samples using a generative adversarial network (GAN) generator, evaluating the generated samples using a plurality of inspectors, each of the plurality of inspectors assessing a respective criterion, and adjusting parameters of the GAN generator based on feedback from the plurality of inspectors to produce diverse and novel design concepts.

[0016] In one embodiment of the second aspect, evaluating the generated samples includes assessing novelty using a Local Outlier Factor (LOF) algorithm.

[0017] In another embodiment of the second aspect, evaluating the generated samples includes assessing diversity using a Covering Radius Upper Bound (CRUB) method.

[0018] In a further embodiment of the second aspect, evaluating the generated samples includes assessing desirability using a Deep Multimodal Design Evaluation (DMDE) model.

[0019] In yet another embodiment of the second aspect, the method further comprises extracting features from the generated samples using a convolutional neural network and performing dimensionality reduction on the extracted features to visualize diversity.

[0020] In an additional embodiment of the second aspect, the convolutional neural network is a VGG16 network and the dimensionality reduction is performed using Principal Component Analysis (PCA).

[0021] In a further embodiment of the second aspect, the method includes applying stratified sampling to a pool of random latent codes to select diverse latent codes for generating the design samples.

[0022] In a third aspect, the invention provides a non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations for generating design concepts, the operations comprising generating design samples using a generative adversarial network (GAN) generator, evaluating the generated samples using multiple inspectors, each inspector assessing a different criterion, and adjusting parameters of the GAN generator based on feedback from the multiple inspectors to produce diverse and novel design concepts.

[0023] In one embodiment of the third aspect, evaluating the generated samples includes assessing novelty using a Local Outlier Factor (LOF) algorithm.

[0024] In another embodiment of the third aspect, evaluating the generated samples includes assessing diversity using a Covering Radius Upper Bound (CRUB) method.

[0025] In a further embodiment of the third aspect, evaluating the generated samples includes assessing desirability using a Deep Multimodal Design Evaluation (DMDE) model.

[0026] In yet another embodiment of the third aspect, the operations further comprise extracting features from the generated samples using a convolutional neural network and performing dimensionality reduction on the extracted features to visualize diversity.

[0027] In an additional embodiment of the third aspect, the convolutional neural network is a VGG16 network and the dimensionality reduction uses Principal Component Analysis (PCA).

[0028] The foregoing general description of the illustrative embodiments and the following detailed description thereof are merely exemplary aspects of the teachings of this disclosure and are not restrictive.BRIEF DESCRIPTION OF FIGURES

[0029] Non-limiting and non-exhaustive examples are described with reference to the following figures.

[0030] FIG. 1 is a flow diagram depicting one embodiment of a method for generating and evaluating design concepts based on customer sentiment.

[0031] FIG. 2 depicts a conceptual diagram of a product development process, according to an embodiment.

[0032] FIG. 3 illustrates a conceptual relationship diagram for product evaluation and sentiment analysis, in accordance with example embodiments.

[0033] FIG. 4 illustrates a process flow diagram for analyzing and categorizing design-related information, according to an aspect of the present disclosure.

[0034] FIG. 5 illustrates a latent need conceptual model for analyzing user needs in product design, according to aspects of the present disclosure.

[0035] FIG. 6 illustrates a system diagram of a generative adversarial network architecture for product design and evaluation, according to an embodiment.

[0036] FIG. 7 illustrates a system diagram of a generative adversarial network structure for design concept generation and evaluation, in accordance with example embodiments.

[0037] FIG. 8 illustrates comparative scatter plots of generated design samples, according to aspects of the present disclosure.

[0038] FIG. 9 illustrates a comparison of template matching confidence scores and generated sneaker designs, according to an embodiment.

[0039] FIG. 10 illustrates box plot diagrams comparing novelty and diversity assessments of generated design concepts, in accordance with example embodiments.DETAILED DESCRIPTION

[0040] The following description sets forth exemplary aspects of the present disclosure. It should be recognized, however, that such description is not intended as a limitation on the scope of the present disclosure. Rather, the description also encompasses combinations and modifications to those exemplary aspects described herein.

[0041] The present disclosure relates to an intelligent platform for design concept generation. The intelligent platform may utilize customer sentiment analysis and generative modeling techniques to produce diverse and novel design concepts.

[0042] FIG. 1 illustrates a general flow for a method 100 of generating and evaluating design concepts based on customer sentiment. The method 100 may comprise multiple phases for analyzing customer preferences, generating designs, and optimizing outputs. As shown in FIG. 1, the method 100 begins by exploring and comparing customer sentiment and opinions related to specific product attributes and needs (step 102). Identified customer sentiment may then be evaluated or predicted to create new design concepts (step 104). New designs may be generated or modified based on customer sentiment (step 106). New design concepts may be evaluated or optimized in real-time (step 108)

[0043] Still referring to FIG. 1, and in greater detail, the method 100 depicted begins by exploring and comparing customer sentiment and opinions related to specific product attributes and needs (step 102). In some cases, exploring and comparing customer sentiment may involve: analyzing social media posts and comments related to specific products or product categories; conducting online surveys with targeted customer segments; utilizing natural language processing techniques to extract sentiment from product reviews; implementing focus groups to gather in-depth feedback on product features; analyzing customer support tickets and inquiries for common themes and pain points; monitoring and analyzing search trends related to product attributes; employing sentiment analysis tools on customer emails and chat logs; analyzing purchase behavior and correlating it with expressed opinions; utilizing eye-tracking studies to understand customer attention to product features; implementing A / B testing on product descriptions or images to gauge customer preferences; analyzing competitor products' customer feedback for comparative insights; employing text mining techniques on industry forums and discussion boards; utilizing machine learning algorithms to identify patterns in customer feedback data; implementing interactive feedback mechanisms within product interfaces or apps; or analyzing customer returns data and reasons for product dissatisfaction. Step 102 may alternatively or additionally involve gathering and analyzing data on customer preferences and requirements.

[0044] Identified customer sentiment may then be evaluated or predicted to create new design concepts (step 104). Potential new designs may be analyzed based on the data gathered step 102.

[0045] New designs may be generated or modified based on customer sentiment (step 106). This step 106 may utilize the evaluated customer sentiment to create and refine design concepts. In some cases, new designs are created using a generative adversarial network (GAN) with a generator. In these cases, the GAN may further include a plurality of inspectors, each of the plurality of inspectors evaluating generated samples based on respective criteria.

[0046] In other cases, new designs may be created using evolutionary algorithms that simulate natural selection processes to iteratively improve design concepts based on fitness criteria derived from customer sentiment. In some other cases, new designs may be generated through parametric modeling techniques, where key design parameters are adjusted based on customer feedback and preferences to create variations of existing designs or entirely new concepts. Additionally, new designs may be developed using machine learning-based recommendation systems that analyze patterns in customer preferences and successful product attributes to suggest novel combinations and features for new design concepts.

[0047] New design concepts may be evaluated or optimized in real-time (step 108), allowing for immediate assessment and improvement of the generated designs.

[0048] The system may include a processor executing instructions to perform various functions. The processor may execute instructions to generate design samples using the generator of the GAN.Additionally, the processor may execute instructions to evaluate the generated samples using the plurality of inspectors. The processor may also execute instructions to adjust parameters of the GAN based on feedback from the plurality of inspectors to produce diverse and novel design concepts.

[0049] In some cases, the intelligent platform may be implemented as a non-transitory computer-readable medium storing instructions executed by a processor. The stored instructions may cause the processor to perform operations for generating design concepts when executed.

[0050] FIG. 2 illustrates conceptually a product development process 200 process. As shown in FIG. 2, and in brief overview, the process 200 process may have two main spaces: a need finding space 202 and a generative design space 204.

[0051] Still referring to FIG. 2, and in greater detail, the need finding space 202 may begin by generating a wide range of initial problem ideas 206. As the process 200 process progresses, the diamond may narrow, indicating a potential convergence of ideas. As shown in the embodiment depicted in FIG. 2, initial problem ideas may be sourced from product images, product descriptions, or customer reviews. In some of these cases, natural language processing may be used to elicit initial problem ideas. In addition to natural language processing, other techniques that may be used to identify problem ideas include: sentiment analysis tools to extract insights from customer emails, chat logs, and social media posts; eye-tracking studies to understand which product features or design elements capture customer attention; machine learning algorithms to identify patterns in large datasets of customer feedback, purchase behavior, and product usage; interactive feedback mechanisms within product interfaces or apps to gather real-time data on user behavior and preferences; analysis of customer returns data and reasons for product dissatisfaction; implementing A / B testing on product descriptions or images to gauge customer preferences; text mining techniques applied to industry forums and discussion boards to extract insights from informal customer conversations and discussions; and analyzing competitor products' customer feedback for comparative insights.

[0052] As noted above, in some cases, natural language processing (NLP) may be used to elicit initial problem ideas. Referring now to FIG. 3, a conceptual model useful in NLP processing for sentiment analysis and aspect extraction in product evaluations is illustrated. Sentiment analysis may also involve determining the overall emotional tone or attitude expressed in a piece of text, such as a product review 300. Aspect extraction may involve identifying specific features or characteristics of a product that are being discussed.

[0053] In some cases, sentiment analysis and aspect extraction may be used to break down direct opinions about a product into specific aspects and associate these aspects with performance or support characteristics. For example, as shown in FIG. 3, a direct opinion statement 304 such as "A lot of flexibility is provided with shoes" may be analyzed to extract the aspect 302 of "flexibility" and associate it with the performance characteristic of "Support / Stability".

[0054] In some aspects, a new annotated dataset may be created to facilitate use of the model in specific product design domains. First, a categorical ontology may be established, such as a collection of words and phrases that recurred most frequently in the product review. In some specific cases, annotators first, at the sentence level, annotate the aspect term in the sentence which may be either a noun or a verb. This represents the objective target of the review sentence. The opinion span may be the subjective portion of the review sentence and can be a word or phrase that directly or indirectly refers to the user's feelings about the aspect. These two categories are then connected together by their associated category and sentiment. It may be noted that a review sentence can contain multiple aspects and opinions and so multiple meanings can exist. In such cases, the sentence may be annotated more than once to capture the full content of the review sentence.

[0055] When performed, sentiment analysis process may involve categorizing the overall sentiment of the statement 300 as positive, negative, or neutral. In the example shown in FIG. 3, the statement expresses a positive sentiment about the flexibility of the shoes.

[0056] Aspect extraction may involve identifying specific product features or attributes mentioned in the text. In the example depicted in FIG. 3, "flexibility" may be extracted as a key aspect of the shoes being discussed.

[0057] In some cases, the extracted aspects may be associated with higher-level performance categories. As illustrated in FIG. 3, the aspect of "flexibility" is linked to the broader category 306 of "PerformanceSupport / Stability". This association may help in organizing and analyzing product feedback across multiple reviews and customers.

[0058] FIG. 4 illustrates a process flow for analyzing and categorizing design-related information. The process begins with tokens 404. As shown in FIG. 4, five tokens 404 are input: A (Aspect), C (Category), O (Opinion), S (Sentiment), and I (Implicit Indicator). These inputs may represent various aspects of design knowledge and user feedback.

[0059] The batch inputs can be processed through a Design Knowledge-Guided Position Encoding Algorithm. This algorithm may encode the position and relationships of different design elements and attributes within the input data. The encoded design information may then be processed by a transformer model. In some cases, the transformer model may be a Text-to-Text Transformer, shown as T5 Backbone 402 in FIG. 4. The T5 Backbone may utilize natural language processing techniques to understand and analyze the design-related data. The T5 model may treat all text-related tasks in a sequence-to-sequence manner. For example, classification tasks such as sentiment analysis may output strings like 'positive' or 'negative'. The model may even handle regression tasks in this format by predicting numbers as strings. The T5 model 402 may include an encoder-decoder structure with a predetermined number of attention heads (for example, 8), layers (for example, 6) and hidden feature size (for example 768). The decoder may generate an autoregressive prediction, with each subsequent prediction based on the encoder output and previous outputs. This architecture may allowthe T5 Backbone 402 to effectively process and transform the encoded design information into structured outputs containing aspects, categories, opinions, sentiments and implicit indicators related to the design concepts.

[0060] After processing by the T5 Backbone 402, the system generates multiple output labels 406. As shown in FIG. 4, each output label contains an Aspect, Category, Opinion, and Sentiment, numbered sequentially from 1 to n. These structured outputs may provide a comprehensive analysis of the design concepts, categorizing various aspects and associated sentiments. Below are three examples of generated annotation results.

[0061] (1) “Bought them out of nostalgia. They almost look the same but the leather is not soft as before.”

[0062] Label 1: (NULL, ContextOfUse#PurchaseContext, Positive, “Bought them out of nostalgia,” Direct)

[0063] Label2: (NULL, Appearance#Form, Positive, “almost look the same,” Indirect)Label3 : (“leather,” Appearance#Material, Negative, “not soft as before,” Direct)

[0064] (2) “It is narrow and fits pretty well. However, the tread design traps mud and other debris which I then track into the house. After I started wearing this shoe, I notice bit of dirt and debris on the floor in places where I walk or sit consistently. This has never happen before so I quickly tracked the problem down to the small pockets in this shoes tread design.”

[0065] Label 1: (NULL, Performance#Sizing / Fit, Positive, “narrow and fits pretty well”,Indirect)

[0066] Label2: (“tread design”, Appearance# Shoe Component, Negative, “traps mud”,Indirect)

[0067] Label3 : (NULL, ContextOfUse#UseCase, Negative, “notice bit of dirt and debris on the floor in places”, Direct)

[0068] Label4: (“small pockets”, Appearance#ShoeComponent, Negative, “the problem down to the small”, Direct)

[0069] (3) “The Crocs I purchased for my grandson had to be returned . Unfortunately, they were defective because the back strap that fits, normally, around the back of the ankle was ridiculously short and did not fit around . Measuring them against another pair of the same Crocs he had that I wanted to replace, they were at least one inch shorter! So, regretfully, the pair was returned.”

[0070] Label 1: (“Crocs”, ContextOfUse#Purchase Context, Negative, “I purchased for my grandson had to be returned,” Indirect)

[0071] Label2: (“back”, Appearance#ShoeComponent, Negative, “they were defective because the back,” Indirect)

[0072] Label3 : (“Crocs,” Performance#Sizing / Fit, Negative, “Measuring them against another pair of the same Crocs,” Indirect)

[0073] Label4: (“pair”, ContextOfUse#Purchase Context, Negative, “was returned,” Direct)

[0074] In some aspects, the intelligent platform may employ advanced natural language processing (NLP) techniques to extract both explicit and latent user needs from diverse reviews on e- commerce and social platforms. This process may enhance the understanding of customer preferences and requirements, providing valuable insights for the design concept generation process.

[0075] The NLP -based extraction may involve several steps:

[0076] 1. Data Collection: The system may gather user-generated content from various sources, including product reviews on e-commerce platforms, social media posts, and customer feedback forums. This diverse dataset may provide a comprehensive view of user opinions and experiences.

[0077] 2. Text Preprocessing: The collected text data may undergo preprocessing steps such as tokenization, removal of stop words, and lemmatization to prepare it for analysis.

[0078] 3. Sentiment Analysis: The system may employ sentiment analysis techniques to determine the overall emotional tone of each review or comment. This may involve classifying text as positive, negative, or neutral, and potentially assigning sentiment intensity scores.

[0079] 4. Aspect-Based Sentiment Analysis: To extract explicit needs, the system may perform aspect-based sentiment analysis. This technique may identify specific product features or attributes mentioned in the text and determine the associated sentiment.

[0080] 5. Named Entity Recognition: The system may use named entity recognition to identify and classify named entities in the text, such as product names, brands, or specific features.

[0081] 6. Topic Modeling: Latent Dirichlet Allocation (LDA) or other topic modeling techniques may be applied to uncover hidden themes or topics within the corpus of reviews. This may help identify latent needs that are not explicitly stated.

[0082] 7. Word Embeddings: The system may utilize word embedding techniques such asWord2Vec or GloVe to capture semantic relationships between words and phrases, potentially revealing latent connections between user needs.

[0083] 8. Contextual Analysis: Advanced language models like BERT or GPT may be employed to perform contextual analysis, capturing nuanced meanings and implicit needs based on the surrounding context of words and phrases.

[0084] 9. Usage Context Extraction: The system may analyze text to identify and extract information about the context in which products are used, potentially revealing latent needs related to specific use cases or environments.

[0085] 10. Emotion Detection: Beyond basic sentiment analysis, the system may employ emotion detection techniques to identify specific emotions expressed in the text, which may provide insights into latent emotional needs of users.

[0086] 11. Extreme Use-Case Identification: The system may be designed to identify and flag reviews or comments that describe extreme or unusual use cases, as these may be particularly valuable for uncovering latent needs.

[0087] 12. Clustering and Categorization: The extracted needs, both explicit and latent, may be clustered and categorized to identify patterns and trends across the dataset.

[0088] 13. Temporal Analysis: The system may analyze changes in user needs and sentiments over time, potentially revealing emerging latent needs or shifts in user preferences.

[0089] 14. Cross-Platform Comparison: By comparing data from different platforms, the system may identify platform-specific needs or validate needs across multiple sources.

[0090] 15. Visualization: The extracted needs may be visualized using techniques such as word clouds, sentiment heat maps, or network graphs to aid in interpretation and analysis.

[0091] By leveraging these NLP techniques, the intelligent platform may provide a comprehensive understanding of both explicit and latent user needs, informing the design concept generation process and potentially leading to more innovative and user-centric product designs.

[0092] Now referring back to FIG. 2, a subset of the range of problem ideas 206 is selected. This is shown in FIG. 2 as the problem selection space 208. Selecting problems from the range of problem ideas 206 may include identifying both explicit and latent user needs. Need finding may seek to understand where the user's problem might arise by developing methods that utilize the user's information and context. Such methods may utilize user information in different forms and processes in order to develop a taxonomy of the user's needs. In some cases, need finding methods can be classified according to three dimensions of user information retrieval: accessibility of the user's information — either existing or newly generated; source of user's information — primary users or experts and information agents; and procedure of user information handling — structured or unstructured procedures.

[0093] Accessibility of user's information may represent the ability to understand a larger portion of the user population. By either increasing the quantity or quality of information, the effectiveness of identifying explicit and latent needs in a user population may also increase. This may allow for many things to be covered that would otherwise not be by statistical analyses based on observations and measurements of reality, or by experiments. Variables affecting user needs may be situation specific and to obtain the appropriate needs of the user, it may be helpful to ascertain an image of the user's experience with the product and the frequency with which the user will undertake the task. The user scenario may generate a picture of the tasks in which they want to accomplish andillustrate the context in which this takes place. Large-scale need finding is one method that may help users articulate their needs using specific types of feedback mechanisms, i.e. image or text, and then collecting this data using online applications in order to develop more detailed user scenarios. In contrast to surveys or other manual collection methods that may be limited to their magnitude of data, by increasing the proportion of user insights, the ability to identify the needs of the user population can increase proportionally. This may be used for crowd-sourcing user scenarios because it can be difficult for users to specify what may be unstructured or unpredictable in their needs, making it difficult to get it right in the early specification stage. Access to more user information may mean generating a more general user scenario from the user population.

[0094] Identifying a type of user or users may alter the way needs can be elicited from the general population. Explicit user needs may be those that will occur more commonly within a user population, and thus users may oftentimes make this more explicit. Latent user needs may be unknown to the general user population, and so identifying types of users that put unique demands on their product may increase the likelihood of identifying unique needs that might be relevant to the general population. To increase the designer's ability to account for these unforeseen needs, identifying a lead user as an information agent may benefit the search, as these users may face some strong needs before the mass customers encounter them. The lead user may try to solve their problems in advance, revealing dimensions of the product that would otherwise be unforeseen by the general user. This lapse in time between when a user's needs are expressed may be either an existing need or future need. The existing need may concern improper or missing need satisfaction by the existing products, which are exposed by the general user and may be more easily revealed. Future needs may concern those which cannot easily be determined and evolve over time due to a users' change in environment or demands. The ability to go beyond the immediate demands of the product and expose future needs may underlie the lead user hypothesis, although, whether these will be future needs of the general user may require further analysis.

[0095] The procedure of handling information may refer to the techniques in which this information is assessed. Such techniques may include manual methods: surveys, focus groups, interviews; or automated methods: sentiment analysis, text analytics / mining / analysis, or the use of deep learning. For example, manual methods such as surveys, may use different probability sampling methods which involve researchers inviting users to respond to open or close-ended questions and then performing some form of data analysis on the results. Sentiment analysis may identify user attitudes towards products by assessing who the user is, what product they are talking about, and whether their comments are negative or positive in order to assess the products performance. The underlying goal of these information handling methods may be to reveal the emotional and rational needs of the user. Rational needs may concern the product's function, performance and operation, and may often be connected to the physical aspects of the product. Emotional user needs may concern novelty, styling, appearance, status, personal value and others connected to the product's perceivedpsychological performance, and may be taken into account when evaluating the ultimate need satisfaction and product value for the user.

[0096] FIG. 5 illustrates a latent need conceptual model 500 for understanding and analyzing user needs in a product design context. As shown in FIG. 5, the latent need conceptual model 500 may comprise three main axes, each representing a different aspect of user needs and information:

[0097] 1. User Scenario, denoted as k(x)

[0098] 2. Need Time, denoted as k(y)

[0099] 3. Emotional / Rational Needs, denoted as k(z)

[0100] Corresponding to each of these axes may be functions that represent the handling or processing of information:

[0101] 1. Accessibility of User Information, denoted as Th(x)

[0102] 2. Source of User Information, denoted as Th(y)

[0103] 3. Handling of Information, denoted as Th(z)

[0104] The central concept of the latent need conceptual model 500 may be the "latent need," which may be positioned at the core of the structure. This latent need may be derived from the interactions between the different aspects and information processing functions.

[0105] The latent need conceptual model 500 may illustrate several paths and connections:

[0106] 1. From k(x) to Th(x), representing the transformation of user scenarios into accessible information.

[0107] 2. From k(y) to Th(y), showing the relationship between need time and the source of user information.

[0108] 3. From k(z) to Th(z), indicating how emotional / rational needs are handled.

[0109] Additional paths such as Th(k(x)), Th(k(y)), and Th(k(z)) may demonstrate more complex interactions within the latent need conceptual model 500.

[0110] The latent need conceptual model 500 may provide a framework for analyzing and understanding user needs in a multidimensional context, considering various factors that contribute to the emergence of latent needs in product design.

[0111] Referring back to FIG. 2, once a need has been identified, the product development process 200 enters the generative design space 204. In brief overview, multiple solutions are generated 210 and then evaluated 212, ultimately producing a product for introduction to the market 214.

[0112] In some cases, the process 200 may utilize a generative adversarial network (GAN) generator, such as the one depicted in FIG. 6, to generate design samples within the concept generationspace 210. The GAN generator may produce diverse and novel design concepts based on the problem ideas 206 identified in the need finding space 202.

[0113] FIG. 6 illustrates a system diagram of a generative adversarial network (GAN) architecture for product design and evaluation. The system comprises several interconnected components that work together to process multimodal product data and generate new designs.

[0114] The system begins with multimodal product data 602, which may include images, text, and numerical data representing various aspects of product designs. This data may be encoded by encoder 604, as described above in connection with FIGs. 3 and 4.

[0115] The latent code from the encoder 604 may then be passed to two main components: a generator 608 and a decoder 606. The generator 606 may also receive input from a latent random variable, which may introduce variability into the generation process. The generator 606 may produce new product designs based on the latent code and random input.

[0116] The decoder 606 may take the latent code and reconstruct the original multimodal product data. This reconstruction process may help ensure that the latent code captures relevant features of the input data. A reconstruction loss may be calculated between the original input and the reconstructed output to measure the accuracy of this process.

[0117] The system may include a discriminator component 612, which may evaluate the output from the generator 608. The discriminator 612 may assess whether the generated designs are real or fake, desirable, and innovative. These evaluations may contribute to the adversarial loss, which may be used to train and improve the generator's performance. In some embodiments, a plurality of discriminators 612 is used, with each of the plurality of inspectors assessing a respective criterion.

[0118] The adversarial loss may be a composite of multiple factors, including the discriminator's ability to distinguish real from fake designs, as well as assessments of desirability and innovation. This multi-faceted loss function may guide the generator to produce designs that are not only realistic but also meet specific criteria for desirability and innovation.

[0119] The system 600 may operate in a cyclical manner, with the generator 608 and discriminator 612 engaging in an adversarial process to improve the quality and relevance of the generated designs. For example, in some cases, parameters of the GAN generator 608 are adjusted based on feedback from the discriminator 612. This adjustment may occur iteratively, allowing the system 600 to refine and improve the generated design concepts overtime.

[0120] Referring now to FIG. 7, one specific embodiment of a generative adversarial network (GAN) structure for design concept generation and evaluation is depicted. The system diagram 700 comprises several interconnected components that work together to generate and assess design samples.

[0121] As shown in FIG. 7, original samples 704 (i.e., product ideas) may be combined with a random noise vector 702 to create new product designs, which are then evaluated by a plurality of discriminators 706, referred to in FIG. 7 as a panel of inspectors.

[0122] As shown in Fig, 7, the inspector panel 706 include five distinct evaluators:

[0123] 1. Realism Evaluator (Discriminator)

[0124] 2. Diversity Evaluator

[0125] 3. Novelty Evaluator

[0126] 4. Customer Satisfaction Evaluator

[0127] 5. Geometrical Constraint Evaluator

[0128] Although FIG. 7 depicts an embodiment in which five evaluators make up the panel of inspectors 706, other embodiments may include more, or fewer, inspectors. Each evaluator in the inspector panel 706 assesses the product ideas based on its specific criteria.

[0129] In some cases, the inspector panel 706 may include a novelty inspector evaluating uniqueness of the generated samples. The novelty inspector may use a Local Outlier Factor (LOF) algorithm to assess novelty of the generated samples. The LOF algorithm may compare the density of a sample to the densities of its neighbors, identifying samples that are substantially different from others in the dataset.

[0130] In some cases, the Local Outlier Factor (LOF) algorithm may be employed to assess the novelty of generated samples. The LOF algorithm may provide a quantitative measure of how different or unique a sample is compared to its neighbors in the dataset. This method may be particularly useful in evaluating the effectiveness of generative models in producing novel designs.

[0131] The LOF algorithm may operate by calculating a local density for each sample and comparing it to the local densities of its neighbors. The algorithm may involve the following steps:

[0132] 1. Defining the neighborhood: For each sample, the algorithm may identify its k-nearest neighbors, where k is a user-defined parameter. This may be achieved by measuring the typical distance at which a point can be reached from its neighbors, known as the reachability distance. This measurement involves calculating the reachability distance between two objects, ensuring that it does not fall below the k distance of the second object, as defined by:

[0133] 2. Calculating reachability distance: For each sample, the algorithm may compute a reachability distance to its neighbors, which may be the maximum of the actual distance and the localreachability density of the neighbor. The local reachability distance (LRD) of a point is then defined as the inverse of the average reachability distance from its neighbors, calculated using:Where Nk represents the set of k neighbors of the generated sample Xgen.

[0134] 3. Determining local reachability density: The algorithm may calculate the local reachability density for each sample, which may be the inverse of the average reachability distance of the sample to its neighbors.

[0135] 4. Computing LOF score: The LOF score for each sample may be calculated as the ratio of the average local reachability density of its neighbors to its own local reachability density:

[0136] Samples with LOF scores significantly higher than 1 may be considered potential novelties or outliers while samples with LOF values below 1 generally indicate inliers, representing data points within denser regions. In the context of design concept generation, samples with high LOF scores may represent designs that are substantially different from existing designs in the dataset.

[0137] The LOF algorithm may offer several advantages for novelty assessment in design concept generation:

[0138] 1. Local context consideration: The algorithm may take into account the local density of samples, allowing it to identify novelties in different regions of the design space.

[0139] 2. Adaptability to different densities: The method may work well in datasets with varying densities, which may be common in design concept spaces.

[0140] 3. Continuous scoring: Instead of binary classification, LOF may provide a continuous score, allowing for more nuanced assessment of novelty.

[0141] In some implementations, the novelty inspector may use the LOF algorithm in conjunction with other novelty metrics to provide a comprehensive assessment of the uniqueness of generated samples. This multi-faceted approach may help ensure that the generative model produces truly novel design concepts, potentially leading to more innovative product designs.

[0142] In some cases, the inspector panel 706 may include a diversity inspector evaluating variety among the generated samples. The diversity inspector may use a Covering Radius UpperBound (CRUB) method to assess diversity of the generated samples. The CRUB method may measure the maximum distance between any point in the design space and its nearest generated sample, providing a metric for how well the generated samples cover the entire design space.

[0143] In some cases, the Covering Radius Upper Bound (CRUB) method may be employed to assess the diversity of generated samples. The CRUB method may provide a quantitative measure of how well the generated samples cover the entire design space. This method may be particularly useful in evaluating the effectiveness of generative models in producing a wide range of diverse designs.

[0144] The CRUB method may operate by calculating the maximum distance between any point in the design space and its nearest generated sample. This distance may be referred to as the covering radius. A smaller covering radius may indicate that the generated samples are we 11 -distributed throughout the design space, potentially representing a more diverse set of designs.

[0145] In some implementations, the CRUB method may involve the following steps:

[0146] 1. Defining the design space: The method may first establish the boundaries and dimensions of the design space in which the samples are generated.

[0147] 2. Generating samples: The generative model may produce a set of design samples within the defined design space.

[0148] 3. Discretizing the space: The continuous design space may be divided into a finite number of discrete points or regions for computational feasibility.

[0149] 4. Calculating distances: For each point in the discretized space, the method may compute the distance to the nearest generated sample.

[0150] 5. Determining the maximum distance: The largest of these distances may be identified as the covering radius. The covering radius of a generated point set

[0151] may be calculated as follows:

[0152] Then, the upper bound of CR is obtained by

[0153] 6. Comparing to a threshold: The calculated covering radius may be compared to a predetermined threshold or to the covering radii of other sample sets to assess relative diversity.

[0154] In some cases, the CRUB method may be extended to incorporate weighted distances, allowing certain regions of the design space to be prioritized based on their importance or relevance to the specific design problem.

[0155] The diversity inspector may use the CRUB method in conjunction with other diversity metrics to provide a comprehensive assessment of the variety among generated samples. This multifaceted approach may help ensure that the generative model produces a rich and diverse set of design concepts, potentially leading to more innovative and effective product designs.

[0156] In some cases, the inspector panel 706 may include a geometrical constraint inspector that evaluates compatibility of generated samples with predefined silhouettes using Structural Similarity Index Measure (SSIM). The SSIM may compare the structural information between the generated sample and a set of predefined silhouettes, ensuring that the generated designs maintain certain geometric constraints.

[0157] The Structural Similarity Index Measure (SSIM) may be used to evaluate the similarity between generated samples and predefined silhouettes. This method may provide a way to assess how well the generated designs adhere to certain geometric constraints or desired shapes.

[0158] The SSIM may work by comparing local patterns of pixel intensities across luminance, contrast, and structure. Unlike simple pixel-by-pixel comparison methods, SSIM may take into account the spatial relationships between pixels, which may be particularly relevant for assessing geometric similarities.

[0159] In some implementations, the SSIM calculation may involve the following steps:

[0160] 1. Uuminance comparison: The method may compare the local luminance of the two images, which may be calculated as the mean intensity in a local window.

[0161] 2. Contrast comparison: The method may compare the local contrast of the two images, which may be measured as the standard deviation of pixel intensities in a local window.

[0162] 3. Structure comparison: The method may compare the local structure of the two images, which may be measured by the correlation between the pixels in a local window.

[0163] 4. Combination: These three comparisons may be combined to produce a single similarity score for each local window.

[0164] 5. Pooling: The local similarity scores may be averaged or otherwise combined to produce a global similarity score for the entire image.

[0165] The SSIM may produce a score between -1 and 1, where 1 indicates perfect similarity and -1 indicates perfect dissimilarity. In the context of evaluating generated design samples, a higher SSIM score may indicate that the generated sample more closely matches the predefined silhouette or geometric constraint.

[0166] In some cases, the geometrical constraint inspector may use SSIM in conjunction with other image similarity metrics to provide a comprehensive assessment of how well the generated samples adhere to desired geometric constraints. This multi-faceted approach may help ensure that the generative model produces designs that are not only novel and diverse but also maintain certain desired structural characteristics.

[0167] In some cases, the inspector panel 706 may include a customer satisfaction evaluator (DMDE).

[0168] In some cases, the customer satisfaction evaluator may utilize a Deep Multimodal DesignEvaluation (DMDE) model to assess the potential customer satisfaction of generated design samples. The DMDE model may integrate multiple modalities of design information, such as visual features, textual descriptions, and numerical attributes, to provide a comprehensive evaluation of customer satisfaction.

[0169] The DMDE model may operate by processing different types of input data:

[0170] 1. Visual data: The model may analyze images or 3D renderings of the generated designs using convolutional neural networks (CNNs) to extract relevant visual features.

[0171] 2. Textual data: Natural language processing techniques may be employed to analyze product descriptions, customer reviews, or design specifications.

[0172] 3. Numerical data: Quantitative attributes of the design, such as dimensions, materials, or performance metrics, may be processed using fully connected neural networks.

[0173] The DMDE model may combine these different data modalities using a fusion mechanism, which may involve techniques such as:

[0174] 1. Early fusion: Concatenating features from different modalities before processing them through a joint network.

[0175] 2. Late fusion: Processing each modality separately and combining the outputs.

[0176] 3. Attention mechanisms: Allowing the model to focus on the most relevant features across modalities.

[0177] In some implementations, the DMDE model may be trained on historical data of customer preferences, ratings, and feedback. This training process may enable the model to learn complex relationships between design features and customer satisfaction.

[0178] The output of the DMDE model may be a customer satisfaction score or a probability distribution over different satisfaction levels. This output may be used to rank generated designs or to provide feedback to the generator for improving future design iterations.

[0179] In some cases, the DMDE model may also provide interpretable results, highlighting which aspects of a design contribute positively or negatively to the predicted customer satisfaction. This interpretability may offer valuable insights for designers and decision-makers in the product development process.

[0180] The DMDE approach may offer several advantages in evaluating customer satisfaction:

[0181] 1. Multimodal integration: By considering multiple types of design information, the model may capture a more holistic view of customer preferences.

[0182] 2. Scalability: The model may efficiently evaluate large numbers of generated designs, enabling rapid iteration and exploration of the design space.

[0183] 3. Adaptability: The model may be fine-tuned or retrained as customer preferences evolve or for different product categories.

[0184] 4. Consistency: The DMDE model may provide consistent evaluations across many designs, reducing potential biases or inconsistencies that may occur with human evaluators.

[0185] The cost function 708 aggregates the evaluations from the inspector panel and produces two types of feedback:

[0186] 1. Discriminator Loss, which is fed back to the Realism Evaluator

[0187] 2. Generator Loss, which is fed back to the generator.

[0188] In some embodiments, the loss function may be formulated as follows:where s represents the silhouette. The associated loss functions may bedefined as follows:In these equations, max(SSIM), max(LOF), max(CRUB), and max(DMDE) represent the optimal or maximum values achievable by each function.

[0189] This feedback loop allows the system to iteratively improve the quality, diversity, novelty, customer satisfaction, and geometrical constraints of the generated samples. The generator uses the feedback from the cost function to adjust its parameters and produce better samples in terms of realism, outer shape geometry, novelty, diversity, and desirability in subsequent iterations.

[0190] FIG. 8 illustrates a comparison of sample distributions between a baseline generative model and the DCG-GAN model. The figure presents two scatter plots side by side, labeled "Baseline" and "DCG-GAN" respectively. Each plot displays data points on a two-dimensional graph with axes labeled "First Principal Component" and "Second Principal Component".

[0191] In both plots, there are two types of data points: black dots representing original samples and gray dots representing generated samples. The distribution of these points provides insights into the diversity and coverage of the design space by each model.

[0192] To generate these visualizations, features are extracted from the generated samples using a VGG16 convolutional neural network. The VGG16 network may be employed due to its ability to capture complex visual features from images. In some cases, the VGG16 network may be pre-trained on a large dataset of images to ensure robust feature extraction.

[0193] After feature extraction, dimensionality reduction is performed on the extracted features using Principal Component Analysis (PCA). PCA may be utilized to reduce the high-dimensional feature space to two dimensions, allowing for visualization of the diversity of generated samples. The first two principal components, which capture the most variance in the data, are used as the axes for the scatter plots.

[0194] Examining the Baseline plot, the generated samples (gray dots) appear to be clustered more tightly and occupy a smaller area within the space of the original samples (black dots). This distribution suggests that the baseline model may be generating samples that are less diverse and more concentrated in certain regions of the design space.

[0195] In contrast, the DCG-GAN plot displays generated samples that are more widely dispersed, covering a larger area that extends beyond the boundaries of the original samples in some regions. This distribution indicates that the DCG-GAN model may be producing a more diverse range of design concepts, exploring areas of the design space that are not represented in the original dataset.

[0196] The comparison between these two distributions suggests that the DCG-GAN model may be more effective at generating diverse design concepts compared to the baseline model. The wider spread of DCG-GAN generated samples in the feature space may indicate a greater variety of design attributes and characteristics in the generated concepts.

[0197] In some cases, the ability of DCG-GAN to generate samples that extend beyond the original sample distribution may suggest that the model is capable of producing novel designs that combine features in ways not present in the training data. This capability may be particularly valuable in design concept generation, where exploring new and unexpected combinations of features can lead to innovative solutions.

[0198] The visualization provided by PCA may offer valuable insights into the performance of generative models in terms of diversity. By reducing the complex, high-dimensional feature space to two dimensions, designers and researchers may more easily assess and compare the coverage of the design space achieved by different generative models.

[0199] FIG. 9 illustrates a comparison of novelty between a baseline model and the DCG-GAN model. The figure presents two components: distribution graphs of Template Matching Confidence scores and visual comparisons of generated sneaker designs.

[0200] The top row of FIG. 9 shows two distribution graphs depicting the Template Matching Confidence scores. The baseline graph on the left exhibits a narrower, taller peak centered around 0.85, while the DCG-GAN graph on the right displays a wider, slightly lower peak centered around 0.8. This shift in distribution suggests that the DCG-GAN model may generate samples with lower similarity to existing designs, potentially indicating higher novelty.

[0201] Below the distribution graphs, FIG. 9 presents two pairs of sneaker images. The left pair, associated with the baseline model, shows two very similar black sneakers with white stripes. The similarity score for this pair is 0.84, indicating a high degree of resemblance between the generated sample and the most similar existing design.

[0202] In contrast, the right pair, associated with the DCG-GAN model, displays two gray sneakers that are noticeably different in design. The similarity score for this pair is 0.79, which is lower than the baseline pair. This visual comparison illustrates the DCG-GAN's ability to generate more diverse and novel sneaker designs compared to the baseline model.

[0203] In some cases, the system may assess novelty using a Local Outlier Factor (LOF) algorithm. The LOF algorithm may compare the density of a sample to the densities of its neighbors, identifying samples that are substantially different from others in the dataset.

[0204] The system may use multiple similarity detection methods to assess novelty. These methods may include Root Mean Square Error (RMSE), Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), Signal to Reconstruction Error (SRE), and Spectral Angle Mapper (SAM). Each of these methods may provide a different perspective on the similarity between generated samples and existing designs.

[0205] In some cases, the system may use template matching to identify the most similar existing design for each generated sample. This approach may allow for a direct comparison between generated designs and the closest existing designs in the dataset, providing a quantitative measure of novelty.

[0206] The combination of distribution analysis, visual comparison, and multiple similarity detection methods may provide a comprehensive assessment of the novelty of generated designs. The results presented in FIG. 9 suggest that the DCG-GAN model may be more effective at generating novel sneaker designs compared to the baseline model.

[0207] FIG. 10 presents two box plot diagrams comparing the performance of DCG-GAN and a baseline model in terms of novelty and diversity assessment. The left diagram, labeled "Novelty Assessment," shows box plots for 20 design concept samples, with the first 10 representing DCG- GAN results and the last 10 representing baseline results. The right diagram, labeled "Diversity Assessment," compares the diversity ratings between DCG-GAN and the baseline model using two box plots.

[0208] In the Novelty Assessment diagram, the DCG-GAN samples generally show higher novelty ratings compared to the baseline samples. The median values for DCG-GAN samples are consistently higher, and the interquartile ranges (represented by the boxes) are generally smaller, indicating more consistent novelty scores. This suggests that DCG-GAN may be more effective at generating novel design concepts compared to the baseline model.

[0209] The Diversity Assessment diagram shows that the DCG-GAN box plot has a higher median and smaller range compared to the baseline, suggesting improved diversity in the DCG-GAN generated samples. In some cases, the method may assess diversity using a Covering Radius Upper Bound (CRUB) method. The CRUB method may measure how well the generated samples cover the entire design space, providing a quantitative metric for diversity.

[0210] The results presented in FIG. 10 have several implications for the effectiveness of DCG- GAN in generating novel and diverse design concepts:

[0211] 1. Improved Novelty: The consistently higher novelty ratings for DCG-GAN samples suggest that the model may be more capable of generating unique and innovative design concepts compared to the baseline model. This may lead to a broader range of creative solutions in the design process.

[0212] 2. Enhanced Diversity: The higher median and smaller range in the diversity assessment for DCG-GAN indicate that the model may produce a more varied set of design concepts. This increased diversity may provide designers with a wider array of options to consider during the concept selection phase.

[0213] 3. Consistency: The smaller interquartile ranges for DCG-GAN in both novelty and diversity assessments suggest that the model may generate consistently novel and diverse concepts. This consistency may be valuable in maintaining a high standard of innovation across multiple design iterations.

[0214] 4. Potential for Innovation: The combination of higher novelty and diversity scores forDCG-GAN may indicate an increased potential for generating innovative design solutions. By exploring a broader and more diverse design space, DCG-GAN may uncover unexpected or unconventional concepts that could lead to breakthrough innovations.

[0215] 5. Efficiency in Concept Generation: The improved performance of DCG-GAN in both novelty and diversity may lead to more efficient concept generation processes. Designers may need to generate fewer concepts to achieve a desired level of novelty and diversity, potentially saving time and resources in the early stages of product development.

[0216] In some cases, the quantitative assessment provided by methods such as CRUB may offer designers a more objective way to evaluate and compare the performance of different generative models. This objective evaluation may help in selecting the most effective tools for design concept generation and in refining the models to better meet the specific needs of various design projects.

[0217] The results presented in FIG. 10 suggest that DCG-GAN may offer significant advantages over the baseline model in generating novel and diverse design concepts. These improvements may contribute to more innovative and effective product design processes.

[0218] A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the disclosure. Accordingly, other implementations are within the scope of the following claims.

Claims

We claim:

1. A system for generating design concepts, comprising: a generative adversarial network (GAN) including a generator; a plurality of inspectors, each of the plurality of inspectors evaluating generated samples based on a respective criteria; and a processor executing instructions to: generate design samples using the generator of the GAN; evaluate the generated samples using the plurality of inspectors; and adjust parameters of the GAN based on feedback from the plurality of inspectors to produce diverse and novel design concepts.

2. The system of claim 1, wherein the plurality of inspectors includes a novelty inspector evaluating uniqueness of the generated samples.

3. The system of claim 2, wherein the novelty inspector uses a Local Outlier Factor (LOF) algorithm to assess novelty of the generated samples.

4. The system of claim 1, wherein the plurality of inspectors includes a diversity inspector evaluating variety among the generated samples.

5. The system of claim 4, wherein the diversity inspector uses a Covering Radius Upper Bound (CRUB) method to assess diversity of the generated samples.

6. The system of claim 1, wherein the plurality of inspectors includes a desirability inspector evaluating user satisfaction with the generated samples.

7. The system of claim 6, wherein the desirability inspector uses a Deep Multimodal Design Evaluation (DMDE) model to assess desirability of the generated samples.

8. A method for generating design concepts, comprising: generating design samples using a generative adversarial network (GAN) generator; evaluating the generated samples using a plurality of inspectors, each of the plurality of inspectors assessing a respective criterion; and adjusting parameters of the GAN generator based on feedback from the plurality of inspectors to produce diverse and novel design concepts.

9. The method of claim 8, wherein evaluating the generated samples includes assessing novelty using a Local Outlier Factor (LOF) algorithm.

10. The method of claim 8, wherein evaluating the generated samples includes assessing diversity using a Covering Radius Upper Bound (CRUB) method.

11. The method of claim 8, wherein evaluating the generated samples includes assessing desirability using a Deep Multimodal Design Evaluation (DMDE) model.

12. The method of claim 8, further comprising: extracting features from the generated samples using a convolutional neural network; and performing dimensionality reduction on the extracted features to visualize diversity.

13. The method of claim 12, wherein the convolutional neural network is a VGG16 network and the dimensionality reduction is performed using Principal Component Analysis (PCA).

14. The method of claim 8, further comprising applying stratified sampling to a pool of random latent codes to select diverse latent codes for generating the design samples.

15. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations for generating design concepts, the operations comprising: generating design samples using a generative adversarial network (GAN) generator; evaluating the generated samples using multiple inspectors, each inspector assessing a different criterion; and adjusting parameters of the GAN generator based on feedback from the multiple inspectors to produce diverse and novel design concepts.

16. The non-transitory computer-readable medium of claim 15, wherein evaluating the generated samples includes assessing novelty using a Local Outlier Factor (LOF) algorithm.

17. The non-transitory computer-readable medium of claim 15, wherein evaluating the generated samples includes assessing diversity using a Covering Radius Upper Bound (CRUB) method.

18. The non-transitory computer-readable medium of claim 15, wherein evaluating the generated samples includes assessing desirability using a Deep Multimodal Design Evaluation (DMDE) model.

19. The non-transitory computer-readable medium of claim 15, the operations further comprising: extracting features from the generated samples using a convolutional neural network; and performing dimensionality reduction on the extracted features to visualize diversity.

20. The non-transitory computer-readable medium of claim 19, wherein the convolutional neural network is a VGG16 network and the dimensionality reduction uses Principal Component Analysis (PCA).

Images