Multimodal Design Graph Reasoning for Semantic Scheme Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-modal large language models fail to deeply understand semantic relationships among multi-modal design information, leading to inadequate feedback and optimization of design schemes.
Innovation Solution
A method that transforms design information into a graph network, using a large language model to extract and update structural, functional, and behavioral nodes and edges, comparing graph networks at different moments to optimize design schemes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing multi-modal large language models are used to process design information, then the models can understand multi-modal data through modality alignment, but they fail to deeply understand semantic relationships among the multi-modal design information
Solution Approach 1:
The patent introduces a graph network as an intermediary structure to represent semantic relationships among multi-modal design information. The graph network captures entities, attributes, and relationships from text, images, and other modalities, enabling the large language model to deeply understand semantic connections rather than merely aligning modalities. This intermediary structure transforms raw multi-modal data into a structured knowledge representation that preserves semantic relationships.
Solution Approach 2:
The patent segments the complex task of understanding multi-modal design information into distinct components: extracting entities, attributes, and relationships from each modality, organizing them into a graph network structure, and then processing the structured representation. This segmentation allows the system to handle semantic relationships systematically by breaking down the complex understanding task into extractable and processable elements.
2Adaptability or versatility
If traditional reasoning methods are used for design scheme optimization, then the methods can follow existing design rules and cases, but they lack generalization ability and are only applicable to specific products
Solution Approach 1:
The patent creates a universal reasoning framework that can handle multiple product types and design domains through the graph network representation. The system extracts general design patterns, relationships, and knowledge from diverse design information sources, storing them in a modality-agnostic graph structure. This universal representation enables the large language model to apply learned design principles across different product types, achieving generalization while maintaining reliability through structured semantic relationships.
Solution Approach 2:
The patent transforms design information from fixed, modality-specific formats into a flexible graph network representation where entities, attributes, and relationships can be dynamically adjusted. This parameter transformation allows the system to adapt to different design domains by changing the graph structure and content rather than requiring domain-specific reasoning algorithms, thereby achieving both generalization and reliability.
3Productivity
If multi-modal large language models focus only on modality alignment, then they can map multi-modal data to text space, but they fail to provide timely and accurate feedback on design ideas and current design progress
Solution Approach 1:
The patent performs preliminary processing of multi-modal design information by extracting entities, attributes, and relationships into a graph network structure before feeding it to the large language model. This preliminary structuring of semantic relationships enables the model to quickly access and understand design information without performing extensive modality alignment during the feedback generation process, thereby improving feedback timeliness while maintaining understanding accuracy.
Data Source
AI summary
The present invention discloses a multi-modal design information unified expression and reasoning method based on a large language model. The present invention utilizes the large language model to extract corresponding information of function-behavior-structure from a design scheme, so as to construct a graph network of the function-behavior-structure, thereby achieving a relatively accurate unified expression of multi-modal design information. The present invention obtains differences in three dimensions of the function-behavior-structure based on comparison of graph networks at different moments. Through the difference in each dimension, the large language model is used to obtain an optimized scheme corresponding to the dimension to reason out a design idea and a design process of a designer in each dimension through the large language model, and the design scheme in each dimension is optimized based on the design idea and the design process, and then, the optimized schemes corresponding to the three dimensions are aggregated to obtain the optimized design scheme.