Workflow Chart Processing via Graph Neural Network Soft Prompts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in directly inputting graph representations of workflow charts into language models like GPT, as this requires encoding graph data into long text prompts, which is a significant technical hurdle.
Innovation Solution
The method involves using a graph neural network (GNN) to encode structural information from workflow charts into a soft prompt, which can be understood by a textual language model. This soft prompt, combined with a hard prompt for specific steps, is used to pretrain and fine-tune the language model, enabling it to process and generate effective natural language outputs for customer service.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If graph representation of workflow chart is directly input into language model, then the language model can process workflow information, but it requires encoding graph data into long text prompts which is complex and inefficient
Solution Approach 1:
The patent introduces an intermediary encoding layer that transforms graph representations into language model-compatible formats. This intermediary process includes: (1) extracting structural information from graph nodes and edges, (2) generating path descriptions that capture workflow logic, and (3) formatting these paths as structured text prompts. This intermediary transformation layer resolves the incompatibility between graph data structures and language model input requirements without requiring direct input of raw graph data.
Solution Approach 2:
The patent segments the workflow graph into discrete path components that can be independently processed. Each path from start to end node is extracted and described separately, allowing the language model to process complex workflows through multiple manageable path descriptions rather than attempting to process the entire graph structure at once. This segmentation reduces the complexity of the encoding task while preserving the essential workflow logic.
2Loss of information
If graph information is encoded into long text prompts, then the language model can understand workflow structure, but the processing time and computational cost increase significantly
Solution Approach 1:
The patent extracts only the essential structural information from the graph representation, specifically focusing on path descriptions that capture the critical workflow logic. Rather than encoding all graph details into text prompts, the method selectively extracts start nodes, end nodes, and intermediate path relationships. This extraction approach retains the necessary workflow information while significantly reducing the length of text prompts, thereby decreasing processing time and computational overhead.
3Ease of operation
If conventional workflow charts are used with customer service staff, then solutions can be provided, but efficiency is low and costs are high due to manual processing
Solution Approach 1:
The patent enables the system to automatically process customer inquiries by integrating the workflow graph processing capability with a language model that can generate responses. Instead of requiring customer service staff to manually interpret workflow charts and formulate solutions, the system autonomously processes the encoded workflow information and generates appropriate customer service responses. This self-service automation dramatically improves productivity while maintaining ease of operation through natural language interaction.
Data Source
AI summary
Example embodiments of the present disclosure provide a method, a device, and a computer program product for processing a workflow chart. The method includes encoding structural information of the workflow chart including a plurality of nodes and a plurality of edges by using a graph neural network to acquire a vector representation of the structural information; acquiring textual description data about the workflow chart at the nodes; training a language model based on the acquired textual description data and the acquired vector representation to acquire a pretrained language model; and fine-tuning the pretrained language model through training data of a specific task to acquire a fine-tuned language model. Through the method for processing the workflow chart of the present disclosure, the combination of the graph neural network and the language model not only can process a large number of complex workflow charts, but also can generate effective natural language outputs.