Automated Compliance Review Using LLM Question Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for reviewing marketing materials for compliance are time-consuming, costly, and inefficient due to the reliance on manual efforts by legal experts, with limitations in the use of large language models (LLMs) such as inability to process charts, token limits, and pre-processing requirements.
Innovation Solution
A method utilizing a large language model combined with a question generator system to automate the review of documents for compliance. The system generates relevant questions based on document data and legal questions, processes these questions using the LLM, and provides user-friendly output, overcoming token limits and enhancing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual compliance review by legal experts is used, then review accuracy and reliability are maintained, but review time and cost increase significantly
Solution Approach 1:
The patent introduces an intermediary system consisting of a question generator and large language model that mediates between the compliance review process and legal experts. The system automatically generates compliance questions based on document content and regulatory requirements, then uses an LLM to provide preliminary answers and identify potential issues. This intermediary layer handles routine compliance checking, allowing legal experts to focus only on complex cases requiring human judgment, thereby reducing overall review time while maintaining reliability through expert oversight of critical decisions.
Solution Approach 2:
The compliance review system implements self-service capabilities through automated question generation and LLM-based preliminary review. The system autonomously analyzes document content, identifies relevant regulatory requirements, generates appropriate compliance questions, and provides initial assessment answers without requiring continuous human intervention. This self-service approach handles standard compliance scenarios automatically, significantly reducing the time legal experts need to spend on routine reviews while maintaining accuracy through systematic automated checking.
2Productivity
If large language models are used for automated review, then review speed and efficiency improve, but current LLM limitations such as token limits and inability to process charts reduce accuracy
Solution Approach 1:
The patent applies segmentation by dividing the document processing into multiple manageable components that respect LLM token limits. The system segments documents into logical sections, extracts text and chart information separately, and processes them through multiple sequential steps. The question generator creates focused compliance questions based on specific document segments, and the LLM processes these segmented inputs in manageable chunks, avoiding the token limit constraints while maintaining comprehensive review coverage and accuracy.
Solution Approach 2:
The patent uses an intermediary question generator system that bridges the gap between document analysis and LLM processing. The question generator extracts relevant information from documents including charts and graphs, formulates precise compliance questions, and prepares optimized inputs for the LLM. This intermediary layer translates complex document content into LLM-friendly question formats, enabling the LLM to process charts and complex data structures that would otherwise exceed token limits or be difficult to interpret directly.
3Device complexity
If basic machine-aided analysis such as keyword searching is used, then implementation complexity is low, but analysis depth and compliance coverage remain insufficient
Solution Approach 1:
The patent replaces basic mechanical keyword searching with an advanced system that uses natural language processing and large language models. Instead of relying on simple text matching algorithms, the system uses the question generator to create context-aware compliance questions and employs LLMs to analyze document content, extract insights from charts and graphs, and provide comprehensive compliance assessments. This substitution of mechanical search with intelligent LLM-based analysis dramatically increases information coverage and compliance depth while the automated question generation keeps the overall system implementation manageable.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
The present invention relates to a method for automated review of documents (200) to be checked for compliance by using a large language model. The method comprises the steps of providing a collection of legal questions (202) to an electronic question generator system (300); receiving, with the question generator system (300), electronic document data representing at least a part of a document (200) to be reviewed; generating (140), with the question generator system (300), at least one question message (240) for the large language model based on at least one part of the document data and at least one of the legal question (202) from the collection of legal questions (202); providing the question message (240) from the question generator system (300) to the large language model for processing the question message (240) in view of reference material (230); and receiving, with the question generator system (300), a response of the large language model to the question message (240). The invention further relates to a system for performing such method.