Generated Content Signature Matching for Training Data Attribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large Language Models (LLMs) generate content that may inadvertently include copyrighted training data, lacking clear indications of training data presence or copyright status, leading to potential legal risks and compliance issues.
Innovation Solution
A system evaluates generated content by tokenizing it into signature shingles, applying a hash function to create signatures, and matching these against a training data signature database to identify and attribute copyrighted content, providing compliance status and source attribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If LLMs generate content using training data, then content generation capability is improved, but copyright compliance and traceability deteriorate
Solution Approach 1:
The system performs preliminary actions by creating signatures of training data before the LLM generates content. These signatures are stored in a database in advance, enabling subsequent comparison and verification of generated content against training data sources, thus ensuring copyright compliance while maintaining generation capability
Solution Approach 2:
The patent introduces an intermediary mechanism - a signature database and comparison system - that mediates between the LLM's content generation process and copyright compliance requirements. This intermediary enables traceability without directly interfering with the generation capability
2Adaptability or versatility
If LLMs use trillions of training tokens, then model intelligence is improved, but detecting training data presence in generated content becomes more difficult
Solution Approach 1:
The system extracts essential identifying features from training data by generating signatures (hashes) of training tokens. This extraction process condenses trillions of training tokens into manageable signature representations that can be efficiently stored and compared, making detection feasible despite the large scale of training data
Solution Approach 2:
The patent transforms the detection problem by changing parameters - instead of comparing actual training data against generated content (which would be computationally infeasible), it compares cryptographic signatures or hashes. This parameter transformation makes detection of training data presence practical even when the model has been trained on trillions of tokens
Data Source
AI summary
Evaluating the presence of training data in generated content by defining a first signature of a portion of the generated content by loading the generated content into memory registers, dividing the generated content into tokens, designating a sequential group of tokens as a signature shingle, and defining the first signature as a hash function value for the signature shingle. The evaluation also including matching the first signature to a training data signature in a training data signature database, updating a database record for the generated content to include the training data associated with the training data signature, and providing an output comprising data associated with the training data signature over a network.