- PatentBench-Novelty Search
- Methodology
- Use Cases
Understanding Novelty Search
Novelty search is a key patent task that involves systematically identifying prior art worldwide to determine whether a technical solution is new and inventive under patent law.
It plays a critical role throughout the innovation process, including:
R&D planning: guiding the direction and feasibility of new developments
Pre-filing: verifying that an invention is patentable before submission
Patent examination: helping examiners assess the novelty of applications
Key Findings
This benchmark evaluates seven AI tools for patent novelty search: Patsnap's Novelty Search AI Agent, Claude Opus 4.8 (with web search), Perplexity Pro (with web search), Kimi-k3 (with web search), DeepSeek-v4-flash (with web search), ChatGPT 5.6 (with web search), and Gemini 3.1 Pro (with web search).
The evaluation uses a curated cross-jurisdiction patent family dataset of 340 test samples. Each sample contains a problem statement and a standard answer: the family set of X references cited by examiners across different patent offices. This design creates a practical benchmark answer that closely reflects real-world novelty search requirements.
Benchmark results show that Patsnap's Novelty Search AI Agent leads the evaluated tools, achieving an 85% X Hit Rate, a 60% X Recall (Original), and a 37% X Recall (Expanded) within the top 100 results. Among general-purpose AI tools, ChatGPT 5.6 performed best, with a 71.76% X Hit Rate, a 33.40% X Recall (Original), and a 30.51% X Recall (Expanded). X Hit Rate is evaluated against the original reference set. These results highlight the value of domain-specific AI in patent novelty search.
The evaluation dataset is evenly distributed across IPC classifications, covering both mainstream technologies and niche domains. In terms of language, 68% of the data is in English and 32% of the data is in Chinese, ensuring the model performs well across multilingual patent content.
For receiving-office distribution, applications from United States (US) and China (CN) each make up about 32%, while those from the European Patent Office (EP) and WIPO (WO) each account for roughly 18%. This balanced mix reflects the different examination styles across major patent jurisdictions and ensures more realistic, globally representative evaluation.
Language distribution of patent texts in 340 test samples
Distribution of IPC samples across 340 test samples
Note: Percentages may not sum to 100% due to rounding to one decimal place.
Distribution of receiving offices for 340 test samples
1) X Hit Rate
Patsnap’s Novelty Search AI Agent successfully identified at least one relevant X document in 85% of test cases—an essential capability for speeding up decision-making in patent examination and early-stage R&D.
2) X Recall Rate — Original Reference Set (Top 100)
Patsnap’s Novelty Search AI Agent retrieved, on average, 60% of the X references in each sample’s original ground-truth reference set within the top 100 results.
This metric measures how completely the agent retrieves the original benchmark X references. Recall is calculated separately for each sample, then averaged across all valid samples. Higher recall indicates more complete coverage of these known references, helping teams identify relevant prior art for technical review and patent claim drafting.
3) X Recall Rate — Expanded Reference Set (Top 100)
Patsnap’s Novelty Search AI Agent retrieved 37% of all relevant X documents, enabling more thorough analysis and more informed patent claim drafting.
A high X Recall Rate is key during R&D planning and before filing a patent. Patsnap’s Novelty Search AI Agent helps teams—whether in-house researchers, patent professionals, or external agents—find more relevant X documents. This supports better technical decisions and stronger patent claims, increasing the chances of patent approval.
4) Typical Test Result Sample
In this test, the patent specification, or problem statement, was submitted to each AI tool. The returned results were evaluated against a predefined standard-answer set of X references.
The sample below shows how returned patent references are evaluated against both the original and expanded reference sets. Green references match families in the original reference set; blue references match additional families found only in the expanded reference set. X Hit Rate indicates whether at least one original-set family was retrieved. X Recall (Original) measures coverage of the one original-set family, while X Recall (Expanded) measures coverage of all four expanded-set families. Recall counts distinct patent families, so multiple references from the same family count as a single match.
In this sample, Patsnap’s Novelty Search AI Agent achieved a 100% X Hit Rate and 100% X Recall (Original), retrieving the single patent family in the original reference set. It also retrieved two additional families from the expanded reference set, achieving the highest X Recall (Expanded) at 75% (3/4). Claude Opus 4.8 and ChatGPT 5.6 each achieved a 100% X Hit Rate, 100% X Recall (Original), and 50% X Recall (Expanded), retrieving two of the four expanded-set families. Claude’s matching references appeared at positions 3 and 8 among its eight returned results. Perplexity Pro retrieved the original reference family, achieving a 100% X Hit Rate, 100% X Recall (Original), and 25% X Recall (Expanded). DeepSeek-v4-flash, Kimi-k3, and Gemini 3.1 Pro scored 0% across all three metrics.
These sample-level results are illustrative. The overall benchmark conclusions are based on the full dataset of 340 cross-jurisdiction patent family samples.
Future Research
Future benchmarks will continue to expand the dataset and refine the evaluation methods for greater accuracy, coverage, and representativeness. As the dataset grows, the benchmark will provide a more robust view of how AI tools perform in professional patent novelty search.
