July 2026

Patsnap PatentBench for Novelty Search

  • PatentBench-Novelty Search
  • Methodology
  • Use Cases

Understanding Novelty Search

Novelty search is a key patent task that involves systematically identifying prior art worldwide to determine whether a technical solution is new and inventive under patent law.

It plays a critical role throughout the innovation process, including:

R&D planning: guiding the direction and feasibility of new developments

Pre-filing: verifying that an invention is patentable before submission

Patent examination: helping examiners assess the novelty of applications

Key Findings

This benchmark evaluates seven AI tools for patent novelty search: Patsnap's Novelty Search AI Agent, Claude Opus 4.8 (with web search), Perplexity Pro (with web search), Kimi-k3 (with web search), DeepSeek-v4-flash (with web search), ChatGPT 5.6 (with web search), and Gemini 3.1 Pro (with web search).

The evaluation uses a curated cross-jurisdiction patent family dataset of 340 test samples. Each sample contains a problem statement and a standard answer: the family set of X references cited by examiners across different patent offices. This design creates a practical benchmark answer that closely reflects real-world novelty search requirements.

Benchmark results show that Patsnap's Novelty Search AI Agent leads the evaluated tools, achieving an 85% X Hit Rate, a 60% X Recall (Original), and a 37% X Recall (Expanded) within the top 100 results. Among general-purpose AI tools, ChatGPT 5.6 performed best, with a 71.76% X Hit Rate, a 33.40% X Recall (Original), and a 30.51% X Recall (Expanded). X Hit Rate is evaluated against the original reference set. These results highlight the value of domain-specific AI in patent novelty search.

The evaluation dataset is evenly distributed across IPC classifications, covering both mainstream technologies and niche domains. In terms of language, 68% of the data is in English and 32% of the data is in Chinese, ensuring the model performs well across multilingual patent content.

For receiving-office distribution, applications from United States (US) and China (CN) each make up about 32%, while those from the European Patent Office (EP) and WIPO (WO) each account for roughly 18%. This balanced mix reflects the different examination styles across major patent jurisdictions and ensures more realistic, globally representative evaluation.

Language distribution of patent texts in 340 test samples

Distribution of IPC samples across 340 test samples

Note: Percentages may not sum to 100% due to rounding to one decimal place.

Distribution of receiving offices for 340 test samples

1) X Hit Rate

Patsnap’s Novelty Search AI Agent successfully identified at least one relevant X document in 85% of test cases—an essential capability for speeding up decision-making in patent examination and early-stage R&D.

X Hit Rate: Patsnap 85%, ChatGPT 5.6 71.76%, Claude Opus 4.8 52.37%, Perplexity Pro 39.16%, Kimi-k3 33.53%, DeepSeek-v4-flash 22.09%, Gemini 3.1 Pro 14.24%.

2) X Recall Rate — Original Reference Set (Top 100)

Patsnap’s Novelty Search AI Agent retrieved, on average, 60% of the X references in each sample’s original ground-truth reference set within the top 100 results.

This metric measures how completely the agent retrieves the original benchmark X references. Recall is calculated separately for each sample, then averaged across all valid samples. Higher recall indicates more complete coverage of these known references, helping teams identify relevant prior art for technical review and patent claim drafting.

X Recall Rate, Original Reference Set: Patsnap 60%, ChatGPT 5.6 33.40%, Claude Opus 4.8 17.58%, Kimi-k3 15.35%, DeepSeek-v4-flash 8.38%, Gemini 3.1 Pro 4.79%, Perplexity Pro 3.60%.

3) X Recall Rate — Expanded Reference Set (Top 100)

Patsnap’s Novelty Search AI Agent retrieved 37% of all relevant X documents, enabling more thorough analysis and more informed patent claim drafting.
A high X Recall Rate is key during R&D planning and before filing a patent. Patsnap’s Novelty Search AI Agent helps teams—whether in-house researchers, patent professionals, or external agents—find more relevant X documents. This supports better technical decisions and stronger patent claims, increasing the chances of patent approval.

X Recall Rate, Expanded Reference Set: Patsnap 37%, ChatGPT 5.6 30.51%, Kimi-k3 12.09%, Claude Opus 4.8 11.68%, Perplexity Pro 6.4%, DeepSeek-v4-flash 6.25%, Gemini 3.1 Pro 2.39%.

4) Typical Test Result Sample

In this test, the patent specification, or problem statement, was submitted to each AI tool. The returned results were evaluated against a predefined standard-answer set of X references.

The sample below shows how returned patent references are evaluated against both the original and expanded reference sets. Green references match families in the original reference set; blue references match additional families found only in the expanded reference set. X Hit Rate indicates whether at least one original-set family was retrieved. X Recall (Original) measures coverage of the one original-set family, while X Recall (Expanded) measures coverage of all four expanded-set families. Recall counts distinct patent families, so multiple references from the same family count as a single match.

Test question US20180138692A1: an undervoltage release circuit. The standard answer contains four X-reference patent families, with one family in the original reference set and three additional families in the expanded set.Typical test result: Patsnap, Claude, ChatGPT, and Perplexity each achieve 100% X Hit Rate and Original Recall. Expanded Recall is 75% for Patsnap, 50% for Claude and ChatGPT, and 25% for Perplexity. DeepSeek, Kimi, and Gemini score 0% on all three metrics.

In this sample, Patsnap’s Novelty Search AI Agent achieved a 100% X Hit Rate and 100% X Recall (Original), retrieving the single patent family in the original reference set. It also retrieved two additional families from the expanded reference set, achieving the highest X Recall (Expanded) at 75% (3/4). Claude Opus 4.8 and ChatGPT 5.6 each achieved a 100% X Hit Rate, 100% X Recall (Original), and 50% X Recall (Expanded), retrieving two of the four expanded-set families. Claude’s matching references appeared at positions 3 and 8 among its eight returned results. Perplexity Pro retrieved the original reference family, achieving a 100% X Hit Rate, 100% X Recall (Original), and 25% X Recall (Expanded). DeepSeek-v4-flash, Kimi-k3, and Gemini 3.1 Pro scored 0% across all three metrics.

These sample-level results are illustrative. The overall benchmark conclusions are based on the full dataset of 340 cross-jurisdiction patent family samples.

Future Research

Future benchmarks will continue to expand the dataset and refine the evaluation methods for greater accuracy, coverage, and representativeness. As the dataset grows, the benchmark will provide a more robust view of how AI tools perform in professional patent novelty search.