Site Originality Score via N-gram Sampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods fail to effectively detect and differentiate between original and copied content on websites, leading to challenges in ranking search results based on content authenticity.
Innovation Solution
A system that calculates a site originality score by sampling n-grams from resources, determining their origin, and computing a ratio of unique to total n-grams, which is used to rank search results and identify original content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If content scraping detection is not implemented, then search results can be ranked simply, but original content cannot be distinguished from copied content
Solution Approach 1:
The patent segments content into n-grams (overlapping sequences of n words) to analyze originality at a granular level. By breaking down web pages into smaller linguistic units and comparing them across sites, the system can precisely detect copied content without requiring complex full-page analysis, thus improving measurement precision while managing system complexity
Solution Approach 2:
The patent creates a digital copy of content in the form of n-gram signatures and hash values. By computing hashes of n-grams and storing them in a database with origin site information, the system enables efficient comparison and detection of copied content across different websites without repeatedly analyzing full original pages
2Measurement precision
If all n-grams are analyzed to determine originality, then detection accuracy is maximized, but processing time and computational resources increase
Solution Approach 1:
The patent analyzes only a sample of n-grams from each web page rather than every possible n-gram. By selecting representative n-grams strategically, the system achieves sufficient detection accuracy while significantly reducing computational overhead and processing time compared to exhaustive analysis of all content
3Reliability
If site originality scores are used to rank search results, then original content is promoted, but the ranking system becomes more complex
Solution Approach 1:
The patent merges the site originality score with existing search ranking signals by combining it with page relevance scores. This integration allows originality consideration to be incorporated into the ranking process without requiring a completely separate ranking system, thus improving result quality while managing complexity through signal combination
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for identifying a plurality of n-grams in a plurality of resources found in a particular site; determining, for each of the plurality of resources, a count of n-grams that originated in the resource; determining, based on counts of n-grams that originated in the resources, a first aggregate count of n-grams that originated in the particular site; determining a second aggregate count of the plurality of n-grams that were identified in the plurality of resources found in the particular site; and determining, based on the first and second aggregate counts, a site originality score for the particular site.