Webpage Content Verification via Server-Client Data Comparison

Solution Overview

Problem

Webpage content indexing systems are vulnerable to malicious content and software threats, allowing websites with malicious content to spoof or misrepresent their content, potentially affecting search result rankings and user privacy.

Innovation Solution

A method and system for verifying webpage content by comparing indicating data from a hosting server and a client terminal, using cryptographic hashes and other analyses to determine the accuracy and integrity of the content, and removing invalid parts to provide verified content for indexing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If a crawler resides on a static IP address or subnet to enable content indexing, then the indexing system can operate efficiently, but malicious websites can spoof or misrepresent their content more easily

Engineering Contradiction:

Improvecontent indexing efficiencyVSAvoidcontent authenticity

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system performs preliminary verification by obtaining indicating data from multiple sources (hosting server and client terminals) before indexing content. This advance verification prevents malicious content from being indexed, resolving the contradiction between indexing efficiency and content authenticity by establishing trust before the indexing operation occurs.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary verification mechanism that compares indicating data from the hosting server with indicating data from multiple client terminals. This intermediary comparison process acts as a mediator to verify content authenticity, allowing the system to maintain both indexing efficiency and reliability by filtering out spoofed content through the intermediary verification layer.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Speed

If a crawler uses high-bandwidth connections for content retrieval, then indexing speed is improved, but the system becomes more vulnerable to exploitation by malicious websites

Engineering Contradiction:

Improveindexing speedVSAvoidmalicious content exploitation

Core Design Contradiction:

SpeedVSObject-affected harmful factors

Solution Approach 1:

The system performs preliminary verification by obtaining indicating data from multiple sources (hosting server and client terminals) before indexing content. This advance verification prevents malicious content from being indexed, resolving the contradiction between indexing efficiency and content authenticity by establishing trust before the indexing operation occurs.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary verification mechanism that compares indicating data from the hosting server with indicating data from multiple client terminals. This intermediary comparison process acts as a mediator to verify content authenticity, allowing the system to maintain both indexing efficiency and reliability by filtering out spoofed content through the intermediary verification layer.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If content is indexed without verification to improve productivity, then search result ranking accuracy deteriorates due to potential spoofing

Engineering Contradiction:

Improveindexing throughputVSAvoidsearch ranking accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The system performs preliminary verification by obtaining indicating data from multiple sources (hosting server and client terminals) before indexing content. This advance verification prevents malicious content from being indexed, resolving the contradiction between indexing efficiency and content authenticity by establishing trust before the indexing operation occurs.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary verification mechanism that compares indicating data from the hosting server with indicating data from multiple client terminals. This intermediary comparison process acts as a mediator to verify content authenticity, allowing the system to maintain both indexing efficiency and reliability by filtering out spoofed content through the intermediary verification layer.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS8307276B2Distributed content verification and indexing

Publication Date: 2012.11.06 GEN DIGITAL INC

AI summary

A method of verifying content, for example webpage content, including receiving, at an indexing system (100), a first copy (240) of a document (220) from a hosting server (210), then processing the first copy (240) to obtain first indicating data including one or more properties of the first copy (240). Also obtaining second indicating data, the second indicating data obtained by processing a second copy (255) of the document (220) from the hosting server (210), the second copy (255) received by a client terminal (250), the second indicating data including one or more properties of the second copy (255). Then the first indicating data and the second indicating data can be compared, and the first copy (240) of the document (220) received by the indexing system (100) can be determined to be verified or not verified based on a comparison between the first indicating data and the second indicating data.