Webpage Type Identification via URL Rule Matching and ML

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current methods for identifying webpage types are either manpower-intensive and limited in scalability or computationally intensive and not suitable for real-time processing, failing to efficiently classify webpages across various fields in a timely manner.

Innovation Solution

A method and apparatus that analyze web addresses to match components with webpage classification rules, using a combination of rule-based classification and machine learning classifiers to determine webpage types, allowing for fast and efficient identification of webpage types based solely on the web address.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual classification by professional knowledge is used, then identification accuracy and speed are improved, but extendibility and scalability deteriorate due to manpower limitations

Engineering Contradiction:
Improveidentification accuracyVSAvoidextendibility
Core Design Contradiction:
Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent segments the classification task into two distinct stages: rule-based filtering (using URL patterns, domain names, and path structures) and machine learning classification (using SVM or other classifiers). This segmentation allows the system to leverage the accuracy of manual classification rules for obvious cases while using automated ML for ambiguous cases, thereby improving both accuracy and scalability simultaneously.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediary mechanism - the rule-based filtering system - that acts as a gateway between the incoming URL and the ML classifier. This intermediary pre-processes URLs by applying classification rules first, only passing uncertain cases to the ML model, thus reducing computational overhead while maintaining high accuracy.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Adaptability or versatility

If text classification methods like SVM are used, then coverage of various fields is improved, but computation time increases making real-time identification impossible

Engineering Contradiction:
Improvecoverage of various fieldsVSAvoidcomputation time
Core Design Contradiction:
Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent applies preliminary action by performing rule-based classification before invoking the computationally intensive ML classifier. The rule-based system quickly filters and classifies URLs that match known patterns (news, shopping, entertainment, etc.), only passing ambiguous cases to the ML model. This preliminary filtering significantly reduces the number of URLs requiring ML processing, thereby reducing overall computation time while maintaining broad field coverage.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent implements partial action by applying the full ML classification process only to the subset of URLs that fail to match rule-based criteria. Instead of applying ML to all URLs, the system performs partial classification - using simple rules for clear cases and reserving complex ML processing for ambiguous cases, thus optimizing computation time while maintaining accuracy.

Inventive Principle:
Principle #16Partial or excessive action

3Productivity

If only rule-based classification is used, then real-time processing speed is improved, but accuracy and coverage for unknown fields deteriorate

Engineering Contradiction:
Improveprocessing speedVSAvoididentification accuracy
Core Design Contradiction:
ProductivityVSMeasurement precision

Solution Approach 1:

The patent merges two classification approaches - rule-based classification and machine learning classification - into a unified hybrid system. The rule-based component provides fast processing for known patterns, while the ML component supplements it with learned patterns from training data, handling unknown or ambiguous cases. This merging allows the system to achieve both high processing speed and high accuracy simultaneously.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The patent creates a universal classification system that can handle both known and unknown webpage types. The rule-based portion handles conventional, well-documented URL patterns, while the ML classifier provides universal pattern recognition that adapts to various fields and unknown types. This multi-functional approach ensures the system maintains high speed for common cases while achieving high accuracy across diverse and unknown fields.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Data Source

PatentUS10311120B2Method and apparatus for identifying webpage type
Publication Date: 2019.06.04 TENCENT TECHNOLOGY (SHENZHEN) CO LTD
  • US10311120B2 patent drawing
  • US10311120B2 patent drawing
  • US10311120B2 patent drawing

AI summary

Various embodiments provide a method and an apparatus for identifying webpage type. The method includes: judging whether a web address to be classified matches with a webpage classification rule in at least two webpage classification rules; and determining the type of the webpage to be a type corresponding to a webpage classification rule which matches with the web address.