Website Address Identification Using Hidden Markov Model Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for identifying spam website addresses are inadequate, as they rely on webpage whitelisting, blacklisting, or content crawling, which fail to accurately detect newly generated spam sites or those that mimic legitimate sites to steal user information.
Innovation Solution
A website address identification method using a hidden Markov model that segments website addresses into type identifiers, calculates probability values based on these identifiers, and generates identification information, allowing for the classification of websites without pre-storing historical addresses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If webpage whitelisting or blacklisting is used for spam identification, then the identification process is simple to implement, but newly generated spam website addresses cannot be identified
Solution Approach 1:
The patent transforms website addresses into type identifier sequences by segmenting the address and converting each part to a type identifier. This parameter transformation enables the system to identify spam websites based on structural patterns rather than relying on pre-stored lists, thus expanding the identification range to include newly generated spam addresses while maintaining implementation simplicity through automated processing.
2Measurement precision
If website address crawling is used for identification, then the identification can be based on actual content, but spam websites that set crawling prohibition or clone ordinary sites cannot be accurately identified
Solution Approach 1:
The patent segments website addresses into multiple parts (such as protocol, domain, path, query parameters) and converts each part to a type identifier. This segmentation allows the system to analyze the structural characteristics of website addresses without needing to crawl the actual webpage content, thereby accurately identifying spam websites that prohibit crawling or clone legitimate sites while expanding the identification range.
3Reliability
If traditional spam identification methods are used, then the system requires pre-stored historical website addresses, but this limits the identification range and requires continuous updates
Solution Approach 1:
The patent replaces the mechanical approach of storing and comparing historical website address strings with a probabilistic model (Hidden Markov Model) that analyzes the structural patterns of addresses. This substitution eliminates the need for continuous updates of pre-stored historical data while maintaining identification reliability, as the model learns from training data and generalizes to identify both known and newly generated spam addresses.
Data Source
AI summary
The present application discloses a website address identification method and apparatus. A specific embodiment of the method includes: obtaining a website address to be identified; segmenting the to-be-identified website address into a plurality of website address parts according to a preset rule, and converting each of the segmented website address parts into a type identifier corresponding to a content type of the each of the segmented website address parts; determining a probability value of the to-be-identified website address as a non-spam website address according to a type identifier sequence corresponding to the to-be-identified website address; and generating website address identification information according to the probability value. According to the embodiment, the website address identification range is expanded.