Method and system for identifying web crawler, storage medium and electronic apparatus
A technology for identifying networks and web crawlers, applied in the field of information processing, can solve problems such as failure to identify crawlers, and achieve the effect of good identification and realization of identification
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment Construction
[0034] The present invention will be described in further detail below in conjunction with the accompanying drawings and embodiments.
[0035] see figure 1 As shown, the embodiment of the present invention provides a method for identifying a web crawler, the method includes the following steps:
[0036] S1. Create multiple invalid links for detecting web crawler behavior.
[0037] The embodiment of the present invention will first create some invalid links. Invalid means that the links will not provide any information for normal webpage access, and normal users basically cannot see the links and will not click on the links. Invalid Links are only used to detect crawler behavior. Usually invalid links will be hidden as much as possible to prevent normal users from clicking and accessing.
[0038] The specific link address of the invalid link can be to add a field to the existing address to indicate the invalid link, or to change a certain field of the normal address to an in...
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com