Unlock instant, AI-driven research and patent intelligence for your innovation.

Reptile Protocol Management Method and Device, Reptile System

A crawler system and protocol management technology, applied in the Internet field, can solve problems such as unified management of unfavorable robots information, and achieve the effect of improving processing capabilities

Active Publication Date: 2021-08-17
BEIJING SOGOU TECHNOLOGY DEVELOPMENT CO LTD
View PDF11 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This method is not conducive to the unified management of robots information

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Reptile Protocol Management Method and Device, Reptile System
  • Reptile Protocol Management Method and Device, Reptile System
  • Reptile Protocol Management Method and Device, Reptile System

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0054] In order to better understand the embodiments of the embodiments of the present invention, the following detailed description will be described in detail below with reference to the drawings and embodiments.

[0055]Embodiments of the present invention provide a reptile protocol management method and apparatus, which will independently become a separate service of the management of the reptile protocol, and the reptile protocol documents of each site are managed and operated. Specifically, the reptile protocol file of the acquired site is added to the message queue, and the message queue is monitored in real time. Once you monitor the newly added reptile protocol file in the message queue, the resolution process is parsed, and the resolution The post-reptile protocol file is saved to the database.

[0056] like figure 1 As shown, it is a flow chart of a reptile protocol management method according to an embodiment of the present invention, including the following steps:

[...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a crawler protocol management method and device. The method includes: acquiring a crawler protocol file of a site, and adding the crawler protocol file into a message queue; monitoring the message queue in real time, and checking the message queue The newly added crawler protocol file is parsed to obtain the parsed crawler protocol file, which includes the corresponding relationship between the site and the crawler protocol; and the parsed crawler protocol file is stored in the database. The invention also provides a crawler system. The invention can realize the unified management of the reptile protocol.

Description

Technical field [0001] The present invention relates to the field of Internet, and more particularly to a reptile protocol management method and apparatus, further involving a reptile system. Background technique [0002] Network crawler, also called the web spider, which is based on the address of the web page to find web URL (UniformResource Locator, Uniform Resource Locator), and network crawler obtains web information based on the URL. Because network reptiles can climb various content from the server, because there may be content related to personal privacy or commercial confidentiality, it will bring unnecessary troubles and disputes to users and server managers, and through the Robots protocol ( Also known as reptile agreements, robot protocols, etc.) to specification. [0003] The Robots protocol is an agreement between the website with the reptile, telling the corresponding reptile permissions with a simple and direct TXT format text. When a network crawler accesses a si...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/955
Inventor 李超李毅陈飞
Owner BEIJING SOGOU TECHNOLOGY DEVELOPMENT CO LTD