Unlock instant, AI-driven research and patent intelligence for your innovation.

Allocation method, distributor and url detection system of url detection tasks

A technology of detection tasks and distribution methods, applied in the Internet field, can solve problems such as detection failures, reduce the risk of shielding, and prevent frequent request tasks

Active Publication Date: 2018-12-25
BEIJING QIHOO TECH CO LTD
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The developer of the Internet platform needs to detect the link address (URL) of the idea to ensure that the user can correctly link to the content of the idea. However, the existing method is generally to uniformly assign the link address of the idea to a web crawler, and the web crawler For detection, it does not consider the risk that the crawler frequently requests the website of the merchant or enterprise and is blocked by the firewall of the merchant or enterprise, resulting in detection failure

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Allocation method, distributor and url detection system of url detection tasks
  • Allocation method, distributor and url detection system of url detection tasks
  • Allocation method, distributor and url detection system of url detection tasks

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0030] figure 1 A flow chart of a method for allocating URL detection tasks according to an embodiment of the present invention is shown. Among them, URL is the link address of the material. The material here can be ideas or keywords. Each material corresponds to a URL, and you can view the specific content by clicking on the URL, such as title, description, etc. Such as figure 1 As shown, the method includes the following step...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a URL detecting task distributing method, a distributor and a URL detecting system. A URL is a material chained address. The method comprises the steps that URL detecting tasks which are obtained from a database are stored in a URL task list; requests for obtaining the URL detecting tasks are received, wherein the requests are sent by multiple web crawlers; the requests are responded, and according to a pre-set distributing strategy, the URL detecting tasks in the URL task list are distributed to one of the multiple web crawlers so that the connectivity of the URL can be detected by the one of the web crawlers. According to the pre-set distributing strategy, the URL detecting tasks are distributed to the web crawlers, it can be avoided that the web crawlers request the tasks frequently, and the risk that the tasks are shielded by a user firewall is lowered.

Description

technical field [0001] The invention relates to the technical field of the Internet, in particular to a URL detection task distribution method, a distributor and a URL detection system. Background technique [0002] With the rapid development and popularization of Internet technology, more and more merchants or enterprises put their ideas on Internet-based platforms, use the Internet to promote information on web pages, and attract more people to understand and pay attention to their products and services, etc. On the one hand, merchants or enterprises can obtain more conversion rates through the click-through rate, and then profit from it; on the other hand, using Internet promotion information can reduce promotion costs and save promotion costs for merchants or enterprises. [0003] The developer of the Internet platform needs to detect the link address (URL) of the idea to ensure that the user can correctly link to the content of the idea. However, the existing method is ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
Inventor 桂成林
Owner BEIJING QIHOO TECH CO LTD