Unlock instant, AI-driven research and patent intelligence for your innovation.

A data processing method and client device

A client device and data processing technology, applied in the client field, can solve problems such as reducing crawling efficiency

Active Publication Date: 2021-12-24
BEIJING GRIDSUM TECH CO LTD
View PDF12 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, in practical applications, users may not need to crawl all urls under the target domain name, but only want to crawl some urls under the target domain name, such as urls under certain subdirectories or subdomain names. At this time, If the web crawler still crawls all urls, the crawling efficiency will be reduced

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A data processing method and client device
  • A data processing method and client device
  • A data processing method and client device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0054] The embodiment of the present invention provides a data processing method and a client device that allows the user to filter URLs under the target domain by simple expressions.

[0055] The description and claims of the present application and the terms "first", "second", "third", "fourth", etc. in the above drawings are used to distinguish similar objects without having to use Describe a specific order or intern order. It should be understood that the data such as use can be interchangeable in appropriate, so that the embodiments described herein can be implemented in the order other than the content illustrated or described herein. Moreover, the terms "include" and and any of their deformations are intended to cover the inclusion of the inclusion, for example, the processes, methods, systems, products, or devices that contain a series of steps or units, are not necessarily limited to those steps or clearly listed. Unit, but may include other steps or units that are not cl...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

Embodiments of the present invention provide a data processing method and a client device, which enable users to filter URLs under a target domain name through simple expressions. An embodiment of the present invention provides a data processing method, including: a client device acquires template information input by a user, the template information is used to describe the matching rules of url, and the url is an URL under the target domain name corresponding to the template information the url; the client device converts the template information into a regular expression according to preset rules; the client device acquires the first target url matching the regular expression in the url; the The client device adds the first target url to a queue to be crawled.

Description

Technical field [0001] The present invention relates to the field of client, and more particularly to a data processing method and a client device. Background technique [0002] Network crawler is a program or script that automatically captures the web information in accordance with certain rules. [0003] During using the network reptile, the client device obtains the target domain name that needs to be climbed. The client device obtains all the URLs under the target domain name and add all URLs to the crawling queue. [0004] However, in practical applications, users may not need to climb all URLs under the target domain name, but only want to climb some of the URLs under the target domain, such as the URL under certain subdirectories or subdomains, at this time, If the network reptile still crawls all URLs will reduce climb efficiency. Inventive content [0005] The embodiment of the present invention provides a data processing method and a client device that allows the user ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/951G06F16/9535G06F16/955
CPCG06F16/9535
Inventor 何熠皓
Owner BEIJING GRIDSUM TECH CO LTD