Unlock instant, AI-driven research and patent intelligence for your innovation.

Network data crawling method and device, storage medium and terminal equipment

A network data and data capture technology, applied in the computer field, can solve problems such as the inability to crawl data resources

Pending Publication Date: 2019-08-23
ONE CONNECT SMART TECH CO LTD SHENZHEN
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

It can be seen that traditional crawlers crawl data based on URLs, but now a large number of data resources are stored in APPs, and traditional crawlers cannot crawl these data resources

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Network data crawling method and device, storage medium and terminal equipment
  • Network data crawling method and device, storage medium and terminal equipment
  • Network data crawling method and device, storage medium and terminal equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0024] Embodiments of the present invention provide a network data crawling method, device, storage medium, and terminal equipment, capable of acquiring data resources stored in an APP.

[0025] In order to make the purpose, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the following The described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0026] see figure 1 , the first embodiment of a network data crawling method in the embodiment of the present invention includes:

[0027] 101. After ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to the technical field of computers, and provides a network data crawling method and device, a storage medium and terminal equipment. The network data crawling method comprises the following steps: calling an APP associated with identification information after receiving a data crawling request which is sent by a server and contains the identification information; controllingthe called APP to skip to each display page in sequence; when jumping to one display page every time, identifying a control contained in the page; extracting content data packaged in the control; andreturning the content data to the server. The above process can automatically call the APP installed on the terminal device and control the APP to skip among the display pages. In the page jumping process, content data contained in each page control is identified and extracted so as to obtain data resources stored in the APP.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a network data crawling method, device, storage medium and terminal equipment. Background technique [0002] A crawler is a program that automatically extracts web pages. It downloads web pages from the World Wide Web for search engines and is an important part of search engines. Traditional crawlers start from the URL of one or several initial webpages, obtain the URLs on the initial webpage, and continuously extract new URLs from the current page and put them into the queue during the process of crawling webpages until certain stop conditions are met. It can be seen that traditional crawlers crawl data based on URLs, but now a large number of data resources are stored in APPs, and traditional crawlers cannot crawl these data resources. Contents of the invention [0003] In view of this, the embodiments of the present invention provide a network data crawling method...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/951G06F9/4401
CPCG06F9/4416G06F16/951
Inventor 廖盼盼
Owner ONE CONNECT SMART TECH CO LTD SHENZHEN