Scraping method based on application interface, terminal and storage medium

An application interface and crawler technology, applied in the field of information security, can solve the problems of poor data stability and a lot of investment, and achieve the effect of reducing time and efficient deduplication processing.

Pending Publication Date: 2020-12-18
IND INTERNET INNOVATION CENT (SHANGHAI) CO LTD
View PDF6 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The inventors have found that there are at least the following problems in the related technologies: the stability of the required data obtained from the webpage is poor, if the style of the webpage changes, the position of the extracted data also changes, if the crawler technology is used in the original settings Analytical technology cannot extract accurate data, and it takes a lot of time to adapt and maintain it

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Scraping method based on application interface, terminal and storage medium
  • Scraping method based on application interface, terminal and storage medium
  • Scraping method based on application interface, terminal and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art can understand that in each embodiment of the present invention, many technical details are provided for readers to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in this application can also be realized.

[0021] The division of the following embodiments is for the convenience of description, and should not constitute any limitation to the specific implementation of the present invention, and the various embodiments can be combined and referred to each other on the premise of no contradiction.

[0022] The first embodiment of the present invention relates t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

Embodiments of the invention relate to the field of information security, and discloses a scraping method based on an application interface, a terminal and a storage medium. The method comprises the steps of determining a target application interface according to to-be-acquired data; calling the target application interface to establish a data interaction channel; and collecting a first data packet in the data interaction channel by utilizing a scraping technology, and acquiring to-be-obtained data in the first data packet. The application interface of an application program is utilized to realize data acquisition, and the data for calling the application interface to perform interaction is relatively fixed, so the required data can still be stably acquired from the target application interface after the application program is updated, and time required for maintenance is reduced.

Description

technical field [0001] The embodiments of the present invention relate to the field of information security, in particular to a crawler method, terminal and storage medium based on an application interface. Background technique [0002] With the rapid development of the information society, there are more and more data in various fields. In order to obtain useful information, web crawler technology is often used to crawl useful data. Crawler technology can automatically crawl data in web pages, reducing It reduces the consumption of manpower and improves the efficiency of data crawling. When crawler technology obtains data from web pages, it first simulates the requested url, sends a request to the server, and obtains the web page display data after obtaining the response, and then uses xpath, bs4 and other analysis technologies to extract corresponding valuable data, and completes Acquisition of web data. [0003] The inventors have found that there are at least the follo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/951G06F21/60
CPCG06F16/951G06F21/602
Inventor 郑忠斌王朝栋彭新张雪帆
Owner IND INTERNET INNOVATION CENT (SHANGHAI) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap