Unlock instant, AI-driven research and patent intelligence for your innovation.

Crawling method, device, computer equipment and storage medium for website resources

A website and resource technology, applied in the direction of network data indexing, network data retrieval, and other database retrieval, etc., can solve the problems of reducing the development efficiency of R&D personnel, reducing the enthusiasm of R&D personnel, time-consuming and labor-intensive, etc., to save labor costs and time costs, The effect of improving flexibility and effectiveness and ensuring accuracy

Active Publication Date: 2022-01-11
BEIJING KINGSOFT INTERNET SECURITY SOFTWARE CO LTD
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the problem is that R&D personnel can still configure a small number of data sources, and for large data needs, mechanical repetitive activities will also reduce the enthusiasm of R&D personnel, and this development method is time-consuming and laborious, which seriously reduces Improve the development efficiency of R&D personnel

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Crawling method, device, computer equipment and storage medium for website resources
  • Crawling method, device, computer equipment and storage medium for website resources
  • Crawling method, device, computer equipment and storage medium for website resources

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0043] Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

[0044] The crawling method, device, computer equipment and computer-readable storage medium of the embodiments of the present invention will be described below with reference to the accompanying drawings.

[0045] figure 1 It is a flowchart of a method for crawling website resources according to an embodiment of the present invention. It should be noted that the website resource crawling method in the embodiment of the present invention can be applied to the website resource crawling device in the embodiment of the present inven...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a website resource crawling method, device, computer equipment and storage medium. Wherein, the method includes: determining a flow chart designed by the user; wherein, the flow chart includes a plurality of nodes and connection relationships between nodes, and each node corresponds to a control; based on the controls corresponding to the nodes in the flow chart and the The connection relationship between nodes generates crawling configuration rules for the target website; crawls the corresponding resources in the target website according to the crawling configuration rules to obtain corresponding crawling result information. This method allows users to design corresponding flow charts according to their own needs, streamlines the process of configuring crawler rules based on the flow charts, improves configuration flexibility, effectiveness, and crawling accuracy, and can effectively save labor costs and time costs .

Description

technical field [0001] The invention relates to the field of computer applications, in particular to a method, device, computer equipment and storage medium for crawling website resources. Background technique [0002] With the rapid development of Internet technology, there are massive amounts of data on the Internet. In order to conveniently provide users with search functions, search engines often need to search and analyze massive amounts of data on the Internet. The emergence of crawler technology has effectively improved search efficiency. Crawler technology mainly extracts effective information by identifying, crawling, and cleaning specific resources. With the development of the times, crawler technology will also develop rapidly and be applied to more application fields to improve the utilization rate of data and promote the development of society. [0003] In related technologies, resources on the crawled webpage are identified and crawled mainly by manually view...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/951
CPCG06F16/951
Inventor 孙加亮
Owner BEIJING KINGSOFT INTERNET SECURITY SOFTWARE CO LTD
PatSnap Eureka logo

PatSnap Eureka turns technology decisions into work you can execute. Powered by our Innovation Knowledge Graph, it runs expert workflows across engineering, life sciences, materials and intellectual property. Get your review-ready output in minutes.

X (Twitter)LinkedInYouTubeSubstack

© 2026 PatSnap. All rights reserved. 

Terms of Service

 | 

Privacy policy

 | 

Modern Slavery Act Transparency Statement