Crawler technology based on block chain system

A crawler technology, blockchain technology, applied in transmission systems, network data query, network data indexing, etc., to achieve the effect of lowering the threshold

Active Publication Date: 2019-07-05
扬州杜罗新研网络科技有限公司
View PDF3 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

To sum up, the shortcomings of the existing technology: the existing web crawler technology itself is aimed at the Http/Https protocol, and crawls the website information resources, and the website resources often use a centralized network struc

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0014] The content of the present invention will be described below in conjunction with specific embodiments.

[0015] Example method:

[0016] Operation 1: Add a public key open node to the existing nodes of the blockchain, which is only open to trusted crawlers;

[0017] Operation 2: When the blockchain node is started, it needs to actively register with the nearest public key open node; at the same time, the public key open node will also initiate registration work notifications to the entire network from time to time, and give priority to the registered nodes;

[0018] Operation 3: When the crawler is started, it will request verification from the public key open node according to the configuration. After passing the verification and approval, it can download the public key information stored by some nodes from the node;

[0019] Operation 4: The crawler program searches according to the message information clues in the block header data, and decrypts the encrypted data i...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a crawler technology based on a block chain system, which comprises the following steps of: 1, adding a public key open node on an existing node in a block chain network; step2, after a node in the block chain system is started, accessing a nearby public key open node according to self-starting configuration, and carrying out public key information registration on the public key open node; 3, after the crawler is started, searching for nearby public key nodes, and downloading cache public key node information; 4, performing data crawling according to the clue information; and 5, processing crawling of the nodes by the crawler in the system by adopting a flooding algorithm, setting TTL corresponding values, and when deep crawling is carried out according to clue information, and if the TTL set values are exceeded, ending crawling of the clues. The method is mainly used in a block chain system, greatly plays the storage characteristics of the block chain, and isbeneficial to the development of the block chain application.

Description

technical field [0001] The invention relates to a crawler technology based on a blockchain system, which belongs to the technical field of the Internet. Background technique [0002] In the blockchain system, data can be stored in the block header and block body, and each node in the blockchain system can be regarded as an independently running environment and individual, which has an independent network address, The operating system, memory required for operation, computing resources, etc.; at the same time, the data stored in the block header is generally abstract and descriptive, and the data stored in the block body is often encrypted by public and private keys; Nodes perform consensus calculations through continuous interactive calculations. Each node participating in blockchain consensus calculations can have relevant data, so there is a large amount of duplicate data in the blockchain network; since the blockchain is In a decentralized distributed network, there is a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L29/06H04L29/08G06F16/953G06F16/951
CPCH04L63/0442H04L67/02H04L67/10H04L69/26
Inventor 梅一多孙昊马敬东朱晓庆
Owner 扬州杜罗新研网络科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap