A method, a device and equipment for acquiring hot domain name description information, and a storage medium

A technology for describing information and domain names, applied in the Internet field, can solve problems such as slow crawler speed, achieve the effect of increasing crawler speed, reducing overall crawling time, and reducing the number of crawlers

Pending Publication Date: 2021-05-11
PENG CHENG LAB
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The main purpose of the present invention is to provide a method, device, device and storage medium for obtaining hot domain name description information, aiming to solve the technical problem of slow crawler speed when faced with massive hot domain name data that needs to be processed by crawlers

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A method, a device and equipment for acquiring hot domain name description information, and a storage medium
  • A method, a device and equipment for acquiring hot domain name description information, and a storage medium
  • A method, a device and equipment for acquiring hot domain name description information, and a storage medium

Examples

Experimental program
Comparison scheme
Effect test

no. 1 example

[0080] Based on the above-mentioned first embodiment, the step S40 of the method for obtaining hotspot domain name description information in this embodiment includes:

[0081] Step S401: Traversing the list to be crawled, invoking a first crawling strategy according to a preset priority order, and crawling the traversed top-level domains according to the first crawling strategy.

[0082] It can be understood that the preset priority order is preset. For example, the first crawling strategy is set to the highest priority, the second crawling strategy is set to the second, and the third crawling strategy is set to the lowest priority. The preset priority Corresponding to multiple crawling strategies, the first crawling strategy may be any one of crawling methods such as Urllib method, requests method, and BS4-BeautifulSoup4 analysis, which is not limited in this embodiment.

[0083] Step S402: When the corresponding description information is not crawled, call the second crawli...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the technical field of Internet, and discloses a method, a device and equipment for acquiring hot domain name description information, and a storage medium. The method comprises the following steps: acquiring a plurality of hotspot domain names with preset time granularity; screening the plurality of hotspot domain names to obtain screened domain names to be processed; grouping is carried out according to the top-level domain of the domain name to be processed, a list to be crawled is obtained, and the list to be crawled comprises the top-level domain and a corresponding domain name list; traversing the list to be crawled, and crawling the traversed top-level domain to obtain corresponding description information; and taking the description information as description information corresponding to each domain name to be processed in the domain name list. According to the mode, the hotspot domain names are screened and grouped, the top-level domains of the to-be-processed domain names are crawled, and the crawled description information of the top-level domains serves as the description information of the to-be-processed domain names in the corresponding groups, so that the crawler number of massive hotspot domain names is greatly reduced, the overall crawler time is shortened, and the crawler speed is increased.

Description

technical field [0001] The present invention relates to the technical field of the Internet, in particular to a method, device, equipment and storage medium for acquiring hot domain name description information. Background technique [0002] Domain name is an important resource in the Internet, and it is the core function to achieve Internet service acquisition and resource access. The normal operation of almost all Internet applications cannot be separated from the support of the Domain Name System (English: Domain Name System, DNS). DNS is the cornerstone of global Internet services and an important guarantee for interconnected network communications. A large number of resource access records are generated on the DNS server every day, and the domain name data in it are sorted by the number of visits to filter out the daily hot domain names. Save the hotspot domain name and the description information of the hotspot domain name obtained through the crawler into the databas...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/951
CPCG06F16/951
Inventor 霍鹏磊张伟哲张宾董国忠刘鹏辉
Owner PENG CHENG LAB
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap