A kind of url data mining method and system

A data mining and data technology, applied in network data retrieval, other database retrieval, special data processing applications, etc., can solve the problems of low mining efficiency and long waiting time, and achieve the effect of improving data mining efficiency

Active Publication Date: 2021-07-27
武汉思普崚技术有限公司
View PDF2 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] This application provides a URL data mining method and system to solve the problems of long waiting time and low mining efficiency in traditional URL data mining methods

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A kind of url data mining method and system
  • A kind of url data mining method and system
  • A kind of url data mining method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0068] The embodiments will be described in detail hereinafter, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following examples do not represent all implementations consistent with this application. These are merely examples of systems and methods consistent with aspects of the present application as recited in the claims.

[0069] see figure 1 , is a schematic flowchart of a URL data mining method of the present application. Depend on figure 1 It can be seen that the URL data mining method provided by this application includes the following steps:

[0070] S1: Obtain URL data, where the URL data includes multiple URL addresses.

[0071] In the technical solution provided by the present application, the URL data mining method can be applied to a ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present application provides a URL data mining method and system. The data mined by the method can be applied to the upgrade of Internet behavior management equipment, including: obtaining URL data, cutting the URL data, and adding them to the coroutine event loop in turn; by running the coroutine The event cycle visits the URL address, and receives the request response fed back by the URL address server, and obtains the response data according to the request response; then checks whether the response data contains JS jump code; if it contains JS jump code, parses the JS jump code, and obtains the Orient the URL address, and receive the response data again; finally, store the URL address and the response data in a file according to the preset format. The method transforms passive acquisition of URL data into active acquisition, and reduces thread and process blockage caused by request response timeout waiting through coroutine events, thereby improving data mining efficiency.

Description

technical field [0001] The present application relates to the technical field of data mining, in particular to a URL data mining method and system. Background technique [0002] URL (Uniform Resource Locator, Uniform Resource Locator), also known as a web page address, is a standard resource address on the Internet, through which you can directly locate a specified resource or web page on the Internet. Online behavior management products refer to helping Internet users control and manage Internet usage. Including web page filtering, network application control, bandwidth flow management, information sending and receiving audit, user behavior analysis, etc. In order to better understand the usage of the Internet, it is necessary to obtain more comprehensive URL data information through the online behavior management product, and reasonably classify the obtained URL data, so that the online behavior management product can match the user's website access classification record ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/955
CPCG06F16/955
Inventor 柳开江
Owner 武汉思普崚技术有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap