A kind of url data mining method and system

A data mining and data technology, applied in network data retrieval, other database retrieval, special data processing applications, etc., can solve the problems of low mining efficiency and long waiting time, and achieve the effect of improving data mining efficiency
CN110287428BActive Publication Date: 2021-07-27武汉思普崚技术有限公司
2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
武汉思普崚技术有限公司
Publication Date
2021-07-27

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The present application provides a URL data mining method and system. The data mined by the method can be applied to the upgrade of Internet behavior management equipment, including: obtaining URL data, cutting the URL data, and adding them to the coroutine event loop in turn; by running the coroutine The event cycle visits the URL address, and receives the request response fed back by the URL address server, and obtains the response data according to the request response; then checks whether the response data contains JS jump code; if it contains JS jump code, parses the JS jump code, and obtains the Orient the URL address, and receive the response data again; finally, store the URL address and the response data in a file according to the preset format. The method transforms passive acquisition of URL data into active acquisition, and reduces thread and process blockage caused by request response timeout waiting through coroutine events, thereby improving data mining efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The present application relates to the technical field of data mining, in particular to a URL data mining method and system. Background technique

[0002] URL (Uniform Resource Locator, Uniform Resource Locator), also known as a web page address, is a standard resource address on the Internet, through which you can directly locate a specified resource or web page on the Internet. Online behavior management products refer to helping Internet users control and manage Internet usage. Including web page filtering, network application control, bandwidth flow management, information sending and receiving audit, user behavior analysis, etc. In order to better understand the usage of the Internet, it is necessary to obtain more comprehensive URL data information through the online behavior management product, and reasonably classify the obtained URL data, so that the online behavior management product can match the user's website access classification record ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More