Structured processing method of distributed network information
A distributed network and information structure technology, applied in the field of distributed network information structured collection and processing, can solve problems such as large storage space and information use barriers, and achieve the effect of effective classification and convenient analysis and processing
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment
[0057] For an electric vehicle website, the user needs to obtain the news webpage and the car model parameter webpage. For these two types of webpages, each type obtains a typical webpage as the target webpage, thus forming a set of target webpages{C 1 , C 2}.
[0058] In step 2), distributed crawling is performed on the electric vehicle website to obtain its webpage data. The time-consuming process is related to the scale of the website to be crawled and the scale of the cluster that executes the crawling task. For a cluster of ten nodes, a fully loaded The crawling rate can reach 100,000 pieces / hour.
[0059] Step 3) The webpages obtained from the electric vehicle website are divided into two types of webpages: news and car model parameters through clustering, and the accuracy rate of clustering in this process can be more than 95%.
[0060] Step 4) carry out structural extraction for these two types of web pages, for news web pages, extract title, text, release date, sour...
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com