A method of crawling and screening financial warehouse receipt risk control information based on stream computing
A stream computing and screening method technology, which is applied in computing, digital data information retrieval, web data retrieval using information identifiers, etc., can solve the real-time requirements of parallel crawlers and financial warehouse receipt risk control for goods valuation Advanced problems, to maximize processing efficiency, prevent local hot spots, and reduce performance impact
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment Construction
[0033] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.
[0034] Such as figure 1 As shown, the method for crawling and screening financial warehouse receipt risk information based on stream computing in the present invention includes the following steps:
[0035] Step 1: Decompose the web crawler task under the streaming computing framework.
[0036] According to the streaming computing framework, the overall process of web crawling is decoupled and decomposed into six sub-processes: URL screening, page analysis, keyword filtering, numerical filtering, feature vector matching filtering, and resource updating. Encapsulate 6 sub-processes into 6 types of logical sub-tasks (Bolt type tasks) according to the streaming computing framework. In addition, data source logic tasks (Spout type tasks) are also required in the streaming computing framework.
[0037] There can be multiple logical subtasks of th...
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com