Method and system for processing huge traffic data
A technology of traffic data and processing method, which is applied in the field of processing method and system of massive traffic data, can solve the problem of high maintenance cost of traffic data processing system, achieve the effect of reducing hardware performance requirements and maintenance cost
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0025] The mass traffic data processing system provided by Embodiment 1 of the present invention, such as figure 1 shown, including:
[0026] A cluster of management nodes 10 and worker nodes 11 connected through a network. in:
[0027] The management node 10 (computer responsible for managing tasks) is used to monitor the working status of each working node 11, monitor the resource usage and task execution of each working node 11, and send information to each working node 11 according to the resource usage of each working node 11. The worker nodes 11 allocate the storage of traffic data and the calculation tasks of the traffic analysis model.
[0028] The management node assigns tasks according to the resource usage of the working nodes, which can ensure that the resource occupancy rate of each working node is equal, so that the data to be stored and computing tasks are too concentrated on a certain working node, causing the working node and even the entire system performa...
Embodiment 2
[0033] Further in the processing system of massive traffic data provided by Embodiment 2 of the present invention, such as figure 2 shown.
[0034] Management nodes include 10:
[0035] The status monitoring unit 101 is configured to regularly collect the resource usage of each working node 11 , and send the resource usage information of each working node 11 to the storage allocation management unit 102 and the computing task allocation scheduling unit 103 .
[0036] The storage allocation management unit 102 is used for segmenting the traffic data to be stored, and assigning traffic data storage tasks to each working node 11 according to the resource usage information of each working node 11 sent by the state monitoring unit 101 .
[0037] The calculation task allocation scheduling unit 103 is configured to allocate the calculation task of the traffic analysis model to each work node 11 according to the resource usage information of each work node 11 sent by the state monit...
Embodiment 3
[0044] The massive traffic data processing method provided by Embodiment 3 of the present invention utilizes the system provided by Embodiment 1, such as image 3 shown, including steps:
[0045] S301. The management node acquires the resource usage of each working node connected to it through the network, and assigns the storage of traffic data and the calculation task of the traffic analysis model to each working node according to the resource usage of each working node.
[0046] S302. The working node receives the traffic data storage and traffic analysis model calculation tasks assigned by the management node, and performs the traffic data storage and traffic analysis model calculation processing.
[0047] In the method for processing massive traffic data provided by Embodiment 3 of the present invention, the management node obtains the resource usage of each working node connected to it through the network, and allocates the storage and storage of traffic data to each wor...
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com