A Spark cluster parallel computing-based traffic jam point discovery method

Abstract

The invention discloses a traffic jam point discovery method based on Spark cluster parallel computing. The invention discloses a trajectory big data mining method, relates to the technical field of trajectory big data mining, solves the technical problem of how to quickly cluster massive trajectory data and find a traffic congestion source of an urban hot region, and comprises the following steps: (1) preprocessing the massive data, including complementing errors between trajectory points and eliminating trajectory point redundancy in a certain region for a period of time; (2) adopting a grid-means clustering algorithm to carry out clustering to obtain a plurality of target data samples; (3) clustering by adopting a neighborhood maximum density grid clustering algorithm to obtain an urbanhot traffic road network model; and (4) calculating the adjacent grid density difference by adopting a neighborhood density difference algorithm to obtain a traffic jam source, and storing the obtained result in a Spark memory. According to the method, mass data can be quickly clustered to obtain an urban traffic network model, and quick discovery of a traffic jam source region is realized.

Description

technical field [0001] The invention relates to the technical field of big data mining, in particular to a method for discovering traffic congestion points based on spark cluster parallel computing. Background technique [0002] With the increasing development of urban traffic and the widespread application of vehicle positioning systems, a large amount of trajectory data can be generated every day, but these massive trajectory data have not been well utilized. At the same time, with the rapid development of urban economy and the needs of smart cities, in the face of increasingly severe traffic congestion problems, how to use the current trajectory big data to find the source of traffic congestion and solve urban traffic congestion problems, and promote the progress of smart cities new research hotspot. [0003] Currently, there are many tools for mining and analyzing big data platforms, among which Hadoop and Spark are the main ones. [0004] Compared with Spark, because ...

AI Technical Summary

Problems solved by technology

Among them, the grid clustering algorithm is the fastest, but the accuracy is not high; the quality of the mean algorithm depends on the selection of the initial clustering center, and it is easy to fall into a local optimal solution; the advantage of the density clustering algorithm is that the clustering effect is good, but the time is complicated high degree

Images