Muck truck track processing method

Abstract

The invention discloses a slag car track processing method. The method comprises the following steps of track data receiving, track information data service building, track data processing task distribution, Kafka message queue cluster building, track data processing service distributed cluster building on multiple servers and track data storage. The method comprises the steps of track data classification, judgment of whether a track point is located on a road, hotspot grid judgment, judgment of whether an accumulated value of a basic grid in a time window exceeds a threshold value, hotspot region merging, hotspot road section analysis, calculation of a road section intersecting with a slag car driving hotspot region surface in urban road network data, and slag car operation point analysis. According to the slag car track processing method, the slag car track processing flow is designed based on a distributed architecture, task distribution is performed through the Kafka message queue, decoupling and load balancing between different track processing tasks are completed, and the requirements of slag car track processing for low delay and real-time performance are met.

Description

Technical Field

[0001] The present application relates to the technical field of tracks of muck trucks, and in particular to a method for processing tracks of muck trucks. Background Art

[0002] Construction dust is one of the main sources of urban air pollution. Muck truck transportation is the main link of construction dust. Illegal transportation and illegal construction have a great impact on the urban environment and human life. The trajectory data of muck trucks contains rich information about the operation of muck trucks. How to analyze the muck transportation behavior through trajectory data and achieve effective and comprehensive supervision of muck trucks is of great significance in dust control.

[0003] The trajectory data of muck trucks has the characteristics of large data volume and high real-time performance. The existing muck truck transportation analysis is mostly carried out using offline past trajectory data. The analysis takes a long time and cannot meet the advanced needs of dust control for real-time analysis of muck truck transportation behavior. In addition, the traditional single-machine operation mode is difficult to cope with the huge throughput requirements of real-time processing of muck truck trajectory data, thus affecting the efficiency of program analysis. Summary of the invention

[0004] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0005] To this end, the first purpose of this application is to provide a method for processing muck truck trajectories, design a muck truck trajectory processing process based on a distributed architecture, distribute tasks through the Kafka message queue, complete the decoupling and load balancing between different trajectory processing tasks, and meet the requirements of muck truck trajectory processing for low latency and real-time performance.

[0006] The second purpose of this application is to propose a method for processing the trajectory of a muck truck, which uses the basic grids divided by Geohash coding as units, identifies the hotspot grids within the time window through distributed streaming processing of the muck truck trajectory data, and merges them into hotspot areas. Through spatial analysis of hotspot areas and existing road network data, the real-time hotspot sections of muck trucks are obtained. Through spatial analysis of hotspot areas and existing data such as existing construction sites, disposal sites, parking lots, and gas stations, real-time hotspot construction sites and disposal sites are obtained. By separately performing density-based aggregation on the trajectory data in the hotspot areas excluding the known source areas, illegal sources such as black construction sites or black disposal sites are obtained.

[0007] To achieve the above-mentioned purpose, the first embodiment of the present application proposes a method for processing a track of a muck truck, and the method comprises the following steps:

[0008] Step 1: Receive trajectory data and build trajectory information data service;

[0009] Step 2: Distribute trajectory data processing tasks, build a Kafka message queue cluster, and build a trajectory data processing service distributed cluster on multiple servers;

[0010] Step 3: Trajectory data storage;

[0011] Step 4: Trajectory data classification to determine whether the trajectory point is on the road;

[0012] Step 5: hotspot grid judgment, judging whether the cumulative value of the basic grid in the time window exceeds the threshold;

[0013] Step 6: Merge hotspot areas;

[0014] Step 7: Hotspot section analysis, calculate the sections in the urban road network data that intersect with the hotspot area of ​​​​the muck truck;

[0015] Step 8: Analyze the operation points of the muck trucks to obtain the target hotspot construction site set, hotspot disposal site set and illegal source point set.

[0016] A method for processing the trajectory of a muck truck in an embodiment of the present application designs a muck truck trajectory processing flow based on a distributed architecture, distributes tasks through a Kafka message queue, completes the decoupling and load balancing between different trajectory processing tasks, and meets the requirements of low latency and real-time performance for muck truck trajectory processing.

[0017] Taking the basic grid divided by Geohash coding as the unit, the hotspot grids within the time window are identified through distributed streaming processing of the trajectory data of the muck truck, and merged into hotspot areas. Through the spatial analysis of the hotspot area and the existing road network data, the real-time hotspot sections of muck trucks are obtained. Through the spatial analysis of the hotspot area and the existing data such as the existing construction sites, disposal sites, parking lots, gas stations, etc., the real-time hotspot construction sites and disposal sites are obtained. By performing density-based aggregation on the trajectory data in the hotspot area after removing the known source area, illegal sources such as illegal construction sites or illegal disposal sites are obtained.

[0018] In addition, the above-mentioned method for processing the track of a muck truck proposed in this application may also have the following additional technical features:

[0019] In one embodiment of the present application, the specific method of receiving the trajectory data in step 1 is:

[0020] The track information data service is constructed to receive in real time the track positioning data of the muck truck transmitted back by the muck truck onboard positioning terminal.

[0021] In one embodiment of the present application, the specific method of distributing the trajectory data processing tasks in step 2 is:

[0022] The Kafka message queue cluster is built to create a topic ic for trajectory data processing. After the service receives the trajectory positioning data, it sends the trajectory data to the topic ic for consumption by consumers. At the same time, the key of the topic ic is set to the license plate number to ensure that the trajectory data of the same vehicle can be distributed to the same partition and processed by consumers in sequence.

[0023] The track data processing service distributed cluster is built on multiple servers, and the track data processing topic ic is subscribed as a Kafka consumer to pull and process the track data, and Kafka completes the balanced distribution of message distribution among the nodes in the cluster.

[0024] In one embodiment of the present application, the specific method of storing the trajectory data in step 3 is:

[0025] Establish a trajectory data storage consumer group and execute the data storage task. After the task obtains the trajectory data, it calculates the 10-bit Geohash code of the point according to the longitude and latitude, and stores it in the distributed column database HBase together with the other received fields.

[0026] In one embodiment of the present application, the specific method of classifying the trajectory data in step 4 is:

[0027] The trajectory data classification is to determine whether the trajectory point is located on the road based on the latitude and longitude of the trajectory point, combined with the urban road network data, and using the spatial database after the task obtains the trajectory data, and divide the trajectory points into two categories: one is the driving data points of the muck truck located on the road, and the other is the earthwork data points of the muck truck not on the road.

[0028] In one embodiment of the present application, the specific method of determining the hotspot grid in step 5 is as follows:

[0029] The Geohash coding grid of a given coding length is used as the basic division grid, and the time window and step length are set. The time window length is an integer multiple of the step length.

[0030] After the task obtains the trajectory data, it calculates the Geohash code of the point based on the longitude and latitude and accumulates it in the Redis cache database;

[0031] Set the hotspot grid threshold, and after each step, determine whether the cumulative value of the basic grid in the time window exceeds the threshold.

[0032] In one embodiment of the present application, the specific method of merging the hotspot areas in step 6 is as follows:

[0033] The entire processing space is divided into multiple processing partitions. Each processing partition contains the hotspot grids falling therein, which are evenly distributed to each node in the cluster for the initial hotspot area merging, and the obtained hotspot sub-areas are merged for the second time.

[0034] In one embodiment of the present application, the specific method of hotspot section analysis in step 7 is as follows:

[0035] The set of hot spots for muck truck driving is obtained to form spatial multi-polygon data. The spatial database is overlaid with the known urban road network data to calculate the road sections in the urban road network data that intersect with the hot spots for muck truck driving, and finally the set of target hot spots is obtained.

[0036] In one embodiment of the present application, the specific method of analyzing the operation point of the muck truck in step 8 is as follows:

[0037] Obtain a set of hotspot areas for muck truck earthwork operations and divide them evenly into various processing nodes for analysis.

[0038] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0040] Figure 1 The present invention is a flowchart of a method for processing a trajectory of a muck truck according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limitations on the present application. On the contrary, the embodiments of the present application include all changes, modifications and equivalents that fall within the spirit and connotation of the appended claims.

[0042] A method for processing the trajectory of a muck truck according to an embodiment of the present application is described below in conjunction with the accompanying drawings.

[0043] like Figure 1 As shown, a method for processing a track of a muck truck in an embodiment of the present application may include the following steps:

[0044] Step 1: Receive trajectory data and build trajectory information data service;

[0045] Step 2: Distribute trajectory data processing tasks, build a Kafka message queue cluster, and build a trajectory data processing service distributed cluster on multiple servers;

[0046] Step 3: Trajectory data storage;

[0047] Step 4: Trajectory data classification to determine whether the trajectory point is on the road;

[0048] Step 5: hotspot grid judgment, judging whether the cumulative value of the basic grid in the time window exceeds the threshold;

[0049] Step 6: Merge hotspot areas;

[0050] Step 7: Hotspot section analysis, calculate the sections in the urban road network data that intersect with the hotspot area of ​​​​the muck truck;

[0051] Step 8: Analyze the operation points of the muck trucks to obtain the target hotspot construction site set, hotspot disposal site set and illegal source point set.

[0052] Specifically, the muck truck trajectory processing process is designed based on a distributed architecture, and tasks are distributed through the Kafka message queue, which completes the decoupling and load balancing between different trajectory processing tasks and meets the requirements of muck truck trajectory processing for low latency and real-time performance.

[0053] Taking the basic grid divided by Geohash coding as the unit, the hotspot grids within the time window are identified through distributed streaming processing of the trajectory data of the muck truck, and merged into hotspot areas. Through the spatial analysis of the hotspot area and the existing road network data, the real-time hotspot sections of muck trucks are obtained. Through the spatial analysis of the hotspot area and the existing data such as the existing construction sites, disposal sites, parking lots, gas stations, etc., the real-time hotspot construction sites and disposal sites are obtained. By performing density-based aggregation on the trajectory data in the hotspot area after removing the known source area, illegal sources such as illegal construction sites or illegal disposal sites are obtained.

[0054] Furthermore, the above-mentioned method for processing the trajectory of a muck truck also includes:

[0055] In an embodiment of the present application, the specific method of receiving the trajectory data in step 1 is:

[0056] Build a trajectory information data service to receive real-time track positioning data of the muck truck sent back by the muck truck’s onboard positioning terminal.

[0057] Specifically, the data returned by a single trajectory mainly includes longitude, latitude, positioning time, speed, direction, and relevant information of the vehicle itself such as license plate number.

[0058] In one embodiment of the present application, the specific method of distributing the trajectory data processing tasks in step 2 is:

[0059] Build a Kafka message queue cluster and create a topic ic for trajectory data processing. After receiving the trajectory positioning data, the service sends the trajectory data to the topic ic for consumers to consume. At the same time, set the key of the topic ic to the license plate number to ensure that the trajectory data of the same vehicle can be distributed to the same partition and processed by consumers in sequence.

[0060] Build a distributed cluster of trajectory data processing services on multiple servers. Subscribe to the trajectory data processing topic ic as a Kafka consumer to pull and process the trajectory data. Kafka completes the balanced distribution of messages among the nodes in the cluster.

[0061] In one embodiment of the present application, the specific method of storing the trajectory data in step 3 is:

[0062] Establish a trajectory data storage consumer group and execute the data storage task. After the task obtains the trajectory data, it calculates the 10-bit Geohash code of the point according to the longitude and latitude, and stores it in the distributed column database HBase together with the other received fields.

[0063] Specifically, HBase establishes a trajectory data table, using the license plate plus time text as the row key, which can not only meet the needs of fast query of time periods, but also avoid the problem of data duplication caused by multiple transmissions or consumption of the same data; at the same time, an index is established for Geohash to match the Geohash code for query.

[0064] In one embodiment of the present application, the specific method of classifying the trajectory data in step 4 is:

[0065] After the task obtains the trajectory data, trajectory data classification is carried out. According to the longitude and latitude of the trajectory point, combined with the urban road network data, and using the spatial database, it is determined whether the trajectory point is located on the road, and the trajectory point is divided into two categories: one is the driving data point of the muck truck on the road, and the other is the earthwork data point of the muck truck not on the road.

[0066] Specifically, the steps are as follows:

[0067] 1. Preprocess urban road network data, build a buffer for each road, and create a spatial index; 2.

[0068] 2. Use spatial database functions to determine the inclusion relationship between trajectory points and road buffers;

[0069] 3. If the trajectory point is included in the buffer zone of a single road and the driving speed is greater than the preset speed threshold, it is classified as a driving data point of a muck truck;

[0070] 4. If the trajectory point is included in the buffer zone of two or more roads, it is judged to be at an intersection and classified as a muck truck driving data point;

[0071] 5. If the trajectory point is not included in any road buffer and the driving speed is less than the preset speed threshold, it is classified as a muck truck earthwork operation data point.

[0072] In one embodiment of the present application, the specific method of hotspot grid determination in step 5 is:

[0073] The Geohash coding grid of a given coding length is used as the basic division grid, and the time window and step length are set. The time window length is an integer multiple of the step length.

[0074] After the task obtains the trajectory data, it calculates the Geohash code of the point based on the longitude and latitude and accumulates it in the Redis cache database;

[0075] Set the hotspot grid threshold, and after each step, determine whether the cumulative value of the basic grid in the time window exceeds the threshold.

[0076] Specifically, Red is uses Geohash code as the key to accumulate the number of trajectory data of each basic grid in the time window and the latest step. The time window cumulative value is stored in a list structure, with each step cumulative value as an element. After each step, the latest step cumulative value is inserted into the head of the list, and the last step cumulative value is removed. At the same time, the category to which the trajectory point belongs is used to identify the category to which the basic grid belongs.

[0077] The steps for merging hotspot areas in a single processing partition are as follows:

[0078] 1. Obtain the hotspot grid set G on the boundary of the processing partition in the entire processing space;

[0079] 2. Obtain all hotspot grid data in the processing partition and traverse all hotspot grids in the area;

[0080] 3. Get the hotspot area R to which the hotspot grid belongs. If it does not belong to any hotspot area, create a new hotspot area R and add the hotspot grid to the hotspot area;

[0081] 4. Traverse the neighborhood grids of the hotspot grid with 8 neighborhoods to check whether there are hotspot grids of the same type. If so, add the hotspot area R;

[0082] 4. If the hotspot grid belongs to the set G, check whether there is a neighborhood grid of the same type as the hotspot grid in other processing partitions in the set G. If so, establish the relationship r(Rg) between the hotspot area R and the neighborhood grid;

[0083] 5. After the traversal is completed, the output stream of the hotspot sub-area set of the processing partition is obtained.

[0084] In one embodiment of the present application, the specific method of merging the hotspot areas in step 6 is as follows:

[0085] The entire processing space is divided into multiple processing partitions. Each processing partition contains the hotspot grids falling therein, which are evenly distributed to each node in the cluster for the initial hotspot area merging, and the obtained hotspot sub-areas are merged for the second time.

[0086] Specifically, the obtained hotspot sub-regions are merged twice, and the steps are as follows:

[0087] 1. Traverse all hotspot sub-areas;

[0088] 2. For the hotspot sub-region R1, check whether there is a relationship r(R1-g). If so, obtain the hotspot sub-region R2 to which the hotspot grid g belongs and establish the relationship r(R1-R2). If not, output it as the final hotspot region;

[0089] 3. Traverse the relationships between all sub-regions, merge the sub-regions with relationships, and form the final hot spot area output;

[0090] 4. After all traversals are completed, the output streams of the hotspot area set of muck truck driving and the hotspot area set of muck truck earthwork operations are obtained.

[0091] In one embodiment of the present application, the specific method of hotspot section analysis in step 7 is as follows:

[0092] The set of hot spots for muck truck driving is obtained to form spatial polygonal surface data (multipolygon). Through superposition analysis of the spatial database and known urban road network data, the road sections in the urban road network data that intersect with the hot spots for muck truck driving are calculated, and finally the set of target hot spots is obtained.

[0093] In one embodiment of the present application, the specific method of analyzing the operation point of the muck truck in step 8 is as follows:

[0094] Obtain a set of hotspot areas for muck truck earthwork operations and divide them evenly into various processing nodes for analysis.

[0095] Specifically, the analysis steps for a single hotspot area are as follows: obtain the hotspot grid set G contained in the hotspot area; convert the hotspot area into spatial data to form spatial polygonal surface data (polygon); perform superposition analysis with known construction site data to obtain the construction site in the area, output it as the hotspot operation construction site data, and remove the grid where the construction site is located in the set G; perform superposition analysis with known disposal field data to obtain the disposal field in the area, output it as the hotspot operation disposal field data, and remove the grid where the disposal field is located in the set G; perform superposition analysis with known parking lots, gas stations and other data to obtain parking lots and gas stations in the area, and remove their grids in the set G; according to the Geohash code and time window of the remaining grids in the set G, obtain the longitude and latitude data set in the time window from HBase; use the density DBSCAN algorithm to cluster the longitude and latitude data set according to the preset parameters radius eps and the minimum number MinPts. If there is a result cluster in the clustering, the core point of the cluster is output as the illegal source point. After the processing is completed, the target hotspot construction site set, hotspot disposal field set and illegal source point set are finally obtained.

[0096] In summary, a method for processing the trajectory of a muck truck in an embodiment of the present application adopts a distributed processing concept, deploys a server cluster, and processes trajectory data in parallel on multiple nodes, thereby improving the system data throughput, improving the real-time nature of the analysis results, and having good scalability; using grid real-time clustering, the overall area is divided into multiple hot spots, and a method of secondary clustering of each hot spot area in parallel reduces the impact of a large number of discrete points on trajectory clustering, greatly reduces the trajectory clustering analysis delay, and realizes real-time clustering of muck truck trajectories; through data processing and analysis, real-time hot spots, hot construction sites, hot disposal sites, illegal sources and other results are obtained, which reduces the workload of on-site inspections by law enforcement personnel and provides sufficient decision-making assistance for dust control and management.

[0097] In the description of this specification, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0098] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0099] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and deform the above embodiments within the scope of the present application.

Claims

1. A method for processing tracks of a muck truck, characterized in that: The following steps are involved: Step 1: Receive trajectory data and build trajectory information data service; Step 2: Distribute trajectory data processing tasks, build a Kafka message queue cluster, and build a trajectory data processing service distributed cluster on multiple servers; Step 3: Trajectory data storage; Step 4: Trajectory data classification to determine whether the trajectory point is on the road; Step 5: hotspot grid judgment, judging whether the cumulative value of the basic grid in the time window exceeds the threshold; Step 6: Merge hotspot areas; Step 7: Hotspot section analysis, calculate the sections in the urban road network data that intersect with the hotspot area of ​​​​the muck truck; Step 8: Analyze the operation points of the muck trucks to obtain the target hotspot construction site set, hotspot disposal site set and illegal source point set.

2. A method for processing tracks of a muck truck according to claim 1, characterized in that: The specific method of receiving the trajectory data in step 1 is as follows: The track information data service is constructed to receive in real time the track positioning data of the muck truck transmitted back by the muck truck onboard positioning terminal.

3. A method for processing tracks of a muck truck according to claim 1, characterized in that: The specific method of distributing the trajectory data processing tasks in step 2 is as follows: The Kafka message queue cluster is built to create a track data processing topic. After the service receives the track positioning data, it sends the track data to the topic for consumption by consumers. At the same time, the key of the topic is set to the license plate number to ensure that the track data of the same vehicle can be distributed to the same partition and processed by consumers in sequence. The track data processing service distributed cluster is built on multiple servers, and the track data processing topic is subscribed as a Kafka consumer to pull and process the track data, and Kafka completes the balanced distribution of message distribution among the nodes in the cluster.

4. A method for processing tracks of a muck truck according to claim 1, characterized in that: The specific method of storing the trajectory data in step 3 is as follows: Establish a trajectory data storage consumer group and execute the data storage task. After the task obtains the trajectory data, it calculates the 10-bit Geohash code of the point according to the longitude and latitude, and stores it in the distributed column database HBase together with the other received fields.

5. A method for processing tracks of a muck truck according to claim 1, characterized in that: The specific method of trajectory data classification in step 4 is: The trajectory data classification is to determine whether the trajectory point is located on the road based on the latitude and longitude of the trajectory point, combined with the urban road network data, and using the spatial database after the task obtains the trajectory data, and divide the trajectory points into two categories: one is the driving data points of the muck truck located on the road, and the other is the earthwork data points of the muck truck not on the road.

6. A method for processing tracks of a muck truck according to claim 1, characterized in that: The specific method of hotspot grid determination in step 5 is as follows: The Geohash coding grid of a given coding length is used as the basic division grid, and the time window and step length are set. The time window length is an integer multiple of the step length. After the task obtains the trajectory data, it calculates the Geohash code of the point based on the longitude and latitude and accumulates it in the Redis cache database; Set the hotspot grid threshold, and after each step, determine whether the cumulative value of the basic grid in the time window exceeds the threshold.

7. A method for processing tracks of a muck truck according to claim 1, characterized in that: The specific method of merging the hotspot areas in step 6 is as follows: The entire processing space is divided into multiple processing partitions. Each processing partition contains the hotspot grids falling therein, which are evenly distributed to each node in the cluster for the initial hotspot area merging, and the obtained hotspot sub-areas are merged for the second time.

8. A method for processing tracks of a muck truck according to claim 1, characterized in that: The specific method of hotspot section analysis in step 7 is as follows: The set of hot spots for muck truck driving is obtained to form spatial multipolygon data. The spatial database is overlaid with the known urban road network data to calculate the road sections in the urban road network data that intersect with the hot spots for muck truck driving, and finally the set of target hot spots is obtained.

9. A method for processing tracks of a muck truck according to claim 1, characterized in that: The specific method of analyzing the operation point of the muck truck in step 8 is as follows: Obtain a set of hotspot areas for muck truck earthwork operations and divide them evenly into various processing nodes for analysis.

Images