Predictive map layer construction system based on reinforcement learning
Through the predictive map layer construction system based on reinforcement learning, the problem of low efficiency and accuracy of intelligent connected data processing is solved, efficient and accurate self-produced map construction and data matching is achieved, data analysis closed loop is supported, and data accuracy is improved and matching efficiency.
Patent Information
- Application Number
- CN202310632889.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-31
- Publication Date
- 2024-12-03
AI Technical Summary
The existing technology is difficult to efficiently and accurately process massive intelligent connected data, resulting in the inability to form an effective closed loop of vehicle operation data, lacking its own multi-attribute map, affecting the accuracy and efficiency of data analysis.
Using a predictive map layer construction system based on reinforcement learning, including data acquisition devices, data collection platforms, vehicle-road matching modules and efficient massive computing modules. Through distributed computing and reinforcement learning feature smoothing modules, the accuracy is higher than that of the original data. It has its own layer library to achieve efficient data matching and query.
It realizes efficient and accurate construction of self-map layers, changes the matching problem of the original single topology structure, efficiently and accurately matches TB-level data, and improves data accuracy through advanced filtering methods, supports configuration optimization and market detail point.
Smart Images

Figure CN119063711A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent networking, and specifically to a predictive map layer construction system based on reinforcement learning. Background Art
[0002] With the development of intelligent networking and the accumulation of big data on vehicle operation, the application of automotive big data has covered the entire automotive industry chain, providing multi-dimensional data decision-making basis for research and development, production, sales, etc. Due to the lack of map-related data, data analysis work cannot form an effective closed loop. Therefore, it is necessary to establish a multi-attribute map to complete the analysis closed loop of the three modules of "people-vehicle-road" in vehicle operation data. At present, there is no efficient and accurate method to process massive data using intelligent network data to build its own map layer. This patent proposes a method for constructing map layers using reinforcement learning, which can achieve efficient and accurate establishment of its own layers. Summary of the invention
[0003] In order to solve the problems existing in the background technology, the present invention aims to provide a predictive map layer construction system based on reinforcement learning, which can quickly establish its own map layer. After the layer is established, the map layer query can be efficiently realized, providing necessary support for configuration optimization and market segmentation definition.
[0004] Solutions for solving problems
[0005] A predictive map layer construction system based on reinforcement learning, the system comprising: a data acquisition device, a data collection platform, a vehicle-road matching module, a reinforcement learning feature smoothing module and an efficient mass computing module:
[0006] The data collection device is used to collect vehicle-road operation data and send the vehicle-road operation data to the data collection platform;
[0007] The data collection platform is used to collect and store the original vehicle-road operation data sent by the data collection device;
[0008] The vehicle-road matching module is used to locate and match the vehicle operation path using the vehicle-road operation data collected by the data collection platform, extract parameters from the matching results and the original vehicle-road operation data, and input them into the efficient mass computing module;
[0009] The efficient massive computing module utilizes a distributed computing system to perform efficient computing, establishes its own layer library, and calibrates the computing results through the reinforcement learning feature smoothing module. Through continuous and autonomous updating, it constructs feature data with significantly higher accuracy than the original data.
[0010] Furthermore, the data acquisition device transmits the collected data to the data collection platform via a wireless communication network.
[0011] Furthermore, the vehicle-road operation data includes: GPS, vehicle speed and fuel consumption data.
[0012] Furthermore, the vehicle-road matching module is used to construct the national road topology and road numbering, and can provide matching functions at the same time. For example, when the GPS data of a vehicle at a certain moment is input into the module, the module outputs the road number, road conditions and surrounding information of the positioning point.
[0013] Furthermore, the efficient massive computing module: utilizes distributed computing to efficiently calculate the stock data, imports data through the JDBC data access module on the module, and creates a new sandbox in the data warehouse for data storage. The administrator account manages the sandbox, ordinary users can apply for the use of the sandbox data in the data warehouse, and users can upload their own data for use.
[0014] Furthermore, the establishment of the own layer database specifically includes: using GPS data to construct a basic database based on vehicle configuration and market segments, such as the entry and exit time points of vehicle A on road numbered g65, the market segment to which vehicle A belongs, and the vehicle operating speed and fuel consumption of vehicle A corresponding to the refined road positioning point.
[0015] Furthermore, by interactively learning with the historical accumulated data through the reinforcement learning feature smoothing module, the historical data of the positioning point and the positioning points around the positioning point are used for collaborative learning according to different weights based on different distances, and continuously and autonomously updated to construct feature data with significantly higher accuracy than the original data.
[0016] Beneficial effects:
[0017] The beneficial effects of the above technical solution are:
[0018] 1. Construct an attribute topology structure based on road network numbering to change the problem that the original single topology structure cannot match massive amounts of data and perform fast retrieval.
[0019] 2. For TB-level data, distributed parallel computing is used to achieve efficient and accurate matching, and advanced filtering and denoising methods are used to improve data accuracy for the same road. This solution has been implemented and used in the existing Shaanxi Automobile's own layer construction. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 It is a process and system for building predictive map layers based on reinforcement learning. DETAILED DESCRIPTION
[0021] The specific implementation mode of the present invention is described below in conjunction with embodiments:
[0022] It should be noted that the structures, proportions, sizes, etc. shown in this specification are only used to match the contents disclosed in the specification so that people familiar with this technology can understand and read them, and are not used to limit the conditions under which the present invention can be implemented. Any structural modification, change in proportional relationship or adjustment of size should still fall within the scope of the technical content disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.
[0023] At the same time, the terms such as "upper", "lower", "left", "right", "middle" and "one" cited in this specification are only for the convenience of description and are not used to limit the scope of implementation of the present invention. Changes or adjustments to their relative relationships should be regarded as the scope of implementation of the present invention without substantially changing the technical content.
[0024] Embodiment 1:
[0025] Data acquisition device: Use data acquisition equipment to obtain vehicle operation data and transmit the data back through the network.
[0026] Data collection platform: The data collection platform is responsible for collecting and storing the returned data.
[0027] Vehicle-road matching module: The vehicle-road matching module is used to construct the national road structure and road numbering, and to obtain information such as road numbers and road conditions based on the latitude and longitude matching function.
[0028] Efficient mass computing module:
[0029] Data management platform: The Cloudera version of Hadoop, which is commonly used in the big data industry, is used as the basic storage framework. Standard Kafka, FTP, and Http are used as access methods. The fully open source Postgres database is used as the transaction database.
[0030] Computing and analysis platform: Hadoop Yarn is used as the underlying resource management and scheduling platform, on which Spark, Storm, SkitLearn, TensorFlow and other mainstream data computing and analysis environments in the industry are established and provided.
[0031] Service governance platform: uses the microservice automated governance environment based on SpringCloud and Docker as the core for business management and customization.
[0032] Analysis and development platform: Use the Azkaban-based task scheduling system for secondary open task orchestration Operation and supervision platform: Perform log collection and statistical analysis based on the log collection system, and provide environment and task monitoring.
[0033] Proprietary layer database: Utilize GPS data to build a basic database based on vehicle configuration and market segments, combining vehicle operation trajectory with basic vehicle information parameters.
[0034] Reinforcement Learning Feature Smoothing Module:
[0035] Basic data processing: read GPS data, sort the longitude and latitude in chronological order from small to large, complete the coordinate conversion of longitude and latitude, and retain 6 decimal places to obtain the processed basic data set;
[0036] Cut off: Cut off the set according to the average moving speed column. Cut off rule: when the average moving speed column has a value outside the threshold (threshold range: 0≤X≤144km / h), delete the corresponding row and cut off the F set; obtain several F subsets (F1, F2, F3...);
[0037] Matching road and link (road number) information: Taking subset F1 as an example, when the number of rows in the subset is 1<X≤5000 (due to the performance of the map location service, if the number of rows is greater than 5000, it is divided into 5000 rows and cut again), the basic map platform vehicle-road matching service is called to match the road name, road grade, link number, link length, and linkGPS sequence for each row of data (for the specific matching method, see the basic map platform interface call example). For rows that are not successfully matched, the road name, road grade, link number, link length, and linkGPS sequence are assigned empty values;
[0038] Link sequence acquisition: After matching the vehicle and the road, the F1 set is deduplicated according to the link sequence. The deduplication rule is: if the link sequence is "1,1,1,2,,2,3,,4,1,,1,1,2,6,7", then the link sequence after deduplication is: "1,2,3,4,1,2,6,7"; after deduplication, the data in the F1 set is extracted according to the link to generate the link sequence metadata set F11, that is, the set F11 contains 8 link sub-units "1,2,3,4,1,2,6,7". Taking the first "1" in the link sequence as an example, it contains the road name, road grade, link number, link length, linkGPS sequence, and the data of the first pass through the link sequence "1" in the F1 set;
[0039] Link longitude and latitude densification: According to the link order in the F11 set, extract the inherent longitude and latitude sequence of each link. Take link "1" in the F11 set "1,2,3,4,1,2,6,7" as an example. When the number of longitude and latitude sequence points is within the threshold range (X>1), estimate the geographical distance between every two adjacent points of the inherent longitude and latitude sequence points of link "1" to obtain the distance sequence; (When the number of longitude and latitude sequence points X≤1, adopt the method of taking points from adjacent links. Based on the F11 set, extract the last longitude and latitude point coordinates on the adjacent links of the link upwards, and extract the first longitude and latitude point coordinates on the adjacent links of the link downwards as data supplement, and then estimate the geographical distance between adjacent longitude and latitude sequence points). Calculate the number of interpolations required based on the mileage of each line of longitude and latitude; When the value of num>0, use one-dimensional linear interpolation to perform equal-distance densification of the longitude and latitude sequence, merge the densified data, and obtain the final trajectory features;
[0040] Associate features: Associate calculated features with road information.
[0041] Reinforcement learning features: Through interactive learning with calculated features and historical accumulated data, the historical data of the positioning point is used for collaborative learning with the positioning points around the positioning point (with different weights according to different distances), and it is continuously updated autonomously to construct feature data with significantly higher accuracy than the original data.
[0042] Embodiment 2:
[0043] Figure 1 A process and method for constructing predictive map layers based on reinforcement learning.
[0044] See also Figure 1 , the predictive map layer construction process provided by this embodiment is described, including: running a vehicle 1-1, a data collection device collecting data 1-2, and transmitting it to a data collection platform 1-3 through a network, using a vehicle-road matching module 1-4 to construct a national road topology structure and road numbering, the model also provides a matching function, combined with 1-3 input to a parameter extraction module 1-5, using an efficient massive computing module 1-6 for distributed computing, storing the calculation results in a proprietary layer database 1-7, using a reinforcement learning feature smoothing module to continuously update and optimize the results, and autonomously updating in 1-7 and 1-8 to construct feature data with significantly higher accuracy than the original data.
[0045] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0046] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0047] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
Claims
1. A predictive map layer construction system based on reinforcement learning, characterized in that: The system includes: a data acquisition device, a data collection platform, a vehicle-road matching module, a reinforcement learning feature smoothing module and an efficient mass computing module: The data collection device is used to collect vehicle-road operation data and send the vehicle-road operation data to the data collection platform; The data collection platform is used to collect and store the original vehicle-road operation data sent by the data collection device; The vehicle-road matching module is used to locate and match the vehicle operation path using the vehicle-road operation data collected by the data collection platform, extract parameters from the matching results and the original vehicle-road operation data, and input them into the efficient mass computing module; The efficient massive computing module utilizes a distributed computing system to perform efficient computing, establishes its own layer library, and calibrates the computing results through the reinforcement learning feature smoothing module. Through continuous and autonomous updating, it constructs feature data with significantly higher accuracy than the original data.
2. According to the reinforcement learning-based predictive map layer construction system of claim 1, it is characterized in that: The data acquisition device transmits the collected data to the data collection platform via a wireless communication network.
3. The system for constructing a predictive map layer based on reinforcement learning according to claim 1, characterized in that: The vehicle-road operation data includes: GPS, vehicle speed and fuel consumption data.
4. According to claim 1, a predictive map layer construction system based on reinforcement learning is characterized in that: The vehicle-road matching module is used to construct the national road topology and road numbering, and can provide matching functions at the same time. For example, the module inputs the GPS data of a vehicle at a certain moment, and the module outputs the road number, road conditions and surrounding information of the positioning point.
5. The system for constructing a predictive map layer based on reinforcement learning according to claim 1, characterized in that: The efficient massive computing module uses distributed computing to efficiently compute stock data, imports data through the JDBC data access module on the module, and creates a new sandbox in the data warehouse for data storage. The administrator account manages the sandbox, and ordinary users can apply for the use of the sandbox data in the data warehouse, and users can upload their own data for use.
6. The system for constructing a predictive map layer based on reinforcement learning according to claim 1, characterized in that: The establishment of the own layer database specifically includes: using GPS data to construct a basic database based on vehicle configuration and market segments, such as the entry and exit time points of vehicle A on road numbered g65, the market segment to which vehicle A belongs, and the vehicle operating speed and fuel consumption of vehicle A corresponding to the refined road positioning point.
7. The system for constructing a predictive map layer based on reinforcement learning according to claim 1, characterized in that: The reinforcement learning feature smoothing module interacts with the historical accumulated data to learn collaboratively with the historical data of the positioning point and the positioning points around the positioning point according to different weights based on different distances, and continuously updates autonomously to construct feature data with significantly higher accuracy than the original data.
