Method and system for generating road data based on AI multi-source data fusion

Abstract

The invention discloses a method and a system for generating road data based on AI multi-source data fusion. An AI model is generated based on road data, road information is extracted from multi-source data, the road data is generated through fusion calculation, and the data is continuously updated and the model is continuously optimized. Compared with the prior art, the system error can be effectively reduced by relying on the multi-source data, and the accuracy and reliability of the data are improved. Moreover, the road data system is not interfered by other people except for manual operation in the data acquisition and selection input stages, so that the labor cost is effectively reduced, and the influence of human factors on the data quality is reduced. In the aspect of data updating and maintenance, a crowdsourcing data source is introduced, and the effectiveness of road data is improved through extraction and fusion of crowdsourcing data information by a system.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent transportation, and in particular to a method and system for generating road data based on AI multi-source data fusion. Background Art

[0002] Accurate road data is the foundation of intelligent transportation systems. Systems that have the ability to generate accurate road data by fusing multi-source data can effectively reduce labor costs, improve work efficiency, and enhance the accuracy and reliability of road data, which will help promote transportation to be safer, more efficient, and smarter. At the same time, it can also drive technological progress and application development in other related fields, such as autonomous driving technology and urban planning, management, and construction.

[0003] The existing methods for producing road data mainly include collecting road centerline data through field surveying and then digitizing the surveying data through professional drawing software such as ArcMap to produce road data, and drawing road data through drawing software based on high-precision maps.

[0004] In the above method, the process of generating road data requires a long period of time and a lot of manpower costs. The update and maintenance of data requires a lot of manpower and repeated operations, and the data iteration cycle is relatively long and the timeliness is not high enough. Due to the single source of data, road data is prone to systematic errors, and the data attribute information is not rich and diverse enough. At the same time, manual operations in various links cannot avoid human factors affecting data quality. Summary of the invention

[0005] Therefore, in response to the above problems, the present invention discloses a method and system for generating road data based on AI multi-source data fusion, which relies on the road data generation AI model to extract road information from multi-source data, and generates road data through fusion calculation. Relying on multi-source data can effectively reduce system errors and improve the accuracy and reliability of data. In addition to manual operations in the data collection and selection entry stages, the system has no other human intervention, which effectively reduces labor costs and reduces the impact of human factors on data quality. In terms of data updating and maintenance, crowdsourcing data sources are introduced, and the effectiveness of road data is improved by extracting and integrating crowdsourcing data information through the system.

[0006] The technical solution to achieve the purpose of the present invention is: a method for generating road data based on AI multi-source data fusion,

[0007] Step (1) selecting or setting a target area for generating road data;

[0008] Step (2) multi-source data selection input: according to the selected target area range, select the original data that meets the requirements of the range from the collected multi-source data;

[0009] Step (3) pre-processing, cleaning, denoising and standardization of multi-source data: cleaning the selected multi-source data and removing data without basic information;

[0010] Step (4) extracting road information features from multi-source data; after the cleansed multi-source data is extracted, different information features are extracted in the road information extraction module according to different types;

[0011] Step (5) AI multi-source data fusion processing, road data generation; after the road information feature extraction and processing of the multi-source data, the road feature data extracted and analyzed from the satellite remote sensing data, vehicle sensor data and crowdsourcing data are integrated by relying on the AI ​​multi-source data fusion module;

[0012] Step (6) road data quality feedback, multi-source data update, and iterative upgrade of AI multi-source data fusion model training.

[0013] Furthermore, in step (2), the multi-source data includes satellite remote sensing image data, data collection vehicle onboard sensor data, and crowd-sourced image text information data.

[0014] 3. The method for generating road data based on AI multi-source data fusion according to claim 1 is characterized in that, in step (3), the multi-source data selected in step (2) is cleaned to remove data without basic information; remove blank or blurred images and picture data, remove abnormal or obviously wrong location information data and abnormal noise data in laser radar scanning, and remove empty data and invalid data in crowdsourced data;

[0015] The cleaned multi-source data mainly includes three standard types: remote sensing images, lidar measurement data and text, which are used for feature information extraction in the subsequent step (4).

[0016] Further, in step (4),

[0017] Information extraction methods include image recognition technology, LiDAR point cloud data processing technology, and natural language processing technology;

[0018] 1) Image recognition technology;

[0019] Convolutional neural networks are trained through image recognition to extract and analyze specific feature data in images;

[0020] Extract the basic road network location information and road connectivity of the target area from satellite remote sensing images to form basic road network data;

[0021] Recognize the images taken by the vehicle camera to obtain road names, road signs, and road type information;

[0022] Identify and extract attribute information from images reported by crowdsourcing data;

[0023] 2) Sensor measurement data;

[0024] The data collection vehicle is equipped with a laser radar to accurately measure the road data in the target area and extract detailed data information of the road from the point cloud data scanned by the laser radar;

[0025] 3) Natural language processing;

[0026] Process and analyze text description information about roads in crowdsourced data to extract timely attribute data of roads.

[0027] Furthermore, the AI ​​multi-source data fusion model construction method in step (5) is:

[0028] 1) In the model construction phase, a road data generation convolutional neural network model is defined using the torch.nn module. According to the needs of data fusion, the three types of data are connected in a certain layer of the model;

[0029] 2) In the model training phase, the three types of data sets extracted in step (4) are input into the model in batches using the DataLoader data loader for model training. The model parameters are updated by defining the loss function and optimizer for forward and backward propagation.

[0030] 3) In the model evaluation phase, the model performance is evaluated based on the test set or test set, and the model structure is adjusted and parameters are optimized.

[0031] Furthermore, in step (5), the data fusion method of the AI ​​multi-source data fusion model is:

[0032] Based on the basic road information data obtained by satellite remote sensing image recognition, the basic location, geometric features and connectivity of road lines are identified;

[0033] Improve the accuracy of road location information, road width, road edge, road sign feature information and clarify the connectivity relationship between roads through Beidou location data and LiDAR measurement data;

[0034] Enrich road attribute information through road feature data extracted through multi-source image recognition;

[0035] Through crowdsourcing information data, road attribute information can be corrected in a timely manner to improve the timeliness of road data.

[0036] Furthermore, step (6) specifically includes:

[0037] Fusion model optimization: Through comprehensive comparison of the generated road data with the actual information, problems in the model are discovered and model parameters are adjusted;

[0038] Fusion model upgrade: By inputting different types of raw data and increasing the types of model training samples, the training of the AI ​​fusion algorithm can be enhanced, thereby outputting more comprehensive and rich road data;

[0039] A system for generating road data based on AI multi-source data fusion, the system mainly includes:

[0040] Multi-source data acquisition module, used for multi-source data acquisition;

[0041] Multi-source data preprocessing module, used to clean and denoise the raw data of various types of data and perform standardization operations on each type of data;

[0042] Road feature extraction module, including image recognition, road information extraction, sensor data extraction, and crowdsourced information road data extraction;

[0043] The AI ​​multi-source data fusion module is used to realize the fusion processing of multi-source information through the AI ​​multi-source data fusion model, and output road data with multi-source information characteristics.

[0044] Compared with the prior art, the present invention has the following significant advantages:

[0045] (1) AI multi-source data fusion is used to integrate road information from multiple types of data, including basic road location information, lane-level data information, road type and other basic attribute information, and crowdsourced road real-time data information. The information is more comprehensive and rich, with spatial location as the combination point of different source information and cross-validation of multi-source data to reduce errors and improve data accuracy and credibility.

[0046] (2) Data generation efficiency is improved. After the user selects the area and inputs multi-source data, the data analysis and processing can be automatically completed, and road data can be generated based on model training calculations. This improves the efficiency of road data production, simplifies the production process, and reduces labor costs and the possibility of human operational errors.

[0047] (3) Improve the real-time characteristics of road data. Relying on the continuously improved AI model, the efficiency and quality of generating road data will gradually improve. With the data support of real-time crowdsourcing information, the model comprehensively considers the priority of crowdsourcing information in the time dimension. Based on the effective content about real-time changes in roads in crowdsourcing information, road data is updated in a timely manner to effectively improve the real-time nature of the data. It can provide more accurate comprehensive considerations for functions that have high requirements for road real-time, such as path planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 Multi-source data fusion generates road data system composition diagram;

[0049] Figure 2 Flowchart of the road data system generated by fusion of multi-source data. DETAILED DESCRIPTION

[0050] The terms used in the present invention are only for the purpose of illustrating the embodiments of the present invention and are not intended to limit the present invention. Figure 1-2 , some embodiments of the present invention are described in detail.

[0051] The present invention discloses a method for generating road data based on AI multi-source data fusion, which can be divided into six main stages from source data selection and classification processing to AI fusion model training to road data generation and road data update:

[0052] like Figure 1 As shown. The system user first selects the target area for road generation and specifies the data generation range; then selects the multi-source data containing the data of the area from the multi-source data for input, and initially obtains the multi-source original data; then, by cleaning and denoising various types of original data, the cleaned multi-source data that can be used to extract road attributes is obtained; then, the road feature data is extracted through comprehensive modules including image recognition, lidar point cloud data processing, and natural language processing; then, the spatial position matching and calibration of road data are realized through the AI ​​multi-source data fusion processing module, and the conflicts on the same road data are resolved through the priority relationship. From the perspective of space, the multi-source data is fused to enhance the position accuracy of the road data, and from the perspective of time, the multi-source data is fused to enhance the timeliness of the road data, thereby generating road data containing multi-source fusion information; finally, the training and upgrading of the fusion model can be completed through the evaluation feedback of the road data and the newly added or revised multi-source data, and new road data can be constructed based on the upgraded AI fusion model.

[0053] A method for generating road data based on AI multi-source data fusion: The specific steps are as follows:

[0054] Step (1) selecting or setting a target area for generating road data;

[0055] System users select the target area range according to road data requirements, draw or select a preset target range on the map operation interface through human-computer interaction, and determine the boundary data required for road data construction.

[0056] Step (2) multi-source data selection and input;

[0057] According to the target area selected in step (1), select the original data that meets the requirements of the range from the collected multi-source data. The multi-source data includes satellite remote sensing image data, data collection vehicle onboard sensor data (Beidou positioning equipment, camera, lidar) and crowd-sourced image text information data.

[0058] 1) Satellite remote sensing image data: Utilize high-resolution satellite images and aerial photography to quickly obtain geographic information of large areas and extract basic location information, direction information, and connectivity node information of roads within a large area.

[0059] 2) Vehicle sensor data:

[0060] Vehicle-mounted Beidou positioning data: can be used to accurately record the location information of each marking point on the route, generate precise location data of the road line, and assist in calibrating the road location data extracted from the image data.

[0061] LiDAR detection data: accurately captures the three-dimensional contours of the road, measures the road width, geometry, road boundaries, road markings, etc., and constructs lane-level road data.

[0062] Camera: Identify traffic signs and road environment information, and enrich road attribute information including road name, road type, and the connectivity between roads.

[0063] 3) Crowdsourced data: road change information reported by non-professional road data collectors, such as route changes, road congestion, road damage, temporary traffic control, etc. Crowdsourced data can greatly enrich the timeliness and accuracy of road data and make up for the problem of untimely information collection.

[0064] Step (3) Multi-source data preprocessing, cleaning, denoising, and standardization. From this stage onwards, the system automatically processes data without manual intervention;

[0065] The multi-source data selected in step (2) is cleaned to remove data without basic information, such as removing blank or blurred images and picture data, removing abnormal or obviously wrong location information data, removing abnormal noise data in LiDAR scanning, and removing invalid data such as empty data in crowdsourced data.

[0066] The cleaned multi-source data mainly includes three standard types: remote sensing images (including pictures), lidar measurement data and text, which are used for feature information extraction in the subsequent step (4).

[0067] Step (4) road information feature extraction from multi-source data;

[0068] For cleaned multi-source data, different information extraction methods can be used in the road information extraction module according to different types, including image recognition technology, lidar point cloud data processing technology, and natural language processing technology.

[0069] 1) Image recognition technology

[0070] Convolutional Neural Network (CNN) is trained through image recognition to extract and analyze specific feature data in images.

[0071] The basic road network location information and road connectivity of the target area can be extracted from satellite remote sensing images to form basic road network data;

[0072] It can identify the road name, road sign and road type information from the pictures taken by the vehicle camera.

[0073] It can identify and extract attribute information such as road names, road changes, road damage and temporary road controls from images reported by crowdsourcing data.

[0074] 2) Sensor measurement data

[0075] The data collection vehicle is equipped with a laser radar to accurately measure road data in the target area. Detailed data information such as the road's geometry, road conditions, traffic signs and markings, and road sidelines can be extracted from the point cloud data scanned by the laser radar.

[0076] 3) Natural Language Processing

[0077] Process and analyze text description information about roads in crowdsourced data to extract time-sensitive attribute data of roads, such as road congestion, road damage, temporary road control, etc.

[0078] Step (5) constructing an AI multi-source data fusion model to process road data;

[0079] After step (4) of extracting and processing the road information features of multi-source data, the road feature data extracted and analyzed from satellite remote sensing data, vehicle-mounted sensor data and crowdsourcing data are integrated based on the AI ​​multi-source data fusion model to generate road data.

[0080] The AI ​​multi-source data fusion model uses the open source Pytorch framework and can be built based on python3 and the corresponding configuration package.

[0081] 1) In the model building phase, a road data generation convolutional neural network model is defined using the torch.nn module. According to the needs of data fusion, three types of data (including pictures, texts, and sensor measurement data) can be connected in a certain layer of the model.

[0082] 2) In the model training phase, the three types of data sets extracted in step (4) are input into the model in batches using the DataLoader data loader for model training. By defining the loss function and optimizer for forward and backward propagation, the model parameters are updated.

[0083] 3) In the model evaluation phase, the model performance is evaluated based on the test set or test set, and the model structure is adjusted and parameters are optimized.

[0084] The data fusion method of AI multi-source data fusion model is:

[0085] Based on the basic road information data obtained from satellite remote sensing image recognition, the basic position, geometric features and connectivity of road lines are identified.

[0086] Improve the accuracy of road location information, road width, road edge, road sign feature information and clarify the connectivity relationship between roads through Beidou location data and LiDAR measurement data;

[0087] Road feature data extracted through multi-source image recognition is used to enrich road attribute information, including road name, road type, and road direction;

[0088] Through crowdsourcing information data, road attribute information can be corrected in a timely manner to improve the timeliness of road data.

[0089] Through the above data fusion operation, the advantages of different data sources can be fully utilized to build a more comprehensive and accurate road data model.

[0090] Step (6) optimizes the AI ​​multi-source data fusion model based on road data quality feedback and multi-source data updates. Specifically, it includes:

[0091] Fusion model optimization: Through comprehensive comparison of the generated road data with the actual information, problems in the model are discovered and model parameters are adjusted. For example, the priority and weight of different types of image recognition data can be modified. Taking crowdsourcing information and satellite image data as an example, the images of crowdsourcing information often have higher effectiveness. On the premise of ensuring that the crowdsourcing information is true, its priority is higher than that of satellite image information. In the fusion, the priority and weight of crowdsourcing information in terms of timeliness should be increased to improve the accuracy of the data.

[0092] Fusion model upgrade: By entering more different types of raw data and increasing the model's training sample categories, the training of the AI ​​fusion algorithm can be enhanced, thereby outputting more comprehensive and rich road data. For example, there is a type of "tidal lane" road, which changes the direction of vehicle travel according to traffic flow needs. The training data when building the fusion model may not contain this type of data. After adding the raw data of the "tidal lane" type in the later stage, the marking type of the model-generated road can be enriched, making the road data attributes more comprehensive.

[0093] The present invention discloses a road data generation system based on AI multi-source data fusion, as follows Figure 1 As shown, the system mainly includes four functional modules, namely multi-source data acquisition module, multi-source data preprocessing module, road feature extraction module and AI multi-source data fusion module.

[0094] The multi-source data acquisition module involves the acquisition of multi-source data, including remote sensing satellite image data, aerial image data, lidar three-dimensional point cloud data based on vehicle-mounted sensors, vehicle-mounted camera shooting data, location data measured by vehicle-mounted Beidou equipment, and crowdsourcing pictures and text information data reported by the crowdsourcing information collection APP that comes with the system.

[0095] The multi-source data preprocessing module includes cleaning and denoising of the original data of each type of data and standardization of each type of data, which facilitates the subsequent extraction of feature information by category.

[0096] The road feature extraction module includes image recognition (satellite aerial image recognition, crowdsourced picture information recognition, and vehicle-mounted camera picture information recognition) to extract road information, sensor data extraction (lidar point cloud measurement road data calculation, GPS location data processing), and crowdsourced information road data extraction (crowdsourced text road information data extraction).

[0097] The AI ​​multi-source data fusion module includes an AI multi-source data fusion model, which realizes the fusion processing of multi-source information and outputs road data with multi-source information characteristics. The quality feedback and model optimization functions under this module rely on the data quality feedback adjustment mechanism and the expansion of data sources to achieve the upgrade and iteration of the fusion model.

[0098] The above is only an implementation mode of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

[0099] The technical solution of the present invention is based on the fact that with the rapid development of convolutional neural networks (CNN) in image recognition and analysis, specific element information in images can be automatically extracted through training sample data. The method of image recognition using convolutional neural networks can extract urban road data information from satellite remote sensing images, and can also identify road attribute information such as road names, road signs, markings and road types from pictures taken by vehicle-mounted cameras and pictures in crowdsourced information.

[0100] In the field of unmanned assisted driving, different on-board sensors can assist in obtaining road environment data information. Among them, the point cloud data processing technology based on LiDAR can realize road marking extraction, road edge information extraction, lane line recognition, etc., thereby obtaining high-precision road data information at the lane level.

[0101] The development of natural language processing (NLP) has been gradually improved, and it is now able to achieve capabilities such as text analysis and understanding, and intelligent question and answer. Relying on the natural language processing model, it is possible to extract road-related information from crowdsourced text information and identify the information.

[0102] AI multi-source data fusion is the process of integrating, analyzing and processing information from different sources, formats and dimensions. By integrating information from different data sources, the accuracy and robustness of artificial intelligence systems can be improved and their application areas can be expanded. AI data fusion methods can improve information integrity and make up for the shortcomings of a single data source by integrating information from multiple data sources. At the same time, cross-validation of multi-source data can be used to reduce errors and improve data accuracy and credibility.

Claims

1. A method for generating road data based on AI multi-source data fusion, characterized in that: Step (1) selecting or setting a target area for generating road data; Step (2) multi-source data selection input: according to the selected target area range, select the original data that meets the requirements of the range from the collected multi-source data; Step (3) preprocessing, cleaning, denoising and standardization of multi-source data; Perform data cleaning on the selected multi-source data to remove data that does not contain basic information; Step (4) extracting road information features from multi-source data; after the cleansed multi-source data is extracted, different information features are extracted in the road information extraction module according to different types; Step (5) AI multi-source data fusion processing, road data generation; after the road information feature extraction and processing of the multi-source data, the road feature data extracted and analyzed from the satellite remote sensing data, vehicle sensor data and crowdsourcing data are integrated by relying on the AI ​​multi-source data fusion module; Step (6) road data quality feedback, multi-source data update, and iterative upgrade of AI multi-source data fusion model training.

2. The method for generating road data based on AI multi-source data fusion according to claim 1 is characterized in that: In step (2), the multi-source data includes satellite remote sensing image data, data collection vehicle onboard sensor data, and crowd-sourced image text information data.

3. The method for generating road data based on AI multi-source data fusion according to claim 1 is characterized in that: In step (3), the multi-source data selected in step (2) is cleaned to remove data without basic information; remove blank or blurred images and picture data; remove abnormal or obviously wrong location information data; remove abnormal noise data in LiDAR scanning; and remove empty data and invalid data in crowdsourced data; The cleaned multi-source data mainly includes three standard types: remote sensing images, lidar measurement data and text, which are used for feature information extraction in the subsequent step (4).

4. The method for generating road data based on AI multi-source data fusion according to claim 1 is characterized in that: In step (4), Information extraction methods include image recognition technology, LiDAR point cloud data processing technology, and natural language processing technology; 1) Image recognition technology; Convolutional neural networks are trained through image recognition to extract and analyze specific feature data in images; Extract the basic road network location information and road connectivity of the target area from satellite remote sensing images to form basic road network data; Recognize the images taken by the vehicle camera to obtain road names, road signs, and road type information; Identify and extract attribute information from images reported by crowdsourcing data; 2) Sensor measurement data; The data collection vehicle is equipped with a laser radar to accurately measure the road data in the target area and extract detailed data information of the road from the point cloud data scanned by the laser radar; 3) Natural language processing; Process and analyze text description information about roads in crowdsourced data to extract timely attribute data of roads.

5. The method for generating road data based on AI multi-source data fusion according to claim 1 is characterized in that: The AI ​​multi-source data fusion model construction method in step (5) is: 1) In the model construction stage, a road data generation convolutional neural network model is defined using the torch.nn module. According to the needs of data fusion, the three types of data are connected in the model; 2) In the model training phase, the data feature set extracted in step (4) is input into the model in batches using the DataLoader data loader for model training. The model parameters are updated by performing forward and backward propagation by defining the loss function and optimizer. 3) In the model evaluation phase, the model performance is evaluated based on the test set or test set, and the model structure is adjusted and parameters are optimized.

6. The method for generating road data based on AI multi-source data fusion according to claim 5 is characterized in that: In step (5), the data fusion method of the AI ​​multi-source data fusion model is: Based on the basic road information data obtained by satellite remote sensing image recognition, the basic location, geometric features and connectivity of road lines are identified; Improve the accuracy of road location information, road width, road edge, road sign feature information and clarify the connectivity relationship between roads through Beidou location data and LiDAR measurement data; Enrich road attribute information through road feature data extracted through multi-source image recognition; Through crowdsourcing information data, road attribute information can be corrected in a timely manner to improve the timeliness of road data.

7. The method for generating road data based on AI multi-source data fusion according to claim 5 is characterized in that: Step (6) specifically includes: Fusion model optimization: Through comprehensive comparison of the generated road data with the actual information, problems in the model are discovered and model parameters are adjusted; Fusion model upgrade: By inputting different types of raw data and increasing the model's training sample categories, the training of the AI ​​fusion algorithm can be enhanced to output more comprehensive and rich road data.

8. A system for generating road data based on AI multi-source data fusion, characterized in that: The method for generating road data based on AI multi-source data fusion according to any one of claims 1 to 8 is applicable; the system mainly comprises: Multi-source data acquisition module, used for multi-source data acquisition; Multi-source data preprocessing module, used to clean and denoise the raw data of various types of data and perform standardization operations on each type of data; Road feature extraction module, including image recognition, road information extraction, sensor data extraction, and crowdsourced information road data extraction; The AI ​​multi-source data fusion module is used to realize the fusion processing of multi-source information through the AI ​​multi-source data fusion model, and output road data with multi-source information characteristics.

Images