A map generation method and system based on intelligent connected vehicles

Abstract

The present invention is applicable to the technical field of map generation, and provides a map generation method and system based on intelligent networked vehicles, which obtain vehicle authority and vehicle data; obtain radar data according to the vehicle authority, and convert the radar data into a three-dimensional model; obtain real-time radar data and vehicle data, compare them with the three-dimensional model, obtain difference points, the difference points are differences with the three-dimensional model, determine the authenticity of the difference points, and update the three-dimensional model according to the difference points; determine whether a traffic jam occurs according to the real-time vehicle data, compare the real-time vehicle data with the three-dimensional model, determine whether there is a new route, and the new route is a route that is not displayed in the three-dimensional model, so as to solve the problem that a three-dimensional map cannot be generated according to the real scene on the street, characteristic buildings on the street cannot be reflected on the map, and the three-dimensional map cannot be updated in real time to supplement the three-dimensional map.

Description

Technical Field

[0001] The present invention belongs to the technical field of map generation, and in particular relates to a map generation method and system based on an intelligent networked vehicle. Background Art

[0002] With the rapid development of science and technology, automobile technology is moving rapidly towards electrification, intelligence, networking and sharing. Related technical fields include electric drive, wire-controlled chassis, integrated control, automatic driving, environmental sensing, wireless communication, etc. Automobiles have gradually formed an intelligent automobile network, and using the intelligent automobile network to generate dynamic maps of automobile driving has become a hot topic in the current research field.

[0003] However, existing map generation has the problems of being unable to generate a three-dimensional map based on the actual street scene, being unable to reflect the characteristic buildings on the street on the map, and being unable to update the three-dimensional map in real time or to supplement the three-dimensional map. Summary of the invention

[0004] The purpose of the embodiment of the present invention is to provide a map generation method based on an intelligent connected vehicle, aiming to solve the problem raised in the third part of the background technology.

[0005] The embodiment of the present invention is implemented as follows: a map generation method based on an intelligent networked vehicle, the method comprising:

[0006] Obtaining car permissions, including lidar permissions and optical radar permissions, and obtaining car data;

[0007] Acquire radar data according to the vehicle authority, the radar data including laser radar data and optical radar data, and convert the radar data into a three-dimensional model, the three-dimensional model including geometric structure and landmark building information;

[0008] Acquire real-time radar data and vehicle data, compare them with the three-dimensional model, obtain difference points, which are differences from the three-dimensional model, determine the authenticity of the difference points, and update the three-dimensional model according to the difference points;

[0009] Determine whether a traffic jam occurs based on real-time vehicle data, and compare the real-time vehicle data with the three-dimensional model to determine whether there is a new route, wherein the new route is a route that is not displayed in the three-dimensional model.

[0010] Preferably, the radar data includes laser radar data and optical radar data, and the steps of converting the radar data into a three-dimensional model, wherein the three-dimensional model includes geometric structure and landmark building information, specifically include:

[0011] Obtain the lidar data and optical radar data according to the vehicle authority, and pre-process the lidar data and optical radar data;

[0012] Acquire the preprocessed first data, identify features in the first data, the features including geographic shapes and building images, and determine a two-dimensional image according to the features;

[0013] A two-dimensional image is acquired, and texture mapping is performed on the two-dimensional image. The texture mapping is used to map the panoramic image generated by image stitching to a three-dimensional point cloud, and a three-dimensional model is generated according to the three-dimensional point cloud.

[0014] Preferably, the step of acquiring real-time radar data and vehicle data, comparing them with the three-dimensional model, obtaining difference points, wherein the difference points are differences from the three-dimensional model, determining the authenticity of the difference points, and updating the three-dimensional model according to the difference points specifically includes:

[0015] Acquire first real-time radar data and vehicle data, acquire calibration parameters, the calibration parameters include internal parameters and external parameters, and compare the calibration parameters with the three-dimensional model;

[0016] Obtaining a comparison result, determining a first difference point through the comparison result, obtaining second real-time radar data and vehicle data, and comparing the second real-time radar data and vehicle data with the three-dimensional model to obtain a second difference point;

[0017] The first difference point is compared with the second difference point to determine the authenticity of the difference point, and the three-dimensional model is updated according to the difference point.

[0018] Preferably, the step of determining whether a traffic jam occurs according to the real-time vehicle data, comparing the real-time vehicle data with the three-dimensional model, and determining whether there is a new route, wherein the new route is a route not displayed in the three-dimensional model, specifically includes:

[0019] Obtain real-time radar data and vehicle data, and determine vehicle speed and trajectory through real-time radar data and vehicle data;

[0020] Compare the vehicle speed and trajectory with the preset values, and determine whether a traffic jam occurs based on the comparison results;

[0021] Obtain the vehicle trajectory, compare the vehicle trajectory with the 3D model, determine whether it is a new route, and if so, upload it to the 3D model.

[0022] Preferably, the vehicle data includes GPS data, vehicle status data and inertial measurement unit data.

[0023] Another object of an embodiment of the present invention is to provide a map generation system based on an intelligent connected vehicle, the system comprising:

[0024] The permission module obtains the car permissions, including the laser radar permissions and the optical radar permissions, and obtains the car data;

[0025] A three-dimensional model module, which obtains radar data according to the vehicle authority, the radar data including laser radar data and optical radar data, and converts the radar data into a three-dimensional model, the three-dimensional model including geometric structure and landmark building information;

[0026] An update module obtains real-time radar data and vehicle data, compares them with the three-dimensional model, obtains difference points, which are differences from the three-dimensional model, determines the authenticity of the difference points, and updates the three-dimensional model according to the difference points;

[0027] The supplementary module determines whether a traffic jam occurs according to the real-time vehicle data, compares the real-time vehicle data with the three-dimensional model, and determines whether there is a new route, wherein the new route is a route not displayed in the three-dimensional model.

[0028] Preferably, the three-dimensional model module includes:

[0029] A preprocessing unit, which obtains the laser radar data and the optical radar data according to the vehicle authority, and preprocesses the laser radar data and the optical radar data;

[0030] A first data unit, which obtains preprocessed first data, identifies features in the first data, wherein the features include geographic shapes and building images, and determines a two-dimensional image according to the features;

[0031] The three-dimensional model unit acquires a two-dimensional image and performs texture mapping on the two-dimensional image. The texture mapping is used to map a panoramic image generated by image stitching to a three-dimensional point cloud, and generate a three-dimensional model according to the three-dimensional point cloud.

[0032] Preferably, the update module includes:

[0033] A comparison unit, which obtains the first real-time radar data and the vehicle data, obtains calibration parameters, the calibration parameters include internal parameters and external parameters, and compares the calibration parameters with the three-dimensional model;

[0034] A difference point unit, which obtains a comparison result, determines a first difference point through the comparison result, obtains second real-time radar data and vehicle data, compares the second real-time radar data and vehicle data with the three-dimensional model, and obtains a second difference point;

[0035] The updating unit compares the first difference point with the second difference point, determines the authenticity of the difference point, and updates the three-dimensional model according to the difference point.

[0036] Preferably, the supplementary module comprises:

[0037] A determination unit, which obtains real-time radar data and vehicle data, and determines the vehicle speed and vehicle trajectory through the real-time radar data and vehicle data;

[0038] The traffic jam unit compares the vehicle speed and vehicle trajectory with the preset values ​​and determines whether a traffic jam occurs based on the comparison results;

[0039] The supplementary unit obtains the vehicle trajectory, compares the vehicle trajectory with the three-dimensional model, determines whether it is a new route, and if it is determined to be a new route, uploads it to the three-dimensional model.

[0040] Preferably, the vehicle data includes GPS data, vehicle status data and inertial measurement unit data.

[0041] The embodiment of the present invention provides a map generation method based on an intelligent networked vehicle, which obtains vehicle permissions. The vehicle is equipped with various sensors and devices for real-time collection of geographic location and environmental data. Therefore, the laser radar permissions and optical radar permissions of the vehicle are obtained, and the vehicle data is obtained at the same time, so that the speed and position of the bicycle can be obtained. Radar data is obtained according to the vehicle permissions, and pre-processing is performed through the laser radar data and the optical radar data to obtain features in the data. A two-dimensional image is determined by the features, and the two-dimensional image is converted into a three-dimensional model through texture mapping. Multiple groups of real-time radar data and vehicle data are obtained, and compared with the three-dimensional model. After the difference points are obtained, the accuracy of the difference points is determined. If it is determined to be accurate, the three-dimensional model is updated. At the same time, it is determined according to the vehicle data whether the vehicle is in a traffic jam, and the rear vehicle can be reminded to change the route in time. By comparing the real-time vehicle data with the three-dimensional model, if a new route is found, it is sent to the relevant vehicle to determine whether the route is accurate. The problem that a three-dimensional map cannot be generated according to the real scene on the street, the characteristic buildings on the street cannot be reflected on the map, and the three-dimensional map cannot be updated in real time to supplement the three-dimensional map is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flowchart of a map generation method based on an intelligent connected vehicle provided by an embodiment of the present invention;

[0043] Figure 2 A flowchart of the steps of acquiring radar data according to vehicle authority and converting the radar data into a three-dimensional model provided in an embodiment of the present invention;

[0044] Figure 3 A flowchart of the steps of obtaining real-time radar data and vehicle data, comparing them with a three-dimensional model to obtain difference points, and updating the three-dimensional model according to the difference points provided by an embodiment of the present invention;

[0045] Figure 4 A flowchart of the steps of determining whether a traffic jam occurs according to real-time vehicle data and comparing the real-time vehicle data with a three-dimensional model to determine whether there is a new route provided by an embodiment of the present invention;

[0046] Figure 5An architecture diagram of a map generation system based on an intelligent connected vehicle provided in an embodiment of the present invention;

[0047] Figure 6 An architectural diagram of a three-dimensional model module provided in an embodiment of the present invention;

[0048] Figure 7 An architectural diagram of an update module provided in an embodiment of the present invention;

[0049] Figure 8 An architectural diagram of a supplementary module provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0051] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0052] like Figure 1 As shown, a map generation method based on an intelligent connected vehicle is provided in an embodiment of the present invention, and the method includes:

[0053] S100, obtaining vehicle permissions, wherein the vehicle permissions include lidar permissions and optical radar permissions, and obtaining vehicle data.

[0054] In this step, the car permissions are obtained. The vehicle is equipped with various sensors and devices for real-time collection of geographic location and environmental data. Therefore, the vehicle's lidar permissions and optical radar permissions are obtained. The lidar is used to collect three-dimensional point cloud data of the environment. These data contain the spatial position and shape information of surrounding objects. The radar detects the distance and speed of the vehicle or obstacle in front.

[0055] Acquire vehicle data, which includes GPS data, vehicle status data and inertial measurement unit data. The precise location of the vehicle is acquired through GPS. The GPS module on the vehicle continuously acquires the precise location coordinates of the vehicle, including longitude and latitude, and generates a real-time location sequence of the vehicle. The inertial measurement unit (IMu) provides the acceleration and angular velocity information of the vehicle to supplement and correct the GPS location data, especially in areas with weak signals.

[0056] S200, acquiring radar data according to the vehicle authority, the radar data including laser radar data and optical radar data, converting the radar data into a three-dimensional model, the three-dimensional model including geometric structure and landmark building information.

[0057] In this step, radar data is obtained according to the car's permissions. Data collected from various sensors on the car, such as GPS, LiDAR (laser radar), camera, and IMu, are used as the basis for map generation. The collected raw data is cleaned and filtered to remove noise, erroneous data, and redundant data. The point cloud data of LiDAR is combined with the vehicle's location information.

[0058] The LiDAR point cloud data is processed into a high-resolution three-dimensional model, which preliminarily reconstructs the initial fuzzy three-dimensional structure of the street. Through texture mapping technology, the image taken by the camera is attached to the three-dimensional point cloud to generate a detail-processed three-dimensional model. The three-dimensional model not only has a geometric structure, but also contains the visual details of the street. The camera can capture surrounding image information, especially the appearance details of landmark buildings, such as station signs, signboard text, building color and shape, etc., to enrich the characteristics of the three-dimensional model.

[0059] S300, real-time radar data and vehicle data are acquired, and compared with the three-dimensional model to obtain difference points, where the difference points are differences from the three-dimensional model, the authenticity of the difference points is determined, and the three-dimensional model is updated according to the difference points.

[0060] In this step, real-time radar data and vehicle data are obtained. By obtaining the radar data and vehicle data in real time, the 3D model can be updated after the 3D model is generated, and the difference points can be obtained by comparing with the 3D model.

[0061] By obtaining external parameters of cameras of different cars, such as positions and orientations, and internal parameters of cameras of different cars, such as focal lengths and lens distortion coefficients, the corresponding position of each pixel in the panoramic image in the three-dimensional model is calculated based on the positions and orientations of the cameras. For example, the position of the left part of building A in the three-dimensional model corresponds to the position in the panoramic image.

[0062] If a mismatch is found, it is determined as a difference point. At the same time, the radar data and car data of other vehicles are obtained and the 3D model is compared again. If the difference point is confirmed, it is determined to be true. At this time, the 3D model is updated according to the difference point.

[0063] S400, determining whether a traffic jam occurs according to the real-time vehicle data, comparing the real-time vehicle data with the three-dimensional model, and determining whether there is a new route, wherein the new route is a route not displayed in the three-dimensional model.

[0064] In this step, the real-time vehicle data is used to determine whether a traffic jam has occurred. Vehicles obtain their speed, location, and trajectory in real time through sensors and communication devices. The system can use this data to determine whether a traffic jam has occurred and detect whether any vehicles have used new routes that have not been recorded. If a new route appears, the system can record and upload the route to the 3D map to help other vehicles optimize navigation.

[0065] like Figure 2 As shown, as a preferred embodiment of the present invention, the step of acquiring radar data according to the vehicle authority, the radar data including laser radar data and optical radar data, converting the radar data into a three-dimensional model, the three-dimensional model including geometric structure and landmark building information, specifically includes:

[0066] S201, obtaining the laser radar data and the optical radar data according to the vehicle authority, and preprocessing the laser radar data and the optical radar data.

[0067] In this step, the laser radar data and optical radar data are obtained according to the vehicle authority, and the laser radar data and optical radar data are preprocessed. The preprocessing includes image processing and point cloud processing. The image processing includes denoising and color space conversion. The point cloud processing includes downsampling and normal vector estimation.

[0068] The data collected by various sensors on the car are used as the basis for map generation, such as GPS, LiDAR (laser radar), camera and IMu, etc., to denoise the image, remove the noise interference in the image, and improve the accuracy of feature extraction. Commonly used methods include Gaussian filtering and median filtering, etc., to convert the image from RGB space to other color spaces, such as HSv, grayscale, etc., to facilitate subsequent feature extraction. For example, edge detection on grayscale images is usually better than on color images;

[0069] The point cloud data is downsampled to reduce the amount of calculation while retaining the necessary spatial structure information. The normal vector is calculated for each point cloud point to describe the direction of the point on the surface of the object, which is helpful for subsequent object recognition.

[0070] S202, obtaining the preprocessed first data, identifying features in the first data, the features including geographic shapes and building images, and determining a two-dimensional image according to the features.

[0071] In this step, the preprocessed first data is obtained, the preprocessed data is used as the first data, and the point cloud data is divided into different parts using the point cloud segmentation technology, the outline and geometric structure of the building are extracted, and the three-dimensional shape of the landmark building, such as size, shape and position, is recognized through the geometric features of the building;

[0072] Using convolutional neural networks (CNNs) to identify landmark buildings in images, such as station signs, signboards, and characteristic buildings, and to identify text on signboards, such as store names and building names, can help further confirm the type of building;

[0073] Use the perspective projection model to convert the 3D coordinates (X, Y, Z) of each LiDAR point into 2D coordinates (u, v) in the camera's image plane, map the 3D point cloud from the LiDAR coordinate system to the camera's coordinate system, and then project it onto the 2D image.

[0074] The images directly captured by the LiDAR are already two-dimensional and contain rich color and texture information. Multiple cameras on the car capture different perspectives of the same scene. Each perspective image provides details of the scene seen from a different position, and these images will be used for texture mapping into the three-dimensional model later.

[0075] S203, acquiring a two-dimensional image, and performing texture mapping on the two-dimensional image, wherein the texture mapping is used to map a panoramic image generated by image stitching to a three-dimensional point cloud, and generate a three-dimensional model according to the three-dimensional point cloud.

[0076] In this step, to obtain a 2D image, it is necessary to combine the 3D geometric data of LiDAR and the 2D image information of the camera, and apply the visual details to the 3D model through texture mapping. Using the point cloud data of LiDAR, it is first meshed to generate a triangular mesh. The triangular mesh represents the 3D surface structure in the scene, such as the wall of a building, the surface of a road, etc.

[0077] Each vertex of the 3D model calculates the corresponding 2D texture coordinates (UV coordinates), which represent how to map the 2D image onto the 3D surface. Specifically, the UV coordinates define the position of the model's vertices in the 2D image plane. If images from multiple perspectives are used for texture mapping, texture fusion is required to eliminate gaps and color differences between images, ensure that the texture is smooth and consistent on the model surface, and attach the image texture to the surface of the 3D model so that the model has a realistic appearance when visualized. Methods such as bilinear interpolation can be used to ensure the smoothness of texture mapping and avoid obvious texture distortion on the surface of the 3D model.

[0078] The edge detection algorithm is used to identify edge information in the image, which usually corresponds to the outline of objects and area boundaries, such as the outline of buildings and the boundaries of lane lines. The point cloud data is first downsampled. At the same time, the normal vector of each point is calculated and the two-dimensional image is projected onto the three-dimensional model. The projection process involves calculating the corresponding position of each three-dimensional point or polygonal face in the image.

[0079] like Figure 3As shown, as a preferred embodiment of the present invention, the steps of acquiring real-time radar data and vehicle data, comparing them with the three-dimensional model, obtaining difference points, the difference points being the differences from the three-dimensional model, determining the authenticity of the difference points, and updating the three-dimensional model according to the difference points specifically include:

[0080] S301, obtaining first real-time radar data and vehicle data, obtaining calibration parameters, the calibration parameters including internal parameters and external parameters, and comparing the calibration parameters with the three-dimensional model.

[0081] In this step, the first real-time radar data and vehicle data are obtained. The camera is calibrated first to obtain the camera's internal parameters, including focal length, principal point coordinates, distortion parameters, and camera's external parameters, including the rotation and translation matrices of the LiDAR. The three-dimensional point cloud is projected onto the two-dimensional image plane using the camera's internal and external parameters. This step maps each point cloud point to a pixel position in the image, and the calibration parameters are compared with the three-dimensional model.

[0082] S302, obtaining a comparison result, determining a first difference point through the comparison result, obtaining second real-time radar data and vehicle data, comparing the second real-time radar data and vehicle data with the three-dimensional model, and obtaining a second difference point.

[0083] In this step, the comparison result is obtained by obtaining the external parameters of the cameras of different cars, such as the position and orientation, and the internal parameters of the cameras of different cars, such as the focal length and the lens distortion coefficient. Based on the position and orientation of the cameras, the corresponding position of each pixel in the panoramic image in the three-dimensional model is calculated. For example, the position of the left part of the building A in the three-dimensional model corresponds to the position in the panoramic image. If it is found that there is no correspondence, it is determined as the first difference point.

[0084] The second real-time radar data and the vehicle data are acquired again, and the corresponding position of each pixel in the panoramic image in the three-dimensional model is calculated. If it is found that there is no correspondence, it is determined as the second difference point.

[0085] S303, comparing the first difference point with the second difference point, determining the authenticity of the difference point, and updating the three-dimensional model according to the difference point.

[0086] In this step, the first difference point is compared with the second difference point. If the first difference point and the second difference point are consistent, both are different from the original three-dimensional model, and the first difference point is consistent with the second difference point, then the three-dimensional model map is updated to ensure that the three-dimensional model map can be guaranteed to be the latest map, and navigation errors will not occur during subsequent use.

[0087] For example, the currently collected point cloud and image are compared with the data in the 3D model, and it is found that the store's sign and appearance are inconsistent with the information in the model. Other vehicles passing through the same section of the road are required to upload real-time data of the same area. Another vehicle then uploads the point cloud and image collected when passing through the section of the road. The system compares the data of multiple vehicles and confirms that the sensors of these vehicles have also detected the same changes, confirming that the store's sign has indeed been updated to "XYZ Supermarket".

[0088] like Figure 4 As shown, as a preferred embodiment of the present invention, the step of determining whether a traffic jam occurs according to real-time vehicle data, comparing the real-time vehicle data with the three-dimensional model, and determining whether there is a new route, wherein the new route is a route not displayed in the three-dimensional model, specifically includes:

[0089] S401, acquiring real-time radar data and vehicle data, and determining vehicle speed and vehicle trajectory through the real-time radar data and vehicle data.

[0090] In this step, real-time radar data and vehicle data are obtained. The vehicle needs to collect its speed, position and trajectory in real time through its own sensors and equipment. These data are the basis for traffic congestion detection and route analysis. The vehicle's own speed sensor can directly measure the vehicle speed and continuously record real-time speed information.

[0091] The real-time collected GPS coordinates are stored in chronological order to form a vehicle trajectory. The trajectory is usually composed of consecutive geographical points, marking the vehicle's starting point to the current location.

[0092] S402, comparing the vehicle speed and vehicle trajectory with preset values, and determining whether a traffic jam occurs based on the comparison result.

[0093] In this step, the vehicle speed and vehicle trajectory are compared with the preset values. If the vehicle speed is at a low value for a period of time, the preset value is set to 10 km / h. If it is lower than 10 km / h and there is no obvious sign of acceleration, it may mean that the vehicle has encountered traffic congestion;

[0094] The relative speed of the vehicle and other surrounding vehicles can be used to further confirm whether congestion has occurred. If most vehicles slow down or stop on the same road section, it can be preliminarily determined that congestion has occurred on that road section.

[0095] S403, obtaining the vehicle trajectory, comparing the vehicle trajectory with the three-dimensional model, determining whether it is a new route, and if it is determined to be a new route, uploading it to the three-dimensional model.

[0096] In this step, the vehicle trajectory is obtained and compared with the 3D model. Through real-time monitoring of vehicle trajectories, the system can detect whether some vehicles are using a new route that is not recorded in the map or an unplanned route. If the vehicle deviates from the known road network, the system will mark the trajectory as a potential new route and determine it as a new route;

[0097] If multiple vehicles travel along this new route at different time periods or at the same time, the existence of the route and its smooth travel will be confirmed by superimposing the trajectories of multiple vehicles. After the new route is confirmed, the route information will be uploaded to the cloud server and the three-dimensional map database will be updated to ensure that all vehicles can use the latest map data for navigation.

[0098] like Figure 5 As shown, a map generation system based on an intelligent networked vehicle is provided in an embodiment of the present invention, and the system includes:

[0099] The permission module 100 is used to obtain vehicle permissions, including lidar permissions and optical radar permissions, and obtain vehicle data.

[0100] In this system, the permission module 100 obtains the car permission. The vehicle is equipped with various sensors and devices for collecting geographic location and environmental data in real time. Therefore, the laser radar permission and optical radar permission of the vehicle are obtained. The laser radar is used to collect three-dimensional point cloud data of the environment. These data contain the spatial position and shape information of the surrounding objects. The radar detects the distance and speed of the vehicle or obstacle in front.

[0101] Acquire vehicle data, which includes GPS data, vehicle status data and inertial measurement unit data. The precise location of the vehicle is acquired through GPS. The GPS module on the vehicle continuously acquires the precise location coordinates of the vehicle, including longitude and latitude, and generates a real-time location sequence of the vehicle. The inertial measurement unit (IMu) provides the acceleration and angular velocity information of the vehicle to supplement and correct the GPS location data, especially in areas with weak signals.

[0102] The three-dimensional model module 200 is used to obtain radar data according to the vehicle authority, the radar data including laser radar data and optical radar data, and convert the radar data into a three-dimensional model, the three-dimensional model including geometric structure and landmark building information.

[0103] In this system, the 3D model module 200 obtains radar data according to the car's authority, and uses the data collected from various sensors on the car as the basis for map generation, such as GPS, LiDAR (laser radar), camera and IMu, etc. The collected raw data is cleaned and filtered to remove noise, erroneous data and redundant data, and the point cloud data of LiDAR is combined with the vehicle's location information.

[0104] The LiDAR point cloud data is processed into a high-resolution three-dimensional model, which preliminarily reconstructs the initial fuzzy three-dimensional structure of the street. Through texture mapping technology, the image taken by the camera is attached to the three-dimensional point cloud to generate a detail-processed three-dimensional model. The three-dimensional model not only has a geometric structure, but also contains the visual details of the street. The camera can capture surrounding image information, especially the appearance details of landmark buildings, such as station signs, signboard text, building color and shape, etc., to enrich the characteristics of the three-dimensional model.

[0105] The updating module 300 is used to obtain real-time radar data and vehicle data, compare them with the three-dimensional model, obtain difference points, which are differences from the three-dimensional model, determine the authenticity of the difference points, and update the three-dimensional model according to the difference points.

[0106] In this system, the update module 300 obtains real-time radar data and vehicle data. By obtaining the radar data and vehicle data in real time, the three-dimensional model can be updated after the three-dimensional model is generated, and the difference points can be obtained by comparing with the three-dimensional model;

[0107] By obtaining external parameters of cameras of different cars, such as positions and orientations, and internal parameters of cameras of different cars, such as focal lengths and lens distortion coefficients, the corresponding position of each pixel in the panoramic image in the three-dimensional model is calculated based on the positions and orientations of the cameras. For example, the position of the left part of building A in the three-dimensional model corresponds to the position in the panoramic image.

[0108] If a mismatch is found, it is determined as a difference point. At the same time, the radar data and car data of other vehicles are obtained and the 3D model is compared again. If the difference point is confirmed, it is determined to be true. At this time, the 3D model is updated according to the difference point.

[0109] The supplement module 400 is used to determine whether a traffic jam occurs according to the real-time vehicle data, and compare the real-time vehicle data with the three-dimensional model to determine whether there is a new route, wherein the new route is a route not displayed in the three-dimensional model.

[0110] In this system, the supplementary module 400 determines whether a traffic jam occurs based on real-time vehicle data. Vehicles obtain their own speed, position and trajectory in real time through sensors and communication devices. The system can use this data to determine whether a traffic jam occurs and detect whether a vehicle uses a new route that is not recorded. If a new route appears, the system can record and upload the route to the three-dimensional map to help other vehicles optimize navigation.

[0111] like Figure 6 As shown, as a preferred embodiment of the present invention, the three-dimensional model module 200 includes:

[0112] The preprocessing unit 201 is used to obtain the laser radar data and the optical radar data according to the vehicle authority, and preprocess the laser radar data and the optical radar data.

[0113] In this module, the preprocessing unit 201 obtains the laser radar data and the optical radar data according to the vehicle authority, and preprocesses the laser radar data and the optical radar data. The preprocessing includes image processing and point cloud processing. The image processing includes denoising and color space conversion, and the point cloud processing includes downsampling and normal vector estimation.

[0114] The data collected by various sensors on the car are used as the basis for map generation, such as GPS, LiDAR (laser radar), camera and IMu, etc., to denoise the image, remove the noise interference in the image, and improve the accuracy of feature extraction. Commonly used methods include Gaussian filtering and median filtering, etc., to convert the image from RGB space to other color spaces, such as HSv, grayscale, etc., to facilitate subsequent feature extraction. For example, edge detection on grayscale images is usually better than on color images;

[0115] The point cloud data is downsampled to reduce the amount of calculation while retaining the necessary spatial structure information. The normal vector is calculated for each point cloud point to describe the direction of the point on the surface of the object, which is helpful for subsequent object recognition.

[0116] The first data unit 202 is used to obtain the pre-processed first data, identify features in the first data, the features include geographic shapes and building images, and determine a two-dimensional image according to the features.

[0117] In this module, the first data unit 202 obtains the preprocessed first data, uses the preprocessed data as the first data, uses the point cloud segmentation technology to divide the point cloud data into different parts, extracts the outline and geometric structure of the building, and recognizes the three-dimensional shape of the landmark building, such as size, shape and position, through the geometric features of the building;

[0118] Using convolutional neural networks (CNNs) to identify landmark buildings in images, such as station signs, signboards, and characteristic buildings, and to identify text on signboards, such as store names and building names, can help further confirm the type of building;

[0119] Use the perspective projection model to convert the 3D coordinates (X, Y, Z) of each LiDAR point into 2D coordinates (u, v) in the camera's image plane, map the 3D point cloud from the LiDAR coordinate system to the camera's coordinate system, and then project it onto the 2D image.

[0120] The images directly captured by the LiDAR are already two-dimensional and contain rich color and texture information. Multiple cameras on the car capture different perspectives of the same scene. Each perspective image provides details of the scene seen from a different position, and these images will be used for texture mapping into the three-dimensional model later.

[0121] The three-dimensional model unit 203 is used to obtain a two-dimensional image and perform texture mapping on the two-dimensional image. The texture mapping is used to map the panoramic image generated by image stitching to a three-dimensional point cloud and generate a three-dimensional model according to the three-dimensional point cloud.

[0122] In this module, the 3D model unit 203 acquires a 2D image, and needs to combine the 3D geometric data of the LiDAR and the 2D image information of the camera, and apply the visual details to the 3D model through texture mapping. The point cloud data of the LiDAR is first meshed to generate a triangular mesh, which represents the 3D surface structure in the scene, such as the wall of a building, the surface of a road, etc.;

[0123] Each vertex of the 3D model calculates the corresponding 2D texture coordinates (UV coordinates), which represent how to map the 2D image onto the 3D surface. Specifically, the UV coordinates define the position of the model's vertices in the 2D image plane. If images from multiple perspectives are used for texture mapping, texture fusion is required to eliminate gaps and color differences between images, ensure that the texture is smooth and consistent on the model surface, and attach the image texture to the surface of the 3D model so that the model has a realistic appearance when visualized. Methods such as bilinear interpolation can be used to ensure the smoothness of texture mapping and avoid obvious texture distortion on the surface of the 3D model.

[0124] The edge detection algorithm is used to identify edge information in the image, which usually corresponds to the outline of objects and area boundaries, such as the outline of buildings and the boundaries of lane lines. The point cloud data is first downsampled. At the same time, the normal vector of each point is calculated and the two-dimensional image is projected onto the three-dimensional model. The projection process involves calculating the corresponding position of each three-dimensional point or polygonal face in the image.

[0125] like Figure 7 As shown, as a preferred embodiment of the present invention, the update module 300 includes:

[0126] The comparison unit 301 is used to obtain the first real-time radar data and the vehicle data, obtain calibration parameters, and the calibration parameters include internal parameters and external parameters. The calibration parameters are compared with the three-dimensional model.

[0127] In this module, the comparison unit 301 obtains the first real-time radar data and vehicle data, first performs camera calibration, obtains the camera's internal parameters, including focal length, principal point coordinates, distortion parameters, and the camera's external parameters, including the rotation and translation matrices of the LiDAR, and uses the camera's internal and external parameters to project the three-dimensional point cloud onto the two-dimensional image plane. This step maps each point cloud point to a pixel position in the image, and the calibration parameters are compared with the three-dimensional model.

[0128] The difference point unit 302 is used to obtain the comparison result, determine the first difference point through the comparison result, obtain the second real-time radar data and the vehicle data, compare the second real-time radar data and the vehicle data with the three-dimensional model, and obtain the second difference point.

[0129] In this module, the difference point unit 302 obtains the comparison result by obtaining the external parameters of the cameras of different cars, such as the position and orientation, and the internal parameters of the cameras of different cars, such as the focal length and the lens distortion coefficient. Based on the position and orientation of the cameras, the corresponding position of each pixel in the panoramic image in the three-dimensional model is calculated. For example, the position of the left part of the building A in the three-dimensional model corresponds to the position in the panoramic image. If it is found that there is no correspondence, it is determined as the first difference point.

[0130] The second real-time radar data and the vehicle data are acquired again, and the corresponding position of each pixel in the panoramic image in the three-dimensional model is calculated. If it is found that there is no correspondence, it is determined as the second difference point.

[0131] The updating unit 303 is used to compare the first difference point with the second difference point, determine the authenticity of the difference point, and update the three-dimensional model according to the difference point.

[0132] In this module, the updating unit 303 compares the first difference point with the second difference point. If the first difference point and the second difference point are consistent, both are different from the original three-dimensional model, and the first difference point is consistent with the second difference point, then the three-dimensional model map is updated to ensure that the three-dimensional model map can be guaranteed to be the latest map, and navigation errors will not occur during subsequent use.

[0133] For example, the currently collected point cloud and image are compared with the data in the 3D model, and it is found that the store's sign and appearance are inconsistent with the information in the model. Other vehicles passing through the same section of the road are required to upload real-time data of the same area. Another vehicle then uploads the point cloud and image collected when passing through the section of the road. The system compares the data of multiple vehicles and confirms that the sensors of these vehicles have also detected the same changes, confirming that the store's sign has indeed been updated to "XYZ Supermarket".

[0134] like Figure 8 As shown, as a preferred embodiment of the present invention, the supplement module 400 includes:

[0135] The determination unit 401 is used to obtain real-time radar data and vehicle data, and determine the vehicle speed and vehicle trajectory through the real-time radar data and vehicle data.

[0136] In this module, the determination unit 401 obtains real-time radar data and vehicle data. The vehicle needs to collect its speed, position and trajectory in real time through its own sensors and equipment. These data are the basis for traffic congestion detection and route analysis. The vehicle's own speed sensor can directly measure the vehicle speed and continuously record real-time speed information.

[0137] The real-time collected GPS coordinates are stored in chronological order to form a vehicle trajectory. The trajectory is usually composed of consecutive geographical points, marking the vehicle's starting point to the current location.

[0138] The traffic jam unit 402 is used to compare the vehicle speed and vehicle trajectory with preset values, and determine whether a traffic jam occurs based on the comparison result.

[0139] In this module, the traffic jam unit 402 compares the vehicle speed and vehicle trajectory with the preset value. If the vehicle speed is at a low value for a period of time, the preset value is set to 10 km / h. If it is lower than 10 km / h and there is no obvious sign of acceleration, it may mean that the vehicle has encountered traffic congestion;

[0140] The relative speed of the vehicle and other surrounding vehicles can be used to further confirm whether congestion has occurred. If most vehicles slow down or stop on the same road section, it can be preliminarily determined that congestion has occurred on that road section.

[0141] The supplement unit 403 is used to obtain the vehicle trajectory, compare the vehicle trajectory with the three-dimensional model, determine whether it is a new route, and if it is determined to be a new route, upload it to the three-dimensional model.

[0142] In this module, the supplementary unit 403 obtains the vehicle trajectory, compares the vehicle trajectory with the three-dimensional model, and through real-time monitoring of the vehicle trajectory, the system can detect whether some vehicles are using a new route that is not recorded in the map or an unplanned route. If the vehicle deviates from the known road network, the system will mark the trajectory as a potential new route and determine it as a new route;

[0143] If multiple vehicles travel along this new route at different time periods or at the same time, the existence of the route and its smooth travel will be confirmed by superimposing the trajectories of multiple vehicles. After the new route is confirmed, the route information will be uploaded to the cloud server and the three-dimensional map database will be updated to ensure that all vehicles can use the latest map data for navigation.

[0144] In one embodiment, a computer device is provided, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0145] Obtaining car permissions, including lidar permissions and optical radar permissions, and obtaining car data;

[0146] Acquire radar data according to the vehicle authority, the radar data including laser radar data and optical radar data, and convert the radar data into a three-dimensional model, the three-dimensional model including geometric structure and landmark building information;

[0147] Acquire real-time radar data and vehicle data, compare them with the three-dimensional model, obtain difference points, which are differences from the three-dimensional model, determine the authenticity of the difference points, and update the three-dimensional model according to the difference points;

[0148] Determine whether a traffic jam occurs based on real-time vehicle data, and compare the real-time vehicle data with the three-dimensional model to determine whether there is a new route, wherein the new route is a route that is not displayed in the three-dimensional model.

[0149] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor performs the following steps:

[0150] Obtaining car permissions, including lidar permissions and optical radar permissions, and obtaining car data;

[0151] Acquire radar data according to the vehicle authority, the radar data including laser radar data and optical radar data, and convert the radar data into a three-dimensional model, the three-dimensional model including geometric structure and landmark building information;

[0152] Acquire real-time radar data and vehicle data, compare them with the three-dimensional model, obtain difference points, which are differences from the three-dimensional model, determine the authenticity of the difference points, and update the three-dimensional model according to the difference points;

[0153] Determine whether a traffic jam occurs based on real-time vehicle data, and compare the real-time vehicle data with the three-dimensional model to determine whether there is a new route, wherein the new route is a route that is not displayed in the three-dimensional model.

[0154] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0155] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0156] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0157] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0158] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A map generation method based on an intelligent connected vehicle, characterized in that: The method comprises: Obtaining car permissions, including lidar permissions and optical radar permissions, and obtaining car data; Acquire radar data according to the vehicle authority, the radar data including laser radar data and optical radar data, and convert the radar data into a three-dimensional model, the three-dimensional model including geometric structure and landmark building information; Acquire real-time radar data and vehicle data, compare them with the three-dimensional model, obtain difference points, which are differences from the three-dimensional model, determine the authenticity of the difference points, and update the three-dimensional model according to the difference points; Acquire first real-time radar data and vehicle data, acquire calibration parameters, the calibration parameters include internal parameters and external parameters, compare the calibration parameters with the three-dimensional model, acquire comparison results, determine a first difference point according to the comparison results, acquire second real-time radar data and vehicle data, compare the second real-time radar data and vehicle data with the three-dimensional model, acquire a second difference point, compare the first difference point with the second difference point, determine the authenticity of the difference point, and update the three-dimensional model according to the difference point; Determine whether a traffic jam occurs according to the real-time vehicle data, and compare the real-time vehicle data with the three-dimensional model to determine whether there is a new route, wherein the new route is a route not displayed in the three-dimensional model; Specifically, it includes: obtaining real-time radar data and vehicle data, determining vehicle speed and vehicle trajectory through real-time radar data and vehicle data; comparing vehicle speed and vehicle trajectory with preset values, and determining whether a traffic jam occurs based on the comparison results; obtaining vehicle trajectory, comparing vehicle trajectory with the three-dimensional model, determining whether it is a new route, and if determined to be a new route, uploading it to the three-dimensional model.

2. A map generation method based on an intelligent connected vehicle according to claim 1, characterized in that: The step of acquiring radar data according to the vehicle authority, the radar data including laser radar data and optical radar data, and converting the radar data into a three-dimensional model, the three-dimensional model including geometric structure and landmark building information, specifically includes: Obtain the lidar data and optical radar data according to the vehicle authority, and pre-process the lidar data and optical radar data; Acquire the preprocessed first data, identify features in the first data, the features including geographic shapes and building images, and determine a two-dimensional image according to the features; A two-dimensional image is acquired, and texture mapping is performed on the two-dimensional image. The texture mapping is used to map the panoramic image generated by image stitching to a three-dimensional point cloud, and a three-dimensional model is generated according to the three-dimensional point cloud.

3. A map generation method based on an intelligent connected vehicle according to claim 1, characterized in that: The vehicle data includes GPS data, vehicle status data and inertial measurement unit data.

4. A map generation system based on intelligent connected vehicles, characterized in that: The system comprises: The permission module obtains the car permissions, including the laser radar permissions and the optical radar permissions, and obtains the car data; A three-dimensional model module, which obtains radar data according to the vehicle authority, the radar data including laser radar data and optical radar data, and converts the radar data into a three-dimensional model, the three-dimensional model including geometric structure and landmark building information; An update module obtains real-time radar data and vehicle data, compares them with the three-dimensional model, obtains difference points, which are differences from the three-dimensional model, determines the authenticity of the difference points, and updates the three-dimensional model according to the difference points; A supplementary module determines whether a traffic jam occurs according to the real-time vehicle data, compares the real-time vehicle data with the three-dimensional model, and determines whether there is a new route, wherein the new route is a route that is not displayed in the three-dimensional model; The supplementary module specifically includes: a determination unit, which obtains real-time radar data and vehicle data, and determines the vehicle speed and vehicle trajectory through the real-time radar data and vehicle data; The traffic jam unit compares the vehicle speed and vehicle trajectory with the preset values ​​and determines whether a traffic jam occurs based on the comparison results; The supplementary unit obtains the vehicle trajectory, compares the vehicle trajectory with the three-dimensional model, determines whether it is a new route, and if it is determined to be a new route, uploads it to the three-dimensional model.

5. A map generation system based on intelligent networked vehicles according to claim 4, characterized in that: The three-dimensional model module includes: A preprocessing unit, which obtains the laser radar data and the optical radar data according to the vehicle authority, and preprocesses the laser radar data and the optical radar data; A first data unit, which obtains preprocessed first data, identifies features in the first data, wherein the features include geographic shapes and building images, and determines a two-dimensional image according to the features; The three-dimensional model unit acquires a two-dimensional image and performs texture mapping on the two-dimensional image. The texture mapping is used to map a panoramic image generated by image stitching to a three-dimensional point cloud, and generate a three-dimensional model according to the three-dimensional point cloud.

6. A map generation system based on intelligent networked vehicles according to claim 5, characterized in that: The update module includes: A comparison unit, which obtains the first real-time radar data and the vehicle data, obtains calibration parameters, the calibration parameters include internal parameters and external parameters, and compares the calibration parameters with the three-dimensional model; A difference point unit, which obtains a comparison result, determines a first difference point through the comparison result, obtains second real-time radar data and vehicle data, compares the second real-time radar data and vehicle data with the three-dimensional model, and obtains a second difference point; The updating unit compares the first difference point with the second difference point, determines the authenticity of the difference point, and updates the three-dimensional model according to the difference point.

7. A map generation system based on intelligent networked vehicles according to claim 6, characterized in that: The vehicle data includes GPS data, vehicle status data and inertial measurement unit data.

Images