High-definition map generation method and system based on point cloud and multi-objective optimization

Abstract

The invention discloses a high-definition map generation method and system based on point cloud and multi-objective optimization, and the method comprises the steps: obtaining sensor data, carrying out the data preprocessing of the sensor data, so as to generate a 3D point cloud image, and carrying out the semantic tag recognition, semantic segmentation and probabilistic reasoning, so as to obtain a semantic tag; the method comprises the following steps: optimizing a 3D point cloud image by using a probabilistic algorithm, carrying out semantic mapping to obtain topological information corresponding to the 3D point cloud image, fusing a semantic tag and the topological information to generate a high-definition map, and planning a safe and efficient path for a vehicle according to the generated high-definition map to complete the generation of the high-definition map. A mobile L i DAR system and probability-based multi-objective optimization are combined, a high-definition map is generated in a complex urban environment, a highly accurate and dense map is realized, key factors such as autonomous ability, rapid data acquisition, minimum error and dense point cloud are considered, and the accuracy of the map is improved. The integration of the multi-objective random model improves the reliability of map generation by incorporating various risk metrics.

Description

Technical Field

[0001] The present invention belongs to the technical field of high-definition map processing, and in particular relates to a method and system for generating high-definition maps based on point cloud and multi-objective optimization. Background Art

[0002] As the demand for accurate and reliable maps for autonomous vehicle development continues to increase, high-definition (HD) maps play a vital role in enabling autonomous vehicles to understand the static elements present in the scene. Due to the characteristics and structure of the data contained in HD maps, much of this map data is no longer useful as roads are built or managed differently. The characteristics and structure of the maps are like centimeter-level descriptions and interpretations of traffic systems, road markings, intersections, traffic lights and signs, traffic signals, and sometimes even speed limits. HD maps provide critical details for autonomous vehicles to understand the scene and the deterministic components of the image. Previous approaches to HD map generation have mainly focused on extracting semantics and attributes from the data. The development of automated systems that can perform this process has the potential to enhance the HD map generation process, reduce labor costs, and further improve driving safety. As autonomous vehicles rely heavily on HD maps, these maps have become an indispensable source of information, providing critical details about the road environment to ensure safe navigation. However, producing HD maps remains a challenging and time-consuming task. In recent years, mobile LiDAR systems have emerged as a promising solution for fast and accurate 3D mapping. They are able to capture large point clouds while driving, thereby generating high-density, high-precision maps.

[0003] In addition to capturing high-density, high-precision maps, mobile LiDAR systems also play a vital role in street tree detection, which is a key component for high-definition map generation. Existing filters face challenges in filtering large-scale and complex urban environments. An adaptive filtering method based on global energy minimization of object primitives is proposed to address the above filtering challenges, which outperforms classical filtering methods. Similarly, extracting street trees from mobile LiDAR point clouds also poses challenges. The method based on spatial geometric features overcomes these challenges, extracts tree trunk points, improves robustness and accuracy, and can successfully separate adjacent trees. Validation on four datasets confirms the effectiveness of the proposed method in achieving individual tree separation and balancing errors.

[0004] Unlike traditional 2D maps, HD maps are created by leveraging mobile laser scanning (MLS), a technology for capturing 3D point cloud data with an absolute positioning accuracy of between 10 and 20 centimeters. MLS systems collect millions of 3D point cloud data at extremely fast speeds in complex urban environments, and these point clouds can display detailed 3D road network topology. Autonomous vehicles (AVs) can benefit greatly from using high-precision HD maps that exceed the capabilities of on-board sensors. Typically, obstacles such as corners, ramps, and moving vehicles, pedestrians, and cyclists limit the detection range of sensors. By combining real-time sensor data with detailed information from HD maps, vehicles can proactively predict and adapt to dynamic elements in the environment. The integration of HD maps and probability-based multi-objective optimization can significantly improve the accuracy, efficiency, and overall effectiveness of autonomous navigation systems.

[0005] However, autonomous vehicles can rely on complementary information from predefined HD maps to extend their sensing capabilities beyond the range of onboard sensors, which enables them to anticipate obstacles and sudden curves ahead, thus facilitating rapid decision-making. The integration of real-time sensory detection with prior map data enriches the localization process, transforming sensor-based environmental detection into simpler map-based operations. Despite the advantages of HD maps in assisting localization services, they still face various challenges and problems. MLS systems are a good way to obtain accurate and clear 3D HD maps. Processing point clouds is not simple due to their large data volume, inherent noise, computational complexity, and geometric distortion. Techniques such as stereo cameras, sensor fusion, machine learning, radar-based mapping, crowdsourcing, and hybrid mapping provide opportunities to improve the accuracy, timeliness, and completeness of HD maps for autonomous vehicles. Combining these techniques with the proposed LiDAR-based approach may result in a more comprehensive and robust HD map generation method. In addition to mobile LiDAR systems, alternative solutions such as aerial LiDAR or photogrammetry methods can also be explored as potential substitutes for MLS data in HD maps for autonomous vehicles. Summary of the invention

[0006] In view of this, this invention focuses on two fundamental aspects of lidar-based mapping: high-definition maps and probability-based multi-objective optimization. By examining existing research and analyzing the interactions between these elements, this invention aims to bridge the knowledge gap, exploit synergies, and propose a comprehensive approach to enhance map generation. This invention can provide a deeper understanding of the complexities of autonomous vehicle maps, ultimately helping to advance this transformative technology. While high-definition maps are critical to improving the capabilities of autonomous vehicles, it must be emphasized that these cars rely on a comprehensive sensor suite for real-time data collection. High-definition maps complement the information collected by on-board sensors, and are specifically implemented using the following technical solutions.

[0007] In a first aspect, the present invention provides a method for generating a high-definition map based on point cloud and multi-objective optimization, comprising the following steps:

[0008] Acquire sensor data, wherein the sensor data includes point cloud data collected by mobile L IDAR, camera, radar and GPS;

[0009] Performing data preprocessing on the sensor data to generate a 3D point cloud image, and performing semantic label recognition, semantic segmentation and probabilistic reasoning on the 3D point cloud image to obtain a semantic label of the 3D point cloud image;

[0010] Optimizing the 3D point cloud image using a probabilistic algorithm and performing semantic mapping on the 3D point cloud image to obtain topological information corresponding to the 3D point cloud image;

[0011] fusing the semantic labels with the topological information to generate a high-definition map, wherein the high-definition map combines the identified semantic labels with the overall structure and layout of the environment;

[0012] Based on the generated HD map, a safe and efficient path is planned for the vehicle to complete the HD map generation.

[0013] In a second aspect, the present invention further provides a high-definition map generation system based on point cloud and multi-objective optimization, comprising:

[0014] A data acquisition module, used to acquire sensor data, wherein the sensor data includes point cloud data collected by mobile L IDAR, camera, radar and GPS;

[0015] A data preprocessing module is used to perform data preprocessing on the sensor data to generate a 3D point cloud image, and perform semantic label recognition, semantic segmentation and probabilistic reasoning on the 3D point cloud image to obtain a semantic label of the 3D point cloud image;

[0016] An information optimization module, used to optimize the 3D point cloud image using a probabilistic algorithm and perform semantic mapping on the 3D point cloud image to obtain topological information corresponding to the 3D point cloud image;

[0017] an information fusion module, for fusing the semantic labels with the topological information to generate a high-definition map, wherein the high-definition map combines the identified semantic labels with the overall structure and layout of the environment;

[0018] The map generation module is used to plan a safe and efficient path for the vehicle based on the generated high-definition map to complete the high-definition map generation.

[0019] The present invention provides a method and system for generating a high-definition map based on point cloud and multi-objective optimization. The method obtains sensor data, performs data preprocessing on the sensor data to generate a 3D point cloud map, performs semantic label recognition, semantic segmentation and probabilistic reasoning on the 3D point cloud map to obtain a semantic label of the 3D point cloud map, optimizes the 3D point cloud map using a probabilistic algorithm and performs semantic mapping on the 3D point cloud map to obtain topological information corresponding to the 3D point cloud map, fuses the semantic label with the topological information to generate a high-definition map, plans a safe and efficient path for a vehicle according to the generated high-definition map to complete the generation of the high-definition map, combines a mobile LiDAR system with probability-based multi-objective optimization, generates a high-definition map in a complex urban environment, and realizes a highly accurate and dense map. At the same time, key factors such as autonomous capability, fast data acquisition, minimum error and dense point cloud are considered. The integration of multi-objective random models improves the reliability of generated maps by combining various risk metrics. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flow chart of a method for generating a high-definition map based on point cloud and multi-objective optimization provided by the present invention;

[0022] Figure 2 A flowchart of probabilistic semantic mapping using the Pareto optimal method provided by the present invention;

[0023] Figure 3 A flowchart of probabilistic semantic mapping using the Pareto optimal method provided by the present invention;

[0024] Figure 4 A flowchart of the asymptotic sampling algorithm provided by the present invention;

[0025] Figure 5 A flowchart of the extrapolation process of the asymptotic sampling algorithm provided by the present invention;

[0026] Figure 6 A flowchart of the extrapolation process of the asymptotic sampling algorithm provided by the present invention;

[0027] Figure 7 A schematic diagram of the centralized module control parameters and position offset of the first pass reliability index provided by the present invention;

[0028] Figure 8A schematic diagram of the position offset of the allowable domain of the centralized module control parameters and the maximum performance problem provided by the present invention;

[0029] Fig. 9 A schematic diagram of a map generated in a local map by the algorithm provided by the present invention;

[0030] Fig.10 The map generated by the algorithm provided by the present invention is located at Fig. 9 Schematic representation of the same area in a global map;

[0031] Fig.11 The effect diagram of the laser radar point cloud map provided by the present invention;

[0032] Fig.12 A test chart for simulating data collection provided by the present invention;

[0033] Fig.13 A probability mixture graph provided by the present invention;

[0034] Fig.14 This is a structural block diagram of the high-definition map generation system based on point cloud and multi-objective optimization provided by the present invention. DETAILED DESCRIPTION

[0035] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0036] See also Figure 1 The present invention provides a method for generating a high-definition map based on point cloud and multi-objective optimization, comprising the following steps:

[0037] S1: Acquire sensor data, wherein the sensor data includes point cloud data collected by mobile LIDAR, camera, radar and GPS;

[0038] S2: performing data preprocessing on the sensor data to generate a 3D point cloud image, and performing semantic label recognition, semantic segmentation and probabilistic reasoning on the 3D point cloud image to obtain a semantic label of the 3D point cloud image;

[0039] S3: Optimizing the 3D point cloud image using a probabilistic algorithm and performing semantic mapping on the 3D point cloud image to obtain topological information corresponding to the 3D point cloud image;

[0040] S4: fusing the semantic labels with the topological information to generate a high-definition map, wherein the high-definition map combines the identified semantic labels with the overall structure and layout of the environment;

[0041] S5: Plan a safe and efficient path for the vehicle based on the generated HD map to complete the HD map generation.

[0042] In this embodiment, probability-based multi-objective optimization has become a widely used method for optimizing complex systems, including systems related to light detection and ranging (LiDAR) technology. LiDAR is a popular remote sensing method used for vegetation mapping, terrain analysis, and even for self-driving cars. However, optimizing LiDAR systems is complicated by the need to consider multiple, often competing objectives such as accuracy, resolution, and cost. Therefore, probability-based multi-objective optimization provides a promising solution to identify the trade-offs between these objectives and find optimal solutions that meet multiple criteria simultaneously. This approach considers the uncertainties in LiDAR observations and model parameters, ultimately improving the design and performance of LiDAR systems. There is increasing interest in optimization under unpredictability and uncertainty. The presence of uncertainty in optimization models requires the management of potential consequences to control the variability of objective functions in uncertain specifications and parameter space. To fill these gaps, this paper focuses on the use of dense MLS point maps and semantically labeled image datasets generated from MLS point clouds. Models are trained on publicly available datasets to generate accurate probabilistic semantic maps (i.e., reliable road labels, lane markings, crosswalks, and sidewalks) for urban driving scenarios. Uncertainty simulation was incorporated through the confusion matrix formulation of the semantic segmentation network to create a bird's-eye view (BEV) semantic map of the environment. In addition, lane marking segmentation accuracy on the road was improved by adopting a probabilistic Pareto-type optimization method and fusing lidar intensity. Next, a multi-objective stochastic model with a range of risk metrics was developed; finally, the generated solutions were evaluated using Pareto optimization, which identifies higher performing solutions in a parameter space with unknown values.

[0043] It should be noted that probability-based multi-objective optimization has been introduced as a technique to improve the quality of maps generated by mobile LiDAR. By considering multiple objectives such as accuracy, density, and efficiency, probability-based multi-objective optimization can provide a comprehensive and adaptable solution for creating high-definition maps. A new method for creating HD maps using mobile LiDAR point clouds and probability-based multi-objective optimization is proposed, and a complete framework is proposed, which combines the advantages of mobile LiLiDAR with probability-based objective optimization to generate high-precision and high-density maps with low computational cost. To evaluate the effectiveness of the method, multiple datasets are evaluated and compared with existing methods. The overall approach of mobile LiDAR and probability-based multi-objective optimization is used to make a significant contribution to HD maps. By combining these advantages, the proposed method has the potential to greatly improve the effectiveness and efficiency of creating HD maps for autonomous vehicles. In addition, the principles of probability-based multi-objective optimization in the context of LiDAR technology are explored, including its potential benefits and challenges.

[0044] Among them, the HD map generation process of the present invention mainly includes extracting geometric parameters, analyzing interface changes, establishing correlations between closed points, utilizing object attributes and features, adopting pattern recognition and feature learning techniques, and establishing interconnections. Point features can be derived from different levels and representative point selections, point features in various measurements and levels can be derived, and the input scene is represented to help reduce computational complexity and cost. A deep hierarchical framework is constructed to meet the above goals, and it is described how the method uses point cloud data to identify the semantic labels of objects and integrates them with topological information to generate a semantic graph. To illustrate the process of creating a block diagram flow chart of HD maps using mobile LiDAR point clouds and probability-based multi-objective optimization in urban autonomous driving, we outline the following steps: (1) Acquire sensor data, (2) Process the data using a probabilistic algorithm to generate a 3D point cloud map. (3) Identify the semantic labels of objects, (4) Optimize the trade-offs between mapping objectives. (5) Integrate the semantic labels with topological information to generate a semantic graph, (6) Use the graph to plan a safe and efficient path for the vehicle. (7) Continuously update the map with new data, and (8) Repeat the process over time to improve accuracy, Figure 2 The overall flow chart of probabilistic semantic mapping using the Pareto optimal approach is shown.

[0045] Figure 2 The flowchart in visually represents the order of these steps in the method of probabilistic semantic mapping using the Pareto optimal approach. It corresponds to the steps described below, as follows:

[0046] Acquire sensor data: This corresponds to the initial step in the flowchart of acquiring sensor data (including mobile LiDAR point clouds). The sensor data used in this study mainly consists of mobile LiDAR (Light Detection and Ranging) point clouds. These point clouds are captured using a mobile LiDAR sensor, which uses laser technology to measure distances and generate a 3D representation of the surrounding environment;

[0047] Processing the data and generating a 3D point cloud: This step is consistent with the description of the method, which involves extracting geometric parameters, interface variation analysis, and closure point correlation to generate a 3D point cloud;

[0048] Identifying semantic labels and optimizing mapping objectives: This step corresponds to the process of identifying semantic labels for objects and optimizing the trade-off between mapping objectives, as described in the method. This involves exploiting object attributes, pattern recognition, and feature learning;

[0049] Integrate semantic labels and generate semantic map: This method integrates semantic labels with topological information to generate a semantic map. This map combines the identified object labels with the overall structure and layout of the environment;

[0050] Using maps for path planning: The generated semantic map is used to plan a safe and efficient path for the autonomous vehicle. It provides the necessary information about the location of objects and their semantic labels to ensure obstacle avoidance and efficient navigation;

[0051] Continuously updating maps: Maps are constantly updated with new sensor data to improve their accuracy. As the vehicle moves and collects new data, the map updates to reflect any changes in the environment.

[0052] Among them, Data collection means data collection, acquire sensor data means acquiring sensor data, data preprocessing with probabilistic algorithms means data preprocessing based on probabilistic algorithms, Generate 3D point cloud map means generating a 3D point cloud map, identify semantic labels means identifying semantic labels, Semantic segmentation means semantic segmentation, probabilistic reasoning means probabilistic reasoning, Optimize with Pareto Approach means optimizing with Pareto approach, Semantic Mapping means semantic mapping, integrate semantic labels with topological info means integrating semantic labels with topological info, Plan safe and efficient paths means planning safe and efficient paths, Update map with realtime feedback means updating the map with real-time feedback, Refine and improve map accuracy means optimizing and improving map accuracy, and control and repeat process means controlling and repeating process.

[0053] It should be understood that, unlike the prior art, this study focuses on the use of mobile LiDAR systems to capture high-density point clouds in complex urban environments and introduces a comprehensive framework that combines the advantages of mobile LiDAR and probability-based multi-objective optimization to enhance map creation, which has several advantages, including the integration of mobile LiDAR technology and probability-based multi-objective optimization to provide highly accurate and dense HD maps for autonomous vehicles. The effectiveness and efficiency of map generation are improved by considering key factors such as autonomous capabilities, fast data acquisition, minimum error, and dense point clouds. In addition, the proposed method combines a new multi-objective stochastic model that integrates various risk metrics to improve the reliability of generated maps. The present invention aims to make a significant contribution to the field of autonomous vehicle mapping by focusing on the integration of high-definition (HD) maps and probability-based multi-objective optimization. By deeply studying the interaction between these elements, a comprehensive method is proposed for advanced map generation. Specifically, the present invention objectives include improving the accuracy, efficiency, and overall effectiveness of HD map creation in complex urban environments using mobile LiDAR systems and probability-based multi-objective optimization. We introduce a comprehensive framework that not only captures high-density point clouds, but also combines a new multi-objective stochastic model and various risk metrics to improve the reliability of maps. We stand out by significantly improving map accuracy, including 11% improvement for roads, 8% improvement for crosswalks, and a substantial 30% improvement for lane markings. These results highlight the effectiveness of our approach and provide valuable insights for advancing the application of probability-based multi-objective optimization in lidar-based HD map generation.

[0054] Specifically, the sensor data is preprocessed to generate a 3D point cloud image, including:

[0055] Using a multi-objective stochastic model to identify geodesic correlations and features between adjacent points of the 3D point cloud in the 3D point cloud image;

[0056] Based on 3D point cloud The Euclidean nearest neighbor graph G = (V, E) is constructed, and the vertices and edges of the graph G are represented by N = {1, 2, ..., n}, Indicates that the neighbor set of each vertex x is represented by y∈N(x), and introduces a set of input vertex features h={h1,h2,...,h N}, where h N ∈R F Represents each graph vertex Unique features, derived from the 3D point cloud P, each vertex in the graph G is assigned a feature represented as h N , the feature dimension of each vertex is given by F, and the output expression of the HD map corresponding to each vertex is:

[0057] h′ y =max(g θ (G(u(x,y))·h y +h y )), y∈N(x) (1)

[0058] Where G(·) is a geometric mapping function R 3 →R, G(·) maps the local Euclidean coordinates u(x, y) = u(y) - u(x) between each vertex and its neighbors to geometric parameters by multiplying G(u(x, y)) by the feature h y To obtain the trainable feature mapping function R F →R K , and multiply the resulting product with Connection, where Represents the maximum aggregation function;

[0059] Calculate the trainable feature mapping function, the corresponding expression is:

[0060]

[0061] in, is a 1×1 learnable parameter, G(u(x, y)) can approximate and estimate various geodesic values ​​of each vertex x using its neighbor set y∈N(x) by modifying the given learnable parameter;

[0062] Using multi-layer perception MLP as the feature mapping function, Indicates that the input feature h y , feature h y The combination of the product with the geometric features G(u(x, y)) is mapped to various feature dimensions R F →R K , use the maximum aggregation algorithm to learn the new features h y Aggregation is performed to determine neighbor features of the 3D point cloud.

[0063] In this embodiment, semantic label recognition, semantic segmentation and probabilistic reasoning are performed on the 3D point cloud image to obtain a semantic label of the 3D point cloud image, including:

[0064] Use PointNet to extract object features of 3D point cloud images, where the object features have coordinates r i =[x i ;f i ]、x i ∈R 3 and f i ∈R C Representative points

[0065] The representative points of the object features Offset adjustment is performed to make each representative point centered around the shape and center of the object in the point cloud data, where the representative point The characteristic f i ∈R C As input, the 3D position offset Δx in the Euclidean domain i ∈R 3 and characteristic shift Δf i ∈R C As output,

[0066] From the representative point r i Start offset adjustment to get c i =[y i ; g i ], where y i =x i +Δx i and g i =f i +Δf i , the estimated 3D offset Δx i It is controlled by the following loss function:

[0067]

[0068] Among them, 1[r i on the object] represents the representative point r i Whether it is on the surface of the object, N pos is the total number of representative points on the surface of the object, Δx i Indicates the position of the representative point x i and the ground truth displacement between the center of the object bounding box, and the scale control parameter γ is set to 0.1.

[0069] It should be noted that the identification of correlations and features between neighboring points in a point cloud aims to develop a stochastic model that emphasizes simple geometric properties, reduces parameters, and exploits internal features of objects. A comprehensive approach to extracting meaningful geometric information from point clouds aims to achieve accurate mapping for autonomous driving by leveraging graph-based techniques and geometric and feature mapping functions. It also emphasizes the potential of sensor fusion, where data from LiDAR, cameras, radar, and GPS can be combined with data from a variety of sensors to improve segmentation and mapping accuracy. HD maps attempt to identify geodesic correlations and features between nearby neighbors of each point, but provide a large number of parameters. In addition, most points and their local neighbors in indoor scenes lie on planes or surfaces with simple regular shapes, which can be characterized by simplified parameterized geometric mathematical functions. Therefore, a new multi-objective stochastic model is developed that focuses on regular and simple geometric properties to reduce the number of parameters and exploits geodesic internal features and features of environmental objects.

[0070] A set of input vertex features are introduced to construct a graph from a 3D point cloud according to its respective Euclidean nearest neighbors. It is worth noting that the word “feature” refers to a specific characteristic or property associated with a single vertex in the graph, which comes from a 3D point cloud. Each vertex in the graph is assigned a feature, which is unique to that vertex. These features represent specific properties or qualities of the vertex and are used as input in subsequent computations and mapping processes. The local Euclidean coordinates between each vertex and the Euclidean coordinates of its neighbors are mapped to a geometric parameter. Trainable feature mapping function: is then obtained by multiplying the features, and the resulting product is concatenated with a maximum aggregation function and represents the maximum aggregation function. The process involves estimating the unknown values ​​of the vertex set using a set of discretized Taylor Gaussian filters, capturing the geometric spatial features within a defined region, as extensively discussed in TGConv, which stands for Taylor Gaussians filters, a component of the proposed structural model, and these filters are described as consisting of geometric features based on intrinsic coordinates and characterized by a series of Gaussian weighted Taylor kernel functions as well as local neighbor point features.

[0071] Optionally, optimizing the 3D point cloud image using a probabilistic algorithm and performing semantic mapping on the 3D point cloud image to obtain topological information corresponding to the 3D point cloud image includes:

[0072] The default input is the representative point c i The output is the module (r i ,y i , g i ) generates a 3D position offset of a representative point with random variables x1,…,x in n-dimensional space. n The failure probability PF for:

[0073]

[0074] in, Denotes the joint probability function of random variables, D F Represents the failure domain of the n-dimensional random variable space. The limit state function g(X) is determined based on the failure probability and failure domain. The reliability index β corresponding to the limit state function * The expression is:

[0075] β * =φ -1 (1-P F ) (5)

[0076] Among them, Φ -1 (·) represents the inverse normalized Gaussian distribution function. Preset position offset X i is a Gaussian random variable with a mean and standard deviation σ of zero. If the limit state function g(X) is highly nonlinear, g < 0 can be used to indicate failure. If the limit state function has the same distribution X k Gaussian variable, then k = 1, 2…, n is σ;

[0077] The Monte Carlo-based asymptotic sampling algorithm is used to adjust the standard deviation in the n-dimensional Gaussian space to accurately control the failure probability. The asymptotic sampling algorithm is used to determine the reliability index β * , reliability index β * The dependence on the standard deviation of the random variable is expressed by function (6):

[0078]

[0079] Among them, when f→0 satisfies the necessary asymptotic linear behavior, it can be satisfied by focusing on σ→0; the coefficients A and B are defined using regression analysis of least squares fitting, based on the corresponding β values ​​of different support points satisfying f<1 * The Monte Carlo estimate of is adjusted according to the scaled reliability index provided by expression (7):

[0080]

[0081] In this embodiment, by modifying the given learnable parameters, various geodesic values ​​of each vertex can be approximated and estimated using its neighboring set. Considering that multi-layer perception (MLP) may approximate any continuous function while maintaining the same weight distribution as traditional convolution. Using a shared MLP as our feature mapping function, the initial input features and the combination multiplied by the geometric features are mapped to various feature dimensions: the learned new features are then aggregated using the maximum aggregation method, which can utilize the most useful features and iteratively explore the relevant neighbor features. However, due to the high point cloud density in the sampling layer, the identified local neighbors of each point form part of the object surface. In order to extract object properties, including geometric patterns, surface variability, closed positions, and point correlations, we use GeoConv in these sampling layers. When points are carefully collected and sampled, object properties such as location or layout are improved, while geometric properties such as size, shape, and shape between retrieved neighbors are reduced. Since GeoConv cannot retrieve object features from all layers, it improves detection performance. Therefore, in the following sampling layers, PointNet is used to extract object features and characteristics, and PointNet is used to combine and interpolate various features in subsequent sampling layers. A series of representative points with and coordinates is the output of the program.

[0082] It should be noted that while LiDAR is used to create semantic segmentation of BEVLiDAR projections, other sensor data such as cameras, radar, and GPS can also be used in combination to produce more accurate and precise segmentations. Multi-sensor fusion is a typical autonomous driving approach, which is processed by deep learning-based algorithms to provide semantic information for maps. This approach helps overcome the limitations of individual sensors and ensure reliable and precise mapping results. Therefore, although LiDAR point clouds are used, other sensors are used to obtain more reliable and accurate outputs. The adopted structural model integrates various deep learning techniques, each of which plays a different role. The architecture starts with a Taylor-Gaussian filter, which uses a combination of Taylor expansion and Gaussian smoothing to enhance input data or perform feature extraction. The feature mapping function uses MLP, a neural network architecture with multiple layers of interconnected nodes, to learn a complex mapping from input to desired features. In addition, GeoConv is used as a convolutional layer to operate on geometric structures such as point clouds or graphs to extract features from geometric data. Finally, PointNet, a deep learning architecture specifically designed for processing point cloud data, is used in the proposed network for object features and feature extraction. PointNet is able to capture local and global information from point clouds without relying on a predefined structure. These techniques together constitute the proposed structural model, which provides a comprehensive approach for deep learning based feature extraction.

[0083] Optionally, the Monte Carlo based asymptotic sampling algorithm includes:

[0084] Select the initial factor f0, the useful set is f0 = 1, the number of samples in the failure domain N0, and the reliability index β * The accuracy of F Reduce the coefficient f and iterate until an acceptable number of N is obtained F ;

[0085] After identifying the parameters, regression analysis is performed on the data pairs to obtain the functional relationship β * (f), β * (f) Extrapolate to f = 1 to obtain the reliability index.

[0086] In this embodiment, independent and identically distributed random variables W are used. k Extract the object features of the point cloud data, preset to keep a constant value within the discrete time interval Δt, the random variable W k The variance and mean of the feature has zero mean and variance Object feature F x(max) The intensity can be expressed in terms of variance and time interval:

[0087]

[0088] The independent and identically distributed value sequence can be represented as the object surface point of the object feature, and the object surface point has a random variable U with unit standard deviation k The random variable W k The expression is:

[0089]

[0090] It should be noted that the object surface points are relocated to the center of the object, and the background points of the object features remain sparsely distributed, including:

[0091] According to the representative point position in 3D Euclidean space Select and sample a subset of K points, using the index of K points to obtain To determine the offset representative point like By Find nearby points in To cluster N groups of points, the matching features of each group point are specified as

[0092] A shared MLP is used to map the surrounding points of each object feature, and the feature aggregation uses the maximum operation. The corresponding expression is:

[0093]

[0094] in, is the relationship between adjacent positions and points with respect to F k ∈R (3+C) Comparing coordinates, most of the surrounding points are from the same object, and the aggregated features show the intra-object properties.

[0095] Since depth sensors mainly capture surface points of objects, there are few or no points around the center of shapes and objects. Existing point-based networks have difficulty in obtaining meaningful context around the center of elements and objects. This problem is overcome by generating new points (votes) at the central core of objects. Voting is performed using features and representative point features. Afterwards, the collected votes can be systematically organized using trainable modules to generate proposals that meet the context. A voting loss function is designed to estimate the displacement and bias of the votes using Euclidean distance. The network is good at identifying and discovering 3D objects. The voting loss function does not take into account the scaling issue. Larger objects may be more effectively returned in time than smaller elements and objects. To improve this, the centralization module is built following VoteNet, but a scaling control parameter is introduced when defining the loss function. Based on a series of representative points, the centralization module makes an offset adjustment from each representative point to its center. The module consists of a shared MLP module with three fully connected layers, ReLU and batch and subsampling normalization. A shared MLP module refers to a multi-layer perceptron (MLP) module that is shared between multiple components or parts of a neural network. It is used to offset each representative point so that the points can be concentrated around the center of the shape and object in the point cloud data. The features of the representative points are taken as input, and the 3D position offset and feature offset in the Euclidean domain are taken as output. Therefore, this module starts with the representative point, where and.

[0096] Optionally, construct a 3D bounding box (x, y, z, h, w, l, θ), where (x, y, z) are the coordinates of the center of the object, (h, w, l) are the height, width, and length of the object, and θ is the orientation of the object;

[0097] Given an object feature F k , and use three layers with full connectivity to estimate the bounding box parameters B k,1 (x k,1 ,y k,1 ,z k,1 ,h k,1 ,w k,1 ,l k,1 ,θ k,1 ), using the projected bounding box center coordinates and position (x k,1 ,y k,1 ,z k,1), identify the neighbors of each object, and the relationship between the object and its surrounding and adjacent objects is represented as a region-to-region optimal path and an undirected graph, using the symbol Indicates; among them, and denote vertices and edges respectively;

[0098] Determine the relationship parameter α i，k ∈R 1×(3+C) (i=1,...,S) and the relationship feature F of the object rk , F i，k represents the feature of the i-th adjacent object, and the corresponding expression is:

[0099]

[0100] in, is a trainable parameter with a value of 1×(3+C), related to the feature F rk will be passed to three fully connected layers, designated as B k，2 (x k，2 ,y k，2 , z k，2 ,h k，2 , w k，2 , l k，2 ,θ k，2 ), B k，1 and B k，2 The sum of is taken as the final output of the network.

[0101] In this embodiment, the 3D point cloud image is optimized using a probabilistic algorithm and semantic mapping is performed on the 3D point cloud image to obtain topological information corresponding to the 3D point cloud image, including:

[0102] Then use a set of The representative points and scaling control parameters of the 3D point cloud are optimized.

[0103] By starting from the representative point r i To its center independently generate and expand c i =[y i ; g i ], and uses γ as the proportional control parameter to determine the offset Δf generated from each representative point position i ;

[0104] Calculate the offset Δf i The reliability index r corresponding to the first pass probability i , and correspondingly generate representative y i Values ​​and representative g i value, where r i ,y i , g iThe maximum offset position is fixed at 0.01m.

[0105] In order to accurately extract the scene context around the center of the object while maintaining the integrity of the context, it is necessary to resolve the trade-off through an optimization process. Given that the target points are regarded as non-stationary random processes, the design of the algorithm for extracting the scene context can incorporate appropriate probabilistic features of the dynamic response, such as the first-pass probability of the critical response level. This probabilistic approach enables a more accurate representation of the scene context and improves the reliability of the extraction process. In this case, the input is the features of the representative points (the output is the 3D position offset of the representative points generated by the module).

[0106] The first pass probability in a random point cloud can be effectively evaluated using a common technique called first-order reliability method. To further enhance this method, it has been shown to be beneficial to exploit importance in sampling. The first-order reliability method calculates and defines the failure probability based on generalized linear limit state functions, especially at the most probable failure point. The failure cases and conditions in the traditional Gaussian space are given by the expression where is the most probable failure point with the minimum deviation from the origin. The problem should be defined and developed using optimal solutions and optimization methods. The optimization process may become asymptotically relevant or the global minimum may not converge. This concept is considered for different algorithms, NLPQL, L-BFGS, and CONMIN, which are tested in this application. A considerable number of random variables are required to accurately describe the characteristics of the object. Although efficient and effective convergence is possible, the computational process is long. The estimation of the gradient requires a lot of nonlinear dynamic analysis. It is challenging to apply the first-order reliability method (FORM) to evaluate the safety indicator for the entire variable design space. The computational process, especially in the evaluation and estimation of the gradient, has significant limitations. To overcome this, a new method using Monte Carlo-based asymptotic sampling is used to quickly determine the reliable safety indicator for the first pass. The method is used in n-dimensional Gaussian space, where the variables are all equally spread and relatively distributed. In n-dimensional Gaussian space, the variables are uniformly spread and distributed, and the failure probability can be precisely controlled by adjusting the standard deviation, providing accurate results by fine-tuning parameters such as limit state geometry, sample size, and failure probability. Asymptotic sampling uses extrapolation methods to efficiently determine the reliability index, which addresses the complexity of the method.

[0107] When the necessary asymptotic linear behavior is satisfied by focusing on this point, the coefficients A and B are usually defined using regression analysis with least squares fitting, which is based on Monte Carlo estimation of different values ​​of the support points that satisfy this. A simple rewriting of equation (6) allows the regression analysis to give the same weight to each individual support point (e.g. Figure 3 ). The asymptotic sampling method has developed to its main stage and is Figure 3It is presented in the form of a flowchart in . In the case of the attention algorithm, the initial factors are selected, and since there is no prior knowledge, a useful set is selected. Another important measurable factor is the number of samples present in the fault domain, which affects the accuracy of the safety index. In the fault domain of the main and starting steps, the actual number of samples is usually very limited and small. Therefore, in this case, one should start reducing the factors (or reducing the factors), and iterate this process until an acceptably large value is obtained. In addition, the simulation model and the gradual decline are followed until a sufficient number of pairings are obtained. And was selected as the object of this study. By applying regression to the data pairs after identifying the parameters, the functional relationship is obtained. The safety index is obtained by extrapolation (such as Figure 4 ), Figure 4 The discretization process shown corresponds to the consistent and continuous time of the object extraction cycle.

[0108] The non-stationarity of target points is addressed by exploiting the probabilistic characteristics of dynamic responses, ensuring accurate representation of variability and patterns within the scene. It introduces the concept of first-pass probability and assumes that the target points are Gaussian random variables, evaluating the safety metric through standard deviation. By adopting a Monte Carlo-based asymptotic sampling method, the algorithm evaluates the safety metric across variables and quantifies the relationship between the safety metric and the standard deviation. Overall, this approach captures variability and complexity, improving the accuracy and reliability of scene context extraction while maintaining the integrity of the context.

[0109] The centralized module effectively relocates the object surface points to the center of the object, while the background points remain sparsely distributed. Therefore, it is crucial to consider density fluctuations and select sampling and clustering procedures based on spatial proximity to ensure that the proposals can represent a sufficient number of potential objects. Using Farthest Point Sampling (FPS), a portion of points are selected and sampled according to the typical representative point locations in 3D Euclidean space. The indices of these points are then used to obtain proposals among the moved representative points. Then, for each proposal, we cluster the grouped points by finding nearby points in (if any). Therefore, the matching features of each grouped point are assigned as. Although a narrower radius can include cleaner neighbors (from the same item), the context of the background points is lost. Increasing may corrupt and contaminate the nearby area due to the inclusion of more surrounding objects and cluttered sites. A shared MLP is adopted to perform feature mapping on the surrounding points of each proposal. Feature aggregation utilizes the maximum operation.

[0110] In selective 3D object detection, the features of the object play a key role in accurate recognition. The aforementioned aggregated proposal features capture the intra-object features, especially the features derived from points located on the same object surface. However, there are connections between objects in real-world scenes. In order to enhance the reasoning ability, an object relationship learning module is proposed, which exploits the features of objects in terms of co-occurrence and object location. By considering the nearest neighbor proposals of each proposal, using their relationship features, the neighboring proposals are determined based on the predicted bounding box center location. These neighboring proposals are searched based on the predicted bounding box center location. The present invention proposes a 3D bounding box containing the coordinates of the object center, the object size (height, width, length), and the object orientation. Given an object feature, three layers with full connectivity are used to estimate the bounding box parameters. Using the projected bounding box center coordinates and location, the neighbors of each proposal are identified. The relationship between a proposal and its surrounding and neighboring proposals is represented as a region-to-region optimal path and an undirected graph to determine the relationship parameters and the possible relationship and connection features of the object.

[0111] See also Fig.14 The present invention also provides a high-definition map generation system based on point cloud and multi-objective optimization, comprising:

[0112] A data acquisition module, used to acquire sensor data, wherein the sensor data includes point cloud data collected by mobile LIDAR, camera, radar and GPS;

[0113] A data preprocessing module is used to perform data preprocessing on the sensor data to generate a 3D point cloud image, and perform semantic label recognition, semantic segmentation and probabilistic reasoning on the 3D point cloud image to obtain a semantic label of the 3D point cloud image;

[0114] An information optimization module, used to optimize the 3D point cloud image using a probabilistic algorithm and perform semantic mapping on the 3D point cloud image to obtain topological information corresponding to the 3D point cloud image;

[0115] an information fusion module, for fusing the semantic labels with the topological information to generate a high-definition map, wherein the high-definition map combines the identified semantic labels with the overall structure and layout of the environment;

[0116] The map generation module is used to plan a safe and efficient path for the vehicle based on the generated high-definition map to complete the high-definition map generation.

[0117] In this example, the MLS dataset for Zhenjiang City, Jiangsu Province, eastern China, was collected using the RIEGL VMX-450 system, a mobile laser scanning system widely used for 3D data collection and mapping in a variety of applications. The system focuses on providing precise and accurate point cloud data and is well known for its reliability. Notably, it exhibits an accuracy level of 8 mm, ensuring that the captured measurements are close to the true value. In addition, the 5 mm accuracy can produce highly detailed and consistent point clouds with reduced random errors. The RIEGL VMX-450 has an impressive maximum effective measurement rate of 1.1 million measurements per second, which is achieved by utilizing a dual scanning mechanism for outward and return. This dual scanning approach enables comprehensive data collection and improves the overall qualitative output. In addition, the line scan speed can be adjusted up to 400 lines per second (2x200 lines per second), providing versatility according to specific requirements. To improve the accuracy and geo-referencing of the collected data, the system is supplemented with fixed GPS stations. These stations play a vital role in establishing control points and facilitating the precise positioning and alignment of the scanned data within the geographic coordinate system.

[0118] The MLS system moves at speeds up to 60 km / h, resulting in a point density between 7500 and 8500 points / hour. A total of four MLS test datasets from LAS files were used in our study. Dataset 1 is a short and simple section of about 30 meters long along a straight urban road corridor extending along Xuefu Road. Dataset 1 contains 1827963 points with a point density of 8500 points / . Datasets 2, 3, and 4 cover the other three sections of lengths of 176.5 meters, 175.5 meters, and 176.4 meters, respectively, and contain 6875, 450, 9850, 578 and 8475, 868 points, respectively. The median density is 7500 points / . Dataset 2 is a two-way road with one-way traffic flow, while datasets 3 and 4 are three-way roads with lane restrictions and reduction zones. The four data sets have different types of lane markings, roadside bushes and trees, traffic and warning signs, and buildings. The effectiveness of the proposed method is evaluated using the four datasets and compared with other current methods.

[0119] To derive semantic segmentation from 2D images, the network architecture provided by the DeepLabV3Plus system was used, with a lightweight PointNet based on VoteNet as the backbone of the feature extraction system. It can achieve the same mean intersection over union (mIoU) as other backbone networks (such as ResNet101

[56] ) without requiring as many parameters and faster inference time. The decoder layer also incorporates depthwise separable convolutions to further reduce inference time while maintaining the same efficiency and performance. A semantic segmentation network was developed and trained in the MLS dataset. This dataset provides a large number of pixel-level semantic segmentation photos of autonomous driving scenes and scenes with 66 different types of labels. The labels were divided into 19 groups (e.g. zebra crossing and pedestrian crossing) by eliminating unnecessary labels for driving conditions and environments, such as snow, and combining labels with comparable semantic definitions and meanings. This decision was made because some classes are unlikely to be learned, practiced, and appear in our test simulation environment. Datasets 1 to 4, collected along Xuefu Road, provide a total of 18,000 frames and images for training, with an additional 2,000 frames for verification and validation. During the integration process, we merged terrain, landforms, and vegetation, classified various rider categories into the "human" category, merged the front and rear traffic signs under "traffic signs", included bridge data into the "building" category, and merged various pedestrian crossing types into a single "pedestrian crossing" category. Training data augmentation includes random horizontal flipping with probability 0.5, random resizing in the range of 0.5 to 2, and random cropping. In addition, image normalization is performed using a distribution with mean and standard deviation of (0.485, 0.456, 0.406) and (0.229, 0.224, 0.225) respectively. Despite the extensive data augmentation during training, we did not observe significant performance degradation during testing due to the similarity between the Xuefu Road dataset and the road test scenarios conducted.

[0120] We trained the network for 200 epochs and iterations on 8 2080Ti GPUs using a group size of 16 and synchronized group normalization, utilizing an input image size of 640×640 to develop hyperparameters. The network had an output stride of 8. We applied a polynomial learning rate policy and a stochastic gradient descent (SGD) optimization technique with a base learning rate of 0.005 and a power of 0.9. The decay rates for momentum and weight were 0.9 and , respectively. Regarding how the metric is defined, mI oU is used to measure how well the network is working. The ResNeXt50 mI oU on the validation set is 68.32%. ResNeXt50 performs slightly worse than ResNet101, but uses much less memory (from 370MB to 190MB), which is better for onboard hardware since it has a limited amount of memory. The network takes about 0.2 seconds to perform inference on each image.

[0121] The concentration process is discretized at time intervals of s for a total of two seconds. This indicates that there are 1000 random variables used to indicate the concentration. Asymptotic sampling is used to determine and evaluate the first-pass probabilities, with each sample using five support points based on 100 Monte Carlo samples. Figure 3 An automated algorithmic search strategy is shown that aims to gradually improve the factor until five supporting points with stable and reliable probability predictions are identified. This is because the first-pass probability varies widely and is sensitive to changes in system characteristics and parameters. The asymptotic sampling process typically requires less than a thousand Monte Carlo iterations and sampled data points for one of the three reliability indicators and one point (distributed throughout the design space). To illustrate the impact of different and discrete object locations and offset variations of the points, the range of these two variables is addressed by adopting an experimental approach that relies on Latin hypercube sampling with 100 sampling points.

[0122] On a typical laptop, approximately 300,000 dynamic analyses, each lasting about 0.006 seconds, are required to complete the entire optimization process, which requires about 30 minutes of computing time. Figure 5 The first pass reliability indices corresponding to the maximum performance are shown. The figure depicts how these indices vary with the control parameters of the centralized module and the position shift within the design space. The "centralized module" refers to a key component in the method of the present invention and represents the control mechanism in the control optimization process. Due to the inherent statistical unpredictability and uncertainty associated with Monte Carlo based reliability indices, these reliability indices exhibit considerable fluctuations. To account for these fluctuations, techniques for smoothing the data, such as moving least squares, have been employed. Figure 5 The calculated reliability indices are presented visually and the necessity of using smoothing techniques, in particular the well-established moving least squares method, is emphasized to mitigate the fluctuations observed in this study.

[0123] The smoothed reliability index across the design space is Figure 6 As shown in Figure 2, high-scale control parameters are most beneficial in limiting the maximum position offset, while smaller position offsets help reduce the fluctuation of feature points. The feature position is then evaluated using a threshold. Figure 7 The associated first pass safety index is depicted. It can be observed that a small amount of positional offset can have a negative impact on how feature points respond to the surface points of the object. For example, when the scale control parameter is set to a very low level, the feature tends to take more aggressive actions because the feature points do not have enough positional offset. In summary, Figure 7 The first pass safety index of the feature point represented by is depicted in . The main variable to keep the position deviation of the point to a minimum is the finite small-scale control parameter value. Figures 5 to 7It can be clearly seen from the comparison of the reliability indices in that a small part of the entire design space (narrow beams) provides relatively large values ​​for all values. To better describe, if the values ​​of all safety indices are specified to be greater than 4, the allowed design space domain is constrained, such as Figure 8 shown.

[0124] Pareto optimality methods and Pareto analysis-based management procedures are used in high-definition maps (HD Maps) using mobile LiDAR point clouds in the following ways: (1) Identification of important features: Pareto analysis is used to identify the most critical features of HD maps, such as road geometry, lane markings, and signs, based on the Pareto principle, where 80% of the value comes from 20% of the features. This information can help prioritize mapping work and ensure that resources are allocated appropriately. (2) Optimization: Pareto optimization methods are used to optimize the mapping process by finding the best balance between different objectives such as accuracy, data completeness, and data density. This approach can help design a mapping strategy that achieves the best results while minimizing costs. (3) Quality control: Pareto analysis is used to improve quality control by identifying features with the highest error rates. By identifying and resolving these issues, the overall accuracy of the HD map can be improved. (4) Update cycle planning: Pareto analysis-based management procedures are also used to plan the update cycle of HD maps. This allows us to focus on parts of the map that need to be updated immediately more quickly than less critical parts. Overall, applying Pareto optimality and management procedures based on Pareto analysis in HD maps can help optimize the mapping process, identify key features that require accuracy, improve quality control, and save time and effort while ensuring high accuracy of HD maps.

[0125] By evaluating and determining the relative depth of semantic pixel data for a given semantic image, 3D scenes can be reconfigured with semantic labels. However, this data is often inaccessible. It is important to understand the most obvious features and characteristics to interpolate and estimate from multi-view geometry, which is difficult to do when driving or in complex and difficult lighting situations. LiDAR sensors can easily capture depth information and feature sets of objects. Dedicated underlying geometry can be challenging in real-time environments because they usually have a limited number of optical channels (e.g., 16). The method of the present invention uses centimeter-level positioning to identify and extract discrete dense areas from an already constructed dense point cloud map, which helps to solve this problem. These smaller areas are then hypothesized and projected into the semantic image segmentation to extract and collect depth information.

[0126] Automation is a viable option for creating such dense point maps as it only requires driving through the area once and is therefore less expensive than manual labeling. After localization, the point cloud data is projected into the 2D image space using the intrinsic camera settings and parameters and settings, as well as the relative and comparative transformations between the LiDAR and the camera. Nearest neighbor search is used to semantically associate and relate the point cloud data

[64] . Furthermore, the intrinsic matrix of the camera is determined using the traditional chessboard technique, while the relative transformation between the camera and the LiDAR is evaluated and calculated using the PnP method. It should be mentioned that the LiDAR point cloud map and the semantic segmentation of 2D camera images are two different sets of data that provide unique information and serve different purposes. The LiDAR point cloud provides real-time distance, position, and orientation information of the environment, which can be used for obstacle detection, localization, and mapping. On the other hand, the semantic segmentation of 2D camera images is able to identify and classify objects in the image, which is very useful for object detection, tracking, and scene understanding. Here, they are used together in autonomous driving to provide different data types.

[0127] While point clouds with semantic labels inherently preserve the 3D geometry of the environment, it is important to note that such representations of the scene are still susceptible to noisy sensor estimates and slight changes or fluctuations in the semantic labels. A map with regional or global probability distributions (probabilistic maps) is retained to properly address this issue, where local maps do provide dense semantic clues about the vehicle at close range, while global maps can help automatically create HD maps. The reference frame is the main difference between local and global semantic maps using semantic occupancy grids. Computational juxtaposition and comparison are performed within the globalized frame. The local probability map seems to be a bird's eye view of the vehicle's body frame (rear axle). We make a local map for a particular frame and keep it up to date with the semantic point cloud. The acceptable poses set the origin of that frame. When the difference between the new pose and the previous pose exceeds a certain threshold, a new map is generated and the previous map is modified to compensate for vehicle traffic flow and movement. The local probability map is similar to a bird's eye view centered on the rear axle of the vehicle and is continuously updated based on the semantic point cloud data aligned with the acceptable poses, ensuring accurate representation. When the difference between the new and previous poses exceeds a predefined threshold, a new map is initiated and the previous map is adjusted to accommodate changes in vehicle traffic flow and motion. On the other hand, the global probability map works in a global framework and does not require any map modification or revision.

[0128] Fig. 9 and Fig.10Graphical representations and visible juxtapositions are shown simultaneously, with the top image depicting local frame features and descriptions and the bottom graph and image depicting the global frame representation. The semantic occupancy grid has three dimensions: the semantic class of the scene is associated with each channel. After creating the semantic point cloud, it is projected onto the grid using the x and y components. The semantic labels of the neighboring cells will be assumed to represent the semantic label of the point. Each cell in the grid represents a physical region with dimensions (in meters) of square area, where d is some discrete factor. The probabilistic approach to minimizing the prediction error using semantic and LiDAR intensity information as well as points of interest in the point cloud improves the reliability and applicability of the semantic occupancy grid prediction. The semantic labels for all channels are denoted by the observed semantic labels and the LiDAR intensity is denoted by . Therefore, is determined and estimated based on previous observations, i.e., the probability distribution of the semantic probability is updated using the following procedure, which is based on the Markov assumption and the conditional independence of the observed semantic labels and LiDAR intensity.

[0129] When the normalization factor is displayed, the statistical likelihood and probability of a hypothetical label being identified as a label is represented by a component in the row and column of the 2D matrix used for modeling. It describes the degree of certainty in the prediction so that the probability update is more accurate. The rate is modeled based on how well each class in the scene reflects. The LiDAR sensor intensity provides comprehensive and clear clues about the various materials present in the scene. For example, in Fig.11 In Figure 2, the top image shows a BEV intensity map on a stretch of road. Lane markings can reflect higher intensity light because they are painted white; therefore, they can be identified and distinguished by a threshold. This threshold serves as an initial justification for scene arrangement and may be useful in situations where semantic segmentation cannot accurately capture the true labels due to low light. It is noted that simultaneous localization and mapping (SLAM) and real-time kinematics (RTK-GPS) methods are beneficial for obtaining highly accurate position information during mapping processes for autonomous systems. SLAM uses data from multiple sensors to create an accurate map of the environment and the robot's position, making it applicable to a variety of environments. On the other hand, RTK-GPS uses a base station to transmit correction data to the rover receiver, providing highly accurate positioning in outdoor environments. Therefore, a combination of SLAM and RTK-GPS methods is used here to improve accuracy and reliability, making it the preferred method for navigation in mapping applications.

[0130] A 2.5 km long section of Xuefu Road was chosen to test the effectiveness of our mapping transfer data and generate results. This location contains manual annotations on the HD map. It has been tested in a real-world environment and includes road information and traffic data such as crosswalks, intersections, sidewalks, and lane center marking boundaries. Using these labels and data, we developed a semantic map with a resolution of meters and five channels, including roads, crosswalks, lane markings, vegetation, and sidewalks. HD map labeling is not a simple task. However, it shows how useful it is to be able to automate the entire process. Our evaluation uses mIoU and pixel accuracy as evaluation metrics. Since mIoU measures and evaluates the segmentation memory, recall, and accuracy of our model, it is important to remember that these metrics are affected by the sparsity of the LiDAR point cloud. Therefore, even if the model output is very reliable, it may still contain a large number of unclassified cells (holes). To address this issue, we use a smoothing kernel to interpolate the missing labels in the map. The local dense point maps obtained for all tests in Table 1 were cropped and constrained to have a maximum distance of 15 meters on the longitudinal axis of the vehicle and a maximum range of -10 to 10 meters on the horizontal axis, as longer distances result in significantly worse semantic segmentation performance. The Vanilla and Vanilla Modified methods efficiently and economically create HD maps using existing sensors. On the other hand, the CFN and CFN Modified methods provide higher accuracy, faster processing speed, flexibility, and lower cost in creating HD maps compared to traditional methods. Both methods have their advantages, but the CFN and CFN Modified methods provide a highly accurate, cost-effective, fast and flexible solution for HD map creation, making them an excellent choice for advanced applications such as autonomous driving and intelligent transportation systems.

[0131] Table 1 Quantitative evaluation on training and labeled datasets

[0132]

[0133] Modeling observation uncertainty is a key task in generating high-quality maps. We focus on designing a matrix that models the observation uncertainty of a given semantic label, which can be numerically stable via the logarithmic representation of the elements of M. Two different approaches are considered to explore the best strategy for modeling observation uncertainty. The first approach, called Vanilla, employs a normalization factor μ and a hyperparameter λ, denoted as μ(I+λ1), where I is the matrix of input observations supplemented by a vector of 1s. The second approach, called CFN, involves leveraging the confusion matrix of the semantic segmentation network in the Mapillary validation dataset. During inference time, each cell is assigned the label with the highest probability, and the numerical accuracy of these methods is quantitatively evaluated and compared.

[0134] It is observed that CFN performs significantly better than vanilla, especially in the case of crosswalks and lane markings. This finding highlights the importance of utilizing confusion matrices to model prediction errors in semantic segmentation, ultimately leading to better map generation results. It is noted that modeling the uncertainty in the semantic labels based on the input observations, the parameters used to ensure numerical stability, the resulting values ​​can be scaled and adjusted. This expression is used in map generation to improve the quality of the map by taking into account the uncertainty in the semantic labels. The semantic map development process did not prioritize the evaluation of vegetation and sidewalk classes due to various reasons. These include focusing on developing accurate maps of key features such as roads, crosswalks, and lane markings, sensor limitations, limited use cases, and time constraints. However, future updates may include these classes.

[0135] This comparison demonstrates the benefits of using the network's confusion matrix to model and represent the estimation errors in semantic segmentation, which leads to better map generation results. According to Table 1, the proposed road extraction method has an accuracy of 89.16%, a recall of 88.74%, and a score of 88.95%; for crosswalks, it can reach an accuracy of 80.64%, a recall of 80.24%, and a score of 80.44%; for lane markings, the accuracy is 76.57%, the recall is 76.19%, and the score is 76.38%. It should be noted that all score values ​​in Table 1 have four decimal places to ensure high precision in reporting performance metrics. Score or -score is a key metric for balancing precision and recall, and the inclusion of four decimal places allows for a more detailed and nuanced evaluation of model performance. This precision is critical in academic and research environments to clearly and accurately represent the model's ability to balance false positives and false negatives. The increase in granularity improves the reliability of the results and promotes more precise comparisons and repeatability in the evaluation.

[0136] Initially, intensity data below the normalization threshold were filtered out (see Fig.11 ), this threshold is manually tuned and calibrated for the Velodyne VLP-16 lidar to exploit the fact that different road materials have variable reflectivity. If the labels are projected as lane markings in the semantic mapping step, their log-likelihood can be improved by controlling the factor, which reduces the ability to predict other classes while improving the ability to predict lane markings. Modified Vanilla shows better results than Vanilla in terms of accuracy and IoU of lane markings; however, the number of roads and crosswalks decreases slightly, which shows the advantage of intensity interconnection and integration in predicting and projecting lane markings. But this trend is not constant for Modified CFN compared to CFN. More comprehensive features for modeling lidar intensity may be needed to obtain further improvements. Fig.12The figure shows a global map generated by our improved CFN model for the design validation test area. It also shows how the model can accurately represent static features on the road by zooming in on a portion of the map. The point cloud map is preprocessed and thresholded using point intensity values. Points with intensities below k are eliminated. This threshold makes the road lane markings and crosswalk patterns visible. The dense point cloud map is overlaid with our semantic map. To show the details of the map, a section is zoomed in. Color provides the best view. It is worth mentioning that point cloud data produced in real time using LiDAR is another way to connect semantic images with depth and comprehensive information. Point clouds can be projected into semantic image frames and used to generate semantic maps using similar mapping methods that provide real-time performance and accuracy. In addition, point cloud scans are not sufficient for the 16-channel LiDAR required to generate semantic maps at larger and more diverse distances.

[0137] The situation gets worse when the car is driving faster, and semantic maps can be created at longer ranges and distances using pre-built dense point cloud maps. The 2D semantic image representation can be back-projected into 3D space by exploiting homography, while assuming that the ground is smooth and level, and the back-projection method does not produce any black holes. In urban driving situations, the flat assumption fails on steep hills or at road intersections, which can cause severe deformations and deviations at longer distances. The first section of the analysis examines how modifying map parameters affects common localization errors such as position shift and centralization modules. The grid resolution and number are the two main map settings that are user-configurable. The global probability map is effectively executed in a global framework. It generates a reflectance map with a full 3D image representation or with only a ground plane representation. In addition, the first two options (structure vs. appearance) can be changed and adjusted for any map type. Due to all these variations, the search space of potential map combinations is large. Five different versions of the centralization module as well as localization bias and shift are evaluated, modifying the number of first-pass reliability procedures for each unit from one to five as preliminary examples to qualitatively illustrate what can be captured in our map. Snapshots of these maps are as follows Fig.13 shown.

[0138] Components and elements of the probabilistic mixture map that capture the appearance of the ground plane, vegetation, and other objects are included in the parameters of the centralized module and the position offset. (From left to right). Fig.13In the , each column depicts a different map, from a probabilistic mixture map with only one component per cell to a map with five components per cell (from left to right). It shows the average of the position shifts organized by the centralized module. Therefore, this seems to be particularly evident in the ground plane and superstructure, where several components have the same average. There is a large qualitative improvement between the first and second probability maps. In the first probability scenario, a clear blurring can be seen between the ground plane and the superstructure (such as plants, trees, and lamp posts). However, the second probability scenario can easily capture the patterns close to the ground plane and supporting the superstructure. The high-fidelity map follows this pattern, showing that the ground is captured in the lowest average component and may overfit the building facades and other superstructures due to the increase in the number of first-pass reliability practices and operations. It is expected that there will be a clear improvement in positioning between the first probability map and the larger number of maps, as keeping these patterns different and separated is crucial to eliminate possible obstacles between the ground and the structure.

[0139] The proposed method combines a mobile LiDAR system and a probability-based multi-objective optimization for generating high-definition (HD) maps in complex urban environments. By leveraging the strengths of these techniques, our approach achieves highly accurate and dense maps while considering key factors such as autonomous capabilities, fast data acquisition, minimal error, and dense point clouds. The integration of multi-objective stochastic models improves the reliability of the generated maps by incorporating various risk metrics. Compared to the existing literature, our study stands out by proposing a comprehensive framework that integrates a mobile LiDAR system and a probability-based multi-objective optimization specifically tailored for urban environments. This integration brings significant advantages, including improved map accuracy, efficient map generation, and the ability to balance the key factors required for autonomous vehicles, demonstrating significant improvements in map accuracy for road features such as roads, crosswalks, and lane markings. An 11% improvement in accuracy for roads, an 8% improvement for crosswalks, and a substantial 30% improvement for lane markings were achieved, thereby promoting the potential and applicability of probability-based multi-objective optimization for generating LiDAR-based HD maps.

[0140] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.

[0141] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0142] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A method for generating high-definition maps based on point cloud and multi-objective optimization, characterized in that: The following steps are involved: Acquire sensor data, wherein the sensor data includes point cloud data collected by mobile LIDAR, camera, radar and GPS; Performing data preprocessing on the sensor data to generate a 3D point cloud image, and performing semantic label recognition, semantic segmentation and probabilistic reasoning on the 3D point cloud image to obtain a semantic label of the 3D point cloud image; Optimizing the 3D point cloud image using a probabilistic algorithm and performing semantic mapping on the 3D point cloud image to obtain topological information corresponding to the 3D point cloud image; fusing the semantic labels with the topological information to generate a high-definition map, wherein the high-definition map combines the identified semantic labels with the overall structure and layout of the environment; Based on the generated HD map, a safe and efficient path is planned for the vehicle to complete the HD map generation.

2. The method for generating a high-definition map based on point cloud and multi-objective optimization according to claim 1, characterized in that: The sensor data is preprocessed to generate a 3D point cloud image, including: Using a multi-objective stochastic model to identify geodesic correlations and features between adjacent points of the 3D point cloud in the 3D point cloud image; Based on 3D point cloud The Euclidean nearest neighbor graph G = (V, E) is constructed, and the vertices and edges of the graph G are N = {1, 2, ..., n}, Indicates that the neighbor set of each vertex x is represented by y∈N(x), and introduces a set of input vertex features h={h1,h2,…,h N }, where h N ∈R F Represents each graph vertex Unique features, derived from the 3D point cloud P, each vertex in the graph G is assigned a feature represented as h N , the feature dimension of each vertex is given by F, and the output expression of the HD map corresponding to each vertex is: h' y =max(g θ (G(u(x,y))·h y +h y )),y∈N(x) (1) Where G(·) is a geometric mapping function R 3 →R, G(·) maps the local Euclidean coordinates u(x,y)=u(y)-u(x) between each vertex and its neighbors to geometric parameters by multiplying G(u(x,y)) by the feature h y To obtain the trainable feature mapping function R F →R K , and multiply the resulting product with Connection, where Represents the maximum aggregation function; Calculate the trainable feature mapping function, the corresponding expression is: in, is a 1×1 learnable parameter, G(u(x,y)) can approximate and estimate various geodesic values ​​of each vertex x using its neighbor set y∈N(x) by modifying the given learnable parameter; Using multi-layer perception MLP as the feature mapping function, Indicates that the input feature h y , feature h y The combination of the product with the geometric features G(u(x,y)) is mapped to various feature dimensions R F →R K , use the maximum aggregation algorithm to learn the new features h y Aggregation is performed to determine neighbor features of the 3D point cloud.

3. The method for generating a high-definition map based on point cloud and multi-objective optimization according to claim 2, characterized in that: Performing semantic label recognition, semantic segmentation, and probabilistic reasoning on the 3D point cloud image to obtain a semantic label of the 3D point cloud image includes: PointNet is used to extract object features of a 3D point cloud image, where the object features have coordinates r i =[x i ;f i ]、x i ∈R 3 and f i ∈R C Representative points The representative points of the object features Offset adjustment is performed to make each representative point centered around the shape and center of the object in the point cloud data, where the representative point The characteristic f i ∈R C As input, the 3D position offset Δx in the Euclidean domain i ∈R 3 and characteristic shift Δf i ∈R C As output, From the representative point r i Start offset adjustment to get c i =[y i ; g i ], where y i =x i +Δx i and g i =f i +Δf i , the estimated 3D offset Δx i It is controlled by the following loss function: Among them, 1[r i on the object] represents the representative point r i Whether it is on the surface of the object, N pos is the total number of representative points on the surface of the object, Δx i Indicates the position of the representative point x i and the ground truth displacement between the center of the object bounding box, and the scale control parameter γ is set to 0.

1.

4. The method for generating a high-definition map based on point cloud and multi-objective optimization according to claim 3, characterized in that: Optimizing the 3D point cloud image using a probabilistic algorithm and performing semantic mapping on the 3D point cloud image to obtain topological information corresponding to the 3D point cloud image includes: The default input is the representative point c i The output is the module (r i ,y i ,g i ) generates a 3D position offset of a representative point with random variables x1,…,x in n-dimensional space. n The failure probability P F for: in, Denotes the joint probability function of random variables, D F Represents the failure domain of the n-dimensional random variable space. The limit state function g(X) is determined based on the failure probability and failure domain. The reliability index β corresponding to the limit state function * The expression is: b * =Φ -1 (1-P F ) (5) Among them, Φ -1 (·) represents the inverse normalized Gaussian distribution function, with a preset position offset X i is a Gaussian random variable with a mean and standard deviation σ of zero. If the limit state function g(X) is highly nonlinear, g<0 can be used to indicate failure. If the limit state function has the same distribution X k Gaussian variable, then k=1,2…,n is σ; The Monte Carlo-based asymptotic sampling algorithm is used to adjust the standard deviation in the n-dimensional Gaussian space to accurately control the failure probability. The asymptotic sampling algorithm is used to determine the reliability index β * , reliability index β * The dependence on the standard deviation of the random variable is expressed by function (6): Among them, when f→0 satisfies the necessary asymptotic linear behavior, it can be satisfied by focusing on σ→0; the coefficients A and B are defined using regression analysis of least squares fitting, based on the corresponding β values ​​for different support points satisfying f<1 * The Monte Carlo estimate of is adjusted according to the scaled reliability index provided by expression (7):

5. The method for generating a high-definition map based on point cloud and multi-objective optimization according to claim 4, characterized in that: Monte Carlo based asymptotic sampling algorithms include: Select the initial factor f0, the useful set is f0 = 1, the number of samples in the failure domain N0, and the reliability index β * The accuracy of F Reduce the coefficient f and iterate until an acceptable number of N is obtained F ; After identifying the parameters, regression analysis is performed on the data pairs to obtain the functional relationship β * (f), β * (f) Extrapolate to f = 1 to obtain the reliability index.

6. The method for generating a high-definition map based on point cloud and multi-objective optimization according to claim 4, characterized in that: Also includes: Using independent and identically distributed random variables W k Extract the object features of the point cloud data, preset to keep a constant value within the discrete time interval Δt, the random variable W k The variance and mean of the feature has zero mean and variance Object feature F x(max) The intensity can be expressed in terms of variance and time interval: The independent and identically distributed value sequence can be represented as the object surface point of the object feature, and the object surface point has a random variable U with unit standard deviation k , random variable W k The expression is:

7. The method for generating a high-definition map based on point cloud and multi-objective optimization according to claim 6, characterized in that: The object surface points are relocated to the center of the object, and the background points of the object features are kept sparsely distributed, including: According to the representative point position in 3D Euclidean space Select and sample a subset of K points, using the index of K points to obtain To determine the offset representative point like By Find nearby points in To cluster N groups of points, the matching features of each group point are specified as A shared MLP is used to map the surrounding points of each object feature, and the feature aggregation uses the maximum operation. The corresponding expression is: in, is the relationship between adjacent positions and points with respect to F k ∈R (3+C) Comparing coordinates, most of the surrounding points are from the same object, and the aggregated features show the intra-object properties.

8. The method for generating a high-definition map based on point cloud and multi-objective optimization according to claim 7, characterized in that: Also includes: Construct a 3D bounding box (x, y, z, h, w, l, θ), where (x, y, z) are the coordinates of the center of the object, (h, w, l) are the height, width, and length of the object, and θ is the orientation of the object. Given an object feature F k , and use three layers with full connectivity to estimate the bounding box parameters B k,1 (x k,1 ,y k,1 ,z k,1 ,h k,1 ,w k,1 ,l k,1 ,θ k,1 ), using the projected bounding box center coordinates and position (x k,1 ,y k,1 ,z k,1 ), identify the neighbors of each object, and the relationship between the object and its surrounding and adjacent objects is represented as a region-to-region optimal path and an undirected graph, using the symbol Indicates; among them, and denote vertices and edges respectively; Determine the relationship parameter α i,k ∈R 1×(3+C) (i=1,…,S) and the object’s relational features F rk , F i,k represents the feature of the i-th adjacent object, and the corresponding expression is: in, is a trainable parameter with a value of 1×(3+C), related to the feature F rk will be passed to three fully connected layers, designated as B k,2 (x k,2 ,y k,2 ,z k,2 ,h k,2 ,w k,2 ,l k,2 ,θ k,2 ), B k,1 and B k,2 The sum of is taken as the final output of the network.

9. The method for generating a high-definition map based on point cloud and multi-objective optimization according to claim 8, characterized in that: Optimizing the 3D point cloud image using a probabilistic algorithm and performing semantic mapping on the 3D point cloud image to obtain topological information corresponding to the 3D point cloud image includes: Then use a set of The representative points and scaling control parameters of the 3D point cloud are optimized. By starting from the representative point r i To its center independently generate and expand c i =[y i ; g i ], and uses γ as the proportional control parameter to determine the offset Δf generated from each representative point position i ; Calculate the offset Δf i The reliability index r corresponding to the first pass probability i , and correspondingly generate representative y i Values ​​and representative g i value, where r i ,y i , g i The maximum offset position is fixed at 0.01m.

10. A high-definition map generation system based on point cloud and multi-objective optimization, applied to the high-definition map generation method based on point cloud and multi-objective optimization as claimed in any one of claims 1 to 9, characterized in that: include: A data acquisition module, used to acquire sensor data, wherein the sensor data includes point cloud data collected by mobile LIDAR, camera, radar and GPS; A data preprocessing module is used to perform data preprocessing on the sensor data to generate a 3D point cloud image, and perform semantic label recognition, semantic segmentation and probabilistic reasoning on the 3D point cloud image to obtain a semantic label of the 3D point cloud image; An information optimization module, used to optimize the 3D point cloud image using a probabilistic algorithm and perform semantic mapping on the 3D point cloud image to obtain topological information corresponding to the 3D point cloud image; an information fusion module, for fusing the semantic labels with the topological information to generate a high-definition map, wherein the high-definition map combines the identified semantic labels with the overall structure and layout of the environment; The map generation module is used to plan a safe and efficient path for the vehicle based on the generated high-definition map to complete the high-definition map generation.

Images