High-definition map processing method and device based on mobile laser radar data

Abstract

The invention discloses a high-definition map processing method and device based on mobile laser radar data, and the method comprises the steps: obtaining sensor data collected by a plurality of sensors of an autonomous vehicle, carrying out the probability data processing, obtaining a structured 3D point cloud, carrying out the recognition of the 3D point cloud, distributing a semantic tag, and creating a geometric mapping function; the method comprises the following steps: constructing a 3D point cloud image, converting local Euclidean coordinates by adopting a geometric mapping function to obtain a semantic segmentation result, classifying and correcting the semantic segmentation result by adopting a confusion network to obtain object attributes, constructing a semantic graph of the 3D point cloud image based on the object attributes and the geometric mapping function, and outputting a high-definition map corresponding to the semantic graph. According to the method, the high-definition map is evaluated and optimized to obtain a path navigation result of the autonomous vehicle, multi-target optimization is applied to processing the mobile laser radar point cloud to create the high-definition map, a confusion matrix is used to merge uncertainty simulation and probability semantic mapping, and the reliability and accuracy of map generation are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of laser radar point cloud processing, and in particular relates to a high-definition map processing method and device based on mobile laser radar data. Background Art

[0002] The development of high-definition (HD) maps is critical to the advancement of autonomous vehicle technology, providing a detailed representation of the road environment to complement real-time sensor data and facilitate active navigation. However, existing approaches face significant gaps, especially in processing mobile LiDAR point clouds, which have complex nature and uncertainty under different conditions. These challenges include managing the dynamic nature of the road that needs to be constantly updated, such as dealing with data noise, computational complexity, and geometric distortion. Therefore, by leveraging uncertainty trade-offs and optimization frameworks, more accurate, timely, and complete maps can be generated, significantly improving the capabilities of autonomous vehicle navigation systems. The production of high-definition (HD) maps remains problematic, especially when processing mobile LiDAR point clouds, where the integrity of this data can be compromised due to vibrations, such as sudden movements, speed changes, data volume, or background noise. HD maps provide highly detailed information about various elements of the transportation system, extending all the way to the centimeter level. These details include descriptions of roads, road markings, intersections, traffic lights, signs, traffic signals, and speed limits. This precise and structured information is critical to autonomous vehicles as it helps to accurately understand and navigate the driving environment. Accurate, real-time, and reliable localization is a basic but important task for downstream functions such as target tracking, trajectory prediction, and motion planning and control of autonomous vehicles (AVs). By combining real-time sensor data with detailed information from high-definition maps, vehicles can proactively predict and adapt to dynamic elements in the environment. Adding real-time sensor data to existing map data can improve the localization process and transform sensor-based environmental detection into a simpler map-based process. Advanced sensors in aircraft usually consist of global navigation satellite systems (GNSS), inertial measurement units (IMUs), cameras, lidars, and radars, which are usually used to design accuracy and localization algorithms. Since the mentioned sensors will more or less have errors and accumulated errors in visual or lidar odometry, multi-sensor fusion methods are preferred and have great potential to achieve high localization accuracy.

[0003] Corners, ramps and obstacles such as moving vehicles, pedestrians and cyclists often limit the detection range of sensors. Creating HD maps also requires collecting and processing large amounts of data. This task involves complex mobile lidar systems and encounters technical obstacles such as data noise, computational complexity and geometric distortion. This data-intensive process highlights the significant technical challenges involved in producing accurate and up-to-date maps. Based on distance sensors, lidar was developed to locate vehicles. The cumulative error of visual or lidar odometry increases with the distance traveled, which is different from the operating time of the inertial navigation system (INS). Therefore, when GNSS signals are unavailable or erroneous, they can be combined to limit the cumulative error of INS.

[0004] Therefore, there is an urgent need to provide a high-definition map processing method and device based on mobile lidar data to solve the above-mentioned technical problems. Summary of the invention

[0005] In view of this, the present invention provides a high-definition map processing method and device based on mobile lidar data, which can generate high-definition (HD) maps in complex urban environments, achieve highly accurate and dense maps, and improve the reliability of the generated maps. The following technical solutions are specifically used to achieve this.

[0006] In a first aspect, the present invention provides a method for processing a high-definition map based on mobile laser radar data, comprising the following steps: Acquire sensor data collected by multiple sensors of the autonomous vehicle, wherein the sensor data includes 3D point cloud data of a radar sensor, visual data of a camera, and radar signals and geographic coordinates of a GPS; Performing probabilistic data processing on the sensor data to obtain a structured 3D point cloud, wherein the probabilistic data processing includes extracting geometric parameters, evaluating interface changes, and identifying correlations between points, wherein the geometric parameters include edges, surfaces, and corners of the 3D point cloud; Performing semantic label recognition on the 3D point cloud to assign semantic labels, and creating a geometric mapping function according to the semantic labels; Constructing a 3D point cloud image of the 3D point cloud, and using the geometric mapping function to transform local Euclidean coordinates of the 3D point cloud image to obtain a semantic segmentation result of the 3D point cloud image; Using a confusion fusion network to classify and correct the semantic segmentation results to obtain object attributes of the 3D point cloud image, wherein the classification correction includes integrating sensor data, using a confusion fusion network to correct misclassification, adapting to real-time changes in the environment, a geometric prediction model, and object feature extraction; A semantic map of the 3D point cloud map is constructed based on the object attributes and the geometric mapping function, and a high-definition map corresponding to the semantic map is output, and the high-definition map is evaluated and optimized to obtain a path navigation result of the autonomous driving vehicle.

[0007] In a second aspect, the present invention further provides a high-definition map processing device based on mobile laser radar data, which is applied to the above-mentioned high-definition map processing method based on mobile laser radar data, comprising: A data acquisition unit, configured to acquire sensor data collected by a plurality of sensors of the autonomous vehicle, wherein the sensor data includes 3D point cloud data of a radar sensor, visual data of a camera, and radar signals and geographic coordinates of a GPS; A probability processing unit, configured to perform probability data processing on the sensor data to obtain a structured 3D point cloud, wherein the probability data processing includes extracting geometric parameters, evaluating interface changes, and identifying correlations between points, and the geometric parameters include edges, surfaces, and corners of the 3D point cloud; A semantic recognition unit, configured to perform semantic label recognition on the 3D point cloud to assign semantic labels, and to create a geometric mapping function according to the semantic labels; A semantic segmentation unit, used to construct a 3D point cloud image of the 3D point cloud, and convert the local Euclidean coordinates of the 3D point cloud image using the geometric mapping function to obtain a semantic segmentation result of the 3D point cloud image; A classification correction unit, used for classifying and correcting the semantic segmentation result by using a confusion fusion network to obtain the object attributes of the 3D point cloud image, wherein the classification correction includes integrating sensor data, correcting misclassification by using a confusion fusion network, adapting to real-time changes in the environment, a geometric prediction model, and object feature extraction; An evaluation and optimization unit is used to construct a semantic map of the 3D point cloud map based on the object attributes and the geometric mapping function, output a high-definition map corresponding to the semantic map, and evaluate and optimize the high-definition map to obtain a path navigation result of the autonomous driving vehicle.

[0008] The present invention provides a high-definition map processing method and device based on mobile laser radar data, which obtains sensor data collected by multiple sensors of an autonomous driving vehicle, performs probabilistic data processing on the sensor data to obtain a structured 3D point cloud, performs semantic label recognition on the 3D point cloud to assign semantic labels, creates a geometric mapping function based on the semantic labels, constructs a 3D point cloud map of the 3D point cloud, and uses the geometric mapping function to convert the local Euclidean coordinates of the 3D point cloud map to obtain a semantic segmentation result of the 3D point cloud map, uses a confusion fusion network to classify and correct the semantic segmentation result to obtain object attributes of the 3D point cloud map, constructs a semantic map of the 3D point cloud map based on the object attributes and the geometric mapping function, outputs a high-definition map corresponding to the semantic map, evaluates and optimizes the high-definition map to obtain a path navigation result of the autonomous driving vehicle, applies multi-objective optimization to process mobile laser radar point clouds to create high-definition maps, and uses a confusion matrix to merge uncertainty simulation and probabilistic semantic mapping, which helps to improve the accuracy and reliability of a semantic segmentation network used for mapping, enhances the ability to extract and express map features, and improves the reliability and accuracy of generated maps. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A high-definition map processing method based on mobile laser radar data provided by the present invention; Figure 2 The reliability index provided by the present invention Depends on centralized module Control parameters and position offset Schematic diagram of the relationship between Figure 3 The reliability index provided by the present invention Depends on centralized module Control parameters and position offset Schematic diagram of the relationship between Figure 4 The reliability index provided by the present invention Depends on centralized module Control parameters and position offset Schematic diagram of the relationship between Figure 5 A schematic diagram of simulation data collection based on the BEV map provided by the present invention; Figure 6A probabilistic mixture map that captures the appearance of ground plane, vegetation, and other objects provided by the present invention; Figure 7 This is a structural block diagram of the high-definition map processing device based on mobile laser radar data provided by the present invention. DETAILED DESCRIPTION

[0011] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0012] To address the accumulated errors, another source that can provide global position, such as GNSS, is needed to help INS. In this regard, HD maps have been studied in the literature to complement the situation where GNSS is not available. Different types of HD maps have been developed based on various sensors such as magnetometers and lidars, where magnetic maps are created using crowdsourced data and INS / RSS / magnetometer integrated systems, and edge maps are generated from lidar 3D point cloud data using beam models. Point cloud maps are generated by converting high-end GNSS / INS integrated systems with lidar. A large graph optimization algorithm fuses lidar with coarse GNSS positions to create point cloud maps, which are then used for vehicle localization through map matching; however, problems such as high computational load and inaccuracy of moving objects are addressed by extracting semantic features to create segment maps and developing map matching localization based on global scan descriptors and local features. Evaluation of point cloud maps using map matching algorithms shows that merging 1D and 2D features has better correspondence due to the higher certainty of these features.

[0013] Therefore, by integrating techniques to address the above challenges, more robust HD maps can be generated to enhance the capabilities of autonomous vehicles. Various techniques have been explored to enhance the creation of HD maps, such as stereo cameras, sensor fusion, machine learning, radar-based maps, crowdsourcing, and hybrid maps. These methods aim to improve the accuracy, timeliness, and completeness of HD maps for autonomous vehicles. Mobile LiDAR technology and deep learning address the challenges in HD map creation. Mobile LiDAR systems help to quickly acquire high-density, high-precision point clouds, simplifying the map creation process. Advanced techniques such as multi-objective optimization can be explored. Multi-objective optimization can simultaneously optimize accuracy, efficiency, and completeness, which is crucial for producing high-quality HD maps. In addition, integrating probabilistic methods within the optimization framework effectively provides the advantage of managing the uncertainty inherent in LiDAR observations. Therefore, probability-based multi-objective optimization can identify trade-offs between objectives and find the optimal solution with minimum error while satisfying multiple criteria. This approach helps to consider the uncertainty in LiDAR observations and model parameters, ultimately leading to improved LiDAR system design and performance.

[0014] A framework for creating HD maps that uses probabilistic methods to effectively manage uncertainty and find high-performance error-reducing solutions in areas where parameter values ​​are unknown is described below: (1) Probabilistic methods are used to handle uncertainty in the creation of HD maps. This involves using probability-based methods to better manage unknowns and variability in the data. (2) Probabilistic semantic mapping creates probabilistic semantic mapping of bird's-eye view (BEV) probabilistic semantic mapping. This mapping includes modeling uncertainty using confusion matrices, which helps understand and improve the performance of semantic segmentation networks. (3) Integration with multi-objective optimization, that is, integrating probabilistic methods within a multi-objective optimization framework, provides an effective way to manage uncertainty. This involves using a stochastic model to find the optimal parameters with the minimum error to address the challenge of optimizing the lidar system. (4) Pareto-based optimization uses Pareto-based optimization to enhance the 3D map generated by mobile lidar data. This optimization method is effective for managing uncertainty because it finds the optimal trade-off between competing objectives such as accuracy, resolution, and cost while considering the same probabilistic nature of the input data and the model. (5) Uncertainty-aware mapping segmentation improves the accuracy of environmental mapping by fusing lidar intensity information. This involves using probabilistic Pareto-type optimization methods to balance segmentation accuracy and computational efficiency.

[0015] This paper aims to link the latest advances in HD maps with multi-objective optimization techniques, and by integrating the latest techniques from HD maps and multi-objective optimization, attempts to create more accurate, efficient and comprehensive HD maps with a focus on managing uncertainty. In summary, the main contributions of this paper are: (1) A comprehensive framework is proposed to improve the effectiveness of HD map creation, which is the first to apply multi-objective optimization to the creation of HD maps by processing mobile LiDAR point clouds. This method allows simultaneous optimization of multiple objectives such as accuracy, efficiency and completeness while taking into account uncertainties in the data and process. (2) A method is proposed to generate BEV probabilistic semantic maps from semantic point cloud data, which includes incorporating uncertainty simulation using confusion matrices, which helps to improve the accuracy and reliability of the semantic segmentation network used for mapping. (3) A method is proposed to improve the mapping segmentation accuracy by integrating LiDAR intensity information and utilizing a probabilistic Pareto-type optimization method, which effectively enhances the extraction and representation capabilities of map features.

[0016] To demonstrate the effectiveness of the above approach, a comprehensive evaluation was performed using various datasets and a comparative analysis with existing methods was provided. Through this analysis, the potential for improving map accuracy was demonstrated, achieving an impressive 11% improvement on roads, 8% on crosswalks, and a significant 30% improvement on lane markings, these results demonstrate its ability to advance the application of Pareto probability-based multi-objective optimization in the field of LiDAR-based HD map generation.

[0017] See also Figure 1 The present invention provides a method for processing high-definition maps based on mobile laser radar data, comprising the following steps: S1: Acquire sensor data collected by multiple sensors of the autonomous driving vehicle, wherein the sensor data includes 3D point cloud data of a radar sensor, visual data of a camera, and radar signals and geographic coordinates of a GPS; S2: performing probabilistic data processing on the sensor data to obtain a structured 3D point cloud, wherein the probabilistic data processing includes extracting geometric parameters, evaluating interface changes, and identifying correlations between points, and the geometric parameters include edges, surfaces, and corners of the 3D point cloud; S3: performing semantic label recognition on the 3D point cloud to assign semantic labels, and creating a geometric mapping function according to the semantic labels; S4: constructing a 3D point cloud image of the 3D point cloud, and using the geometric mapping function to transform the local Euclidean coordinates of the 3D point cloud image to obtain a semantic segmentation result of the 3D point cloud image; S5: using a confusion fusion network to classify and correct the semantic segmentation result to obtain the object attributes of the 3D point cloud image, wherein the classification correction includes integrating sensor data, using a confusion fusion network to correct misclassification, adapting to real-time changes in the environment, a geometric prediction model, and object feature extraction; S6: Constructing a semantic map of the 3D point cloud map based on the object attributes and the geometric mapping function, outputting a high-definition map corresponding to the semantic map, and evaluating and optimizing the high-definition map to obtain a path navigation result of the autonomous driving vehicle.

[0018] In this embodiment, the semantic label includes at least one of a lane, a crosswalk, a car, and a navigation indicator. The HD map processing method of the present invention mainly includes extracting geometric parameters, analyzing interface changes, establishing correlations between closed points, utilizing object attributes and features, using pattern recognition and feature learning techniques, and establishing interconnections, which can obtain point features at different levels and have representative point selection, obtain point features of different measurements and levels, and represent input scenes, helping to reduce computational complexity and cost. A deep hierarchical framework is constructed to meet the above goals, and how the method uses point cloud data to identify semantic labels of objects and integrates them with topological information to generate semantic maps is described. In order to illustrate the process of creating a block diagram flow chart of HD maps using mobile lidar point clouds and probability-based multi-objective optimization in urban autonomous driving, the following steps are outlined: (1) sensor data collection; (2) probabilistic data processing; (3) semantic label identification; (4) map refinement and update; (5) final semantic map generation. First, the process of collecting sensor data requires obtaining 3D point cloud data from LiDAR sensors, visual data from cameras, radar signals, and geographic coordinates from GPS. The unprocessed sensor data undergoes probabilistic data processing, including filtering and denoising, to remove noise and irrelevant information. This process generates a structured 3D point cloud and extracts geometric parameters such as edges, surfaces, and corners. It can also identify correlations between adjacent points to improve data accuracy and reduce the spatial size of parameters while maintaining data integrity. The process then turns to semantic label recognition, which is to identify important features in the point cloud and classify them into specific groups, such as lanes, crosswalks, cars, and other navigation indicators. These identified objects are assigned semantic labels, creating a meaningful mapping. A graph-based approach is used to construct a graph from the 3D point cloud, collecting geometric spatial features and connections. Geometric mapping functions are used to transform local Euclidean coordinates and features, improving semantic segmentation.

[0019] It should be noted that the map is continuously refined and updated by integrating new sensor data, correcting misclassifications using the Confusion Fusion Network (CFN) method, adapting to real-time changes in the environment, applying geometric prediction models, and extracting additional object features to improve mapping accuracy. The final stage of generating a semantic map requires merging all processed and improved data into a complete, high-resolution, detailed and accurate map. Subsequently, the map is evaluated by comparative analysis using different data sets and indicators. The application of optimization algorithms aims to improve the effectiveness and efficiency of the map by utilizing probability-based multi-objective optimization techniques. Probabilistic multi-objective optimization plays a vital role in this case, achieving a good balance between multiple mapping objectives such as accuracy, density, and computational efficiency, which leads to more complex and more effective decision-making procedures. The map is used for path planning and autonomous navigation, calculating safe paths, avoiding obstacles, selecting the best route, and making dynamic decisions based on the vehicle's current state and environment. The accuracy of semantic segmentation is improved by systematically handling and adjusting misclassifications. In urban autonomous driving situations, this integration plays a vital role in generating reliable HD maps and optimizing probability-based multi-objective methods. The CFN method improves the quality and reliability of the generated maps by effectively solving and correcting misclassifications by leveraging confusion matrix data.

[0020] It should be understood that by acquiring sensor data collected by multiple sensors of an autonomous vehicle, probabilistic data processing is performed on the sensor data to obtain a structured 3D point cloud, semantic label recognition is performed on the 3D point cloud to assign semantic labels, and a geometric mapping function is created based on the semantic labels to construct a 3D point cloud map of the 3D point cloud, and the local Euclidean coordinates of the 3D point cloud map are converted using the geometric mapping function to obtain the semantic segmentation result of the 3D point cloud map, a confusion fusion network is used to classify and correct the semantic segmentation result to obtain the object attributes of the 3D point cloud map, a semantic map of the 3D point cloud map is constructed based on the object attributes and the geometric mapping function, and a high-definition map corresponding to the semantic map is output, the high-definition map is evaluated and optimized to obtain the path navigation result of the autonomous vehicle, multi-objective optimization is applied to process mobile lidar point clouds to create high-definition maps, and a confusion matrix is ​​used to merge uncertainty simulation and probabilistic semantic mapping, which helps to improve the accuracy and reliability of the semantic segmentation network used for mapping, enhances the ability to extract and express map features, and improves the reliability and accuracy of generated maps.

[0021] Optionally, performing probabilistic data processing on the sensor data to obtain a structured 3D point cloud includes: A multi-objective random model is constructed, and the Euclidean nearest neighbors corresponding to the features of the multi-objective random model are obtained from the 3D point cloud. Build the graph , where the vertices and edges of the graph are represented by and Indicates that the neighbor set of each vertex x is composed of express; Introducing a set of input vertex features ,in, Represents each graph vertex , the features or attributes associated with each vertex in the graph G come from the 3D point cloud P, and each vertex in the graph is assigned a feature representation , the feature dimension of each vertex is given by F, and the output of the high-definition map HD The expression is: (1) in, is a geometric mapping function , the local Euclidean coordinates between each vertex and its adjacent Euclidean coordinates to a geometric parameter; Multiply the features To obtain a trainable feature mapping function , and multiply the resulting product with association, Represents the maximum aggregation function; Using a mapping function with trainable parameters The geometric prediction model expresses the structure of the HD map; (2) in, yes Learnable parameters, , By modifying the given learnable parameters, the adjacent set Estimate various geodesic values ​​for each vertex x; use multi-layer perception MLP as the feature mapping function by Indicates that the initial input features , With geometric features The combination of products is mapped to various feature dimensions , and use the maximum aggregation algorithm to learn the new features Perform aggregation.

[0022] In this embodiment, GeoConv is used for detection in the sampling layer of the geometric prediction model, and PointNet is used in t sampling layers to extract object features and to combine and interpolate various features in subsequent sampling values. The output of PointNet is a coordinate , and Representative points , where GeoConv is used for convolutional layers and PointNet is used to capture local and global information from point clouds; According to the representative point Offset each representative point to its center, and adjust the features of the representative point As input, the 3D position offset in the Euclidean domain and feature offset As output, from the representative point generate ,in, , , the estimated 3D offset It is controlled by the following loss function: (3) in, Indicates representative point Is it on the surface of the object? Represents the total number of representative points on the surface of the object, Indicates the position of the representative point The ground truth displacement from the center of the object's bounding box.

[0023] It should be noted that by utilizing complex deep learning frameworks and hierarchical structures to improve semantic mapping and navigation for autonomous driving in urban environments, complex deep learning including Taylor Gaussian filters, GeoConv, MLP (multi-layer perceptron) and PointNet are added to the basic hierarchical approach. This combination plays a key role in data processing from raw sensor input to complex semantic understanding of the surrounding environment, ensuring that each stage of data processing adds a layer of specificity, resulting in rich, useful and meaningful semantic maps. Data processing provides a unique probabilistic mechanism to manage inherent uncertainty and maximize the balance between different mapping objectives, playing a vital role in improving the reliability of the overall map and semantic labels, providing a solid foundation for subsequent navigation and path planning operations. This probabilistic feature enables more complex handling of uncertainty, ensuring that the system can make wise decisions even in complex urban environments. The method also uses probabilistic data processing to extract the scene context around the center of the object. A deep understanding of the surrounding environment is essential to ensure the precise location of the object in the semantic map, thereby achieving more effective path planning and navigation. The approach goes from collecting raw data to producing detailed semantic maps and puts effective path planning into action by combining these modules into a single well-organized framework, where each module has its own function in the overall goal of creating customized high-resolution maps designed specifically for autonomous driving in urban environments. The system's hierarchical structure, enhanced by advanced probabilistic optimization and state-of-the-art deep learning techniques, ensures not only adaptability and accuracy, but also efficiency in processing and utilizing data to effectively navigate the complex dynamics of urban environments, thus validating its contribution to the advancement of autonomous vehicle technology.

[0024] Specifically, due to the high density of point clouds in the sampling layers, the local neighbors of each point establish a part of the object surface. In order to extract object properties, including geometry, surface variability, closed position, and point correlation, GeoConv is used in these sampling layers. When a small number of points are collected and sampled, object properties such as location or layout are improved, while geometric properties such as size, shape, and shape between retrieved neighbors are reduced. Since GeoConv cannot retrieve object features from all layers, it improves detection performance.

[0025] It should be noted that while LiDAR is used to create semantic segmentation of BEV LiDAR projections, other sensor data such as cameras, radars, and GPS can also be used in combination to produce more accurate and precise segmentations. Multi-sensor fusion is a typical autonomous driving approach, and deep learning-based algorithms provide semantic information that helps overcome the limitations of individual sensors and ensure reliable and precise mapping results. Therefore, while LiDAR point clouds are used, other sensors are used for more reliable and accurate outputs. Starting with the Taylor-Gaussian filter, which combines Taylor expansion and Gaussian smoothing to enhance input data or perform feature extraction, the feature mapping function uses an MLP, a neural network architecture with multiple layers of interconnected nodes, to learn a complex mapping from input to desired features. In addition, GeoConv is used as a convolutional layer to operate on geometric structures such as point clouds or graphs to extract features from geometric data. Finally, the PointNet network, a deep learning architecture specifically designed for processing point cloud data, is used for object features and feature extraction. The PointNet network is able to capture local and global information from point clouds without relying on predefined structures. Together, these constitute the proposed structural model, which provides a comprehensive approach for deep learning-based feature extraction.

[0026] Optionally, the probabilistic data processing includes: Input representative point features And output the 3D position offset of the generated representative point , a random variable with n-dimensional space The probability of failure for: (4) in, Represents the joint probability function of random variables, and the failure region of the n-dimensional random variable space is given by express, is the failure domain; the joint probability function is used to extract the scene context around the object center to provide a threshold, and the corresponding reliability index The expression is: (5) in, Represents the inverse normalized Gaussian distribution function, default is a Gaussian random variable with mean and standard deviation is zero, considering the limit state function is highly nonlinear. Can be used to represent failure; if the standard deviation of a Gaussian variable with the same distribution is ,in, As , reliability index Depends on the standard deviation .

[0027] In this embodiment, geometric parameters are extracted in a hierarchical manner and the newly proposed probabilistic data processing is adopted, which mainly includes sensor data acquisition, probabilistic data processing, semantic label recognition, and continuous improvement of the map over time to improve accuracy. This study combines these elements to achieve a holistic and adaptive approach to semantic mapping and path planning, combining object features and graph-based techniques. Identifying the correlation and features between adjacent points in a point cloud is to develop a stochastic model that emphasizes simple geometric properties, reduces parameters, and utilizes internal features of objects. Extracting meaningful geometric information from point clouds aims to achieve accurate mapping for autonomous driving by utilizing graph-based techniques and geometric and feature mapping functions. It also emphasizes the potential of sensor fusion, where data from LiDAR, cameras, radars, and GPS can be combined with data from various sensors to improve segmentation and mapping accuracy. Among them, HD maps attempt to identify geodesic correlations and features between neighbors near each point, providing a large number of parameters, while most points and their local neighbors in indoor scenes are located on planes or surfaces with simple regular shapes, which can be characterized by simplified parameterized geometric mathematical functions. The multi-objective stochastic model adopted focuses on regular and simple geometric properties to reduce the number of parameters and exploits the features within geodesic objects and the characteristics of surrounding objects.

[0028] Specifically, since depth sensors mainly capture surface points of objects, there are few or no points around the centers of shapes and objects, and existing point-based networks have difficulty obtaining meaningful context around the centers of elements and objects. As for VoteNet, the project uses the Hough voting module to overcome this problem by generating new points (votes) in the central core of the object. Voting is performed using features and representative point features, after which the collected votes can be systematically organized using trainable modules to generate contextual items. A voting loss function is designed that uses Euclidean distance to estimate the displacement and deviation of votes, and the network is good at identifying and discovering 3D objects. However, the voting loss function does not take into account the scaling issue, and larger objects may be returned in time more effectively than smaller elements and objects.

[0029] To improve this, the centralization module is built following VoteNet, but a scaling control parameter is introduced when defining the loss function. Based on a series of representative points, the centralization module performs an offset adjustment from each representative point to its center. The module consists of a shared MLP module with three fully connected layers, ReLU, and batch and subsampling normalization. "Shared MLP module" refers to a multi-layer perceptron (MLP) module shared between multiple components or parts of a neural network, which is used to perform an offset adjustment on each representative point so that the points can be concentrated around the center of shapes and objects in point cloud data. The features of the representative points are taken as input, and the 3D position offset and feature offset in the Euclidean domain are taken as output.

[0030] Optionally, the failure probability is calculated and defined based on the linear limit state function, and the failure cases and conditions in the Gaussian space are given by Given, where is the most likely failure point, defined using the optimal solution and ; The reliable safety index of the first channel of probabilistic data processing is determined by Monte Carlo asymptotic sampling algorithm. In n-dimensional Gaussian space, the variables are all equally expanded and relatively distributed. The failure probability is controlled by adjusting the standard deviation. The reliability index and the standard deviation of the random variable The correlation can be expressed as: (6) when When, through When the necessary asymptotic behavior is satisfied, Different support points, based on The Montero estimation uses a least squares regression analysis to define the coefficients A and B, and the fitting process is adjusted according to the scaled reliability index provided by expression (6) as follows: (7) Here, rewriting formula (6) gives the same weight to each individual support point in the regression analysis, and choosing the initial factor ,Pick A useful collection of The number of samples in the The accuracy is reduced by factor f and iterated to obtain an acceptable ; The simulation model and f are gradually reduced to obtain the paired f and After identifying the parameters, regression is applied to the data to obtain the functional relationship The reliability index is Push Got it.

[0031] In this embodiment, in order to accurately extract the scene context around the center of the object while maintaining the integrity of the context, it is necessary to solve the trade-off problem through an optimization process. Given that the target point is regarded as a non-stationary random process, the design of the algorithm for extracting the scene context can incorporate appropriate probabilistic features of the dynamic response, such as the first-pass probability of the critical response level. This probabilistic approach can more accurately represent the scene context and improve the reliability of the extraction process. The input is the characteristics of the representative point (the output is the 3D position offset of the representative point generated by the module). The first-pass probability in the random point cloud can be effectively evaluated using a commonly used method called first-order reliability. To further enhance this method, it has been proven to be beneficial to utilize importance in sampling. The first-order reliability method calculates and defines the failure probability based on the generalized linear limit state function, especially at the most likely failure point. Considering the so-called object characteristic design point, which is caused by the first article problem, with object points, the optimal solution and optimization method should be used to define and develop the problem. It should be noted that the optimization process may gradually become relevant or the global minimum point may not converge. This concept is considered for different algorithms. The NLPQL, L-BFGS and CONMIN algorithms are tested in this application. A considerable number of random variables are required to accurately describe the characteristics of the object. Although efficient and effective convergence is possible, the computational process is long and the estimation of the gradient requires a lot of nonlinear dynamic analysis. Therefore, it is challenging to apply the first-order reliability method (FORM) to evaluate the safety index of the entire variable design space. The computational process, especially in the evaluation and estimation of the gradient, has great limitations. To overcome this, a new method using Monte Carlo-based asymptotic sampling is used to quickly determine the reliable safety index of the first pass.

[0032] It should be noted that the non-stationary nature of the object points is addressed by exploiting the probabilistic characteristics of the dynamic response, ensuring the accurate representation of the variability and patterns in the scene. It combines the concept of first-pass probability and assumes that the target points are Gaussian random variables, and evaluates the safety indicators through standard deviation. By adopting a Monte Carlo-based asymptotic sampling method, the algorithm evaluates the safety indicators across variables and quantifies the relationship between the reliability index and the standard deviation. In general, the method captures variability and complexity, improves the accuracy and reliability of scene context extraction, and maintains the integrity of the context.

[0033] Optionally, the semantic segmentation result is classified and corrected using a confusion fusion network to obtain object attributes of the 3D point cloud image, further comprising: The object extraction process is represented by independent and identically distributed IID random variables as , which is assumed to be constant over discrete event intervals ∆t, and has zero mean and variance , object characteristics The intensity variance and time interval are available ∆t It is expressed as: (8) The independent and identically distributed IID value sequences represent the surface points of the object and are random variables with unit standard deviation ,variable The expression is: (9).

[0034] In this embodiment, the sampling and clustering procedures are selected based on spatial proximity to ensure that the items can represent potential objects with enough data. The farthest point sampling FPS is used according to the typical representative point position in 3D Euclidean space. , select a subset of K points for sampling and calculate the typical representative points The index of the moving representative point is obtained ; By Find nearby points in Cluster N grouping points. If and , then the matching feature of each group point is specified as , where r is the radius; A shared MLP is used to map the features of the surrounding points of each item, and the feature aggregation uses the maximum operation: (10) in, is the comparison coordinate between adjacent positions and points, and For,the result features of the aggregation, display the object properties.

[0035] Wherein, constructing the semantic graph of the 3D point cloud graph based on the object attributes and the geometric mapping function includes: Determine the neighboring items based on the predicted bounding box center position, using a 3D bounding box as Perform object feature recognition, where are the coordinates of the object center, are the height, width and length of the object size, is the object direction; Preset Object Features , using three layers with full connectivity to estimate bounding box parameters ; Use the projected bounding box center coordinates and position The neighbors of each item are identified, and the relationship between the item and its surrounding and adjacent items is represented as an optimal path from region to region and an undirected graph, using the symbol It indicates that, and denote vertices and edges respectively, Represents the characteristics of the i-th adjacent item and determines the relationship parameters Possible relationship characteristics with objects ; (11) (12) (13) in, Is has trainable parameters with values, , the relationship features is passed to three fully connected layers, designated as , and The sum of is the final output of the network.

[0036] Specifically, in selective 3D object detection, the features of the object play a key role in accurate recognition. The aggregated item features mentioned above capture the intra-object features, especially the features derived from points located on the surface of the same object. There are connections between objects in real-world scenes. By considering the nearest neighbor items of each item and utilizing their relationship features, the neighboring items are determined based on the predicted bounding box center position, and these neighboring items are searched based on the predicted bounding box center position.

[0037] See also Figure 7 The present invention also provides a high-definition map processing device based on mobile laser radar data, which is applied to the above-mentioned high-definition map processing method based on mobile laser radar data, comprising: A data acquisition unit, used to acquire sensor data collected by multiple sensors of the autonomous driving vehicle, wherein the sensor data includes 3D point cloud data of a radar sensor, visual data of a camera, and radar signals and geographic coordinates of a GPS; A probability processing unit, configured to perform probability data processing on the sensor data to obtain a structured 3D point cloud, wherein the probability data processing includes extracting geometric parameters, evaluating interface changes, and identifying correlations between points, and the geometric parameters include edges, surfaces, and corners of the 3D point cloud; A semantic recognition unit, configured to perform semantic label recognition on the 3D point cloud to assign semantic labels, and to create a geometric mapping function according to the semantic labels; A semantic segmentation unit, used to construct a 3D point cloud image of the 3D point cloud, and to convert the local Euclidean coordinates of the 3D point cloud image using the geometric mapping function to obtain a semantic segmentation result of the 3D point cloud image; A classification correction unit, configured to classify and correct the semantic segmentation result using a confusion fusion network to obtain object attributes of the 3D point cloud image, wherein the classification correction includes integrating sensor data, correcting misclassification using a confusion fusion network, adapting to real-time changes in the environment, a geometric prediction model, and object feature extraction; An evaluation and optimization unit is used to construct a semantic map of the 3D point cloud map based on the object attributes and the geometric mapping function, and output a high-definition map corresponding to the semantic map, and evaluate and optimize the high-definition map to obtain a path navigation result of the autonomous driving vehicle.

[0038] In this embodiment, the execution process of the high-definition map processing device of the present invention mainly includes acquiring sensor data, processing data and producing 3D point cloud maps, identifying semantic labels and optimizing mapping targets, integrating semantic labels, using maps for path planning, and continuously updating maps with new data. Acquiring sensor data: corresponds to the initial step of acquiring sensor data (including mobile LiDAR point clouds) in the flowchart. The sensor data used is mainly composed of mobile LiDAR (light detection and ranging) point clouds, which are captured using mobile laser radar sensors that use laser technology to measure distances and generate three-dimensional representations of the surrounding environment; processing data and generating 3D point cloud maps: processing raw LiDAR data by extracting geometric parameters, evaluating interface changes, and establishing correlations between adjacent positions, and using sensor data to create a structured 3D point cloud map that accurately reflects the physical environment in a digital format; identifying semantic labels and optimizing mapping targets: by utilizing object attributes, Feature and pattern recognition techniques are used to identify semantic identifier labels of objects in the point cloud, while optimizing the mapping goal to ensure a balance between computational efficiency and accuracy. This optimization is critical to a successful semantic mapping process and depends on the 3D point cloud data that has been processed; Integrate semantic labels to generate semantic maps with topological information: Combine semantic labels with topological information to create semantic maps. Integrating this information is critical to understanding the layout of the environment and the interconnections between objects, which is necessary for autonomous navigation; Use maps for path planning: The generated semantic map is used to plan a safe and efficient path for the self-driving car, providing the necessary information about the location of objects and their semantic labels to ensure obstacle avoidance and efficient navigation; Continuously update the map with new data: As the self-driving car collects new data, this module updates the semantic map to reflect changes in the environment, ensuring that the map remains accurate over time, enhancing the vehicle's ability to adapt to new obstacles and changes in the surrounding environment. Repeated process of improving accuracy over time: Focuses on the continuous process of data collection, processing, and updating to continuously improve the correctness of the semantic map. It emphasizes how the mapping process is dynamic and changes with each iteration to improve navigation and path planning.

[0039] Specifically, the MLS dataset was collected using the RIEGL VMX-2HA system, a mobile laser scanning system widely used for 3D data collection and mapping in a variety of applications, which focuses on providing precise and accurate point cloud data. Notably, it exhibits an accuracy level of 8mm, ensuring that the captured measurements are close to the true value. In addition, the 5mm accuracy allows for the generation of highly detailed and consistent point clouds, reducing random errors. The RIEGL VMX-2HA boasts an impressive maximum effective measurement rate of 1.1 million measurements per second, which is achieved by utilizing a dual scanning mechanism of outward and return, this dual scanning approach enables comprehensive data collection and improves the overall qualitative output. The line scan speed can be adjusted up to 400 lines per second (2x200 lines per second), providing versatility according to specific requirements. To improve the accuracy and geo-referencing of the collected data, the system is supplemented with fixed GPS stations, which play a vital role in establishing control points, facilitating the precise positioning and alignment of the scanned data within the geographic coordinate system.

[0040] The MLS system moves at speeds of up to 60 km / h, resulting in a point density between 7500 and 8500 points / h. Four MLS test datasets from LAS files were used. Dataset 1 is a short and simple section of about 30 meters long along a straight urban road corridor extending along the road. Dataset 1 contains 1827963 points with a point density of 8500 points / m 2 Datasets 2, 3, and 4 cover three other road segments with lengths of 176.5 m, 175.5 m, and 176.4 m, respectively, and contain 6875, 450, 9850, 578, and 8475868 points, respectively. The median density is 7500 points / m 2 ; Dataset 2 is a two-way road with one-way traffic flow, while Datasets 3 and 4 are three-way roads with lane restrictions and reduction zones. The four datasets have different types of lane markings, roadside bushes and trees, traffic and warning signs, and buildings. The effectiveness of the proposed method is evaluated using these four datasets and compared with other current methods.

[0041] Specifically, in order to extract semantic segmentation from 2D images, the network architecture provided by the DeepLabV3Plus system was used, and a lightweight PointNet based on VoteNet was used as the backbone of the feature extraction system, which can achieve the same mean intersection over union (mIoU) as other backbone networks (such as ResNet101) without requiring so many parameters and faster inference time. The layers and decoder layers also combine depthwise separable convolutions to further reduce inference time while maintaining the same efficiency and performance. The semantic segmentation network was trained in the MLS dataset, which provides a large number of pixel-level semantic segmentation photos in self-driving car scenes and scenes with 66 different types of labels, and the labels are divided into 19 groups (such as zebra crossings and pedestrian crossings) by eliminating unnecessary labels for driving conditions and environments, such as snow, and combining labels with comparable semantic definitions and meanings.

[0042] Datasets 1 to 4 collected along the road provide a total of 18,000 frames and images for training, with an additional 2,000 frames verified and validated. During the integration process, terrain, landforms, and vegetation were fused, various rider categories were grouped into the "person" category, front and rear traffic signs were merged under "traffic signs", bridge data was included in the "building" category, and various pedestrian crossing types were merged into a single "pedestrian crossing" category. Training data augmentation includes random horizontal flipping with a probability of 0.5, random resizing in the range of 0.5 to 2, and random cropping. In addition, image normalization was performed using a distribution with a mean and standard deviation of (0.485, 0.456, 0.406) and (0.229, 0.224, 0.225) respectively. Despite the extensive data augmentation during training, no significant performance degradation was observed during testing due to the similarity between the road dataset and the road test scenarios conducted.

[0043] We trained the model for 200 iterations on 8 2080Ti GPUs, using a group size of 16 and synchronized group normalization. We used an input image size of 640 × 640 to develop the hyperparameters. The output stride of the network was 8. We applied a polynomial learning rate policy and stochastic gradient descent (SGD) optimization technique with a base learning rate of 0.005 and a power of 0.9. The momentum and weight decay rates were 0.9 and Regarding how the metric is defined, mIoU is used to measure how well the network is working, and the ResNeXt50 mIoU on the validation set is 68.32%. ResNeXt50 performs slightly worse than ResNet101, but uses much less memory (from 370MB to 190MB), which is better for onboard hardware because it has a limited amount of memory. The network takes about 0.2 seconds to perform inference on each image.

[0044] In addition, VoteNet is used to build and improve the centralized module, which is then used with a set of The representative feature points and scaling control parameters are optimized by Independent generation and development to its center , and Used as a proportional control parameter to generate an offset from each representative point position Due to the different offsets between different object positions, a value of 0.1 was set in the experimental test. To meet the main design specifications and requirements, the reliability index corresponding to the first pass probability was calculated for each displacement: Representative value , representative value and the resulting representative value For the maximum offset position , the limit of the first channel analysis in the numerical study is fixed at 0.01 m, and these reliability indices are , and .

[0045] In a total of two seconds, the centralization process The time intervals are discretized at s, which indicates that 1000 random variables are used to represent the centralization and asymptotic sampling is used to determine and evaluate the first pass probability, each sample using 5 support points based on 100 Monte Carlo samples. The asymptotic sampling process typically requires less than a thousand Monte Carlo iterations and sampled data points for one of the three reliability indices and one point (distributed throughout the design space). To illustrate the impact of different and discrete object locations and offset variations of the points, the range of these two variables is addressed by adopting an experimental method that relies on Latin hypercube sampling with 100 sampling points.

[0046] The preferred and selected intervals are and On a typical laptop, approximately 300,000 dynamic analyses, each lasting approximately 0.006 seconds, are required to complete the entire optimization process, including the variables , and , which takes about 30 minutes of computation time. Figure 2 The first channel reliability index corresponding to the maximum performance is shown （ ), which depicts how these indicators change with the control parameters of the centralized module ( ) and positional offsets within the design space While variations, the “centralized module” refers to a key component in the method of the present invention, representing the control mechanism that governs the optimization process. Due to the inherent statistical unpredictability and uncertainty associated with Monte Carlo based reliability indices, these exhibit considerable fluctuations. To address these fluctuations, techniques for smoothing the data, such as the moving least squares method, have been employed, the calculated reliability indices and emphasize the need to use smoothing techniques, particularly the well-established moving least squares method, to mitigate the fluctuations observed in this study.

[0047] Smoothed reliability index across the design space like Figure 3 As shown in Figure 2, high-scale control parameters are most beneficial in limiting the maximum position offset, while smaller position offsets help reduce the number of feature points. The feature position is then evaluated using a threshold of 0.01m. , Figure 4 describes the associated first-pass reliability index ( ), a small amount of positional shift can be observed This can have a negative impact on how feature points respond to the surface points of the object. For example, when the scale control parameter is set to a very low level, the feature will tend to act more aggressively because the feature points do not have enough positional offset.

[0048] In summary, Figure 4 Depicted in The first channel reliability index of the feature point represented by , point The main variable for keeping the position offset to a minimum is the finite small-scale control parameter value, from Figures 2 to 4 It can be clearly seen from the comparison of reliability index in and A small part of the entire design space (narrow beams) is all The value provides a relatively large value. For better description, if the value of all safety indicators is specified to be greater than 4, the allowed design space domain is constrained. It can be seen that Pareto optimality methods and management procedures based on Pareto analysis are used in high-definition maps (HD Maps) using mobile LiDAR point clouds in the following ways: (1) Identification of important features: Pareto analysis is used to identify the most critical features of HD maps, such as road geometry, lane markings, and signs, based on the Pareto principle, where 80% of the value comes from 20% of the features. This information can help prioritize mapping work and ensure that resources are appropriately allocated; (2) Optimization: Pareto optimization methods are used to optimize the mapping process by finding the best balance between different objectives such as accuracy, data completeness, and data density. This approach can help design a mapping strategy that achieves the best results while minimizing costs; (3) Quality control: Pareto analysis is used to improve quality control by identifying features with the highest error rates. By identifying and resolving these issues, the overall accuracy of the HD map can be improved; (4) Update cycle planning: Management procedures based on Pareto analysis are also used to plan the update cycle of HD maps, which allows focusing on the parts of the map that need to be updated immediately, much faster than less critical parts. Overall, the application of Pareto optimality methods and management procedures based on Pareto analysis in HD maps can help optimize the mapping process, identify key features that require accuracy, improve quality control, save time and effort, and at the same time ensure high accuracy of HD maps.

[0049] By evaluating and determining the relative depth of semantic pixel data given a semantic image, it is possible to reconfigure the 3D scene with semantic labels, but this data is often inaccessible. It is important to understand the most obvious features and characteristics for interpolation and estimation from multi-view geometry, which is difficult to do when driving or in complex and difficult lighting situations. LiDAR sensors can easily capture depth information and feature sets of objects. However, dedicated underlying geometry can be challenging in real-time environments because they usually have limited optical channels (e.g. 16). Using cm-level localization to identify and extract discrete dense regions from the already constructed dense point cloud helps to solve this problem. These smaller regions are then hypothesized and projected into the semantic image segmentation to extract and collect depth information.

[0050] Automation is a viable option for creating such dense point maps, as it only requires moving a driver around the area and, therefore, is cheaper than manual marking. After localization, the point cloud data is projected into a 2D image space using the intrinsic camera settings, parameters and settings, as well as the relative and comparative transformations between the LiDAR and the camera, and the nearest neighbor search method is used to semantically associate and relate the point cloud data. In addition, the intrinsic matrix of the camera is determined through the traditional chessboard technique, while the relative transformation between the camera and the LiDAR is evaluated and calculated using the PnP method. It is important to point out that the LiDAR point cloud map and the semantic segmentation of the 2D camera image are two different sets of data that provide unique information and serve different purposes. The LiDAR point cloud provides real-time distance, position, orientation, and direction information of the environment, which is very useful for obstacle detection, localization, and mapping. On the other hand, the semantic segmentation of the 2D camera image can identify and classify objects in the image, which is very useful for object detection, tracking, and scene understanding. Here, they are used together in autonomous driving to provide different data types.

[0051] While point clouds with semantic labels inherently preserve the 3D geometry of the environment, it is important to note that such representations of scenes are still susceptible to noisy sensor estimates and slight changes or fluctuations in semantic labels. A map with regional or global probability distributions (probabilistic maps) is maintained to properly address this issue, where local maps can indeed provide dense semantic clues about the vehicle at close range, while global maps can help automatically create HD maps. The reference frame is the main difference between local and global semantic maps using semantic occupancy grids. The computational juxtaposition and comparison are performed within the globalized frame. The local probability map seems to be a bird's-eye view of the vehicle's body frame (rear axle). A local map is made for a specific frame and kept up to date with the semantic point cloud. The acceptable pose sets the origin of that frame. When the difference between the new pose and the previous pose exceeds a certain threshold, a new map is generated and the previous map is modified to compensate for vehicle traffic flow and movement. The local probability map is similar to a bird's-eye view centered on the rear axle of the vehicle, which is continuously updated based on the semantic point cloud data aligned with the acceptable poses to ensure accurate identification. When the difference between the new pose and the previous pose exceeds a predefined threshold, a new map is started and the previous map is adjusted to accommodate changes in vehicle traffic flow and movement. On the other hand, the global probabilistic map works in a global framework and does not require any map modification or revision. High-resolution (0.05m) drone imagery showcases the fine-grained spatial details recorded in the data. This resolution makes it possible to accurately view and analyze the scene, laying a solid foundation for the subsequent semantic mapping process. The graph enables close attention to the complex properties of the environment, which is essential for accurate semantic labeling and creating credible probabilistic maps. For applications in vehicle dynamics and HD map development, this high-resolution imaging helps minimize uncertainties and errors associated with sensor noise and semantic label fluctuations, ensuring a more reliable representation of the surrounding environment.

[0052] The semantic occupancy grid has three dimensions: , and , the semantic class of the scene is associated with each channel. After creating the semantic point cloud, it is projected onto the grid using the x and y components, and the semantic labels of the neighboring cells will be assumed to represent the semantic label of the point. Each cell in the grid represents a physical region whose size (in m) is the square area, where is a discrete factor, which uses semantic and LiDAR intensity information and interest points in the point cloud to minimize the probability of prediction error, improving the reliability and applicability of semantic occupancy grid prediction. The semantic labels of all channels are Denotes that the observed semantic labels are expressed as The laser radar intensity is expressed by Determine and estimate based on previous observations ,Right now The probability distribution of is based on the Markov assumption and the observed semantic labels and the given The conditional independence of the lidar intensity updates the semantic probability as follows: (14) When a normalization factor Z is displayed, it is assumed that , the statistical likelihood and probability that label i is identified as label j is represented by a component in the first row and jth column of the two-dimensional matrix M, which is used to model , which describes the degree of certainty in a prediction so that a probability update can be more accurate. Modeled as a function of how much each class in the scene reflects the rate, the intensity of the lidar sensor provides a comprehensive and clear clue about the various materials present in the scene, e.g. Figure 5 In Figure 2, the top image shows a BEV intensity map for a road segment. Lane markings can reflect higher intensity light because they are painted white. Therefore, they can be identified and distinguished using a threshold k, which can serve as an initial proof of scene arrangement and may be useful in situations where semantic segmentation cannot accurately capture the true labels due to low light conditions.

[0053] The CFN and CFN-improved methods are used, which are crucial for creating HD maps and optimizing probability-based multi-objective processes in urban autonomous driving scenarios. The accurate identification of road features relies on semantic segmentation. However, deep learning models often have difficulties distinguishing visually identical categories, resulting in classification errors that negatively affect map extraction. The CFN technique effectively addresses these difficulties by obtaining valuable information about the model's deficiencies, leveraging the confusion matrix of the semantic segmentation network. The technique consists of two main steps: first, analyzing the confusion matrix to find common misclassifications, and second, changing the network's output according to these patterns to minimize repeated misclassifications. Implementing such a systematic correction procedure greatly improves the accuracy of semantic segmentation, resulting in more reliable and complex maps. By integrating CFN, we can align it with our hierarchical approach, which improves the accuracy of semantic labeling and makes it easier to extract precise geometric and semantic information from LiDAR point clouds. This enhancement is crucial to improving the trade-off between mapping goals, such as accuracy and processing efficiency. Ultimately, it helps create HD maps, which is essential to ensure safe navigation of autonomous vehicles in urban areas. CFN improves segmentation accuracy, allowing optimization algorithms to make more accurate judgments, resulting in accurate and computationally feasible HD maps. Both methods have their advantages, but CFN and CFN-improved methods provide high-precision, cost-effective, faster, and flexible solutions for the creation of HD maps, making them excellent choices for advanced applications such as autonomous driving and intelligent transportation systems. Modeling of observation uncertainty is a key task in generating high-quality maps. A matrix M is designed to simulate the observation of a given Semantic tags The uncertainty is expressed in logarithmic form by M to achieve numerical stability. Two different approaches are considered to explore the best strategy for modeling observation uncertainty: the first approach, called vanilla, uses a normalization factor μ and a hyperparameter λ, denoted as ,in is an input observation matrix supplemented by a vector of 1s; the second method, called CFN, involves exploiting the confusion matrix of the semantic segmentation network in an auxiliary validation dataset. Each cell is assigned the label with the highest probability during inference. The numerical accuracy of these methods is quantitatively evaluated and compared, as shown in Table 1: Table 1. Training and labeled datasets for quantitative evaluation

[0054] Through empirical analysis, we observe that CFN achieves significantly better performance, especially in the case of crosswalks and lane markings. This finding highlights the importance of modeling prediction errors in semantic segmentation using confusion matrices, which ultimately leads to superior map generation results. Note that Expressions are used to model uncertainty in semantic labels based on input observations. , λ and 1 are parameters used in the expression to ensure numerical stability. μ By multiplying, the resulting value can be scaled and adjusted. This expression is used for map generation, which improves the quality of the map by taking into account the uncertainty in the semantic labels. This comparison demonstrates the benefits of using the network's confusion matrix to model and represent the estimation error in semantic segmentation, which leads to better map generation results. As can be seen in Table 1, the road extraction method has an accuracy of 89.16%, a recall of 88.74%, and a FI score of 88.95%; the pedestrian crossing has an accuracy of 80.64%, a recall of 80.24%, and a FI score of 80.44%, and the lane marking has an accuracy of 76.57%, a recall of 76.19%, and a FI score of 76.38%. It should be noted that all F1-score values ​​in Table 1 are displayed to four decimal places to ensure a high level of accuracy in reporting performance metrics. The FI score is a key metric for balancing precision and recall. The inclusion of four decimal places allows for a more detailed and nuanced evaluation of the model performance. It provides a clear and accurate representation of the model's ability to balance false positives and false negatives.

[0055] The point cloud is pre-processed and thresholded using point intensity values, eliminating points with intensities below k, a threshold that makes the road lane markings and crosswalk patterns clearly visible. The dense point cloud is displayed as an overlay on the semantic map. Using point cloud data generated in real time by LiDAR is another way to link semantic images with depth and comprehensive information. Point clouds can be projected into a semantic image framework and a similar mapping method can be used to generate semantic maps, providing real-time performance and accuracy. Semantic maps can be created at longer ranges and distances using pre-built dense point clouds, and homogeneity can be used to back-project the 2D semantic image representation into 3D space, assuming that the ground is smooth and level, and the back-projection method does not produce any black holes. In urban driving situations, the plane presumption fails on steep slopes or road intersections, causing significant distortions and deviations at longer ranges.

[0056] The grid resolution and number are the two main user-configurable map settings. The global probability map is effectively represented in the global coordinate system, which generates reflectance maps with either a full 3D image representation or just a ground plane representation. The first two alternatives (structure and appearance) can also be changed and adapted to any map type. Due to all these variations, there is a large search space for potential map combinations, and five different versions of the centralized module were evaluated. and localization bias and offset, we modified the number of first-pass reliability procedures per cell from 1 to 5 as a preliminary example to qualitatively illustrate what we were able to capture in our maps, snapshots of which are in Figure 6It is described visually in Figure 6 Each column describes a different and distinct mapping, from a probabilistic mixture mapping with only one component per unit to a probabilistic mixture mapping with five components per unit, showing the probability of the central module Positional offset of tissue (left to right) of the averages, so this seems to be particularly noticeable in the ground plane and superstructure, where several components have the same average. There is a large qualitative improvement between the first and second probability maps; in the first probability case, a clear blurring between the ground plane and the superstructure (e.g., plants, trees, and lamp posts) can be seen. However, the second probability scenario easily captures patterns close to the ground plane and supports the superstructure. The high-fidelity map follows this pattern, showing that the ground is captured in the lowest average component. The overfitting of building facades and other superstructure is due to the increased number of first-pass reliability practices and operations. There will be a clear improvement in positioning between the first probability map and the larger number of maps, as keeping these patterns distinct and separated is critical to eliminating obstacles that may develop in the space between the ground and the structure.

[0057] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.

[0058] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0059] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A high-definition map processing method based on mobile laser radar data, characterized in that: The following steps are involved: Acquire sensor data collected by multiple sensors of the autonomous vehicle, wherein the sensor data includes 3D point cloud data of a radar sensor, visual data of a camera, and radar signals and geographic coordinates of a GPS; Performing probabilistic data processing on the sensor data to obtain a structured 3D point cloud, wherein the probabilistic data processing includes extracting geometric parameters, evaluating interface changes, and identifying correlations between points, wherein the geometric parameters include edges, surfaces, and corners of the 3D point cloud; Performing semantic label recognition on the 3D point cloud to assign semantic labels, and creating a geometric mapping function according to the semantic labels; Constructing a 3D point cloud image of the 3D point cloud, and using the geometric mapping function to transform local Euclidean coordinates of the 3D point cloud image to obtain a semantic segmentation result of the 3D point cloud image; Using a confusion fusion network to classify and correct the semantic segmentation results to obtain object attributes of the 3D point cloud image, wherein the classification correction includes integrating sensor data, using a confusion fusion network to correct misclassification, adapting to real-time changes in the environment, a geometric prediction model, and object feature extraction; A semantic map of the 3D point cloud map is constructed based on the object attributes and the geometric mapping function, and a high-definition map corresponding to the semantic map is output, and the high-definition map is evaluated and optimized to obtain a path navigation result of the autonomous driving vehicle.

2. The method for processing high-definition maps based on mobile laser radar data according to claim 1, characterized in that: Probabilistic data processing is performed on the sensor data to obtain a structured 3D point cloud, including: A multi-objective random model is constructed, and the Euclidean nearest neighbors corresponding to the features of the multi-objective random model are obtained from the 3D point cloud. Build the graph , where the vertices and edges of the graph are represented by and Indicates that the neighbor set of each vertex x is composed of express; Introducing a set of input vertex features ,in, Represents each graph vertex , the features or attributes associated with each vertex in the graph G come from the 3D point cloud P, and each vertex in the graph is assigned a feature representation , the feature dimension of each vertex is given by F, and the output of the high-definition map HD The expression is: （1） in, is a geometric mapping function , the local Euclidean coordinates between each vertex and its adjacent Euclidean coordinates to a geometric parameter; Multiply the features To obtain a trainable feature mapping function , and multiply the resulting product with association, Represents the maximum aggregation function; Using a mapping function with trainable parameters The geometric prediction model expresses the structure of the HD map; （2） in, yes Learnable parameters, , By modifying the given learnable parameters, the adjacent set Estimate various geodesic values ​​for each vertex x; use multi-layer perception MLP as the feature mapping function by Indicates that the initial input features , With geometric features The combination of products is mapped to various feature dimensions , and use the maximum aggregation algorithm to learn the new features Perform aggregation.

3. The method for processing high-definition maps based on mobile laser radar data according to claim 2, characterized in that: GeoConv is used for detection in the sampling layer of the geometric prediction model, and PointNet is used in t sampling layers to extract object features and to combine and interpolate various features in subsequent sampling values. The output of PointNet is a vector with coordinates , and Representative points , where GeoConv is used for convolutional layers and PointNet is used to capture local and global information from point clouds; According to the representative point Offset each representative point to its center, and adjust the features of the representative point As input, the 3D position offset in the Euclidean domain and feature offset As output, from the representative point generate ,in, , , the estimated 3D offset It is controlled by the following loss function: （3） Among them, 1[ on object] represents the representative point Is it on the surface of the object? Represents the total number of representative points on the surface of the object, Indicates the position of the representative point The ground truth displacement from the center of the object's bounding box.

4. The method for processing high-definition maps based on mobile laser radar data according to claim 3 is characterized in that: Probabilistic data processing includes: Input representative point features And output the 3D position offset of the generated representative point , a random variable with n-dimensional space The probability of failure for: （4） in, Represents the joint probability function of random variables, and the failure region of the n-dimensional random variable space is given by express, is the failure domain; the joint probability function is used to extract the scene context around the object center to provide a threshold, and the corresponding reliability index The expression is: （5） in, Represents the inverse normalized Gaussian distribution function, default is a Gaussian random variable with mean and standard deviation is zero, considering the limit state function is highly nonlinear. Can be used to represent failure; if the standard deviation of a Gaussian variable with the same distribution is ,in, As , reliability index Depends on the standard deviation .

5. The method for processing high-definition maps based on mobile laser radar data according to claim 4 is characterized in that: Also includes: Based on the linear limit state function, the failure probability is calculated and defined. The failure conditions and failure scenarios in Gaussian space are given by Given, where is the most likely failure point, defined using the optimal solution and ; The reliable safety index of the first channel of probabilistic data processing is determined by Monte Carlo asymptotic sampling algorithm. In n-dimensional Gaussian space, the variables are all equally expanded and relatively distributed. The failure probability is controlled by adjusting the standard deviation. The reliability index and the standard deviation of the random variable The correlation can be expressed as: （6） when When, through When the necessary asymptotic behavior is satisfied, Different support points, based on The Montero estimation uses a least squares regression analysis to define the coefficients A and B, and the fitting process is adjusted according to the scaled reliability index provided by expression (6) as follows: （7） Here, rewriting formula (6) gives the same weight to each individual support point in the regression analysis, and choosing the initial factor ,Pick A useful collection of The number of samples in the The accuracy is reduced by factor f and iterated to obtain an acceptable ; The simulation model and f are gradually reduced to obtain the paired f and After identifying the parameters, regression is applied to the data to obtain the functional relationship The reliability index is Push Got it.

6. The method for processing high-definition maps based on mobile laser radar data according to claim 5, characterized in that: The confusion fusion network is used to classify and correct the semantic segmentation result to obtain the object attributes of the 3D point cloud image, and further includes: The object extraction process is represented by independent and identically distributed IID random variables as , which is assumed to be constant over discrete event intervals ∆t, and has zero mean and variance , object characteristics The intensity variance and time interval are available ∆t It is expressed as: （8） The independent and identically distributed IID value sequences represent the surface points of the object and are random variables with unit standard deviation ,variable The expression is: （9）。 7. The method for processing high-definition maps based on mobile laser radar data according to claim 6, characterized in that: The sampling and clustering procedures are selected based on spatial proximity to ensure that the project can represent enough potential objects with data, using the farthest point sampling FPS according to the typical representative point position in 3D Euclidean space. , select a subset of K points for sampling and calculate the typical representative points The index of the moving representative point is obtained ; By Find nearby points in Cluster N grouping points. If and , then the matching feature of each group point is specified as , where r is the radius; A shared MLP is used to map the features of the surrounding points of each item, and the feature aggregation uses the maximum operation: （10） in, is the comparison coordinate between adjacent positions and points, and For,the result features of the aggregation, display the object properties.

8. The method for processing high-definition maps based on mobile laser radar data according to claim 1, characterized in that: Constructing a semantic graph of the 3D point cloud graph based on the object attributes and the geometric mapping function, comprising: Determine the neighboring items based on the predicted bounding box center position, using a 3D bounding box as Perform object feature recognition, where are the coordinates of the object center, are the height, width and length of the object size, is the object direction; Preset Object Features , using three layers with full connectivity to estimate bounding box parameters ; Use the projected bounding box center coordinates and position The neighbors of each item are identified, and the relationship between the item and its surrounding and adjacent items is represented as an optimal path from region to region and an undirected graph, using the symbol It indicates that, and denote vertices and edges respectively, Represents the characteristics of the i-th adjacent item and determines the relationship parameters Possible relationship characteristics with objects ; （11） (12） （13） in, Is has trainable parameters with values, , the relationship features is passed to three fully connected layers, designated as , and The sum of is the final output of the network.

9. The method for processing high-definition maps based on mobile laser radar data according to claim 1, characterized in that: The semantic label includes at least one of a lane, a crosswalk, a car, and a navigation indicator.

10. A high-definition map processing device based on mobile laser radar data, characterized in that: The method for processing a high-definition map based on mobile laser radar data as described in any one of claims 1 to 9 comprises: A data acquisition unit, used to acquire sensor data collected by multiple sensors of the autonomous driving vehicle, wherein the sensor data includes 3D point cloud data of a radar sensor, visual data of a camera, and radar signals and geographic coordinates of a GPS; A probability processing unit, configured to perform probability data processing on the sensor data to obtain a structured 3D point cloud, wherein the probability data processing includes extracting geometric parameters, evaluating interface changes, and identifying correlations between points, and the geometric parameters include edges, surfaces, and corners of the 3D point cloud; A semantic recognition unit, configured to perform semantic label recognition on the 3D point cloud to assign semantic labels, and to create a geometric mapping function according to the semantic labels; A semantic segmentation unit, used to construct a 3D point cloud image of the 3D point cloud, and to convert the local Euclidean coordinates of the 3D point cloud image using the geometric mapping function to obtain a semantic segmentation result of the 3D point cloud image; A classification correction unit, configured to classify and correct the semantic segmentation result using a confusion fusion network to obtain object attributes of the 3D point cloud image, wherein the classification correction includes integrating sensor data, correcting misclassification using a confusion fusion network, adapting to real-time changes in the environment, a geometric prediction model, and object feature extraction; An evaluation and optimization unit is used to construct a semantic map of the 3D point cloud map based on the object attributes and the geometric mapping function, and output a high-definition map corresponding to the semantic map, and evaluate and optimize the high-definition map to obtain a path navigation result of the autonomous driving vehicle.

Images