Road three-dimensional model construction method, device, computer equipment and storage medium

Abstract

The embodiments of the present application belong to the field of intelligent traffic management and control technology, and relate to a method, device, computer equipment and storage medium for constructing a three-dimensional road model. The method uses lidar technology to obtain point cloud data while collecting GNSS data, and adopts deep learning technology to realize the joint processing of the two, thereby improving the accuracy and robustness of the data; based on the real-time fusion of multi-source data, a dynamic road model is generated to enhance the system's perception of the road environment; the constructed high-precision dynamic three-dimensional model can realize real-time tracking and prediction of dynamic objects, providing the system with accurate environmental understanding and inference of future motion, and is suitable for more complex scenarios.

Description

Technical Field

[0001] The present application relates to the field of intelligent traffic management and control technology, and in particular to a method, device, computer equipment and storage medium for constructing a three-dimensional road model. Background Art

[0002] Smart transportation is a key approach to solving traffic problems and improving transportation efficiency. With the acceleration of urbanization and the continued growth of traffic volume, intelligent transportation systems are playing an increasingly critical role. By integrating advanced information technology, intelligent transportation systems can optimize the allocation of transportation resources, improve road capacity, and provide strong support for the sustainable development of cities.

[0003] Traditional traffic management systems rely primarily on GNSS data. However, in practice, GNSS data is susceptible to interference, exhibits high levels of noise, and suffers from insufficient positioning accuracy. Furthermore, the system's coordinate conversion capabilities are weak, making it difficult to accurately match GNSS data with three-dimensional road models. These issues prevent the system from accurately representing road conditions, making it particularly difficult to provide timely and effective information support when responding to sudden traffic incidents. Furthermore, the static road models typically used in conventional traffic systems struggle to cope with the complexity and variability of traffic events. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to propose a method, apparatus, computer equipment and storage medium for constructing a three-dimensional road model to solve the problem that traditional traffic management systems have difficulty in achieving accurate matching of GNSS data with three-dimensional road models.

[0005] In order to solve the above technical problems, the present application provides a method for constructing a three-dimensional road model, which adopts the following technical solutions:

[0006] Receiving a road model building request sent by a user terminal, wherein the road model building request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign;

[0007] performing a feature fusion operation on the GNSS data and the point cloud data according to a deep learning technique to obtain fused feature data;

[0008] Inputting the fused feature data into a deep learning model to perform dynamic object recognition and segmentation operations to obtain dynamic object data;

[0009] Performing real-time tracking processing on the dynamic object data according to a multi-target tracking algorithm to obtain dynamic vehicle tracking data;

[0010] A static road model corresponding to the target road sign is obtained, and the static road model and the dynamic vehicle tracking data are combined to obtain a three-dimensional model of the target road.

[0011] Furthermore, the step of performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain fused feature data specifically includes the following steps:

[0012] Performing encoding operation on the GNSS data to obtain vehicle position information;

[0013] performing a feature extraction operation on the point cloud data according to a three-dimensional convolutional neural network to obtain point cloud feature data;

[0014] The vehicle position information and the point cloud feature data are input into a deep learning network for joint processing to obtain the fused feature data.

[0015] Furthermore, before the step of performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain fused feature data, the following steps are also included:

[0016] A data optimization operation is performed on the point cloud data according to the bilateral filtering method and the MLS algorithm.

[0017] Furthermore, before the step of performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain fused feature data, the following steps are also included:

[0018] Decomposing the GNSS data to obtain component GNSS data at multiple scales;

[0019] Perform feature extraction operations on the component GNSS data of each scale to obtain local feature data corresponding to each scale;

[0020] Inputting the local feature data corresponding to each scale into the SVR model to perform smoothing parameter prediction operation to obtain the predicted smoothing parameters of each scale;

[0021] Adaptively smoothing the component GNSS data at the plurality of scales according to the prediction smoothing parameters at the respective scales to obtain smoothed component GNSS data at the respective scales;

[0022] A reconstruction operation is performed on the smoothed component GNSS data of each scale to obtain smoothed GNSS data.

[0023] Furthermore, the step of adaptively smoothing the component GNSS data of the plurality of scales according to the prediction smoothing parameters of the respective scales to obtain smoothed component GNSS data of the respective scales specifically includes the following steps:

[0024] The adaptive smoothing process is optimized according to an incrementally optimized coordinate transformation method to obtain the smoothed component GNSS data of each scale.

[0025] Furthermore, the step of encoding the GNSS data to obtain vehicle location information specifically includes the following steps:

[0026] The GNSS data is calculated according to the vehicle trajectory GNSS algorithm to obtain the vehicle position information, wherein the vehicle position information is expressed as:

[0027]

[0028] in, represents the trajectory similarity function of the vehicle between time i-1 and time i, v represents the velocity vector of the actual vehicle movement, represents the velocity vector of the reference vehicle motion, k represents the total number of trajectory data, represents the dot product of the actual velocity vector and the reference velocity vector at all trajectory points, and Represent the modulus of the actual velocity vector and the reference velocity vector respectively.

[0029] In order to solve the above technical problems, the present application also provides a device for constructing a three-dimensional road model, which adopts the following technical solution:

[0030] a request receiving module, configured to receive a road model building request sent by a user terminal, wherein the road model building request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign;

[0031] A feature fusion module is used to perform a feature fusion operation on the GNSS data and the point cloud data based on deep learning technology to obtain fused feature data;

[0032] A dynamic object recognition and segmentation module is used to input the fused feature data into a deep learning model to perform dynamic object recognition and segmentation operations to obtain dynamic object data;

[0033] A real-time tracking module is used to perform real-time tracking processing on the dynamic object data according to a multi-target tracking algorithm to obtain dynamic vehicle tracking data;

[0034] The combining module is used to obtain a static road model corresponding to the target road sign, and combine the static road model with the dynamic vehicle tracking data to obtain a three-dimensional model of the target road.

[0035] Furthermore, the feature fusion module includes:

[0036] An encoding submodule, configured to perform encoding operations on the GNSS data to obtain vehicle position information;

[0037] A feature extraction submodule, configured to perform a feature extraction operation on the point cloud data according to a three-dimensional convolutional neural network to obtain point cloud feature data;

[0038] The combined submodule is used to input the vehicle position information and the point cloud feature data into a deep learning network for combined processing to obtain the fused feature data.

[0039] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0040] The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the above-mentioned method for constructing a three-dimensional road model when executing the computer-readable instructions.

[0041] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0042] The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the method for constructing a three-dimensional road model.

[0043] The present application provides a method for constructing a three-dimensional road model, comprising: receiving a road model construction request sent by a user terminal, wherein the road model construction request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign; performing a feature fusion operation on the GNSS data and the point cloud data based on deep learning technology to obtain fused feature data; inputting the fused feature data into a deep learning model to perform dynamic object recognition and segmentation operations to obtain dynamic object data; performing real-time tracking processing on the dynamic object data based on a multi-target tracking algorithm to obtain dynamic vehicle tracking data; obtaining a static road model corresponding to the target road sign, and combining the static road model and the dynamic vehicle tracking data to obtain a target road three-dimensional model. Compared with the existing technology, the present application uses lidar technology to obtain point cloud data while collecting GNSS data, and adopts deep learning technology to realize joint processing of the two, thereby improving the accuracy and robustness of the data. Based on the real-time fusion of multi-source data, a dynamic road model is generated, enhancing the system's perception of the road environment. The constructed high-precision dynamic three-dimensional model can realize real-time tracking and prediction of dynamic objects, providing the system with accurate environmental understanding and inference of future motion, and is suitable for more complex scenarios. In addition, this application proposes a smoothing algorithm based on wavelet transform, which is more robust than traditional filtering methods. By performing multi-resolution analysis on vehicle trajectory data and combining it with a fixed threshold method, the denoised trajectory data is ensured to maintain details while improving smoothness and stability. This application introduces a weighted sum of translation vectors and a nonlinearly corrected rotation matrix. By dynamically adjusting weights and angle corrections, it solves the low-precision problem of traditional coordinate transformation in multi-axis rotation and complex environments. This method effectively responds to different distance conversion requirements and improves accuracy and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the solutions in this application, a brief introduction will be given below to the drawings required for use in the description of the embodiments of this application. Obviously, the drawings described below are some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0045] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0046] Figure 2 This is a flowchart of the method for constructing a three-dimensional road model provided in an embodiment of the present application;

[0047] Figure 3 Schematic diagram of the structure of the device for constructing a three-dimensional road model provided in an embodiment of the present application;

[0048] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of the application are for the purpose of describing specific embodiments only and are not intended to limit this application. The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0050] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0051] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0052] like Figure 1 As shown, system architecture 100 may include a terminal device 101, a network 102, and a server 103. Terminal device 101 may be a laptop computer 1011, a tablet computer 1012, or a mobile phone 1013. Network 102 is a medium for providing a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0053] The user can use the terminal device 101 to interact with the server 103 via the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0054] The terminal device 101 can be various electronic devices with a display screen and supporting web browsing. In addition to the laptop computer 1011, the tablet computer 1012 or the mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop computer, a desktop computer, etc.

[0055] The server 103 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal device 101 .

[0056] It should be noted that the road three-dimensional model construction method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the road three-dimensional model construction device is generally set in the server / terminal device.

[0057] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0058] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for constructing a three-dimensional road model according to the present application. The method for constructing a three-dimensional road model includes: step S201, step S202, step S203, step S204 and step S205.

[0059] In step S201 , a road model building request sent by a user terminal is received, wherein the road model building request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign.

[0060] In the embodiments of the present application, the user terminal refers to a terminal device used to execute the image processing method for preventing document abuse provided by the present application. The user terminal can be a mobile terminal such as a mobile phone, a smart phone, a laptop computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a navigation device, etc., as well as a fixed terminal such as a digital TV, a desktop computer, etc. It should be understood that the examples of user terminals here are only for convenience of understanding and are not used to limit the present application.

[0061] In the embodiment of the present application, the timestamp of the sensor data is used for time alignment during data synchronization. Assume that the time of the lidar data is t LiDAR , the time of GNSS data is t GNSS , time alignment is achieved through linear interpolation:

[0062] (Formula 1);

[0063] in, Represents the time offset between two sensors, which is corrected by interpolation.

[0064] In step S202, a feature fusion operation is performed on the GNSS data and the point cloud data according to deep learning technology to obtain fused feature data.

[0065] In the embodiment of the present application, the feature fusion operation can be to jointly process the data of the two sensors through a deep learning network. A multi-layer perceptron (MLP) is used, and the input is GNSS data x gnss and point cloud data x lidra The output is the fused location information and environment information:

[0066] (Formula 2);

[0067] Among them, y fusion Represents the fused features, including location and environment information; f fusion Represents the MLP model that completes the fusion operation;

[0068] Through neural network learning, optimize the loss function of data fusion, such as minimizing the position estimation error:

[0069] (Formula 3);

[0070] Among them, L represents the total error of model optimization, y fusion,i Represents the predicted position after data fusion, y groundtruth,i Indicates the real location.

[0071] In some optional implementations of the embodiments of the present application, the neural network can be trained using multiple optimization algorithms to achieve joint processing of GNSS data and lidar data, ensuring the robustness and accuracy of the fusion results under various environmental conditions.

[0072] In the embodiments of this application, the multi-sensor data fusion method proposed in this application fully utilizes the advantages of multiple sensors to achieve data complementarity. Through the deep learning model, the deep features of multi-sensor data are further extracted, thereby ensuring the accuracy of perception, reducing the deviation of a single sensor, and significantly improving the robustness of the system.

[0073] In the embodiment of the present application, after pre-processing the GNSS data and point cloud data, the data is optimized, laying the foundation for subsequent three-dimensional modeling.

[0074] In step S203, the fused feature data is input into a deep learning model to perform dynamic object recognition and segmentation operations to obtain dynamic object data.

[0075] In this embodiment, a deep learning model is used to process point cloud data to identify and segment dynamic vehicles. The model inputs the fused features of point cloud data and GNSS data (the features obtained from the previous fusion step) and outputs labels that distinguish between static and dynamic objects. The calculation formula is as follows:

[0076] (Formula 4);

[0077] Among them, y dynamic represents the segmentation result of dynamic vehicles; f segmentation represents the deep learning model that implements the segmentation task; x fusion Represents the fused feature data.

[0078] In step S204, the dynamic object data is tracked in real time according to a multi-target tracking algorithm to obtain dynamic vehicle tracking data.

[0079] In this embodiment of the application, after obtaining the dynamic object, it is necessary to use a multi-target tracking algorithm (Kalman filter) to track the dynamic vehicle in real time. GNSS data can be used for initial positioning, and the lidar point cloud is used for fine-grained motion estimation of the vehicle:

[0080] (Formula 5);

[0081] Among them, P t+1 represents the predicted position of the vehicle at time t+1, P t represents the position of the object at the current time t, v t represents the speed of the object at the current moment, a t Indicates the acceleration of the object at the current moment.

[0082] In step S205, a static road model corresponding to the target road sign is obtained, and the static road model and the dynamic vehicle tracking data are combined to obtain a three-dimensional model of the target road.

[0083] In the embodiment of the present application, the geometric features of static objects are extracted by clustering and segmenting the point cloud data, and the overall static map is constructed by the ICP registration algorithm:

[0084] (Formula 6);

[0085] Among them, M static Represented as a three-dimensional model of a static road, x lidar Represents point cloud data obtained from a lidar sensor.

[0086] In the embodiment of the present application, the static road model and dynamic vehicle information are combined to generate a real-time updated three-dimensional model of the road scene:

[0087] (Formula 7);

[0088] Among them, M road Represents the final three-dimensional scene, M static Represents a three-dimensional model of a static road; P vehicle (t) represents the position of the dynamic vehicle at time t.

[0089] In the embodiments of this application, the multimodal fusion 3D modeling method proposed in this application can provide more complete scene understanding and modeling results through the collaborative work of multiple sources of information in complex environments. This multimodal-driven 3D modeling method can accurately capture the geometric information of the surrounding environment, significantly improving data accuracy. In addition, the method can distinguish dynamic objects in the environment in real time and predict and track the trajectories of these dynamic objects through a Kalman filter, thereby enhancing the perception of the road environment.

[0090] In the embodiment of the present application, the construction of a high-precision three-dimensional road model has been completed. The next step is to accurately match the on-board GNSS data with the model to achieve accurate positioning of the vehicle in three-dimensional space.

[0091] In an embodiment of the present application, a method for constructing a three-dimensional road model is provided, comprising: receiving a road model construction request sent by a user terminal, wherein the road model construction request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign; performing a feature fusion operation on the GNSS data and point cloud data according to deep learning technology to obtain fused feature data; inputting the fused feature data into a deep learning model to perform dynamic object recognition and segmentation operations to obtain dynamic object data; performing real-time tracking processing on the dynamic object data according to a multi-target tracking algorithm to obtain dynamic vehicle tracking data; obtaining a static road model corresponding to the target road sign, and combining the static road model and the dynamic vehicle tracking data to obtain a target road three-dimensional model. Compared with the prior art, the present application uses lidar technology to obtain point cloud data while collecting GNSS data, and adopts deep learning technology to realize joint processing of the two, thereby improving the accuracy and robustness of the data. Based on the real-time fusion of multi-source data, a dynamic road model is generated, enhancing the system's perception of the road environment. The constructed high-precision dynamic three-dimensional model can realize real-time tracking and prediction of dynamic objects, providing the system with accurate environmental understanding and inference of future motion, and is suitable for more complex scenarios. In addition, this application proposes a smoothing algorithm based on wavelet transform, which is more robust than traditional filtering methods. By performing multi-resolution analysis on vehicle trajectory data and combining it with a fixed threshold method, the denoised trajectory data is ensured to maintain details while improving smoothness and stability. This application introduces a weighted sum of translation vectors and a nonlinearly corrected rotation matrix. By dynamically adjusting weights and angle corrections, it solves the low-precision problem of traditional coordinate transformation in multi-axis rotation and complex environments. This method effectively responds to different distance conversion requirements and improves accuracy and adaptability.

[0092] In some optional implementations of the embodiments of the present application, the step of performing a feature fusion operation on the GNSS data and the point cloud data based on the deep learning technology to obtain fused feature data specifically includes the following steps:

[0093] Encode the GNSS data to obtain vehicle location information;

[0094] Perform feature extraction on point cloud data according to a three-dimensional convolutional neural network to obtain point cloud feature data;

[0095] The vehicle location information and point cloud feature data are input into the deep learning network for joint processing to obtain fused feature data.

[0096] In the embodiment of the present application, the GNSS data is encoded and converted into the vehicle's position information, which is represented by x gnss :

[0097] x gnss =f gnss (longitude, latitude, altitude) (Formula 8);

[0098] A 3D convolutional neural network is used to extract features from the lidar point cloud to capture local and global spatial information. The feature representation of the point cloud is x lidar :

[0099] x lidar =f lidra (Point Cloud Data) (Formula 9);

[0100] Among them, x lidar Represents the characteristics of the point cloud; f lidra represents a three-dimensional convolutional neural network for feature extraction;

[0101] In some optional implementations of the embodiments of the present application, before performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain fused feature data, the following steps are also included:

[0102] Data optimization operations are performed on point cloud data based on bilateral filtering and MLS algorithm.

[0103] In the embodiment of the present application, bilateral filtering can be used to remove outliers, but it has limitations in retaining model details. The MLS algorithm recalculates the coordinates of the points by polynomial fitting, which can better retain detailed features. To this end, the present application combines bilateral filtering and the MLS algorithm to optimize point cloud data processing to improve the ability to retain details. The improved filter weight formula is:

[0104] (Formula 10);

[0105] Where W represents the filter weight; exp represents the natural exponential function, Represents the distance between a point and the center point, Represents the standard deviation of the Gaussian distribution.

[0106] By combining the two algorithms, point cloud data processing is further optimized, and experimental comparative analysis is conducted.

[0107] In some optional implementations of the embodiments of the present application, before performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain fused feature data, the following steps are also included:

[0108] Decompose the GNSS data to obtain component GNSS data of several scales;

[0109] Perform feature extraction operations on the component GNSS data of each scale to obtain local feature data corresponding to each scale;

[0110] The local feature data corresponding to each scale is input into the SVR model for smoothing parameter prediction operation to obtain the predicted smoothing parameters of each scale;

[0111] Adaptively smoothing the component GNSS data of several scales according to the prediction smoothing parameters of each scale to obtain smoothed component GNSS data of each scale;

[0112] The smoothed component GNSS data of each scale is reconstructed to obtain smoothed GNSS data.

[0113] In this application, an adaptive smoothing algorithm combining support vector regression (SVR) is proposed to address the issues of high noise and low accuracy in GNSS data. This algorithm dynamically adjusts smoothing parameters based on input data characteristics (such as noise levels and signal variations) through machine learning, achieving adaptive optimization. This method effectively addresses changes in signal characteristics in a variety of scenarios, providing enhanced smoothing performance and data recovery capabilities. Specifically:

[0114] ① Perform a scale analysis transformation on the input signal x(t) and decompose it into components x at different scales s s (t). The calculation formula is as follows:

[0115] (Formula 11);

[0116] Where x(t) represents the component of the signal at scale s.

[0117] ②At each scale s, for the component x s (t) Perform feature extraction to capture the local characteristics of the signal (such as noise level, gradient, etc.) as input for support vector regression. The calculation formula is as follows:

[0118] (Formula 12);

[0119] Among them, f s (t) is the feature vector extracted at scale s and time t, containing n features.

[0120] ③Use the extracted features f s (t) is used as input and the SVR model is used to predict the adaptive smoothing parameters for each scale and position :

[0121] (Formula 13);

[0122] Among them, SVR s is a support vector regression model trained on scale s, is the smoothing parameter of the forecast.

[0123] ④Use the predicted smoothing parameter , for each scale component x s (t) uses Gaussian filtering for adaptive smoothing. The calculation formula is as follows:

[0124] (Formula 14);

[0125] in, is the signal component after smoothing at scale s, is the time variable, exp is the natural exponential function, It is a component of the integral symbol, indicating that the integral variable is .

[0126] ⑤Smooth the signal components at each scale Reconstruct and obtain the final smooth signal :

[0127] (Formula 15);

[0128] ⑥Output the reconstructed smooth signal , completing the adaptive smoothing of the input signal.

[0129] In some optional implementations of the embodiments of the present application, the step of adaptively smoothing the component GNSS data of multiple scales according to the prediction smoothing parameters of each scale to obtain smoothed component GNSS data of each scale specifically includes the following steps:

[0130] Adaptive smoothing is optimized according to the incremental optimization coordinate transformation method to obtain smoothed component GNSS data of various scales.

[0131] In the embodiments of this application, in order to further improve the adaptability and accuracy of the algorithm and better cope with environmental changes and data inconsistencies, this application proposes an incremental optimization coordinate transformation method. This method combines graph optimization technology to process the coordinate transformation of GNSS and LiDAR data, and continuously iteratively adjusts the node positions through the optimization algorithm to achieve high-precision coordinate transformation. Specifically:

[0132] ① Initial coordinate transformation:

[0133] Determine the initial transformation relationship between the GNSS and LiDAR coordinate systems, including the initial rotation matrix R and translation vector t.

[0134] ②Build a graph model:

[0135] Node: Each position or posture is represented as a node v i ;

[0136] Edges: Constraints between nodes are represented as edges, including:

[0137] Constraints between GNSS observations and LiDAR poses;

[0138] Motion model constraints between consecutive LiDAR poses.

[0139] ③Define the error function:

[0140] For each edge, define the error function e i , which measures the degree of constraint satisfaction between nodes.

[0141] 1. Error between GNSS and LiDAR:

[0142] For GNSS observations P GNSS and the corresponding LiDAR pose T i , the error function is:

[0143] (Formula 16);

[0144] Among them, e GNSS represents the position error vector; R i and t i is the rotation and translation of the LiDAR at time i, and P is the reference point in the LiDAR coordinate system.

[0145] 2. Error of motion model:

[0146] For continuous LiDAR pose T i and T i+1 , the error function is:

[0147] (Formula 17);

[0148] Among them, e motion represents the error of the motion model;

[0149] Indicates the pose T at the next moment i+1 The inverse transformation matrix of

[0150] T i represents the LiDAR pose transformation matrix at time i; u i is the predicted increment of the motion model.

[0151] ④Construct optimization objective function

[0152] Establish an overall error metric and optimize the coordinate transformation parameters by minimizing this error. Accumulate the sum of squares of all errors to form a scalar function that needs to be minimized.

[0153] (Formula 18);

[0154] Where E(R,t) represents the objective function; e i represents the specific error term.

[0155] ⑤Incremental optimization

[0156] Through iterative updates, the unknown rotation matrix and translation vector are adjusted in the direction of reducing the error, and the optimal value is finally found.

[0157] 1. Linearize the error function: approximate the nonlinear error function into a linear form to facilitate the solution.

[0158] Perform a first-order Taylor expansion of the error function at the current estimate.

[0159] (Formula 19);

[0160] Where R represents the currently estimated rotation matrix; represents the incremental adjustment of the rotation matrix; t represents the current estimated translation vector; Indicates the incremental adjustment of the translation vector; J i The Jacobian matrix representing the error function with respect to its parameters.

[0161] 2. Construct a system of linear equations:

[0162] The linear approximations of all errors are combined to form a linear system.

[0163] Summarizing all errors, we get the normal equation:

[0164] (Formula 20);

[0165] in, Include and , H represents the information matrix, the product of the summary Jacobian matrix, and b represents the accumulation of error terms.

[0166] 3. Solve the increment :

[0167] By using the least squares method to solve the above equations, we can find the updated amount of parameters and reduce the error.

[0168] 4. Update parameters:

[0169] Update the rotation matrix and translation vector:

[0170] (Formula 21);

[0171] (Formula 22);

[0172] 5. Iterative Optimization

[0173] Continue to reduce the error until the convergence condition is met.

[0174] Repeat the above steps until the increment is small enough or the maximum number of iterations is reached.

[0175] 6. Apply optimal conversion parameters

[0176] The optimal parameters obtained by optimization are used to convert LiDAR data into the global coordinate system of GNSS to achieve accurate alignment.

[0177] (Formula 23);

[0178] Among them, R * and t * Represents the optimal rotation matrix and translation vector obtained by optimization, P LiDAR represents the coordinates of the LiDAR point, P global Represents the transformed global coordinates.

[0179] Through the above steps, this study achieved high-precision coordinate conversion between GNSS and LiDAR data. Compared with traditional translation and rotation processing methods, this method has higher adaptability and accuracy, and can more effectively cope with environmental changes and data inconsistencies.

[0180] In some optional implementations of the embodiments of the present application, the step of encoding the GNSS data to obtain the vehicle location information specifically includes the following steps:

[0181] The GNSS data is calculated according to the vehicle trajectory GNSS algorithm to obtain the vehicle position information, where the vehicle position information is expressed as:

[0182] (Formula 24);

[0183] in, represents the trajectory similarity function of the vehicle between time i-1 and time i, represents the position or trajectory state of the vehicle in the target coordinate system at time i, v represents the velocity vector of the actual vehicle motion, represents the velocity vector of the reference vehicle motion, k represents the total number of trajectory data, represents the dot product of the actual velocity vector and the reference velocity vector at all trajectory points, and Represent the modulus of the actual velocity vector and the reference velocity vector respectively.

[0184] In the embodiment of the present application, the algorithm will use GNSS data to obtain the real-time position information of the vehicle, and combine it with the road environment data to ensure the accurate match between the vehicle trajectory and the road model.

[0185] In the embodiments of this application, by denoising and smoothing the GNSS data, more accurate and stable vehicle trajectory information is obtained, achieving a precise match between the GNSS data and the 3D model. To better display and analyze this data and enable real-time monitoring and management of road traffic, this application constructs a digital twin holographic road traffic platform.

[0186] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve results.

[0187] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0188] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware using computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium. When executed, the program can include the processes in the above-described method embodiments. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0189] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0190] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a device for constructing a three-dimensional road model. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0191] like Figure 3 As shown, the road three-dimensional model construction device 200 of the embodiment of the present application includes:

[0192] The request receiving module 210 is configured to receive a road model building request sent by a user terminal, wherein the road model building request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign;

[0193] A feature fusion module 220 is configured to perform feature fusion operations on the GNSS data and the point cloud data based on deep learning technology to obtain fused feature data;

[0194] The dynamic object recognition and segmentation module 230 is used to input the fused feature data into the deep learning model to perform dynamic object recognition and segmentation operations to obtain dynamic object data;

[0195] A real-time tracking module 240 is used to perform real-time tracking processing on dynamic object data according to a multi-target tracking algorithm to obtain dynamic vehicle tracking data;

[0196] The combining module 250 is used to obtain a static road model corresponding to the target road sign, and combine the static road model with the dynamic vehicle tracking data to obtain a three-dimensional model of the target road.

[0197] In an embodiment of the present application, a road three-dimensional model construction device 200 is provided, comprising: a request receiving module 210 for receiving a road model construction request sent by a user terminal, wherein the road model construction request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign; a feature fusion module 220 for performing a feature fusion operation on the GNSS data and point cloud data based on deep learning technology to obtain fused feature data; a dynamic object recognition and segmentation module 230 for inputting the fused feature data into a deep learning model for dynamic object recognition and segmentation operations to obtain dynamic object data; a real-time tracking module 240 for performing real-time tracking processing on the dynamic object data based on a multi-target tracking algorithm to obtain dynamic vehicle tracking data; and a combining module 250 for obtaining a static road model corresponding to the target road sign and combining the static road model with the dynamic vehicle tracking data to obtain a target road three-dimensional model. Compared with the prior art, the present application uses lidar technology to obtain point cloud data while collecting GNSS data, and adopts deep learning technology to achieve joint processing of the two, thereby improving the accuracy and robustness of the data. Based on the real-time fusion of multi-source data, a dynamic road model is generated to enhance the system's perception of the road environment. The constructed high-precision dynamic three-dimensional model can realize real-time tracking and prediction of dynamic objects, provide the system with accurate environmental understanding and inference of future movements, and is suitable for more complex scenarios. In addition, this application proposes a smoothing algorithm based on wavelet transform, which is more robust than traditional filtering methods. By performing multi-resolution analysis on vehicle trajectory data and combining it with a fixed threshold method, the denoised trajectory data is ensured to improve smoothness and stability while maintaining details; this application introduces a weighted sum of translation vectors and a nonlinearly corrected rotation matrix. By dynamically adjusting weights and angle corrections, the low-precision problem of traditional coordinate transformation in multi-axis rotation and complex environments is solved. This method effectively responds to different distance conversion requirements and improves accuracy and adaptability.

[0198] In some optional implementations of the embodiments of the present application, the feature fusion module includes:

[0199] The encoding submodule is used to encode the GNSS data to obtain the vehicle position information;

[0200] The feature extraction submodule is used to perform feature extraction operations on point cloud data based on a three-dimensional convolutional neural network to obtain point cloud feature data;

[0201] The joint submodule is used to input vehicle location information and point cloud feature data into the deep learning network for joint processing to obtain fused feature data.

[0202] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device according to an embodiment of the present application.

[0203] The computer device 300 includes a memory 310, a processor 320, and a network interface 330 that are interconnected through a system bus. It should be noted that the figure only shows the computer device 300 having components 310-330, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0204] The computer device may be a desktop computer, notebook computer, PDA, cloud server, etc. The computer device may interact with the user via a keyboard, mouse, remote control, touchpad, or voice control device.

[0205] The memory 310 includes at least one type of readable storage medium, including flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, a magnetic disk, an optical disk, etc. In some embodiments, the memory 310 may be an internal storage unit of the computer device 300, such as a hard disk or memory of the computer device 300. In other embodiments, the memory 310 may also be an external storage device of the computer device 300, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Of course, the memory 310 may also include both the internal storage unit of the computer device 300 and its external storage device. In the embodiment of the present application, the memory 310 is generally used to store an operating system and various application software installed on the computer device 300, such as computer-readable instructions for a method for constructing a three-dimensional road model. In addition, the memory 310 can also be used to temporarily store various data that has been output or is about to be output.

[0206] In some embodiments, the processor 320 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 320 is generally used to control the overall operation of the computer device 300. In the embodiment of the present application, the processor 320 is used to execute computer-readable instructions or process data stored in the memory 310, such as computer-readable instructions for executing the method for constructing a three-dimensional road model.

[0207] The network interface 330 may include a wireless network interface or a wired network interface. The network interface 330 is generally used to establish a communication connection between the computer device 300 and other electronic devices.

[0208] The computer device provided in this application uses lidar technology to obtain point cloud data while collecting GNSS data, and adopts deep learning technology to realize the joint processing of the two, thereby improving the accuracy and robustness of the data. Based on the real-time fusion of multi-source data, a dynamic road model is generated to enhance the system's perception of the road environment. The constructed high-precision dynamic three-dimensional model can realize real-time tracking and prediction of dynamic objects, provide the system with accurate environmental understanding and inference of future motion, and is suitable for more complex scenarios. In addition, this application proposes a smoothing algorithm based on wavelet transform, which is more robust than traditional filtering methods. By performing multi-resolution analysis on vehicle trajectory data and combining it with a fixed threshold method, it ensures that the denoised trajectory data improves smoothness and stability while maintaining details; this application introduces a weighted sum of translation vectors and a nonlinearly corrected rotation matrix. By dynamically adjusting the weights and angle corrections, it solves the low-precision problem of traditional coordinate transformation in multi-axis rotation and complex environments. This method effectively responds to different distance conversion requirements and improves accuracy and adaptability.

[0209] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the road three-dimensional model construction method as described above.

[0210] The computer-readable storage medium provided by this application uses lidar technology to obtain point cloud data while collecting GNSS data, and adopts deep learning technology to realize the joint processing of the two, thereby improving the accuracy and robustness of the data. Based on the real-time fusion of multi-source data, a dynamic road model is generated to enhance the system's perception of the road environment. The constructed high-precision dynamic three-dimensional model can realize real-time tracking and prediction of dynamic objects, providing the system with accurate environmental understanding and inference of future motion, and is suitable for more complex scenarios. In addition, this application proposes a smoothing algorithm based on wavelet transform, which is more robust than traditional filtering methods. By performing multi-resolution analysis on vehicle trajectory data and combining it with a fixed threshold method, the denoised trajectory data is ensured to improve smoothness and stability while maintaining details; this application introduces a weighted sum of translation vectors and a nonlinearly corrected rotation matrix. By dynamically adjusting the weights and angle corrections, the low-precision problem of traditional coordinate transformation in multi-axis rotation and complex environments is solved. This method effectively responds to different distance conversion requirements and improves accuracy and adaptability.

[0211] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of this application.

[0212] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the scope of protection of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of protection of the present application.

Claims

1. A method for constructing a three-dimensional road model, characterized in that: The steps include: Receiving a road model building request sent by a user terminal, wherein the road model building request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign; The GNSS data and the point cloud data are subjected to feature fusion operation according to deep learning technology to obtain fused feature data, wherein the fused feature data y fusion Expressed as: y fusion =f fusion (x gnss ,x lidar ) Among them, y fusion Represents the fused features, including location and environment information; f fusion Represents the MLP model that completes the fusion operation; Inputting the fused feature data into a deep learning model to perform dynamic object recognition and segmentation operations to obtain dynamic object data; Performing real-time tracking processing on the dynamic object data according to a multi-target tracking algorithm to obtain dynamic vehicle tracking data; A static road model corresponding to the target road sign is obtained, and the static road model and the dynamic vehicle tracking data are combined to obtain a three-dimensional model of the target road.

2. The method for constructing a three-dimensional road model according to claim 1, wherein: The step of performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain fused feature data specifically includes the following steps: performing an encoding operation on the GNSS data to obtain vehicle position change information; performing a feature extraction operation on the point cloud data according to a three-dimensional convolutional neural network to obtain point cloud feature data; The vehicle position change information and the point cloud feature data are input into a deep learning network for joint processing to obtain the fused feature data.

3. The method for constructing a three-dimensional road model according to claim 1, wherein: Before the step of performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain fused feature data, the following steps are also included: A data optimization operation is performed on the point cloud data according to the bilateral filtering method and the MLS algorithm.

4. The method for constructing a three-dimensional road model according to claim 1, wherein: Before the step of performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain fused feature data, the following steps are also included: Decomposing the GNSS data to obtain component GNSS data at multiple scales; Perform feature extraction operations on the component GNSS data of each scale to obtain local feature data corresponding to each scale; Inputting the local feature data corresponding to each scale into the SVR model to perform smoothing parameter prediction operation to obtain the predicted smoothing parameters of each scale; Adaptively smoothing the component GNSS data at the plurality of scales according to the prediction smoothing parameters at the respective scales to obtain smoothed component GNSS data at the respective scales; A reconstruction operation is performed on the smoothed component GNSS data of each scale to obtain smoothed GNSS data.

5. The method for constructing a three-dimensional road model according to claim 4, characterized in that: The step of adaptively smoothing the component GNSS data of the plurality of scales according to the prediction smoothing parameters of the respective scales to obtain smoothed component GNSS data of the respective scales specifically includes the following steps: The adaptive smoothing process is optimized according to an incrementally optimized coordinate transformation method to obtain the smoothed component GNSS data of each scale.

6. The method for constructing a three-dimensional road model according to claim 2, wherein: The step of encoding the GNSS data to obtain vehicle position change information specifically includes the following steps: The GNSS data is calculated according to the vehicle trajectory GNSS algorithm to obtain the vehicle position change information, wherein the vehicle position change information is expressed as: in, represents the position change state of the vehicle between time i-1 and time i, v represents the velocity vector of the actual vehicle movement, represents the velocity vector of the reference vehicle motion, k represents the total number of trajectory data, represents the dot product of the actual velocity vector and the reference velocity vector at all trajectory points, and Represent the modulus of the actual velocity vector and the reference velocity vector respectively.

7. A device for constructing a three-dimensional road model, characterized in that: include: a request receiving module, configured to receive a road model building request sent by a user terminal, wherein the road model building request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign; A feature fusion module is used to perform feature fusion operation on the GNSS data and the point cloud data according to deep learning technology to obtain fused feature data, wherein the fused feature data y fusion Expressed as: y fusion =f fusion (x gnss ,x lidar ) Among them, y fusion Represents the fused features, including location and environment information; f fusion Represents the MLP model that completes the fusion operation; A dynamic object recognition and segmentation module is used to input the fused feature data into a deep learning model to perform dynamic object recognition and segmentation operations to obtain dynamic object data; A real-time tracking module is used to perform real-time tracking processing on the dynamic object data according to a multi-target tracking algorithm to obtain dynamic vehicle tracking data; The combining module is used to obtain a static road model corresponding to the target road sign, and combine the static road model with the dynamic vehicle tracking data to obtain a three-dimensional model of the target road.

8. The road three-dimensional model construction device according to claim 7, characterized in that: The feature fusion module includes: An encoding submodule, configured to perform encoding operations on the GNSS data to obtain vehicle position change information; A feature extraction submodule, configured to perform a feature extraction operation on the point cloud data according to a three-dimensional convolutional neural network to obtain point cloud feature data; The combined submodule is used to input the vehicle position change information and the point cloud feature data into a deep learning network for combined processing to obtain the fused feature data.

9. A computer device comprising a memory and a processor, characterized in that: The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the method for constructing a three-dimensional road model according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the method for constructing a three-dimensional road model according to any one of claims 1 to 6.

Images