Road three-dimensional model construction method and device, computer equipment and storage medium

Abstract

The embodiment of the invention belongs to the technical field of intelligent traffic management and control, and relates to a road three-dimensional model construction method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining point cloud data through employing a laser radar technology while collecting GNSS data, and achieving the combined processing of the two through employing a deep learning technology, therefore, the accuracy and robustness of the data are improved; based on real-time fusion of multi-source data, a dynamic road model is generated, and the perception ability of the system to the road environment is enhanced; the constructed high-precision dynamic three-dimensional model can realize real-time tracking and prediction of a dynamic object, provides accurate environmental understanding and future motion inference for a system, and is suitable for more complex scenes.

Description

Technical Field

[0001] The present application relates to the field of intelligent traffic management and control technology, and in particular to a method, device, computer equipment and storage medium for constructing a three-dimensional road model. Background Art

[0002] Smart transportation is an important way to solve traffic problems and improve traffic efficiency. With the acceleration of urbanization and the continuous growth of traffic volume, intelligent transportation systems are increasingly playing a key role. By integrating advanced information technology, intelligent transportation systems can optimize the allocation of traffic resources, improve road capacity, and provide strong support for the sustainable development of cities.

[0003] Traditional traffic management systems mainly rely on GNSS data, but in actual applications, GNSS data is easily interfered with, has a lot of noise, and has insufficient positioning accuracy. In addition, the system's coordinate conversion capability is weak, making it difficult to accurately match GNSS data with three-dimensional road models. These problems make it difficult for the system to accurately present the actual road conditions, especially when responding to sudden traffic incidents, and it is difficult to provide effective information support in a timely manner. In addition, the static road model established by the general traffic system is also difficult to cope with the complexity and variability of traffic events. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a method, device, computer equipment and storage medium for constructing a three-dimensional road model to solve the problem that it is difficult for traditional traffic management systems to achieve accurate matching of GNSS data with three-dimensional road models.

[0005] In order to solve the above technical problems, the present application provides a method for constructing a three-dimensional road model, which adopts the following technical solution: Receiving a road model building request sent by a user terminal, wherein the road model building request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign; Performing a feature fusion operation on the GNSS data and the point cloud data according to a deep learning technology to obtain fused feature data; Inputting the fused feature data into a deep learning model to perform dynamic object recognition and segmentation operations to obtain dynamic object data; Performing real-time tracking processing on the dynamic object data according to a multi-target tracking algorithm to obtain dynamic vehicle tracking data; A static road model corresponding to the target road sign is acquired, and the static road model and the dynamic vehicle tracking data are combined to obtain a target road three-dimensional model.

[0006] Furthermore, the step of performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain fused feature data specifically includes the following steps: Performing encoding operation on the GNSS data to obtain vehicle position information; Performing a feature extraction operation on the point cloud data according to a three-dimensional convolutional neural network to obtain point cloud feature data; The vehicle position information and the point cloud feature data are input into a deep learning network for joint processing to obtain the fused feature data.

[0007] Furthermore, before the step of performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain fused feature data, the following steps are also included: The point cloud data is optimized according to the bilateral filtering method and the MLS algorithm.

[0008] Furthermore, before the step of performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain fused feature data, the following steps are also included: Decomposing the GNSS data to obtain component GNSS data of several scales; Perform feature extraction operations on the component GNSS data of each scale respectively to obtain local feature data corresponding to each scale; Inputting the local feature data corresponding to each scale into the SVR model to perform a smoothing parameter prediction operation to obtain a predicted smoothing parameter of each scale; Adaptively smoothing the component GNSS data at the plurality of scales according to the prediction smoothing parameters at the respective scales to obtain smoothed component GNSS data at the respective scales; A reconstruction operation is performed on the smoothed component GNSS data of each scale to obtain smoothed GNSS data.

[0009] Furthermore, the step of adaptively smoothing the component GNSS data of the plurality of scales according to the prediction smoothing parameters of the various scales to obtain the smoothed component GNSS data of the various scales specifically includes the following steps: The adaptive smoothing process is optimized according to an incrementally optimized coordinate transformation method to obtain the smoothed component GNSS data of each scale.

[0010] Furthermore, the step of encoding the GNSS data to obtain the vehicle location information specifically includes the following steps: The GNSS data is calculated according to the vehicle trajectory GNSS algorithm to obtain the vehicle position information, wherein the vehicle position information is expressed as:

[0011] in, represents the trajectory similarity function of the vehicle between time i-1 and time i, v represents the velocity vector of the actual vehicle movement, represents the velocity vector of the reference vehicle motion, k represents the total number of trajectory data, represents the dot product of the actual velocity vector and the reference velocity vector at all trajectory points, and They represent the modulus of the actual velocity vector and the reference velocity vector respectively.

[0012] In order to solve the above technical problems, the embodiment of the present application further provides a road three-dimensional model construction device, which adopts the following technical solution: A request receiving module, configured to receive a road model building request sent by a user terminal, wherein the road model building request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign; A feature fusion module, used for performing a feature fusion operation on the GNSS data and the point cloud data according to a deep learning technology to obtain fused feature data; A dynamic object recognition and segmentation module, used for inputting the fused feature data into a deep learning model to perform dynamic object recognition and segmentation operations to obtain dynamic object data; A real-time tracking module, used to perform real-time tracking processing on the dynamic object data according to a multi-target tracking algorithm to obtain dynamic vehicle tracking data; The combining module is used to obtain a static road model corresponding to the target road sign, and combine the static road model with the dynamic vehicle tracking data to obtain a three-dimensional model of the target road.

[0013] Furthermore, the feature fusion module includes: The encoding submodule is used to encode the GNSS data to obtain vehicle position information; A feature extraction submodule, used for performing a feature extraction operation on the point cloud data according to a three-dimensional convolutional neural network to obtain point cloud feature data; The joint submodule is used to input the vehicle position information and the point cloud feature data into a deep learning network for joint processing to obtain the fused feature data.

[0014] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the method for constructing a three-dimensional road model as described above when executing the computer-readable instructions.

[0015] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the method for constructing a three-dimensional road model as described above are implemented.

[0016] The present application provides a method for constructing a three-dimensional road model, comprising: receiving a road model construction request sent by a user terminal, wherein the road model construction request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign; performing feature fusion operation on the GNSS data and the point cloud data according to deep learning technology to obtain fused feature data; inputting the fused feature data into a deep learning model to perform dynamic object recognition and segmentation operations to obtain dynamic object data; performing real-time tracking processing on the dynamic object data according to a multi-target tracking algorithm to obtain dynamic vehicle tracking data; obtaining a static road model corresponding to the target road sign, and combining the static road model and the dynamic vehicle tracking data to obtain a target road three-dimensional model. Compared with the prior art, the present application uses laser radar technology to obtain point cloud data while collecting GNSS data, and uses deep learning technology to realize joint processing of the two, thereby improving the accuracy and robustness of the data. Based on the real-time fusion of multi-source data, a dynamic road model is generated to enhance the system's perception of the road environment. The constructed high-precision dynamic three-dimensional model can realize real-time tracking and prediction of dynamic objects, provide the system with accurate environmental understanding and inference of future motion, and is suitable for more complex scenarios. In addition, this application proposes a smoothing algorithm based on wavelet transform, which is more robust than traditional filtering methods. By performing multi-resolution analysis on vehicle trajectory data and combining it with a fixed threshold method, it ensures that the denoised trajectory data improves smoothness and stability while maintaining details; this application introduces weighted summation of translation vectors and nonlinearly corrected rotation matrices, and solves the low-precision problem of traditional coordinate transformation in multi-axis rotation and complex environments by dynamically adjusting weights and angle corrections. This method effectively responds to different distance conversion requirements and improves accuracy and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 is an exemplary system architecture diagram to which the present application may be applied; Figure 2 It is a flowchart of the implementation method of the road three-dimensional model construction method provided in the embodiment of the present application; Figure 3 It is a structural schematic diagram of a road three-dimensional model building device provided in an embodiment of the present application; Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0020] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0022] like Figure 1As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0023] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0024] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to the laptop computer 1011, the tablet computer 1012 or the mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0025] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0026] It should be noted that the road three-dimensional model construction method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the road three-dimensional model construction device is generally set in the server / terminal device.

[0027] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0028] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for constructing a three-dimensional road model according to the present application. The method for constructing a three-dimensional road model includes: step S201, step S202, step S203, step S204 and step S205.

[0029] In step S201, a road model building request sent by a user terminal is received, wherein the road model building request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign.

[0030] In the embodiments of the present application, the user terminal refers to a terminal device used to execute the image processing method for preventing document abuse provided by the present application. The user terminal can be a mobile terminal such as a mobile phone, a smart phone, a laptop computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a navigation device, etc., as well as a fixed terminal such as a digital TV, a desktop computer, etc. It should be understood that the examples of user terminals here are only for the convenience of understanding and are not used to limit the present application.

[0031] In the embodiment of the present application, during the data synchronization process, the timestamp of the sensor data is used for time alignment. Assume that the time of the laser radar data is t LiDAR , the time of GNSS data is t GNSS , then time alignment is achieved through linear interpolation: (Formula 1);

[0032] in, Represents the time offset between two sensors, which is corrected by interpolation.

[0033] In step S202, a feature fusion operation is performed on the GNSS data and the point cloud data according to deep learning technology to obtain fused feature data.

[0034] In the embodiment of the present application, the feature fusion operation can be to jointly process the data of the two sensors through a deep learning network. Using a multi-layer perceptron (MLP), the input is GNSS data x gnss and point cloud data x lidra The features of , the output is the fused location information and environment information: (Formula 2);

[0035] Among them, y fusion Represents the fused features, including location and environment information; f fusion Represents the MLP model that completes the fusion operation; Through neural network learning, optimize the loss function of data fusion, such as minimizing the position estimation error: (Formula 3);

[0036] Among them, L represents the total error of model optimization, y fusion,i Represents the predicted position after data fusion, ygroundtruth,i Indicates the real location.

[0037] In some optional implementations of the embodiments of the present application, the neural network can be trained by using a variety of optimization algorithms to achieve joint processing of GNSS data and lidar data, thereby ensuring the robustness and accuracy of the fusion results under a variety of environmental conditions.

[0038] In the embodiment of the present application, the multi-sensor data fusion method proposed in the present application fully utilizes the advantages of multiple sensors and realizes data complementarity. The deep features of multi-sensor data are further extracted through the deep learning model, thereby ensuring the accuracy of perception, reducing the deviation of a single sensor, and significantly improving the robustness of the system.

[0039] In the embodiment of the present application, after preprocessing the GNSS data and point cloud data, the data is optimized, laying the foundation for subsequent three-dimensional modeling.

[0040] In step S203, the fused feature data is input into a deep learning model to perform dynamic object recognition and segmentation operations to obtain dynamic object data.

[0041] In the embodiment of the present application, in order to identify and segment dynamic vehicles, a deep learning model is used to process point cloud data. The model input is the fusion features of point cloud data and GNSS data (the features obtained from the previous fusion step), and the output is the label that distinguishes static and dynamic objects. The calculation formula is as follows: (Formula 4);

[0042] Among them, y dynamic represents the segmentation result of dynamic vehicles; f segmentation represents the deep learning model for segmentation tasks; x fusion Represents the fused feature data.

[0043] In step S204, the dynamic object data is tracked in real time according to a multi-target tracking algorithm to obtain dynamic vehicle tracking data.

[0044] In the embodiment of the present application, after obtaining the dynamic object, a multi-target tracking algorithm (Kalman filter) is required to track the dynamic vehicle in real time. GNSS data can be used for initial positioning, and the lidar point cloud is used for fine-grained motion estimation of the vehicle: (Formula 5);

[0045] Among them, P t+1 represents the predicted position of the vehicle at time t+1, P t represents the position of the object at the current time t, v trepresents the speed of the object at the current moment, a t Indicates the acceleration of the object at the current moment.

[0046] In step S205, a static road model corresponding to the target road sign is obtained, and the static road model and the dynamic vehicle tracking data are combined to obtain a three-dimensional model of the target road.

[0047] In the embodiment of the present application, the geometric features of static objects are extracted by clustering and segmenting the point cloud data, and the overall static map is constructed by the ICP registration algorithm: (Formula 6);

[0048] Among them, M static Represented as a three-dimensional model of a static road, x lidar Represents point cloud data obtained from a lidar sensor.

[0049] In the embodiment of the present application, the static road model and dynamic vehicle information are combined to generate a real-time updated three-dimensional model of the road scene: (Formula 7);

[0050] Among them, M road Represents the final three-dimensional scene, M static Represents a three-dimensional model of a static road; P vehicle (t) represents the position of the dynamic vehicle at time t.

[0051] In the embodiments of the present application, the multimodal fusion 3D modeling method proposed in the present application can provide more complete scene understanding and modeling results through the collaborative work of multi-source information in complex environments. The multimodal driven 3D modeling method can accurately capture the geometric information of the surrounding environment and greatly improve the data accuracy. In addition, the method can distinguish dynamic objects in the environment in real time, and realize the trajectory prediction and tracking of these dynamic objects through the Kalman filter, thereby enhancing the perception of the road environment.

[0052] In the embodiment of the present application, the construction of a high-precision three-dimensional road model has been completed, and the next step is to accurately match the on-board GNSS data with the model to achieve accurate positioning of the vehicle in three-dimensional space.

[0053] In an embodiment of the present application, a method for constructing a three-dimensional road model is provided, including: receiving a road model construction request sent by a user terminal, wherein the road model construction request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign; performing feature fusion operation on the GNSS data and point cloud data according to deep learning technology to obtain fused feature data; inputting the fused feature data into a deep learning model to perform dynamic object recognition and segmentation operations to obtain dynamic object data; performing real-time tracking processing on the dynamic object data according to a multi-target tracking algorithm to obtain dynamic vehicle tracking data; obtaining a static road model corresponding to the target road sign, and combining the static road model and the dynamic vehicle tracking data to obtain a target road three-dimensional model. Compared with the prior art, the present application uses laser radar technology to obtain point cloud data while collecting GNSS data, and uses deep learning technology to realize joint processing of the two, thereby improving the accuracy and robustness of the data. Based on the real-time fusion of multi-source data, a dynamic road model is generated to enhance the system's perception of the road environment. The constructed high-precision dynamic three-dimensional model can realize real-time tracking and prediction of dynamic objects, provide the system with accurate environmental understanding and inference of future motion, and is suitable for more complex scenarios. In addition, this application proposes a smoothing algorithm based on wavelet transform, which is more robust than traditional filtering methods. By performing multi-resolution analysis on vehicle trajectory data and combining it with a fixed threshold method, it ensures that the denoised trajectory data improves smoothness and stability while maintaining details; this application introduces weighted summation of translation vectors and nonlinearly corrected rotation matrices, and solves the low-precision problem of traditional coordinate transformation in multi-axis rotation and complex environments by dynamically adjusting weights and angle corrections. This method effectively responds to different distance conversion requirements and improves accuracy and adaptability.

[0054] In some optional implementations of the embodiments of the present application, the step of performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain fused feature data specifically includes the following steps: Encode the GNSS data to obtain the vehicle location information; Perform feature extraction operations on point cloud data according to a three-dimensional convolutional neural network to obtain point cloud feature data; The vehicle location information and point cloud feature data are input into the deep learning network for joint processing to obtain fused feature data.

[0055] In the embodiment of the present application, the GNSS data is encoded and converted into the vehicle's position information, which is represented by x gnss : x gnss =f gnss (longitude, latitude, altitude) (Formula 8);

[0056] A 3D convolutional neural network is used to extract features from the lidar point cloud to capture local and global spatial information. The feature representation of the point cloud is x lidar : x lidar =f lidra (Point Cloud Data) (Formula 9);

[0057] Among them, x lidar Represents the characteristics of the point cloud; f lidra represents a three-dimensional convolutional neural network for feature extraction; In some optional implementations of the embodiments of the present application, before performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain the fused feature data, the following steps are also included: The point cloud data is optimized based on the bilateral filtering method and the MLS algorithm.

[0058] In the embodiment of the present application, bilateral filtering can be used to remove outliers, but it has limitations in retaining model details. The MLS algorithm recalculates the coordinates of the points by polynomial fitting, which can better retain the detailed features. To this end, the present application combines bilateral filtering and the MLS algorithm to optimize the point cloud data processing to improve the ability to retain details. The improved filter weight formula is: (Formula 10);

[0059] Where W represents the filter weight; exp represents the natural exponential function, Represents the distance between a point and the center point. Represents the standard deviation of the Gaussian distribution.

[0060] By combining the two algorithms, point cloud data processing is further optimized, and experimental comparative analysis is performed.

[0061] In some optional implementations of the embodiments of the present application, before performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain the fused feature data, the following steps are also included: Decompose the GNSS data to obtain component GNSS data of several scales; Perform feature extraction operations on the component GNSS data of each scale respectively to obtain local feature data corresponding to each scale; The local feature data corresponding to each scale is input into the SVR model for smoothing parameter prediction operation to obtain the predicted smoothing parameters of each scale; Adaptively smoothing the component GNSS data of several scales according to the prediction smoothing parameters of each scale to obtain smoothed component GNSS data of each scale; The smoothed component GNSS data of each scale is reconstructed to obtain smoothed GNSS data.

[0062] In the embodiment of the present application, the present application proposes an adaptive smoothing algorithm combined with support vector regression (SVR) to address the problem of high noise and low precision of GNSS data. The algorithm can dynamically adjust the smoothing parameters through machine learning according to the characteristics of the input data (such as noise level, signal changes, etc.) to achieve adaptive optimization. This method can effectively cope with changes in signal characteristics in a variety of scenarios, providing higher smoothing effects and data recovery capabilities, specifically: ① Perform a scale analysis transformation on the input signal x(t) and decompose it into components x at different scales s s (t). The calculation formula is as follows: (Formula 11);

[0063] Among them, x(t) represents the component of the signal at scale s.

[0064] ②At each scale s, for the component x s (t) Perform feature extraction to capture the local characteristics of the signal (such as noise level, gradient, etc.) as the input of support vector regression. The calculation formula is as follows: (Formula 12);

[0065] Among them, f s (t) is the feature vector extracted at scale s and time t, containing n features.

[0066] ③Use the extracted features f s (t) is used as input and the SVR model is used to predict the adaptive smoothing parameters at each scale and location : (Formula 13);

[0067] Among them, SVR s is a support vector regression model trained for scale s, is the smoothing parameter of the forecast.

[0068] ④Use the predicted smoothing parameter , for each scale component x s (t) Adopt Gaussian filtering for adaptive smoothing. The calculation formula is as follows: (Formula 14);

[0069] in, is the signal component after smoothing at scale s, is the time variable, exp is the natural exponential function, is a component of the integral symbol, indicating that the integral variable is .

[0070] ⑤ Smoothed signal components at each scale Reconstruct and get the final smooth signal : (Formula 15);

[0071] ⑥ Output the reconstructed smooth signal , completing the adaptive smoothing of the input signal.

[0072] In some optional implementations of the embodiments of the present application, the above-mentioned step of adaptively smoothing the component GNSS data of several scales according to the prediction smoothing parameters of each scale to obtain the smoothed component GNSS data of each scale specifically includes the following steps: The adaptive smoothing process is optimized according to the incremental optimization coordinate transformation method to obtain smoothed component GNSS data of each scale.

[0073] In the embodiment of the present application, in order to further improve the adaptability and accuracy of the algorithm and better cope with environmental changes and data inconsistencies, the present application proposes an incremental optimization coordinate transformation method. This method combines graph optimization technology to process the coordinate transformation of GNSS and LiDAR data, and continuously iterates and adjusts the node position through the optimization algorithm to achieve high-precision coordinate transformation. Specifically: ① Initial coordinate transformation: Determine the initial transformation relationship between the GNSS and LiDAR coordinate systems, including the initial rotation matrix R and translation vector t.

[0074] ②Build a graph model: Node: Each position or posture is represented as a node v i ; Edges: Constraints between nodes are represented as edges, including: Constraints between GNSS observations and LiDAR poses; Motion model constraints between consecutive LiDAR poses.

[0075] ③Define the error function: For each edge, define the error function e i , which measures the degree of constraint satisfaction between nodes.

[0076] 1. Error between GNSS and LiDAR: For GNSS observations P GNSS and the corresponding LiDAR pose T i , the error function is: (Formula 16);

[0077] Among them, e GNSS represents the position error vector; R i and t i is the rotation and translation of the LiDAR at time i, and P is the reference point in the LiDAR coordinate system.

[0078] 2. Error of motion model: For continuous LiDAR pose T i and T i+1 , the error function is: (Formula 17);

[0079] Among them, e motion represents the error of the motion model; Indicates the next moment pose T i+1 The inverse transformation matrix of T i represents the LiDAR pose transformation matrix at time i; u i is the predicted increment of the motion model.

[0080] ④Construct optimization objective function Establish an overall error metric and optimize the coordinate transformation parameters by minimizing this error. Accumulate the sum of squares of all errors to form a scalar function that needs to be minimized.

[0081] (Formula 18);

[0082] Where E(R,t) represents the objective function; e i Represents the specific error term.

[0083] ⑤Incremental optimization Through iterative updates, the unknown rotation matrix and translation vector are adjusted in the direction of reducing the error, and finally the optimal value is found.

[0084] 1. Linearized error function: The nonlinear error function is approximated into a linear form to facilitate solution.

[0085] Perform a first-order Taylor expansion of the error function at the current estimate. (Formula 19);

[0086] Among them, R represents the currently estimated rotation matrix; represents the incremental adjustment of the rotation matrix; t represents the currently estimated translation vector; Indicates the incremental adjustment of the translation vector; J i The Jacobian matrix representing the error function with respect to the parameters.

[0087] 2. Construct a system of linear equations: The linear approximations of all errors are combined to form a linear system.

[0088] Summarizing all errors, we get the normal equation: (Formula 20);

[0089] in, Include and , H represents the information matrix, the product of the summary Jacobian matrix, and b represents the accumulation of error terms.

[0090] 3. Solve the increment : By using the least squares method to solve the above equation, we can find the update amount of the parameters and reduce the error.

[0091] 4. Update parameters: Update the rotation matrix and translation vector: (Formula 21); (Formula 22);

[0092] 5. Iterative Optimization Continue to reduce the error until the convergence condition is met.

[0093] Repeat the above steps until the increment is small enough or the maximum number of iterations is reached.

[0094] 6. Apply optimal conversion parameters Using the optimized parameters, the LiDAR data is converted to the global coordinate system of GNSS to achieve accurate alignment.

[0095] (Formula 23); Among them, R * and t * represents the optimal rotation matrix and translation vector obtained by optimization, P LiDAR represents the coordinates of the LiDAR point, P global Represents the transformed global coordinates.

[0096] Through the above steps, this study achieved high-precision coordinate conversion between GNSS and LiDAR data. Compared with traditional translation and rotation processing methods, this method has higher adaptability and accuracy, and can more effectively cope with environmental changes and data inconsistency.

[0097] In some optional implementations of the embodiments of the present application, the above-mentioned step of encoding the GNSS data to obtain the vehicle location information specifically includes the following steps: The GNSS data is calculated according to the vehicle trajectory GNSS algorithm to obtain the vehicle location information, where the vehicle location information is expressed as: (Formula 24); in, represents the trajectory similarity function of the vehicle between time i-1 and time i, represents the position or trajectory state of the vehicle in the target coordinate system at time i, v represents the velocity vector of the actual vehicle motion, represents the velocity vector of the reference vehicle motion, k represents the total number of trajectory data, represents the dot product of the actual velocity vector and the reference velocity vector at all trajectory points, and They represent the modulus of the actual velocity vector and the reference velocity vector respectively.

[0098] In the embodiment of the present application, the algorithm will use GNSS data to obtain the real-time position information of the vehicle, and combine it with the road environment data to ensure the accurate match between the vehicle trajectory and the road model.

[0099] In the embodiment of the present application, by denoising and smoothing the GNSS data, more accurate and stable vehicle trajectory information is obtained, and accurate matching of GNSS data with the three-dimensional model is achieved. In order to better display and analyze these data and realize real-time monitoring and management of road traffic, the present application constructs a digital twin holographic road traffic platform.

[0100] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain results.

[0101] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0102] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0103] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0104] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a road three-dimensional model construction device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0105] like Figure 3 As shown, the road three-dimensional model construction device 200 of the embodiment of the present application includes: The request receiving module 210 is used to receive a road model building request sent by a user terminal, wherein the road model building request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign; A feature fusion module 220 is used to perform feature fusion operation on GNSS data and point cloud data according to deep learning technology to obtain fused feature data; A dynamic object recognition and segmentation module 230 is used to input the fused feature data into a deep learning model to perform dynamic object recognition and segmentation operations to obtain dynamic object data; A real-time tracking module 240 is used to perform real-time tracking processing on dynamic object data according to a multi-target tracking algorithm to obtain dynamic vehicle tracking data; The combining module 250 is used to obtain a static road model corresponding to the target road sign, and combine the static road model with the dynamic vehicle tracking data to obtain a three-dimensional model of the target road.

[0106] In an embodiment of the present application, a road three-dimensional model construction device 200 is provided, including: a request receiving module 210, for receiving a road model construction request sent by a user terminal, wherein the road model construction request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign; a feature fusion module 220, for performing feature fusion operation on GNSS data and point cloud data according to deep learning technology to obtain fused feature data; a dynamic object recognition and segmentation module 230, for inputting the fused feature data into a deep learning model for dynamic object recognition and segmentation operation to obtain dynamic object data; a real-time tracking module 240, for performing real-time tracking processing on dynamic object data according to a multi-target tracking algorithm to obtain dynamic vehicle tracking data; a combining module 250, for obtaining a static road model corresponding to the target road sign, and combining the static road model and the dynamic vehicle tracking data to obtain a target road three-dimensional model. Compared with the prior art, the present application uses laser radar technology to obtain point cloud data while collecting GNSS data, and uses deep learning technology to realize joint processing of the two, thereby improving the accuracy and robustness of the data. Based on the real-time fusion of multi-source data, a dynamic road model is generated to enhance the system's perception of the road environment. The constructed high-precision dynamic three-dimensional model can realize real-time tracking and prediction of dynamic objects, provide the system with accurate environmental understanding and inference of future movements, and is suitable for more complex scenarios. In addition, this application proposes a smoothing algorithm based on wavelet transform, which is more robust than traditional filtering methods. By performing multi-resolution analysis on vehicle trajectory data and combining it with a fixed threshold method, it is ensured that the denoised trajectory data improves smoothness and stability while maintaining details; this application introduces weighted summation of translation vectors and nonlinearly corrected rotation matrices, and solves the low-precision problem of traditional coordinate transformation in multi-axis rotation and complex environments by dynamically adjusting weights and angle corrections. This method effectively responds to different distance conversion requirements and improves accuracy and adaptability.

[0107] In some optional implementations of the embodiments of the present application, the feature fusion module includes: The encoding submodule is used to encode the GNSS data to obtain the vehicle position information; A feature extraction submodule is used to perform feature extraction operations on point cloud data according to a three-dimensional convolutional neural network to obtain point cloud feature data; The joint submodule is used to input the vehicle position information and point cloud feature data into the deep learning network for joint processing to obtain fused feature data.

[0108] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of a computer device according to an embodiment of the present application.

[0109] The computer device 300 includes a memory 310, a processor 320, and a network interface 330 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 300 having components 310-330, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0110] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0111] The memory 310 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 310 can be an internal storage unit of the computer device 300, such as a hard disk or memory of the computer device 300. In other embodiments, the memory 310 can also be an external storage device of the computer device 300, such as a plug-in hard disk equipped on the computer device 300, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. Of course, the memory 310 can also include both the internal storage unit of the computer device 300 and its external storage device. In the embodiment of the present application, the memory 310 is generally used to store the operating system and various application software installed on the computer device 300, such as computer-readable instructions of the road 3D model construction method, etc. In addition, the memory 310 can also be used to temporarily store various data that have been output or are to be output.

[0112] The processor 320 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 320 is generally used to control the overall operation of the computer device 300. In the embodiment of the present application, the processor 320 is used to run the computer-readable instructions or process data stored in the memory 310, such as computer-readable instructions for running the road three-dimensional model construction method.

[0113] The network interface 330 may include a wireless network interface or a wired network interface. The network interface 330 is generally used to establish a communication connection between the computer device 300 and other electronic devices.

[0114] The computer device provided in this application uses laser radar technology to obtain point cloud data while collecting GNSS data, and uses deep learning technology to realize the joint processing of the two, thereby improving the accuracy and robustness of the data. Based on the real-time fusion of multi-source data, a dynamic road model is generated to enhance the system's perception of the road environment. The constructed high-precision dynamic three-dimensional model can realize real-time tracking and prediction of dynamic objects, provide the system with accurate environmental understanding and inference of future motion, and is suitable for more complex scenes. In addition, this application proposes a smoothing algorithm based on wavelet transform, which is more robust than traditional filtering methods. By performing multi-resolution analysis on vehicle trajectory data and combining a fixed threshold method, it is ensured that the denoised trajectory data improves smoothness and stability while maintaining details; this application introduces a weighted sum of translation vectors and a nonlinearly corrected rotation matrix, and solves the low-precision problem of traditional coordinate transformation in multi-axis rotation and complex environments by dynamically adjusting weights and angle corrections. This method effectively responds to different distance conversion requirements and improves accuracy and adaptability.

[0115] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the road three-dimensional model construction method as described above.

[0116] The computer-readable storage medium provided by the present application uses laser radar technology to obtain point cloud data while collecting GNSS data, and uses deep learning technology to realize the joint processing of the two, thereby improving the accuracy and robustness of the data. Based on the real-time fusion of multi-source data, a dynamic road model is generated to enhance the system's perception of the road environment. The constructed high-precision dynamic three-dimensional model can realize real-time tracking and prediction of dynamic objects, provide the system with accurate environmental understanding and inference of future motion, and is suitable for more complex scenes. In addition, the present application proposes a smoothing algorithm based on wavelet transform, which is more robust than traditional filtering methods. By performing multi-resolution analysis on vehicle trajectory data and combining a fixed threshold method, it is ensured that the denoised trajectory data improves smoothness and stability while maintaining details; the present application introduces a weighted sum of translation vectors and a nonlinearly corrected rotation matrix, and solves the low-precision problem of traditional coordinate transformation in multi-axis rotation and complex environments by dynamically adjusting weights and angle corrections. This method effectively responds to different distance conversion requirements and improves accuracy and adaptability.

[0117] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0118] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the scope of protection of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of protection of the present application.

Claims

1. A method for constructing a three-dimensional road model, characterized in that: The steps include: Receiving a road model building request sent by a user terminal, wherein the road model building request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign; Performing a feature fusion operation on the GNSS data and the point cloud data according to a deep learning technology to obtain fused feature data; Inputting the fused feature data into a deep learning model to perform dynamic object recognition and segmentation operations to obtain dynamic object data; Performing real-time tracking processing on the dynamic object data according to a multi-target tracking algorithm to obtain dynamic vehicle tracking data; A static road model corresponding to the target road sign is acquired, and the static road model and the dynamic vehicle tracking data are combined to obtain a three-dimensional model of the target road.

2. The method for constructing a three-dimensional road model according to claim 1, characterized in that: The step of performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain fused feature data specifically includes the following steps: Performing encoding operation on the GNSS data to obtain vehicle position information; Performing a feature extraction operation on the point cloud data according to a three-dimensional convolutional neural network to obtain point cloud feature data; The vehicle position information and the point cloud feature data are input into a deep learning network for joint processing to obtain the fused feature data.

3. The method for constructing a three-dimensional road model according to claim 1, characterized in that: Before the step of performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain fused feature data, the following steps are also included: The point cloud data is optimized according to the bilateral filtering method and the MLS algorithm.

4. The method for constructing a three-dimensional road model according to claim 1, characterized in that: Before the step of performing a feature fusion operation on the GNSS data and the point cloud data according to the deep learning technology to obtain fused feature data, the following steps are also included: Decomposing the GNSS data to obtain component GNSS data of several scales; Perform feature extraction operations on the component GNSS data of each scale respectively to obtain local feature data corresponding to each scale; Inputting the local feature data corresponding to each scale into the SVR model to perform a smoothing parameter prediction operation to obtain a predicted smoothing parameter of each scale; Adaptively smoothing the component GNSS data at the plurality of scales according to the prediction smoothing parameters at the respective scales to obtain smoothed component GNSS data at the respective scales; A reconstruction operation is performed on the smoothed component GNSS data of each scale to obtain smoothed GNSS data.

5. The method for constructing a three-dimensional road model according to claim 4, characterized in that: The step of adaptively smoothing the component GNSS data of the plurality of scales according to the prediction smoothing parameters of the various scales to obtain smoothed component GNSS data of the various scales specifically comprises the following steps: The adaptive smoothing process is optimized according to an incrementally optimized coordinate transformation method to obtain the smoothed component GNSS data of each scale.

6. The method for constructing a three-dimensional road model according to claim 2, characterized in that: The step of encoding the GNSS data to obtain the vehicle location information specifically includes the following steps: The GNSS data is calculated according to the vehicle trajectory GNSS algorithm to obtain the vehicle position information, wherein the vehicle position information is expressed as: ； in, represents the trajectory similarity function of the vehicle between time i-1 and time i, v represents the velocity vector of the actual vehicle movement, represents the velocity vector of the reference vehicle motion, k represents the total number of trajectory data, represents the dot product of the actual velocity vector and the reference velocity vector at all trajectory points, and They represent the modulus of the actual velocity vector and the reference velocity vector respectively.

7. A road three-dimensional model construction device, characterized in that: include: A request receiving module, configured to receive a road model building request sent by a user terminal, wherein the road model building request includes a target road sign and GNSS data and point cloud data corresponding to the target road sign; A feature fusion module, used for performing a feature fusion operation on the GNSS data and the point cloud data according to a deep learning technology to obtain fused feature data; A dynamic object recognition and segmentation module is used to input the fused feature data into a deep learning model to perform dynamic object recognition and segmentation operations to obtain dynamic object data; A real-time tracking module, used to perform real-time tracking processing on the dynamic object data according to a multi-target tracking algorithm to obtain dynamic vehicle tracking data; The combining module is used to obtain a static road model corresponding to the target road sign, and combine the static road model with the dynamic vehicle tracking data to obtain a three-dimensional model of the target road.

8. The road three-dimensional model construction device according to claim 7, characterized in that: The feature fusion module includes: The encoding submodule is used to encode the GNSS data to obtain vehicle position information; A feature extraction submodule, used for performing a feature extraction operation on the point cloud data according to a three-dimensional convolutional neural network to obtain point cloud feature data; The joint submodule is used to input the vehicle position information and the point cloud feature data into a deep learning network for joint processing to obtain the fused feature data.

9. A computer device comprising a memory and a processor, characterized in that: The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the road three-dimensional model construction method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the road three-dimensional model construction method according to any one of claims 1 to 6 are implemented.

Images