Road surface 3D digital high-precision scanning method and system

Abstract

The invention relates to the technical field of 3D scanning, in particular to a road surface 3D digital high-precision scanning method and system. The method comprises the steps of obtaining state information of a scanning area, constructing a dynamic scanning optimization model based on a reinforcement learning algorithm, obtaining multi-dimensional point cloud data of pavement features according to a scanning area division result, carrying out data preprocessing on point cloud after fusion and arrangement, and carrying out feature extraction and matching based on the point cloud data after preprocessing. And performing three-dimensional modeling based on the point cloud data after feature matching, and extracting topographic features and semantic information of the road surface through combination of semantic segmentation and topological relation analysis to complete road surface scanning. According to the invention, three-dimensional modeling is carried out based on the point cloud data after feature matching, and the three-dimensional model is constructed by adopting the improved triangulation algorithm and combining the physical characteristics and topological relation of the pavement, so that the constructed three-dimensional model is more in line with the actual situation of the pavement and can accurately reflect the real form and structure of the pavement.

Description

Technical Field

[0001] The present invention relates to the field of 3D scanning technology, and in particular to a method and system for high-precision 3D digital scanning of a road surface. Background Art

[0002] In the field of road engineering, accurate detection and evaluation of road conditions is a key prerequisite for ensuring road safety, extending road service life, and rationally planning maintenance strategies. With the continuous development of transportation infrastructure and the continuous growth of traffic flow, higher requirements are placed on the accuracy, efficiency, and comprehensiveness of road surface detection technology. However, traditional road surface scanning methods have many insurmountable defects, which seriously restrict the high-quality development of road engineering.

[0003] Traditional road surface scanning methods are mostly based on single-point measurement or simple linear measurement technology. For example, the traditional level measurement method requires manual point-by-point measurement, which is not only inefficient, but also greatly affected by human factors. The measurement accuracy is difficult to guarantee. Especially when inspecting large areas of road surface, this method is time-consuming and labor-intensive and cannot meet the needs of rapid inspection. Although early laser scanning technology can obtain distance information of some roads, due to the lack of effective posture and position perception means, the slightest shake of the equipment during the scanning process will cause data deviation, making it difficult to construct an accurate three-dimensional model of the road surface. In addition, these traditional scanning methods can only obtain limited geometric information, and it is almost impossible to capture semantic features such as the texture and damage type of the road surface, making the detection results unable to fully reflect the true condition of the road surface.

[0004] Although some of the existing slightly more advanced scanning technologies have improved data collection capabilities to a certain extent, they still have obvious shortcomings in data processing and analysis. During data processing, conventional denoising and point cloud registration algorithms are difficult to cope with complex and changeable road data, resulting in low data quality and large point cloud registration errors, which affect subsequent modeling and analysis. In the data analysis stage, traditional methods can often only perform simple terrain feature recognition. It is difficult to accurately extract subtle but critical features such as road cracks and potholes, and it is impossible to provide accurate data support for road maintenance and repair. At this stage, a high-precision 3D digital road surface scanning method and system is needed. Summary of the invention

[0005] In order to solve the problems of low detection accuracy and low detection efficiency in traditional road surface detection, the present invention provides a road surface 3D digital high-precision scanning method and system.

[0006] In a first aspect, the present invention provides a method for high-precision 3D digital scanning of a road surface, which adopts the following technical solution:

[0007] A road surface 3D digital high-precision scanning method, comprising:

[0008] A dynamic scanning optimization model is built based on the reinforcement learning algorithm, and the DBSCAN clustering algorithm is used to divide the scanning area;

[0009] Obtain multi-dimensional point cloud data of road features based on the scan area division results, and use fusion algorithm and PID control algorithm to fuse and organize the multi-dimensional point cloud data;

[0010] Data preprocessing is performed on the fused and organized point cloud, including the introduction of a generative adversarial network based on the Gaussian mixture model to clean and denoise the multi-dimensional data, and then a knowledge graph is constructed to assist in cleaning;

[0011] Feature extraction and matching based on preprocessed point cloud data, including point cloud feature extraction using convolutional neural network, followed by feature matching using a matching algorithm based on attention mechanism;

[0012] 3D modeling based on point cloud data after feature matching, including using an improved Delaunay triangulation algorithm to build a 3D model in combination with the physical characteristics and topological relationships of the road surface;

[0013] The three-dimensional model is analyzed and applied, and the terrain features and semantic information of the road surface are extracted through the combination of semantic segmentation and topological relationship analysis to complete the road surface scanning.

[0014] Furthermore, the dynamic scanning optimization model is constructed based on the reinforcement learning algorithm, including constructing a state input layer, a decision network layer and an action output layer, wherein the state input layer is responsible for receiving and integrating environmental state information and generating a state vector, the decision network layer adopts a multi-layer perceptron, the input layer neurons of the decision network have the same dimension as the state vector, the hidden layer of the decision network processes and extracts features from the input information through a nonlinear activation function, and the action output layer maps the continuous value output by the decision network layer to the actual scanning parameter range.

[0015] Furthermore, the dynamic scanning optimization model constructed based on the reinforcement learning algorithm also includes defining the state space, action space and reward function, then using the Q-learning algorithm and continuously updating the Q value function to learn the optimal strategy, and finally initializing the Q value function to a random value matrix, wherein the number of rows of the matrix is ​​equal to the size of the state space, and the number of columns is equal to the size of the action space, and the dynamic scanning optimization model parameters are initialized by setting the learning rate and discount factor.

[0016] Furthermore, the use of the DBSCAN clustering algorithm to divide the scan area includes defining the neighborhood of any point in the point cloud data set, determining the core point in the point cloud data set by setting a minimum number of points, and dividing the road point cloud data into different clustering areas based on the core point and its neighborhood relationship.

[0017] Furthermore, the adversarial generative network is introduced on the basis of the Gaussian mixture model to clean and denoise the multi-dimensional data, including normalizing the fused point cloud data and processing missing values, estimating the parameters of the Gaussian mixture model using the expectation maximization algorithm, and after obtaining the parameters of the Gaussian mixture model, identifying noise points and abnormal points according to the posterior probability that each data point belongs to different Gaussian components, and then introducing the adversarial generative network composed of a generator and a discriminator. The expectation maximization algorithm formula is expressed as:

[0018]

[0019] Among them, γ ik represents the i-th data point x i The posterior probability of belonging to the kth Gaussian component, π k represents the weight of the kth Gaussian component, Represents the probability density function of the kth Gaussian distribution at the data point x i The value of π l Represents the weight of the lth Gaussian component.

[0020] Furthermore, the construction of the knowledge graph for auxiliary cleaning includes constructing a road surface data knowledge graph, integrating the road surface flatness range, damage type characteristics and historical data knowledge of different road sections, using entity-relationship-entity triples for knowledge representation, and after completing the adversarial generative network denoising, using the road surface data knowledge graph to perform semantic verification on the currently collected data.

[0021] Furthermore, the feature extraction and matching based on the preprocessed point cloud data includes using a convolutional neural network to extract features from the preprocessed point cloud data. After the feature extraction is completed, the cosine similarity is used to calculate the similarity between different point cloud data, and the attention weight is calculated according to the similarity. Feature matching is performed according to the attention weight, and feature points with high similarity and high attention weight are matched. The attention formula is expressed as:

[0022]

[0023] Among them, sim(p i ,p j ) is represented as feature point p i and feature point p j The similarity between them, exp is expressed as an exponential function, ∑ k It is represented as the summation operation of all k feature points.

[0024] Furthermore, the three-dimensional modeling based on the point cloud data after feature matching includes inputting the point cloud data after feature matching and obtaining the physical characteristic parameters of the road surface, performing Delaunay triangulation on the point cloud data after feature matching, using the Bowyer-Watson algorithm to obtain an initial triangular mesh, deforming the mesh according to the physical characteristic parameters and topological relationships of the road surface, calculating the force on each node and the displacement of each node respectively, updating the node position according to the force and displacement, and updating the triangular mesh accordingly.

[0025] Furthermore, the method combines semantic segmentation with topological relationship analysis to extract terrain features and semantic information of the road surface, including using a three-dimensional convolution operation to generate a series of feature maps from a three-dimensional model, inputting the feature maps into a fully convolutional network for semantic segmentation, and outputting a segmentation result of the same size as the input image, training the full convolutional network using cross entropy loss, obtaining topological information using a graph convolutional network, and completing road surface scanning based on the results of semantic segmentation and topological relationship analysis.

[0026] In a second aspect, a road surface 3D digital high-precision scanning system includes:

[0027] Lidar, which emits a laser beam and measures the time difference of reflected light;

[0028] Radar signal processor, used to obtain distance information of each point on the road surface to form point cloud data and construct the three-dimensional geometry of the road surface;

[0029] Differential GPS is used to provide precise geographic location information of the scanning device, including longitude, latitude and altitude. By comparing the differential signal with the ground base station, it provides geographic coordinate reference for the point cloud data.

[0030] High-resolution cameras are used to simultaneously collect visual image data of the road surface and supplement the semantic features of point cloud data;

[0031] An inertial measurement unit, which contains an accelerometer and a gyroscope, and is used to measure the acceleration and angular velocity data of the scanning device;

[0032] The control terminal is used for time synchronization, coordinate conversion and alignment, data association and fusion, and real-time analysis and decision-making of the collected data.

[0033] In summary, the present invention has the following beneficial technical effects:

[0034] 1. The dynamic scanning optimization model constructed by the present invention based on the reinforcement learning algorithm can intelligently adjust the scanning parameters according to the real-time environmental status. Through the collaborative work of the state input layer, decision network layer and action output layer, the model can accurately perceive environmental changes and quickly make the optimal scanning decision, making the scanning process more flexible and efficient, greatly improving the scanning efficiency and quality, and reducing the scanning cost.

[0035] 2. The present invention uses the DBSCAN clustering algorithm to divide the scanning area, and can reasonably divide the road point cloud data into different clustering areas according to the distribution characteristics of the point cloud data. This division method fully considers the inherent structure of the data, provides clear regional guidance for subsequent data collection and processing, and helps to improve the pertinence and accuracy of data collection.

[0036] 3. Based on the Gaussian mixture model, the present invention introduces a generative adversarial network to clean and denoise the multi-dimensional point cloud data. The Gaussian mixture model parameters are first estimated by the expectation maximization algorithm to accurately identify noise points and abnormal points, and then the generative adversarial network is used to further optimize the data quality. The adversarial training mechanism of the generator and the discriminator can effectively remove complex noise, improve the purity of the data, and provide a reliable data basis for subsequent analysis.

[0037] 4. The present invention uses a convolutional neural network to extract features from preprocessed point cloud data, and can automatically learn the deep-level features of the data. The convolutional layer, pooling layer, and fully connected layer of the convolutional neural network work together to effectively extract local and global features of the data, thereby improving the accuracy and robustness of feature extraction and providing rich and effective feature information for subsequent feature matching and three-dimensional modeling.

[0038] 5. The present invention performs three-dimensional modeling based on point cloud data after feature matching, adopts an improved Delaunay triangulation algorithm, and constructs a three-dimensional model in combination with the physical characteristics and topological relationship of the road surface. On the basis of traditional Delaunay triangulation, the elastic deformation and mechanical principles of the road surface are considered to make the constructed three-dimensional model more in line with the actual situation of the road surface and can accurately reflect the real shape and structure of the road surface.

[0039] 6. In the modeling process, the present invention performs mesh deformation according to the physical characteristic parameters and topological relationship of the road surface, calculates the force and displacement of the nodes, and dynamically updates the node positions and triangular meshes. This dynamic optimization modeling method can better simulate the actual changes of the road surface and improve the accuracy and reliability of the three-dimensional model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the overall process of a method for high-precision 3D digital scanning of a road surface according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0042] Example 1

[0043] Reference Figure 1 , a road surface 3D digital high-precision scanning method of this embodiment includes:

[0044] A dynamic scanning optimization model is built based on the reinforcement learning algorithm, and the DBSCAN clustering algorithm is used to divide the scanning area;

[0045] Obtain multi-dimensional point cloud data of road features based on the scan area division results, and use fusion algorithm and PID control algorithm to fuse and organize the multi-dimensional point cloud data;

[0046] Data preprocessing is performed on the fused and organized point cloud, including the introduction of a generative adversarial network based on the Gaussian mixture model to clean and denoise the multi-dimensional data, and then a knowledge graph is constructed to assist in cleaning;

[0047] Feature extraction and matching based on preprocessed point cloud data, including point cloud feature extraction using convolutional neural network, followed by feature matching using a matching algorithm based on attention mechanism;

[0048] 3D modeling based on point cloud data after feature matching, including using an improved Delaunay triangulation algorithm to build a 3D model in combination with the physical characteristics and topological relationships of the road surface;

[0049] The three-dimensional model is analyzed and applied, and the terrain features and semantic information of the road surface are extracted through the combination of semantic segmentation and topological relationship analysis to complete the road surface scanning.

[0050] Specifically, a road surface 3D digital high-precision scanning method includes the following steps:

[0051] S1. Obtain the status information of the scanning area, build a dynamic scanning optimization model based on the reinforcement learning algorithm, and use the DBSCAN clustering algorithm to divide the scanning area;

[0052] During road scanning, different road conditions and environmental conditions have a significant impact on the scanning effect. In order to achieve efficient and accurate scanning, this embodiment constructs a dynamic scanning optimization model based on the reinforcement learning algorithm. First, the state, action and reward function of the model are clarified. In addition, the reinforcement learning dynamic scanning optimization model consists of three core parts: state input layer, decision network layer and action output layer. The state input layer is responsible for receiving and integrating environmental state information to provide a basis for subsequent decision-making. The specific input state information includes vehicle speed v, road bumpiness b, traffic flow t and weather conditions w.

[0053] The decision network layer is implemented using a multi-layer perceptron (MLP). Its main function is to learn and predict the optimal scanning action based on the input state information. The structure of the neural network includes an input layer, a hidden layer, and an output layer: Among them, the input layer: receives the state vector S = {[v, b, t, w]} transmitted by the state input layer. The number of neurons in the input layer is the same as the dimension of the state vector.

[0054] Hidden layer: The hidden layer is composed of multiple neurons, which process the input information and extract features through a nonlinear activation function (ReLU). The number of layers and neurons in the hidden layer can be adjusted according to specific tasks and data to improve the learning and generalization capabilities of the model.

[0055] Output layer: The number of neurons in the output layer is the same as the dimension of the action space. For this model, the action space includes the scanning frequency and angular range of the lidar, so the output layer has two neurons, corresponding to the predicted values ​​of the scanning frequency and angular range respectively.

[0056] The action output layer generates specific scanning actions based on the output of the decision network layer. This layer maps the continuous values ​​output by the decision network layer to the actual scanning parameter range, such as converting the predicted scanning frequency value into the scanning frequency setting acceptable to the lidar, and converting the predicted angle range value into the actual scanning angle of the lidar.

[0057] As mentioned above, the state space consists of vehicle speed, road bumps, traffic flow, and weather conditions, i.e., S = {[v, b, t, w]}. The action space includes the scanning frequency (f) and angle range (θ) of the lidar, i.e., A = {[f, θ]}. The reward function is used to evaluate the pros and cons of each action and guide the model to learn the optimal scanning strategy. The reward function can be defined as r = α 1 q+α 2 e, where q represents the data quality score, e represents the scanning efficiency, and α 1 and α 2 is the weight coefficient. The data quality score can be determined based on indicators such as the density, accuracy, and completeness of the point cloud data. The scanning efficiency is measured by the amount of data collected per unit time.

[0058] Then the Q-learning algorithm is used as the reinforcement learning algorithm, and the Q-value function is continuously updated to learn the optimal strategy. The Q-value function Q(s,a) represents the expected cumulative reward of taking action a in state s. The update formula of the Q-learning algorithm is:

[0059]

[0060] Among them, s tis the state at time t, a t is the action at time t, α is the learning rate, which controls the step size of each update, γ is the discount factor, which is used to weigh the importance of current rewards and future rewards, and r t Take action a at time t t The reward obtained after that is initialized to the Q-value function, which is initialized to a random value matrix. The number of rows of the matrix is ​​equal to the size of the state space, and the number of columns is equal to the size of the action space.

[0061] The model is trained according to the dynamic scanning optimization model. The scanning vehicle is placed on the road surface, and the laser radar and other sensors are started to obtain the initial state s 0 .

[0062] According to the current status t And Q value function, use ∈― greedy strategy to select action a t ,∈-greedy strategy randomly selects actions with probability ∈, and selects the action with the largest Q value with probability 1-∈. In the early stage of training, ∈ is set to a larger value, such as 0.9, to encourage the model to explore; as the training progresses, ∈ gradually decreases, such as linearly decaying to 0.1, to improve the utilization ability of the model.

[0063] Execute the action and get the reward and next state: Execute the selected action a t , the laser radar scans according to the set scanning frequency and angle range, and the sensor obtains the new state s t+1 and reward r t , update the Q-value function according to the update formula of the Q-learning algorithm.

[0064] After determining the scanning strategy, the road surface is divided into reasonable areas for more targeted scanning. The DBSCAN clustering algorithm is used to divide the road surface into different areas according to the density and distribution characteristics of the point cloud data. For a point p in the point cloud data set, its neighborhood is defined as:

[0065] N ∈ (p)={q∈D|dist(p,q)≤∈},

[0066] Among them, ∈ is the neighborhood radius, dist(p,q) is the Euclidean distance between point p and point q, and the neighborhood radius determines the degree of association between points. If the number of points in the neighborhood is greater than or equal to the minimum number of points, then the point is a core point. The core point represents an area with relatively dense data and is the basis for forming clusters. By judging the core point, different cluster centers can be preliminarily determined.

[0067] Different clustering areas are divided according to the core points and their neighborhood relationships. For areas with complex road conditions or key focus areas (such as accident-prone sections, bridge connections, etc.), since the point cloud data density and distribution characteristics of these areas are different from other areas, they will be automatically divided into separate clusters. For these areas, the scanning frequency and resolution are automatically increased to obtain more detailed road surface information; for areas with relatively stable road conditions, the scanning frequency is appropriately reduced to improve the overall scanning efficiency while ensuring data quality.

[0068] S2. Obtain multi-dimensional point cloud data of road features according to the scanning area division results, and fuse and organize the multi-dimensional point cloud data using a fusion algorithm and a PID control algorithm;

[0069] After completing the division of the scanning area, multiple sensors are used to obtain multi-dimensional point cloud data of road features according to the scanning strategies of different areas. Since the data collection methods, time and coordinate systems of different sensors may be different, it is necessary to fuse the multi-dimensional point cloud data to form a unified and accurate data set. Through the hardware clock synchronization mechanism and software time calibration algorithm, ensure that each sensor collects data at the same time scale. Time synchronization is the basis of data fusion. Only by ensuring the consistency of data in time can the data of different sensors be accurately associated and fused, and then the data of different sensors are converted to a unified coordinate system. For example, the polar coordinate data of the lidar and the attitude data of the IMU are converted to the same geographic coordinate system as the differential GPS. Coordinate conversion and alignment can eliminate the spatial deviation between different sensors, so that the point cloud data can accurately correspond in space. Then the Kalman filter algorithm is used for data fusion. The multi-dimensional point cloud data from different sensors are regarded as different observations of the system state. The system state vector x k Contains information such as the position and posture of the road surface, and the observation vector z k The state transfer matrix F is defined as the distance data of the laser radar, the attitude data of the IMU, the position data of the differential GPS, etc. k , observation matrix H k , process noise covariance Q k and the observation noise covariance R k ,The Kalman filter algorithm can fuse these observation data to obtain the optimal estimate of the system state.

[0070] In the road scanning system, according to the road state (such as position, posture, etc.) at the previous moment, combined with the motion model of the scanning device (through the state transfer matrix F kThe road surface state at the current moment can be predicted by using the process noise covariance Q. For example, if the scanning device moves at a certain speed and direction, the current position and attitude can be predicted based on the position and attitude at the previous moment. At the same time, considering the uncertainty that may exist in the system during the movement (such as device jitter, interference from the external environment, etc.), the process noise covariance Q k To compensate, the prediction formula is expressed as:

[0071]

[0072] in, It is the estimated value of the state based on the previous moment The predicted value of the current state, P k|k―1 is the forecast error covariance, F k It is expressed as a state transfer matrix, which describes the change of system state over time, Q k is the process noise covariance, which takes into account the uncertainty that may exist in the system during the state transition process.

[0073] After obtaining the predicted value, compare it with the observation data of each sensor at the current moment (such as the distance data of the laser radar, the attitude data of the IMU, etc.). k Weigh the credibility of the predicted value and the observed value. When the credibility of the observed data is higher, the observed value is used more to update the state estimate; when the credibility of the predicted value is higher, the predicted value is relied on more. If the measurement accuracy of the lidar is higher, the lidar observation data will be given a greater weight when updating the state estimate. At the same time, considering the possible noise in the observation data (such as the measurement error of the lidar, the integration error of the IMU, etc.), the noise covariance R is calculated by observing the noise covariance R. k Compensation is performed, and then the prediction is updated. The update steps include:

[0074]

[0075]

[0076] P k|k =(I-K k H k ) k|k―1 ,

[0077] Among them, K k is the Kalman gain, which weighs the credibility of the predicted value and the observed value, It is the optimal state estimate at the current moment obtained by combining the predicted value and the observed value, P k|k is the optimal estimation error covariance, H kis the observation matrix, which describes the relationship between the observation vector and the system state vector. k is the observation noise covariance, which takes into account the noise present in the observation data.

[0078] Finally, in order to ensure that the scanning angle of the LiDAR is always perpendicular to the road surface and improve the measurement accuracy, the PID control algorithm is used to adjust the scanning angle of the LiDAR. The error e(t) between the set value (the desired angle of the laser beam perpendicular to the road surface) and the actual measurement value (the LiDAR angle corresponding to the current vehicle posture and position calculated by the IMU and differential GPS data) is calculated. The accurate calculation of the error is the key to PID control, which reflects the deviation between the current scanning angle and the desired angle.

[0079] Control quantity calculation: The calculation formula of the output u(t) of the PID controller is:

[0080]

[0081] Among them, K p is the proportionality coefficient, K i is the integration coefficient, K d is the differential coefficient, the proportional coefficient K p Used to adjust the control amount proportionally according to the current error; integral coefficient K i Used to accumulate errors over a period of time and eliminate steady-state errors; differential coefficient K d It is used to predict the trend of future errors based on the rate of change of the errors, and adjust the control amount in advance to make the system have better dynamic response performance. The scanning angle of the laser radar is adjusted by the output control amount to gradually reduce the error and ensure that the laser beam is always perpendicular to the road surface.

[0082] S3, preprocessing the fused and organized point cloud, including introducing a generative adversarial network based on the Gaussian mixture model to clean and denoise the multi-dimensional data, and then constructing a knowledge graph for auxiliary cleaning;

[0083] The fused and organized point cloud data may contain noise and abnormal points, and data preprocessing is required to improve the data quality. First, the data is preliminarily processed based on the Gaussian mixture model. The fused and organized point cloud data X = {x 1 ,x 2 ,…,x n} obeys the Gaussian mixture model, and its probability density function is:

[0084]

[0085] Where K is the number of Gaussian components, π k is the weight of the Kth Gaussian component, μ kis the mean vector, Σ k is the covariance matrix, and The Gaussian mixture model can model the distribution of data, divide the data into different Gaussian components, each Gaussian component represents a data cluster, and use the expectation maximization (EM) algorithm to estimate the parameters of the Gaussian mixture model. The EM algorithm continuously updates the parameters of the model through iteration, making the model fit the data better and better. It identifies possible noise points and outliers based on the probability that each point belongs to different Gaussian components. Noise points and outliers are usually located at the edge of Gaussian components or far away from the main data clusters. By calculating the probability that a point belongs to each Gaussian component, it can be determined whether the point is a noise point or an outlier.

[0086] Based on the preliminary processing of Gaussian mixture model, the Generative Adversarial Network (GAN) is introduced for deep denoising: the generator G learns the distribution characteristics of real road data and inputs noise z to generate false data G(z); the discriminator D distinguishes between real data x and false data G(x).

[0087] Generator loss function Discriminator loss function Among them, p datn (x) is the real data distribution, p z (z) is the noise distribution. The goal of the generator is to generate false data that can deceive the discriminator, while the goal of the discriminator is to accurately distinguish between real data and false data. By minimizing their respective loss functions, the generator and discriminator continuously optimize their own performance.

[0088] By alternately training the generator and the discriminator, the generator can learn a more accurate distribution of road data, thereby identifying and removing noise data. During the training process, the generator continuously adjusts its own parameters to generate false data that is closer to real data; the discriminator also continuously improves its ability to distinguish accurately between real data and false data. After multiple iterations of training, the generator can effectively remove noise from the data.

[0089] After completing the adversarial generative network denoising, a road surface data knowledge graph is constructed for auxiliary cleaning. The road surface data knowledge graph is constructed to integrate knowledge such as road surface flatness range, common damage type characteristics, and historical data of different road sections. 1 ,r,e 2) represents knowledge, such as (a certain road section, historical smoothness range, [0.5, 1.0]). The knowledge graph can structure the professional knowledge and experience in the road surface field and provide semantic information for data cleaning. In the final stage of data cleaning, the knowledge graph is used to perform semantic verification on the data. If the smoothness value f(x) of the current collected data point x is not within the historical smoothness range [a, b] of the road section, it is judged as abnormal data, that is, → Abnormal data: remove data that is obviously unreasonable. Through the auxiliary cleaning of knowledge graphs, the quality and reliability of data can be further improved.

[0090] S4, feature extraction and matching based on the preprocessed point cloud data, including point cloud feature extraction using a convolutional neural network, and then feature matching using a matching algorithm based on an attention mechanism;

[0091] The preprocessed point cloud data contains a lot of information, but feature extraction is required for better subsequent analysis and processing. Convolutional neural network (CNN) is used to extract point cloud features. CNN contains multiple convolutional layers, pooling layers and fully connected layers. The convolutional layer performs convolution operations on the input data through the convolution kernel to extract local features of the data; the pooling layer downsamples the output of the convolutional layer to reduce the dimension of the data and improve computational efficiency; the fully connected layer integrates the output of the pooling layer and outputs the final feature vector. The convolution operation is defined as:

[0092] O(i,j)=∑ m，n K(m,n)I(i+m,j+n),

[0093] Among them, K represents the convolution kernel, I represents the input feature map, O represents the output feature map, i and j are the starting positions of the current convolution kernel on the input feature map, m and n are the indexes of the convolution kernel, and the convolution operation can automatically learn the local features of the data. Different types of features can be extracted through different convolution kernels. Through training on a large number of point cloud data samples, CNN automatically learns the deep features of point cloud data and improves the accuracy and robustness of feature extraction. During the training process, CNN continuously adjusts the parameters of the convolution kernel so that the extracted features can better represent the essential features of the data.

[0094] After feature extraction, it is necessary to match the features between different point cloud data to achieve alignment and splicing of point cloud data. The matching algorithm based on the attention mechanism is used for feature matching. i and p j , the cosine similarity method is used to calculate the similarity sim(p i ,p j ), the similarity calculation can measure the similarity between two feature points. According to the formula Calculate the attention weight, where w ij Indicates the attention weight, which reflects the importance of each feature point in the matching process. The features of key feature points such as road signs and special structures are given higher weights. By giving higher weights to key feature points, the accuracy and efficiency of matching can be improved. Feature matching is performed based on the attention weight, and feature points with high similarity and high attention weight are matched. Through feature matching, the correspondence between different point cloud data can be determined, providing accurate matching information for subsequent 3D modeling.

[0095] S5. Performing 3D modeling based on the point cloud data after feature matching, including using an improved Delaunay triangulation algorithm to build a 3D model in combination with the physical characteristics and topological relationship of the road surface;

[0096] Input the point cloud data after feature matching P = {p 1 ,p 2 ,…,p n}, where p i =(x i ,y i ,z i ) is expressed as the three-dimensional coordinates of the i-th point, and the physical characteristic parameters of the road surface, such as elastic modulus and Poisson's ratio, are determined. The point cloud data after feature matching is subjected to traditional Delaunay triangulation to obtain the initial triangular mesh T = {t 1 ,t 2 ,…,t m}, where t j Represented as the jth triangle, represented by p j1 ,p j2 ,p j3 The traditional Delaunay triangulation can be implemented using a variety of algorithms. This embodiment adopts the Bowyer-Watson algorithm. First, a super triangle containing all points is constructed, and the points are inserted into the super triangle in sequence. After each point is inserted, all triangles whose circumscribed circles contain the point are deleted, and the edges of these triangles are reconnected to form a new Delaunay triangle. Finally, all triangles related to the super triangle are deleted to obtain the final Delaunay triangulation result.

[0097] According to the physical characteristics of the road surface and the terrain features, the force on each node is calculated. Assume that the force F on each node is i By gravity F g,i and elastic force F e,i Composition, namely F i =F g,i +F e,i , gravity F g,i =mi g, where m i is node p i The mass of the pavement unit represented by g is the acceleration due to gravity, and the mass m is i According to the density ρ of the road surface and the triangular area A around the node i Calculated, that is, m i =ρA i , elastic force F e,i According to Hooke's law, assuming that node p i Its neighboring node p i1 ,p i2 ,…There is an elastic connection between them, and the elastic force F e,i It can be expressed as Among them, k ik is node p i With p ik The elastic coefficient between ik is the current node p i With p ik The distance between is the node p in the initial state i With p ik The distance between ik It is from p i Point to p ik The unit vector of each node is calculated according to Newton's second law, and the position of each node is updated to p′ i =p i +Δr i , and update the triangular mesh T accordingly, repeating multiple times until the displacement of the node is less than a preset threshold. At this time, the triangular mesh reaches a stable state and a three-dimensional model that considers the physical characteristics and topological relationship of the road surface is obtained.

[0098] S6. Analyze and apply the three-dimensional model, extract the terrain features and semantic information of the road surface through the combination of semantic segmentation and topological relationship analysis, and complete the road surface scanning.

[0099] The 3D model is converted into a series of feature maps using a 3D convolution operation. The feature maps are input into the full convolution network for semantic segmentation, and the segmentation result with the same size as the input image is output. The full convolution network is trained using the cross entropy loss. The cross entropy loss calculation formula is:

[0100]

[0101] Among them, N is the number of pixels, C is the number of categories, and y i,c is the true label of the i-th pixel belonging to the c-th class, p i,cTo predict the probability, the cross entropy loss can measure the difference between the model prediction result and the true label. By minimizing this loss function, FCN can continuously adjust its own parameters to improve the accuracy of semantic segmentation. A large amount of annotated road 3D model data is used to train FCN. During the training process, the network parameters are continuously adjusted so that the cross entropy loss between the prediction result and the true label gradually decreases until the convergence condition is reached.

[0102] The trained FCN can identify different material areas of the road surface, such as asphalt pavement, cement pavement, etc. There are differences in performance and maintenance requirements for roads of different materials. Accurately distinguishing material areas helps to formulate more reasonable maintenance plans based on the characteristics of different materials. For example, different maintenance methods and cycles are used for asphalt pavement and cement pavement. Finally, the graph convolutional network (GCN) is used to analyze the topological relationship of the 3D model. GCN represents the 3D model as a graph structure, in which nodes represent terrain features, such as damaged areas, different material areas, etc., and edges represent the connection relationship between nodes. Through this graph structure representation, the complex topological information in the 3D model can be abstracted and simplified. GCN learns the topological information between different terrain features, such as the extension direction of cracks, the distance and connectivity between different damaged areas, etc. By processing and analyzing the information of nodes and edges in the graph structure, GCN can mine these topological relationships hidden in the 3D model, providing important information for a comprehensive understanding of the road condition. Combining the terrain semantic information obtained by semantic segmentation and the topological information obtained by GCN, the connection relationship and spatial layout between different terrain features are mined, and the road surface scanning is completed.

[0103] Example 2

[0104] The difference between this embodiment and embodiment 1 is that this embodiment provides a road surface 3D digital high-precision scanning system, including:

[0105] Lidar, which emits a laser beam and measures the time difference of reflected light;

[0106] Radar signal processor, used to obtain distance information of each point on the road surface to form point cloud data and construct the three-dimensional geometry of the road surface;

[0107] The inertial measurement unit includes an accelerometer and a gyroscope, which respectively measure the acceleration and angular velocity data of the scanning device, calculate the attitude change of the scanning device in three axes through an integration algorithm, and determine the real-time attitude of the scanning device;

[0108] Differential GPS is used to provide precise geographic location information of the scanning device, including longitude, latitude and altitude. By comparing the differential signal with the ground base station, it provides geographic coordinate reference for the point cloud data.

[0109] High-resolution cameras simultaneously collect visual image data of the road surface, record visual information such as road surface texture, color, and traffic signs, and supplement the semantic features of point cloud data;

[0110] The control terminal is used for time synchronization, coordinate conversion and alignment, data association and fusion, and real-time analysis and decision-making of the collected data.

[0111] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A road surface 3D digital high-precision scanning method, characterized in that: include: Obtain the status information of the scanning area, build a dynamic scanning optimization model based on the reinforcement learning algorithm, and use the DBSCAN clustering algorithm to divide the scanning area; Obtain multi-dimensional point cloud data of road features based on the scan area division results, and use fusion algorithm and PID control algorithm to fuse and organize the multi-dimensional point cloud data; Data preprocessing is performed on the fused and organized point cloud, including the introduction of a generative adversarial network based on the Gaussian mixture model to clean and denoise the multi-dimensional data, and then a knowledge graph is constructed to assist in cleaning; Feature extraction and matching based on preprocessed point cloud data, including point cloud feature extraction using convolutional neural network, followed by feature matching using a matching algorithm based on attention mechanism; 3D modeling based on point cloud data after feature matching, including using Delaunay triangulation algorithm to build a 3D model in combination with the physical characteristics and topological relationship of the road surface; The three-dimensional model is analyzed and applied, and the terrain features and semantic information of the road surface are extracted through the combination of semantic segmentation and topological relationship analysis to complete the road surface scanning.

2. A road surface 3D digital high-precision scanning method according to claim 1, characterized in that: The dynamic scanning optimization model based on the reinforcement learning algorithm includes constructing a state input layer, a decision network layer and an action output layer, wherein the state input layer is responsible for receiving and integrating environmental state information and generating a state vector, the decision network layer adopts a multi-layer perceptron, the input layer neurons of the decision network have the same dimension as the state vector, the hidden layer of the decision network processes the input information and extracts features through a nonlinear activation function, and the action output layer maps the continuous value output by the decision network layer to the actual scanning parameter range.

3. A road surface 3D digital high-precision scanning method according to claim 2, characterized in that: The dynamic scanning optimization model is constructed based on the reinforcement learning algorithm, and also includes defining the state space, action space and reward function, then using the Q-learning algorithm and continuously updating the Q value function to learn the optimal strategy, and finally initializing the Q value function to a random value matrix, wherein the number of rows of the matrix is ​​equal to the size of the state space, and the number of columns is equal to the size of the action space, and the dynamic scanning optimization model parameters are initialized by setting the learning rate and the discount factor.

4. A road surface 3D digital high-precision scanning method according to claim 3, characterized in that: The DBSCAN clustering algorithm is used to divide the scan area, including defining the neighborhood of any point in the point cloud data set, determining the core point in the point cloud data set by setting a minimum number of points, and dividing the road point cloud data into different clustering areas according to the core point and its neighborhood relationship.

5. A road surface 3D digital high-precision scanning method according to claim 4, characterized in that: The adversarial generative network is introduced on the basis of the Gaussian mixture model to clean and denoise the multi-dimensional data, including normalizing the fused point cloud data and processing missing values, estimating the parameters of the Gaussian mixture model using the expectation maximization algorithm, and identifying noise points and abnormal points according to the posterior probability that each data point belongs to different Gaussian components after obtaining the parameters of the Gaussian mixture model. Then, the adversarial generative network composed of a generator and a discriminator is introduced. The expectation maximization algorithm formula is expressed as: Among them, γ ik represents the i-th data point x i The posterior probability of belonging to the kth Gaussian component, π k represents the weight of the kth Gaussian component, Represents the probability density function of the kth Gaussian distribution at the data point x i The value of π l Represents the weight of the I-th Gaussian component.

6. A road surface 3D digital high-precision scanning method according to claim 5, characterized in that: The construction of the knowledge graph for auxiliary cleaning includes constructing a road surface data knowledge graph, integrating the road surface flatness range, damage type characteristics and historical data knowledge of different road sections, using entity-relationship-entity triples for knowledge representation, and after completing the adversarial generative network denoising, using the road surface data knowledge graph to perform semantic verification on the currently collected data.

7. A road surface 3D digital high-precision scanning method according to claim 6, characterized in that: The feature extraction and matching based on the preprocessed point cloud data includes using a convolutional neural network to extract features from the preprocessed point cloud data. After the feature extraction is completed, the cosine similarity is used to calculate the similarity between different point cloud data, and the attention weight is calculated according to the similarity. Feature matching is performed according to the attention weight, and feature points with high similarity and high attention weight are matched. The attention formula is expressed as: Among them, sim(p i ,p j ) is represented as feature point p i and feature point p j The similarity between them, exp represents an exponential function, Σ k It is represented as the summation operation of all k feature points.

8. A road surface 3D digital high-precision scanning method according to claim 7, characterized in that: The three-dimensional modeling based on the point cloud data after feature matching includes inputting the point cloud data after feature matching and obtaining the physical characteristic parameters of the road surface, performing Delaunay triangulation on the point cloud data after feature matching, using the Bowyer-Watson algorithm to obtain an initial triangular mesh, deforming the mesh according to the physical characteristic parameters and topological relationship of the road surface, calculating the force of each node and the displacement of each node respectively, updating the node position according to the force and displacement, and updating the triangular mesh accordingly.

9. A road surface 3D digital high-precision scanning method according to claim 8, characterized in that: The method combines semantic segmentation with topological relationship analysis to extract terrain features and semantic information of the road surface, including using a three-dimensional convolution operation to generate a series of feature maps from a three-dimensional model, inputting the feature maps into a full convolutional network for semantic segmentation, and outputting a segmentation result of the same size as the input image, training the full convolutional network using cross entropy loss, obtaining topological information using a graph convolutional network, and completing road surface scanning based on the results of semantic segmentation and topological relationship analysis.

10. A road surface 3D digital high-precision scanning system, characterized in that: include: Lidar, which emits a laser beam and measures the time difference of reflected light; Radar signal processor, used to obtain distance information of each point on the road surface to form point cloud data and construct the three-dimensional geometry of the road surface; Differential GPS is used to provide precise geographic location information of the scanning device, including longitude, latitude and altitude. By comparing the differential signal with the ground base station, it provides geographic coordinate reference for the point cloud data. High-resolution cameras are used to simultaneously collect visual image data of the road surface and supplement the semantic features of point cloud data; An inertial measurement unit, which contains an accelerometer and a gyroscope, and is used to measure the acceleration and angular velocity data of the scanning device; The control terminal is used for time synchronization, coordinate conversion and alignment, data association and fusion, and real-time analysis and decision-making of the collected data.

Images