Deep learning-based road and bridge construction measurement data processing method and system

Abstract

The invention discloses a deep learning-based road and bridge construction measurement data processing method and system, and the method comprises the steps: extracting spatial topological features in point cloud data through a GCN graph convolutional network, extracting texture and structural features in image data through improved ResNet-50, extracting time sequence features in sensor data based on an LSTM long-short-term memory network, and carrying out the deep learning-based road and bridge construction measurement data processing. Introducing a multi-head self-attention mechanism to dynamically calculate weights of different modal features; constructing a VAE variational auto-encoder based on the fusion feature vector, judging an abnormal point through a reconstruction error, and performing secondary verification on the abnormal point in combination with a bridge structure mechanical model; and adopting a GAN generative adversarial network to generate alternative data of the abnormal point detection result, and selecting an optimal correction scheme through Bayesian optimization. And the data processing efficiency and the abnormal point detection accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of road and bridge construction measurement, and in particular to a method and system for processing road and bridge construction measurement data based on deep learning. Background Art

[0002] During road and bridge construction, the accuracy of measurement data directly impacts construction quality and safety. Existing technologies primarily rely on traditional algorithms or simple artificial neural networks for data processing. However, the correlations between point clouds, images, and sensor data are not fully explored, leading to redundant or missing information. Furthermore, traditional filtering methods are not sufficiently adaptable to the dynamic noise present in complex construction environments, resulting in inefficient construction measurement data processing and low accuracy in outlier detection. Summary of the Invention

[0003] The purpose of this invention is to solve the above problems and to design a road and bridge construction measurement data processing method based on deep learning.

[0004] To achieve the above-mentioned purpose, the technical solution of the present invention is that, further, in the above-mentioned road and bridge construction measurement data processing method based on deep learning, the road and bridge construction measurement data processing method includes the following steps: Acquire point cloud data, image data, and sensor data during road and bridge construction to obtain multimodal construction data; The GCN graph convolutional network is used to extract spatial topological features from point cloud data. The improved ResNet-50 is used to extract texture and structural features from image data. The LSTM long short-term memory network is used to extract temporal features from sensor data. A multi-head self-attention mechanism is introduced to dynamically calculate the weights of different modal features and generate a fused feature vector. A VAE variational autoencoder is constructed based on the fused feature vector, abnormal points are judged by reconstruction error, and the abnormal points are secondary verified in combination with the bridge structure mechanics model to obtain abnormal point detection results; A GAN generative adversarial network is used to generate alternative data for the outlier detection results, and the optimal correction scheme is selected through Bayesian optimization.

[0005] Furthermore, in the above-mentioned method for processing road and bridge construction measurement data based on deep learning, the point cloud data, image data, and sensor data obtained during the road and bridge construction process to obtain multimodal construction data include: Acquire point cloud data, image data, and sensor data during road and bridge construction to obtain multimodal construction data; denoising the point cloud data using statistical filtering to remove noise points generated by measurement errors and environmental interference, thereby obtaining de-noised point cloud data; Performing noise reduction processing on the image data by Gaussian filtering, and enhancing the noise-reduced image data by histogram equalization to obtain processed image data; The sensor data is cleaned in sequence to remove abnormal values ​​and missing values ​​to obtain processed sensor data.

[0006] Furthermore, in the above-mentioned method for processing road and bridge construction measurement data based on deep learning, the extraction of spatial topological features from point cloud data by using the GCN graph convolutional network includes: The K-nearest neighbor algorithm is used to search the neighborhood of each point in the point cloud data, and the K points closest to each point are selected as neighbor nodes to determine the connection relationship of the graph; Collect the features of all neighboring nodes of each node, combine the summed result with the node's own features through the ReLU function to obtain the updated node features; Through multiple iterations of graph convolutional layers, the spatial topological features in the point cloud data are gradually captured.

[0007] Furthermore, in the above-mentioned method for processing road and bridge construction measurement data based on deep learning, the extraction of texture and structural features from image data using the improved ResNet-50 includes: An attention module is added after the fourth residual block of the ResNet-50 neural network. This module is used to highlight the regional features in the image that are important for road and bridge construction, resulting in an improved ResNet-50 neural network. After processing, the image data is input into the improved ResNet-50 neural network. Through the convolution layer, pooling layer and residual block, the low-level and high-level features of the image are extracted to obtain the texture and structural feature vectors of the image data.

[0008] Furthermore, in the above-mentioned deep learning-based road and bridge construction measurement data processing method, the LSTM long short-term memory network is used to extract the temporal features in the sensor data, and a multi-head self-attention mechanism is introduced to dynamically calculate the weights of different modal features to generate a fused feature vector, including: By stacking multiple layers of LSTM, we can capture temporal features of different time scales in sensor data, including features of short time scales and features of long time scales. Different transformation matrix parameters are set according to the dimensions and importance of different modal features. The spatial topological features of the point cloud, the texture and structural features of the image, and the temporal features of the sensor are respectively used to generate query, key, and value vectors through their corresponding transformation matrices.

[0009] Furthermore, in the above-mentioned road and bridge construction measurement data processing method based on deep learning, the VAE variational autoencoder is constructed based on the fused feature vector, abnormal points are judged by reconstruction error, and the abnormal points are secondary verified in combination with the bridge structure mechanics model to obtain the abnormal point detection results, including: After receiving the fused feature vector, the first fully connected layer of the VAE variational autoencoder converts it into an intermediate feature with lower dimension through linear transformation and Sigmoid nonlinear activation function; The fully connected layer continues to compress and transform the intermediate features and outputs two vectors, corresponding to the mean and variance of the potential probability distribution, to obtain the outliers. For the abnormal points identified by the reconstruction error, the strain value and displacement of the corresponding structure are extracted and input into the bridge structure mechanics model to perform secondary verification on the abnormal points to obtain the abnormal point detection results.

[0010] Furthermore, in the above-mentioned method for processing road and bridge construction measurement data based on deep learning, the GAN generative adversarial network is used to generate alternative data for the outlier detection results, and the optimal correction scheme is selected through Bayesian optimization, including: The Bayesian optimization model is used to analyze the effect of a correction scheme using a certain material strength and construction time on the treatment of abnormal points. The acquisition function is used to select the parameter combination that needs to be evaluated and generate the optimal correction scheme.

[0011] Furthermore, in the road and bridge construction measurement data processing system based on deep learning, the road and bridge construction measurement data processing system includes the following modules: The construction data acquisition module is used to obtain point cloud data, image data, and sensor data during the road and bridge construction process to obtain multimodal construction data; The feature vector fusion module is used to extract spatial topological features from point cloud data through the GCN graph convolutional network, extract texture and structural features from image data using the improved ResNet-50, extract temporal features from sensor data based on the LSTM long short-term memory network, and introduce a multi-head self-attention mechanism to dynamically calculate the weights of different modal features to generate a fused feature vector; An outlier detection module is used to construct a VAE variational autoencoder based on the fused feature vector, determine outliers through reconstruction errors, and perform secondary verification on the outliers in combination with the bridge structure mechanics model to obtain outlier detection results; A correction scheme optimization module is used to generate alternative data for the outlier detection results using a GAN generative adversarial network and select the optimal correction scheme through Bayesian optimization.

[0012] Furthermore, in the system for implementing the above-mentioned road and bridge construction measurement data processing method based on deep learning, the outlier detection module includes the following submodules: The conversion submodule is used to convert the fused feature vector received by the first fully connected layer of the VAE variational autoencoder into an intermediate feature with lower dimension through linear transformation and Sigmoid nonlinear activation function; The output submodule is used by the fully connected layer to further compress and transform the intermediate features and output two vectors, which correspond to the mean and variance of the potential probability distribution, respectively, to obtain the outliers. The verification submodule is used to extract the strain value and displacement of the corresponding structure for the abnormal point determined by the reconstruction error, input the abnormal point into the bridge structure mechanical model for secondary verification, and obtain the abnormal point detection result.

[0013] Furthermore, in the system for implementing the above-mentioned road and bridge construction measurement data processing method based on deep learning, the correction scheme optimization module includes the following submodules: The optimization submodule is used to analyze the treatment effect of the correction scheme using a certain material strength and construction time on the abnormal points through the Bayesian optimization model, and use the acquisition function to select the parameter combination to be evaluated to generate the optimal correction scheme.

[0014] Its beneficial effects are as follows: 1. It utilizes a multi-head self-attention mechanism to dynamically fuse multimodal features, fully tapping into the inherent connections between different types of data, improving data utilization and processing efficiency while also increasing the accuracy of outliers. 2. It not only effectively identifies true outliers, significantly reducing missed and false detection rates, but also interprets them from a structural mechanics perspective, making the determination of outliers more scientific and reliable. 3. It improves the scientific and rationality of correction schemes, helping to reduce construction costs, shorten construction periods, and ensure the quality and safety of road and bridge construction while ensuring correction effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Various other advantages and benefits will become apparent to those skilled in the art by reading the following detailed description of the preferred embodiment.The accompanying drawings are only for the purpose of illustrating the preferred embodiment and are not to be considered as limiting the present invention.

[0016] Figure 1 This is a schematic diagram of a first embodiment of a method for processing road and bridge construction measurement data based on deep learning in an embodiment of the present invention; Figure 2 Schematic diagram of a second embodiment of a method for processing road and bridge construction measurement data based on deep learning in an embodiment of the present invention; Figure 3This is a schematic diagram of the first embodiment of the road and bridge construction measurement data processing system based on deep learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0018] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a," "an," "said," and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0019] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1 As shown, a road and bridge construction measurement data processing method based on deep learning includes the following steps: Step 101: Acquire point cloud data, image data, and sensor data during the road and bridge construction process to obtain multimodal construction data; Specifically, in this embodiment, point cloud data, image data, and sensor data are acquired during the road and bridge construction process to obtain multimodal construction data; Statistical filtering is used to denoise point cloud data, removing noise points caused by measurement errors and environmental interference to obtain de-noised point cloud data; The image data is subjected to noise reduction processing by Gaussian filtering, and the noise-reduced image data is enhanced by using histogram equalization to obtain processed image data; The sensor data is cleaned in turn, outliers and missing values ​​are removed to obtain processed sensor data.

[0020] Specifically: (1) Point cloud data acquisition; Point cloud data can be acquired using terrestrial laser scanners, which can quickly and accurately capture 3D coordinate information on an object's surface. This makes it suitable for small-scale, high-precision measurements of road and bridge components. However, measurement accuracy can be affected in adverse weather conditions such as rain and fog.

[0021] (2) Image data acquisition; Image data can be obtained through aerial photography using drones. Drones can quickly cover large construction areas and obtain wide-area image information, making them suitable for monitoring the overall construction status of roads and bridges. However, the accuracy of aerial images is significantly affected by factors such as flight altitude and camera parameters. The lower the flight altitude, the higher the image accuracy, but the smaller the coverage. Meanwhile, fixed on-site cameras can also be used to obtain image data, enabling real-time monitoring of specific construction areas and obtaining continuous image sequences. These are suitable for detailed observation of key areas during the construction process, but their coverage is limited.

[0022] (3) Sensor data acquisition; Sensor data is primarily collected through various monitoring sensors, such as strain sensors, displacement sensors, and accelerometers. Strain sensors can be installed at key stress-bearing locations on roads and bridges to monitor structural strain in real time; displacement sensors measure structural displacement; and accelerometers capture structural vibration responses. These sensors are typically connected to data acquisition systems via wired or wireless connections, enabling real-time data transmission and storage.

[0023] (4) data preprocessing; Point cloud data preprocessing: First, perform denoising to remove noise points caused by measurement errors and environmental interference. Methods such as statistical filtering and radius filtering can be used. Next, perform point cloud registration. When point cloud data is acquired from different viewpoints or at different times, it must be unified into the same coordinate system. Finally, perform downsampling to reduce the data volume and improve subsequent processing efficiency.

[0024] Image data preprocessing: This includes operations such as image denoising, enhancement, and correction. Image denoising can be performed using methods such as Gaussian filtering and median filtering. Image enhancement is used to improve image contrast and clarity, and can be achieved through methods such as histogram equalization. Image correction eliminates image distortion caused by camera distortion, shooting angle, and other factors.

[0025] Sensor data preprocessing: This primarily involves data cleaning to remove outliers and missing values. Missing values ​​can be filled using interpolation, while outliers can be identified and processed using methods such as setting thresholds. Sensor data also needs to be standardized or normalized to ensure comparability between different sensors.

[0026] Step 102: Extract spatial topological features from point cloud data using a GCN graph convolutional network, extract texture and structural features from image data using an improved ResNet-50, extract temporal features from sensor data based on an LSTM long short-term memory network, and introduce a multi-head self-attention mechanism to dynamically calculate the weights of different modal features to generate a fused feature vector. Specifically, in this embodiment, the K-nearest neighbor algorithm is used to search the neighborhood of each point in the point cloud data, and the K points closest to each point are selected as neighbor nodes to determine the connection relationship of the graph; Collect the features of all neighboring nodes of each node, combine the summed result with the node's own features through the ReLU function to obtain the updated node features; Through multiple iterations of graph convolutional layers, the spatial topological features in the point cloud data are gradually captured.

[0027] An attention module is added after the fourth residual block of the ResNet-50 neural network. This module is used to highlight the regional features in the image that are important for road and bridge construction, resulting in an improved ResNet-50 neural network. After processing, the image data is input into the improved ResNet-50 neural network. Through the convolution layer, pooling layer and residual block, the low-level and high-level features of the image are extracted to obtain the texture and structural feature vectors of the image data.

[0028] By stacking multiple layers of LSTM, we can capture temporal features of different time scales in sensor data, including features of short time scales and features of long time scales. Different transformation matrix parameters are set according to the dimensions and importance of different modal features. The spatial topological features of the point cloud, the texture and structural features of the image, and the temporal features of the sensor are respectively used to generate query, key, and value vectors through their corresponding transformation matrices.

[0029] Specifically: 1. GCN graph convolutional network extracts spatial topological features of point cloud data; When constructing a graph from point cloud data, the neighborhood of each point is first searched, typically using the K-nearest neighbor algorithm. This algorithm selects the K nearest neighbors for each point to determine the graph's connectivity. Edge weights consider not only the distance and angle between points, but also their normal vectors. Points with greater normal vector differences have smaller edge weights, as large differences in normal vectors indicate a weaker spatial relationship between the two points.

[0030] During the convolution operation, for each node, the features of all its neighboring nodes are first collected. During the weighted summation, the edge weights are dynamically adjusted, so that the features of neighboring nodes with closer distances, more reasonable angles, and more similar normal vectors contribute more to the summation. The summation result is then combined with the node's own features using a nonlinear activation function (ReLU function) to obtain the updated node features. By iterating through these graph convolution layers multiple times, the network is able to gradually capture more complex spatial topological features in the point cloud data, such as the connections between bridge components and the spatial layout of the overall structure.

[0031] 2. Improved ResNet-50 extracts texture and structural features of image data; The introduced attention mechanism consists of adding a channel attention module and a spatial attention module after some convolutional layers in the network. The channel attention module evaluates the importance of each feature channel and assigns higher weights to channels that are more critical to structural features (such as those containing crack edge information). The spatial attention module focuses on areas of the image with significant structural changes (such as component joints and areas that may deform), enhancing the feature response in these areas.

[0032] The optimized residual block adds an adaptive adjustment coefficient to the skip connection. This coefficient changes dynamically based on the complexity of the input features. When the input features contain more detailed information, the coefficient increases, allowing more original features to be passed to the next layer via the skip connection, reducing feature loss. During feature extraction, the shallow layers of the network primarily extract texture features of the image, such as the texture of concrete surfaces and the grain of steel. As the network deepens, more advanced structural features are gradually extracted, such as the shape and composition of bridge beams, slabs, and columns.

[0033] 3. LSTM long short-term memory network extracts the temporal features of sensor data; When processing sensor data, the data is first segmented into fixed time intervals. The data for each time period serves as an input sequence for the LSTM network. When deciding to discard historical information, the forget gate considers the rate of change between the current input data and the historical data. A low rate of change indicates that the data is stable, and more historical information will be retained. A high rate of change may indicate a change in the structural state, and some outdated historical information will be discarded.

[0034] When selecting new information, the input gate focuses on data changes related to key structural parameters (maximum allowable strain, normal vibration range), storing this important new information in the cell state. The output gate combines the current cell state with the input information to output features that reflect the changing trends of sensor data over that time period, such as increasing or decreasing strain data or frequency trends in vibration data. By stacking multiple layers of LSTM, it is possible to capture temporal features at different time scales in sensor data, including short-term features such as instantaneous vibration changes and long-term features such as the cumulative changes in strain over construction progress.

[0035] 4. Multi-head self-attention mechanism fusion features; When performing linear transformations on each modal feature vector, different transformation matrix parameters are set according to the dimensions and importance of different modal features. The spatial topological features of the point cloud, the texture and structural features of the image, and the temporal features of the sensor are respectively generated into query, key, and value vectors through their corresponding transformation matrices.

[0036] When calculating attention weights, not only the correlation of features within the same modality is considered, but also the degree of correlation between features from different modalities. For example, the correlation between the structural deformation features shown in the image and the strain increase features in the sensor data is calculated. The higher the correlation, the greater the corresponding attention weight. The multiple heads enable the network to focus on the relationship between features from different perspectives. Some heads may focus more on the correlation between spatial and structural features, while others may focus more on the matching of temporal and spatial features. Finally, through splicing and linear transformation, the correlation information from these different perspectives is fused into a comprehensive fused feature vector that fully reflects the key information in the multimodal data.

[0037] Step 103: construct a VAE variational autoencoder based on the fused feature vector, determine the outliers through reconstruction error, perform secondary verification on the outliers in combination with the bridge structure mechanics model, and obtain the outlier detection result; Specifically, in this embodiment, after receiving the fused feature vector, the first fully connected layer of the VAE variational autoencoder converts it into an intermediate feature with lower dimension through linear transformation and Sigmoid nonlinear activation function. The fully connected layer continues to compress and transform the intermediate features and outputs two vectors, corresponding to the mean and variance of the potential probability distribution, to obtain the outliers. For the abnormal points identified by reconstruction error, the strain value and displacement of the corresponding structure are extracted and input into the bridge structure mechanics model for secondary verification of the abnormal points to obtain the abnormal point detection results.

[0038] Specifically: 1. VAE variational autoencoder construction; The encoder of a VAE variational autoencoder consists of multiple fully connected layers, which progressively process the fused feature vector. The first fully connected layer receives the fused feature vector and converts it into lower-dimensional intermediate features through a linear transformation and a nonlinear activation function (sigmoid function). Subsequent fully connected layers further compress and transform the intermediate features, ultimately outputting two vectors corresponding to the mean and variance of the underlying probability distribution.

[0039] The decoder also consists of multiple fully connected layers, with a structure symmetrical to that of the encoder. First, a sample is randomly sampled from the latent probability distribution output by the encoder. This sample serves as the decoder input. The first fully connected layer of the decoder applies a linear transformation and an activation function to this sample, converting it into a feature of slightly higher dimensionality. Subsequent fully connected layers gradually increase the feature dimensionality. After multiple processing steps, the decoder outputs a reconstructed vector with the same dimensionality as the input fused feature vector.

[0040] During training, in addition to focusing on reconstruction error, the KL divergence must also be considered. Reconstruction error measures the difference between the input fused feature vector and the reconstructed vector. A smaller difference indicates better decoder reconstruction. KL divergence constrains the underlying probability distribution to be as close as possible to a standard Gaussian distribution. This makes the generated latent samples more generalizable and prevents the model from overfitting the training data. VAE training is achieved by continuously adjusting the weights and biases of the fully connected layers of the encoder and decoder to minimize the sum of the reconstruction error and KL divergence.

[0041] 2. Determine outliers based on reconstruction error; When a trained VAE processes a new fused feature vector, the encoder maps it to a potential probability distribution and obtains its mean and variance. The decoder then generates a reconstruction vector based on the sampled data. The reconstruction error between the input fused feature vector and the reconstructed vector is calculated. A common calculation method is the Euclidean distance, which is the square root of the sum of the squared differences between the corresponding elements of the two vectors.

[0042] To determine the threshold for identifying outliers, we first collect a large number of normal fused feature vectors and feed them into the VAE to obtain their respective reconstruction errors. We then perform a statistical analysis of the reconstruction errors of these normal samples, calculating their mean and standard deviation. Typically, the mean plus three times the standard deviation is used as the threshold. When the reconstruction error of a new fused feature vector exceeds this threshold, the construction data corresponding to that feature vector is preliminarily identified as an outlier.

[0043] However, in practice, the threshold is not fixed. It is dynamically adjusted according to the construction stage. For example, during the construction of the bridge foundation, due to the many structural instabilities, the reconstruction error of the normal data fluctuates greatly, so the threshold can be appropriately increased. Meanwhile, in the later stages of the main structure construction, the structure is relatively stable, and the reconstruction error of the normal data fluctuates less, so the threshold can be appropriately lowered.

[0044] 3. Conduct secondary verification of abnormal points based on the bridge structure mechanics model; The bridge structural mechanics model is established based on the design drawings of the road bridge, material properties (compressive strength of concrete, tensile strength of steel), construction technology and other information. It can simulate the stress and deformation of the bridge under different loads (self-weight, vehicle load, wind load).

[0045] For outliers identified through reconstruction errors, corresponding construction data, such as structural strain and displacement, are extracted and input into the bridge structural mechanics model. Based on this input data, the model calculates the theoretical stress distribution and deformation of the structure under that state, including the stress magnitude, displacement direction, and magnitude at each location.

[0046] The theoretical results calculated by the model are compared with the structural conditions monitored during actual construction. If the actual monitored abnormal data deviates significantly from the theoretical results and exceeds a reasonable error range, it indicates that the abnormal point is real and may reflect actual problems with the bridge structure, such as excessive local stress or abnormal structural deformation. If the actual abnormal data deviates from the theoretical results within a reasonable range, it may be a false abnormal point caused by measurement error or other non-structural factors and should be excluded. This secondary verification can improve the accuracy of abnormal point detection and reduce misjudgments.

[0047] Step 104: Generate alternative data for outlier detection results using a GAN generative adversarial network, and select the optimal correction solution through Bayesian optimization.

[0048] Specifically, in this embodiment, the Bayesian optimization model is used to analyze the treatment effect of a correction scheme using a certain material strength and construction time on anomalies, and the acquisition function is used to select the parameter combination to be evaluated to generate the optimal correction scheme.

[0049] Specifically: 1. GAN generates adversarial networks to generate alternative data; The GAN generative adversarial network consists of two core components: the generator and the discriminator, which are jointly optimized through continuous competition. The generator's main function is to learn the characteristics of outlier detection results and generate surrogate data that is similar to real anomaly data; the discriminator is responsible for distinguishing whether the input data is a real outlier detection result or a surrogate data generated by the generator. In the early stages of training, the generator randomly generates data that often differs significantly from actual anomaly data, making it easy for the discriminator to identify it as "fake data." As training progresses, the generator continuously adjusts its parameters based on feedback from the discriminator, gradually generating substitute data that more closely resembles the characteristics of actual anomaly data. For example, it can simulate strain trends and displacement anomaly patterns similar to those of actual anomalies. Simultaneously, the discriminator continuously learns, improving its ability to distinguish between real and generated data. When training reaches a stable state, the generator generates surrogate data that is difficult for the discriminator to distinguish between true and false. Not only is the data distribution of these surrogate data highly consistent with the actual outlier detection results, but they also encompass the anomaly patterns that may occur in different scenarios. For example, for local structural deformation anomalies that may occur during bridge construction, the generator can generate deformation data of varying degrees and locations, providing rich sample support for subsequent correction plan selection.

[0050] Second, Bayesian optimization selects the optimal correction scheme; Bayesian optimization is an optimization method based on probabilistic models that can efficiently find the optimal solution without extensive experimentation. When selecting the optimal correction solution, it's important to first identify the key parameters that influence the correction. These parameters may include the time required for the correction, the strength of the materials used, and the location of the structural stresses to be adjusted. Different parameter combinations correspond to different correction solutions. Bayesian optimization first builds a probabilistic model. This model can predict the effects of different parameter combinations based on existing correction solutions and their effectiveness data. For example, by analyzing the past performance of correction solutions using a certain material strength and construction time on outliers, the model can infer the potential effects of other parameter combinations. Then, based on this probabilistic model, an acquisition function is used to select the next parameter combination to evaluate. The acquisition function balances exploration (trying new parameter combinations that may yield better results) with exploitation (fine-tuning around known good parameter combinations), prioritizing those parameter combinations that are likely to yield the best results for evaluation. By continuously selecting parameter combinations, implementing correction plans, and recording the results, Bayesian optimization continuously updates the probability model, making the model's predictions about the effects of correction plans more accurate. When the model's predictions stabilize and a parameter combination that achieves the best correction results is found, the corresponding correction plan is considered the optimal one. For example, for a structural strain anomaly, Bayesian optimization can determine the use of reinforcement materials of a specific strength and construction within a specific time window, achieving the optimal correction effect with minimal cost and effort, restoring the structure to its normal state.

[0051] Its beneficial effects include: 1. It utilizes a multi-head self-attention mechanism to dynamically fuse multimodal features, fully tapping into the inherent connections between different types of data and improving data information utilization. 2. It not only effectively identifies true outliers, significantly reducing missed and false detection rates, but also interprets outliers from a structural mechanics perspective, making the determination of outliers more scientific and reliable. 3. It improves the scientific and rationality of correction schemes, helping to reduce construction costs, shorten construction periods, and ensure the quality and safety of road and bridge construction while ensuring correction effectiveness.

[0052] See also Figure 2 In the road and bridge construction measurement data processing method based on deep learning, obtaining point cloud data, image data, and sensor data during the road and bridge construction process to obtain multimodal construction data includes the following steps: Step 201: Acquire point cloud data, image data, and sensor data during the road and bridge construction process to obtain multimodal construction data; Step 202: denoising the point cloud data using statistical filtering to remove noise points caused by measurement errors and environmental interference, thereby obtaining de-noised point cloud data. Step 203: performing noise reduction processing on the image data by Gaussian filtering, and enhancing the noise-reduced image data by using histogram equalization to obtain processed image data; Step 204 : Clean the sensor data in sequence, remove outliers and missing values, and obtain processed sensor data.

[0053] See also Figure 3 In the road and bridge construction measurement data processing system based on deep learning, the road and bridge construction measurement data processing system includes the following modules: The construction data acquisition module is used to obtain point cloud data, image data, and sensor data during the road and bridge construction process to obtain multimodal construction data; The feature vector fusion module is used to extract spatial topological features from point cloud data through the GCN graph convolutional network, extract texture and structural features from image data using the improved ResNet-50, extract temporal features from sensor data based on the LSTM long short-term memory network, and introduce a multi-head self-attention mechanism to dynamically calculate the weights of different modal features to generate a fused feature vector; The outlier detection module is used to construct a VAE variational autoencoder based on the fused feature vector, identify outliers through reconstruction errors, and perform secondary verification of outliers in combination with the bridge structure mechanics model to obtain outlier detection results. The correction scheme optimization module is used to generate alternative data for outlier detection results using a GAN generative adversarial network and select the optimal correction scheme through Bayesian optimization.

[0054] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A road and bridge construction measurement data processing method based on deep learning, characterized in that: The road and bridge construction measurement data processing method comprises the following steps: Acquire point cloud data, image data, and sensor data during road and bridge construction to obtain multimodal construction data; The GCN graph convolutional network is used to extract spatial topological features from point cloud data. The improved ResNet-50 is used to extract texture and structural features from image data. The LSTM long short-term memory network is used to extract temporal features from sensor data. A multi-head self-attention mechanism is introduced to dynamically calculate the weights of different modal features and generate a fused feature vector. A VAE variational autoencoder is constructed based on the fused feature vector, abnormal points are judged by reconstruction error, and the abnormal points are secondary verified in combination with the bridge structure mechanics model to obtain abnormal point detection results; A GAN generative adversarial network is used to generate alternative data for the outlier detection results, and the optimal correction scheme is selected through Bayesian optimization.

2. The road and bridge construction measurement data processing method based on deep learning according to claim 1, characterized in that: The point cloud data, image data and sensor data obtained during the road and bridge construction process to obtain multimodal construction data include: Acquire point cloud data, image data, and sensor data during road and bridge construction to obtain multimodal construction data; denoising the point cloud data using statistical filtering to remove noise points generated by measurement errors and environmental interference, thereby obtaining de-noised point cloud data; Performing noise reduction processing on the image data by Gaussian filtering, and enhancing the noise-reduced image data by histogram equalization to obtain processed image data; The sensor data is cleaned in sequence to remove abnormal values ​​and missing values ​​to obtain processed sensor data.

3. The road and bridge construction measurement data processing method based on deep learning according to claim 1, characterized in that: The extraction of spatial topological features from point cloud data through the GCN graph convolutional network includes: The K-nearest neighbor algorithm is used to search the neighborhood of each point in the point cloud data, and the K points closest to each point are selected as neighbor nodes to determine the connection relationship of the graph; Collect the features of all neighboring nodes of each node, combine the summed result with the node's own features through the ReLU function to obtain the updated node features; Through multiple iterations of graph convolutional layers, the spatial topological features in the point cloud data are gradually captured.

4. The road and bridge construction measurement data processing method based on deep learning according to claim 1, characterized in that: The improved ResNet-50 is used to extract texture and structural features from image data, including: An attention module is added after the fourth residual block of the ResNet-50 neural network. This module is used to highlight the regional features in the image that are important for road and bridge construction, resulting in an improved ResNet-50 neural network. After processing, the image data is input into the improved ResNet-50 neural network. Through the convolution layer, pooling layer and residual block, the low-level and high-level features of the image are extracted to obtain the texture and structural feature vectors of the image data.

5. The method for processing road and bridge construction measurement data based on deep learning according to claim 1, wherein: The method extracts temporal features from sensor data based on the LSTM long short-term memory network, introduces a multi-head self-attention mechanism to dynamically calculate the weights of different modal features, and generates a fused feature vector, including: By stacking multiple layers of LSTM, we can capture temporal features of different time scales in sensor data, including features of short time scales and features of long time scales. Different transformation matrix parameters are set according to the dimensions and importance of different modal features. The spatial topological features of the point cloud, the texture and structural features of the image, and the temporal features of the sensor are respectively used to generate query, key, and value vectors through their corresponding transformation matrices.

6. The method for processing road and bridge construction measurement data based on deep learning according to claim 1, wherein: The VAE variational autoencoder is constructed based on the fused feature vector, abnormal points are judged by reconstruction error, and the abnormal points are secondary verified in combination with the bridge structure mechanics model to obtain abnormal point detection results, including: After receiving the fused feature vector, the first fully connected layer of the VAE variational autoencoder converts it into an intermediate feature with lower dimension through linear transformation and Sigmoid nonlinear activation function; The fully connected layer continues to compress and transform the intermediate features and outputs two vectors, corresponding to the mean and variance of the potential probability distribution, to obtain the outliers. For the abnormal points identified by the reconstruction error, the strain value and displacement of the corresponding structure are extracted and input into the bridge structure mechanics model to perform secondary verification on the abnormal points to obtain the abnormal point detection results.

7. The method for processing road and bridge construction measurement data based on deep learning according to claim 1, characterized in that: The method of using a GAN to generate adversarial network to generate alternative data for the outlier detection result and selecting the optimal correction solution through Bayesian optimization includes: The Bayesian optimization model is used to analyze the effect of a correction scheme using a certain material strength and construction time on the treatment of abnormal points. The acquisition function is used to select the parameter combination that needs to be evaluated and generate the optimal correction scheme.

8. The road and bridge construction measurement data processing system based on deep learning is characterized by: The road and bridge construction measurement data processing system includes the following modules: The construction data acquisition module is used to obtain point cloud data, image data, and sensor data during the road and bridge construction process to obtain multimodal construction data; The feature vector fusion module is used to extract spatial topological features from point cloud data through the GCN graph convolutional network, extract texture and structural features from image data using the improved ResNet-50, extract temporal features from sensor data based on the LSTM long short-term memory network, and introduce a multi-head self-attention mechanism to dynamically calculate the weights of different modal features to generate a fused feature vector; An outlier detection module is used to construct a VAE variational autoencoder based on the fused feature vector, determine outliers through reconstruction errors, and perform secondary verification on the outliers in combination with the bridge structure mechanics model to obtain outlier detection results; A correction scheme optimization module is used to generate alternative data for the outlier detection results using a GAN generative adversarial network and select the optimal correction scheme through Bayesian optimization.

9. The road and bridge construction measurement data processing system based on deep learning according to claim 8, characterized in that: The outlier detection module includes the following submodules: The conversion submodule is used to convert the fused feature vector received by the first fully connected layer of the VAE variational autoencoder into an intermediate feature with lower dimension through linear transformation and Sigmoid nonlinear activation function; The output submodule is used by the fully connected layer to further compress and transform the intermediate features and output two vectors, which correspond to the mean and variance of the potential probability distribution, respectively, to obtain the outliers. The verification submodule is used to extract the strain value and displacement of the corresponding structure for the abnormal point determined by the reconstruction error, input the abnormal point into the bridge structure mechanical model for secondary verification, and obtain the abnormal point detection result.

10. The road and bridge construction measurement data processing system based on deep learning according to claim 8, characterized in that: The correction scheme optimization module includes the following submodules: The optimization submodule is used to analyze the treatment effect of the correction scheme using a certain material strength and construction time on the abnormal points through the Bayesian optimization model, and use the acquisition function to select the parameter combination that needs to be evaluated to generate the optimal correction scheme.

Images