A method for calculating structural strain distribution using lidar and deep learning neural networks

Abstract

This invention discloses a method for calculating structural strain distribution using lidar and deep learning neural networks, belonging to the fields of structural health monitoring, experimental mechanics, and non-contact measurement technology. The method first acquires three-dimensional point clouds of the structure at its initial state and in different states using lidar, and obtains the relative displacement field after preprocessing. A deep learning neural network model containing an embedding module, a Transformer module, a downsampling module, an upsampling module, and a fully connected layer is constructed, and the model is trained using data acquired through digital image correlation technology. Finally, the preprocessed lidar point cloud data is input into the trained model, and the full-field strain distribution is output. This invention combines the advantages of lidar and deep learning, overcoming the sparsity and noise effects of point clouds, achieving efficient and high-precision full-field strain measurement, and is suitable for monitoring large or complex structures.

Description

Technical Field

[0001] This invention belongs to the field of structural health monitoring technology, specifically relating to a method for calculating structural strain distribution using lidar and deep learning neural networks. Background Technology

[0002] Strain is a core physical quantity characterizing the degree of deformation of structural materials under stress, and it is directly related to the safety, durability, and performance evaluation of structures. In aerospace, civil engineering (bridges, dams, buildings), wind power generation, shipbuilding, heavy machinery and other fields, accurate and efficient full-field strain monitoring of large or complex structures is crucial, and can be used to verify designs, assess service status, predict remaining life, and prevent catastrophic failures.

[0003] Limitations of existing strain measurement technologies: Traditional contact sensors (such as strain gauges) can only provide discrete point measurements, making it difficult to capture the strain distribution across the entire field. They are also complex to install, susceptible to environmental influences (temperature, humidity), unsuitable for large-scale or hard-to-reach structures, and their long-term monitoring reliability may be limited. Fiber optic sensors (such as FBGs), while capable of quasi-distributed measurements, are still point or line measurements, resulting in high costs, complex deployment, and limited spatial resolution for full field coverage.

[0004] Digital image correlation (DIC) is currently the mainstream optical method for non-contact full-field displacement and strain measurement. It calculates high-resolution displacement and strain fields by tracking the deformation of speckle patterns on an object's surface. However, DIC technology also has significant limitations: Limited field of view and distance; high-precision DIC typically requires close-range (within several meters) imaging, and its high-resolution camera and lens combinations struggle to cover the entire field of ultra-large structures (such as hundred-meter-long bridges, large wind turbine blades, and entire aircraft). Environmental sensitivity; it is sensitive to lighting conditions (requiring stable and uniform illumination), vibration, and obstructions, limiting its application in outdoor, high-altitude, or harsh industrial environments. Equipment cost and complexity; high frame rate, high-resolution cameras and synchronization systems are expensive, and calibration of multi-camera systems for large fields of view is complex. Large data processing volume; calculating full-field displacement and strain requires processing a large amount of image data, resulting in long computation times.

[0005] LiDAR (especially phase-modulated or frequency-modulated continuous wave (FMCW) LiDAR) can measure the three-dimensional coordinates of distant targets (tens to hundreds of meters) with high precision (millimeter-level or even sub-millimeter-level). Its scanning characteristics make it very suitable for rapid, non-contact, full-field three-dimensional shape and displacement measurement of large-scale structures. It is less affected by lighting conditions and has strong robustness in complex outdoor environments.

[0006] The core output of LiDAR is three-dimensional point cloud coordinates (or the displacement field calculated from them). Because the point cloud is randomly distributed during each measurement, it is impossible to track the state at different time steps. Therefore, reliably and accurately calculating the full-field strain directly from sparse or moderately dense LiDAR point cloud displacement data is a significant challenge. Furthermore, slight tilt between the instrument and the ground can complicate the calculation of relative displacement, making it difficult to obtain accurate coordinate values. Traditional numerical differentiation methods (such as derivative calculations based on point clouds or grids) are highly sensitive to noise in the displacement data. Even with high LiDAR displacement measurement accuracy, its spatial resolution (point density) is typically much lower than that of DIC. Noise amplification can lead to a significant decrease in the accuracy of the calculated strain field, or even erroneous results. While DIC can provide high-precision strain ground truth values ​​for training, it is difficult to meet the measurement requirements of ultra-large structures or harsh environments.

[0007] Currently, there is a lack of a practical method that can effectively combine the advantages of LiDAR (Light Detection and Ranging) in large-scale, long-distance, and highly robust displacement measurement, while overcoming the effects of point cloud sparsity and noise, to achieve high-precision, full-field strain calculation. Summary of the Invention

[0008] Based on the above shortcomings, this invention proposes a method for calculating structural strain distribution using lidar and deep learning neural networks. This invention can output high-precision full-field strain distribution, overcoming the problem of long processing time from sparse or medium-density LiDAR point cloud displacement data and DIC data.

[0009] The technology employed in this invention is as follows: A method for calculating structural strain distribution using lidar and deep learning neural networks, comprising the following steps:

[0010] Step 1: Obtain the initial 3D point cloud of the structure and the 3D point cloud under different states. The 3D point cloud is obtained by measuring with a lidar.

[0011] Step 2: Preprocess the initial 3D point cloud and the 3D point clouds under different states to obtain the relative displacement field relative to the initial state;

[0012] Step 3: Construct a deep learning neural network model, including an embedding module, a Transformer module, a downsampling module, an upsampling module, and a fully connected layer. The embedding module processes point cloud data using convolution at the core points of the point cloud. The Transformer module extracts the feature relationships between points through a self-attention mechanism based on a shift window. The downsampling module compresses the point cloud and expands the receptive field through max pooling. The upsampling module is used to aggregate features and restore the point cloud scale. The fully connected layer is used to output the strain distribution.

[0013] Step 4: Train the neural network model using displacement and strain field data obtained by digital image correlation technology;

[0014] Step 5: Input the preprocessed LiDAR point cloud data into the trained model and output the full-field strain distribution of the structure.

[0015] Furthermore, the preprocessing in step 2 as described above includes the following sub-steps:

[0016] Sub-step 2.1: Retain the central 90% area of ​​each point cloud and remove irregular edge areas;

[0017] Sub-step 2.2: Perform principal component analysis on the initial point cloud without load, extract the third principal component: the normal vector of the best-fit plane, and align the third principal component with the Z-axis so that the point cloud is parallel to the ground;

[0018] Sub-step 2.3: Rotate each point cloud captured at different pressure levels by the same rotation angle as the initial point cloud, and calculate the relative displacement field relative to the initial point cloud.

[0019] Furthermore, as described above, the embedding module uses point cloud core point convolution to process point cloud data, and the kernel function of the point cloud core point convolution is defined as:

[0020]

[0021] Where, d i Let W be the Euclidean distance between the kernel point and the local coordinates of the points in its neighborhood, and let W be the dimension transformation matrix. The convolution weights, i.e. the coefficients before the dimension transformation matrix W, are inversely proportional to the Euclidean distance between the kernel point and the local coordinates of the points in its neighborhood.

[0022] Furthermore, as described above, the Transformer module employs a self-attention mechanism based on a shift window, including: first, constructing an inter-point self-attention map within a rectangular window of size 2×l, wherein the self-attention map is calculated as follows:

[0023]

[0024] Where P is the set of points within the rectangular window, n is the number of points within the window, q is the query vector, and k i Let v be the key vector. i It is a value vector;

[0025] Then, a translation window is used to process the translation point set P. s To achieve feature interaction between windows, the self-attention calculation method for the panning window is as follows:

[0026]

[0027] Where m is the number of points within the translation window, and c is the number of points associated with cross-attention;

[0028] Finally, a hierarchical key point sampling strategy is adopted to gradually expand the window size of each Transformer module along the processing hierarchy, thereby expanding the point cloud operation range.

[0029] Furthermore, as mentioned above, the downsampling module embeds a max-pooling operation, defined as follows:

[0030]

[0031] Where N(j) is the set of neighborhood points of point j, f l (j) represents the feature of point i in the previous layer, and the point feature f after processing by the downsampling module. l+1 (j) is the input feature f of the previous layer. l (j) is the maximum value within the neighborhood N(j) centered at point j, and the point set P l+1 It is a point set P l The sampled subset has a size of P. l 1 / 4 of it.

[0032] Furthermore, as described above, the upsampling module maps the output features of the next Transformer module to the scale of the previous layer through interpolation, and aggregates the output features of all Transformer modules, finally inputting the strain distribution of the fully connected layer.

[0033] Furthermore, the training process in step 4 above includes: measuring the displacement field and corresponding strain field of the structure using digital image correlation technology as training data; randomly dividing the point cloud of the training data into multiple sub-point clouds, each sub-point cloud containing a preset number of points; performing random data augmentation on the sub-point clouds, and training the neural network using random deactivation and batch normalization techniques until the model converges.

[0034] This invention also provides a deep learning neural network model for implementing the method described above for calculating structural strain distribution using lidar and a deep learning neural network. The model includes an embedding module, multiple Transformer modules, a downsampling module, an upsampling module, and a fully connected layer connected sequentially. The embedding module employs point cloud core point convolution, the Transformer module employs a self-attention mechanism based on a shift window and a hierarchical keypoint sampling strategy, the downsampling module embeds a max-pooling operation, and the upsampling module employs an interpolation operation.

[0035] Advantages and beneficial effects of this invention: This invention measures the displacement and strain of a specimen surface using the DIC method, and uses the displacement data to train a deep neural network model. During training, the obtained original point cloud is randomly divided into more than one thousand sub-point clouds as training datasets. Each sub-point cloud contains approximately ten thousand points, and each sub-point cloud is independently subjected to random data augmentation, improving the model's stability. Furthermore, the model of this invention employs random deactivation and batch normalization techniques to ensure the network's robustness. After model training is complete, inputting LiDAR into the model directly outputs a high-precision full-field strain distribution. This invention fully utilizes the advantages of DIC and LiDAR, achieving efficient and high-precision measurement of the full-field strain distribution. Attached Figure Description

[0036] Figure 1 This is a diagram of a neural network model framework;

[0037] Figure 2 This is a flowchart of the misalignment correction preprocessing.

[0038] Figure 3 The image shows the third PCA principal component of a certain original data point cloud and the aligned point cloud image, where (a) is the original dataset and its PCA principal components, and (b) is the aligned point cloud.

[0039] Figure 4 This is a schematic diagram of the Transformer module design, where (a) is the displacement and strain fields obtained by DIC measurement, and (b) is the self-attention mechanism based on the shift window.

[0040] Figure 5 The diagram shows the installation method and test setup for FRP specimens, where (a) is an FRP fiber specimen fixed with bolts; Figure 5 (b) is the watertightness test chamber;

[0041] Figure 6 The diagram shows an FRP specimen sealed in a watertightness test chamber, where (a) is a top view of the FRP specimen and (b) is a top view of the watertightness test chamber.

[0042] Figure 7 These are layout diagrams of the LiDAR and DIC measurement equipment, where (a) is the experimental site layout; and (b) is a schematic diagram of the experimental site.

[0043] Figure 8 The comparison chart shows the measurement and prediction results using the DIC method. (a) shows the actual displacement results of SW1 group measured by the DIC method, (b) shows the actual strain results of SW1 group measured by the DIC method, and (c) shows the strain prediction results of SW1 group measured by the DIC method.

[0044] Figure 9 The results are full-field strain predictions based on LiDAR point cloud data, where (a) is the actual displacement results of SW1 group measured by LiDAR method; and (b) is the strain prediction results of SW1 group measured by LiDAR method. Detailed Implementation

[0045] The invention will be further described below with reference to the accompanying drawings:

[0046] Example 1

[0047] This embodiment constructs a neural network model capable of obtaining full-field strain from point cloud data. The model is as follows: Figure 1 As shown, the coordinate information is measured using LiDAR. Inspired by point cloud segmentation (computer vision), the neural network model employs four main functional modules: embedding, transformer, downsampling, and upsampling. Simultaneously, a preprocessing method is proposed to directly calculate the relative displacement with respect to the initial specimen, addressing the problem of point cloud tilt caused by misalignment between the instrument and the ground, thus complicating the calculation.

[0048] The preprocessing flow is as follows: Figure 2 As shown, only the central 90% region of the point cloud is retained to reduce interference from irregular edges, thereby improving the effectiveness of principal component analysis (PCA). Subsequently, the third principal component of the PCA (i.e., the normal vector of the best-fit plane) of the initial point cloud with zero water pressure is aligned with the Z-axis to ensure the point cloud is parallel to the ground. Figure 3 (a) and (b) show the third PCA principal components of a given original point cloud and the aligned point cloud. Finally, each point cloud recorded under different pressures is rotated by the same rotation angle as the initial point cloud, thus enabling easy calculation of the relative displacement with respect to the initial plate.

[0049] At the beginning of the model, point cloud data containing coordinates and hydrostatic pressure is first passed through an embedding module, and then the point cloud data is processed using the Kernel Point Convolution (KPConv) method [2]. Compared with the regularly arranged pixels in a photograph, the inherent sparsity and randomness of point clouds make the computation efficiency of ordinary convolution methods low. To this end, KPConv sets up a convolution kernel that is differentially aligned with the points, and its kernel function g is defined as:

[0050]

[0051] The convolution weights (i.e., the coefficients before the dimensionality transformation matrix W) are inversely proportional to the Euclidean distance between the kernel point and the local coordinates of points in its neighborhood. This improved convolution strategy maintains high computational efficiency while fully leveraging the features of neighboring points, providing more reliable input for subsequent modules.

[0052] Figure 4 (a) shows the displacement and strain of fiber-reinforced polymer (FRP) sheets measured by DIC under specific water pressure. These are typical deformation modes in FRP hydrostatic failure tests, characterized by uniform displacement variation spreading from the center to the edge and longitudinally arranged stripes in the strain field. The goal of this model is to convert the displacement field into a strain field, which requires the model to extract implicit relationships between local and global perspectives from the coordinate information of points in the point cloud. This function is achieved through a self-attention mechanism based on a shift window in the Transformer module, such as... Figure 4 As shown in (b).

[0053] The longitudinal striped strain distribution observed in the measurement results originates from cracks generated along the fiber direction (Y-axis), indicating that for the small rectangular regions at the longitudinal edge of the plate, the boundary constraint in the Y-direction is weaker than the end constraint at the transverse (X-axis) edge. Therefore, as... Figure 4 As shown in (b), a self-attention map is constructed within a rectangular window of size a×l, which conforms to physical properties and reduces the dimensionality of the weight matrix. The window-based self-attention method is described below:

[0054]

[0055] Here, the point set P is the set of points within a rectangular window, with n points in total. The self-attention graph Attention(q,P) is constrained by an a×l rectangular window. The essence of the self-attention graph is the scalar attention weight softmax(q·k). i ) and value vector v i The summation of products, where softmax(q·k) i ) represents the query vector q and the key vector k i The dot product after softmax normalization. The self-attention weight matrix based on the window has dimensions of... The global attention dimension is O(N). 2 Here, n represents the average number of points within a single window, and N represents the total number of points in the point cloud. Furthermore, to ensure a smooth transition of features at the edge points of adjacent windows, the Transformer module uses translation windows of the same size to process the translation point set P. s , where m represents the number of points within the translation window. Shifted-Attention(q,P) s Attention map within the translation window and cross attention weights Composed of two parts, this mechanism enables interaction between windows and maintains the consistency of point cloud segmentation boundaries at the physical level. However, despite using a translation window approach, the model's receptive field remains insufficient, making it difficult to capture long-distance relationships between points. Therefore, a hierarchical keypoint sampling strategy is introduced: such as... Figure 4 (b) shows that Transformer(l) expands the window size to 2a×l after Transformer(l-1), increasing the operational range of the point cloud. Furthermore, thanks to the downsampling module, despite doubling the window size, the dimension of the weight matrix of the single-window self-attention map is reduced to: The proposed DIC-LiDAR strain prediction model contains four Transformer modules. The window lengths of these modules in the horizontal (X-axis) direction are [a, 2a, 4a, 8a].

[0056] The downsampling modules connected after the Transformer modules are designed to compress the point cloud—this not only reduces computational cost but also expands the receptive field. The three downsampling modules in the network also function as encoders. Each downsampling module incorporates max-pooling, defined as:

[0057]

[0058] Among them, the point features f after being processed by the downsampling module l+1 (j) is the input feature f of the previous layer. l (j) is the maximum value within the neighborhood N(j) centered at point j, and the point set P l+1 It is a point set P l The sampled subset has a size of P. l 1 / 4 of it.

[0059] In the final step of this framework—upsampling (as a decoder)—the outputs of all previous Transformer modules are aggregated and then connected to a fully connected (dense) layer to output numerical values ​​predicting the strain field. The order of point features processed in the upsampling module is the reverse of the order of the three downsampling modules; that is, the points output by the last Transformer module are interpolated to the points in the penultimate stage, and so on, up to the initial stage. Finally, all the interpolated point clouds and the original point clouds are combined and then input into the fully connected layer.

[0060] Example 2

[0061] The purpose of hydrostatic pressure failure testing is to simulate the failure behavior of FRP fibers under gradually increasing hydrostatic pressure. For example... Figure 5 As shown: A specimen measuring 609.6mm × 609.6mm × 9.9mm was securely sealed in a watertightness test chamber filled with water, its edges bolted in place. A top view of the specimen and the watertightness test chamber is shown below. Figure 6 As shown. The exposed portion of the specimen is approximately a rounded square, measuring about 304.8 mm × 304.8 mm. The water pressure inside the chamber is gradually increased using a water pump, which can be read using a pressure gauge, until water sprays or droplets appear on the surface of the specimen.

[0062] The experiment used both LiDAR and DIC technologies for recording, such as Figure 7 As shown, both devices are placed directly above the specimen. Before pressurization, the specimen surface is sprayed with a white primer and red random speckle paint with approximately 50% coverage, as shown. Figure 5 As shown, this ensures the optimal contrast required by the DIC post-processing algorithm.

[0063] (1) DIC strain prediction results:

[0064] This study employs the DIC method to perform non-contact measurements of displacement and strain on the surface of FRP specimens under various environmental conditions, ensuring the universality of the trained model. The results for group SW1 are presented here. The measured displacements of group SW1 are input into the model to predict the strain of group SW1. Figure 8 As shown in (a).

[0065] (2) LiDAR strain prediction results:

[0066] The LiDAR-measured SW1 group of point cloud coordinate data is input into the trained model, and the predicted strain is as follows: Figure 9 As shown in (a).

[0067] pass Figure 8 (b) and Figure 9 (b) As shown in the comparison, the strain field predicted by this model based on LiDAR displacement data exhibits significant consistency with the strain field measured by DIC in both spatial distribution and physical magnitude. The prediction results completely reproduce the longitudinal stripe-like strain concentration bands measured by DIC, accurately reflecting the progressive cracking behavior of FRP sheets along the Y-axis fiber direction under hydrostatic pressure. Similar results can be obtained for FRP specimens under other different environments, fully demonstrating the universality of this method.

Claims

1. A method for calculating structural strain distribution using lidar and deep learning neural networks, characterized in that, Includes the following steps: Step 1: Obtain the initial 3D point cloud of the structure and the 3D point cloud under different states. The 3D point cloud is obtained by measuring with a lidar. Step 2: Preprocess the initial 3D point cloud and the 3D point clouds under different states to obtain the relative displacement field relative to the initial state; Step 3: Construct a deep learning neural network model, including an embedding module, a Transformer module, a downsampling module, an upsampling module, and a fully connected layer. The embedding module processes point cloud data using convolutional processing of the core points of the point cloud. The Transformer module extracts inter-point feature relationships through a self-attention mechanism based on a shift window. The downsampling module compresses the point cloud and expands the receptive field through max pooling. The upsampling module aggregates features and restores the point cloud scale. The fully connected layer outputs the strain distribution. The Transformer module employs a self-attention mechanism based on a shift window, including: firstly, at a size of... A self-attention graph is constructed between points within a rectangular window. The self-attention graph is calculated as follows: , Where P is the set of points within the rectangular window, n is the number of points within the window, and q is the query vector. For key vectors, It is a value vector; Then, a translation window is used to process the translation point set. To achieve feature interaction between windows, the self-attention calculation method for the panning window is as follows: ， Where m is the number of points within the translation window, and c is the number of points associated with cross-attention; Finally, a hierarchical key point sampling strategy is adopted to gradually expand the window size of each Transformer module along the processing hierarchy, thereby expanding the point cloud operation range. Step 4: Train the neural network model using displacement and strain field data obtained by digital image correlation technology; Step 5: Input the preprocessed LiDAR point cloud data into the trained model and output the full-field strain distribution of the structure.

2. The method for calculating structural strain distribution using lidar and deep learning neural networks according to claim 1, characterized in that, The preprocessing in step 2 includes the following sub-steps: Sub-step 2.1: Retain the central 90% area of ​​each point cloud and remove irregular edge areas; Sub-step 2.2: Perform principal component analysis on the initial point cloud without load, extract the third principal component: the normal vector of the best-fit plane, and align the third principal component with the Z-axis so that the point cloud is parallel to the ground; Sub-step 2.3: Rotate each point cloud captured at different pressure levels by the same rotation angle as the initial point cloud, and calculate the relative displacement field relative to the initial point cloud.

3. The method for calculating structural strain distribution using lidar and deep learning neural networks according to claim 1, characterized in that, The embedding module processes point cloud data using point cloud core point convolution, and the kernel function of the point cloud core point convolution is defined as: ， in, The Euclidean distance between the core point and its neighboring points is the local coordinates. The dimensionality transformation matrix is ​​the matrix used for convolution weights. The coefficient before is inversely proportional to the Euclidean distance between the local coordinates of the core point and the points in the neighborhood.

4. The method for calculating structural strain distribution using lidar and deep learning neural networks according to claim 1, characterized in that, The downsampling module embeds a max pooling operation, defined as follows: ， in, For point The neighborhood point set, The features of point j in the previous layer are the point features after processing by the downsampling module. It is the input feature of the previous layer. In point The central area Maximum value within a point set It is a point set The size of the sampled subset is 1 / 4 of it.

5. The method for calculating structural strain distribution using lidar and deep learning neural networks according to claim 1, characterized in that, The upsampling module maps the output features of the next Transformer module to the scale of the previous layer through interpolation, and aggregates the output features of all Transformer modules, finally inputting the strain distribution of the fully connected layer.

6. The method for calculating structural strain distribution using lidar and deep learning neural networks according to claim 1, characterized in that, Step 4, the training process, includes: measuring the displacement field and corresponding strain field of the structure using digital image correlation technology as training data; randomly dividing the point cloud of the training data into multiple sub-point clouds, each sub-point cloud containing a preset number of points; performing random data augmentation on the sub-point clouds; and training the neural network using random deactivation and batch normalization techniques until the model converges.

7. A deep learning neural network model, characterized in that, A method for calculating structural strain distribution using lidar and deep learning neural networks as described in any one of claims 1-6 includes an embedding module, multiple Transformer modules, a downsampling module, an upsampling module, and a fully connected layer connected in sequence; the embedding module employs point cloud core point convolution, the Transformer module employs a self-attention mechanism based on a shift window and a hierarchical key point sampling strategy, the downsampling module embeds a max pooling operation, and the upsampling module employs an interpolation operation.

Images