Bridge pier perpendicularity intelligent detection method and system based on deep learning

Abstract

The invention discloses a pier verticality intelligent detection method and system based on deep learning, and relates to the technical field of image processing, and the method comprises the steps: obtaining initial point cloud data obtained after a to-be-detected object is scanned, and the to-be-detected object comprises a pier; identifying target point cloud data belonging to the to-be-detected object in the initial point cloud data through a pre-trained point cloud segmentation model; identifying and removing edge error points in the target point cloud data to obtain processed point cloud data; performing model fitting on the to-be-detected object according to the processed point cloud data, and determining a central axis of the to-be-detected object according to a model fitting result; generating a perpendicularity index of the to-be-detected object according to the central axis; wherein the point cloud segmentation model comprises an NAM, a PTFM and a CM; the NAM is used for representing multi-scale context and local details, the PTFM is used for capturing remote dependency information based on the local details, and the CM is used for connecting the NAM and the PTFM. And the detection precision and efficiency are improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to a method and system for intelligent detection of bridge pier verticality based on deep learning. Background Technology

[0002] In manufacturing and construction scenarios, it is necessary to ensure that the manufactured objects are vertical. For example, in construction scenarios such as walls and bridge piers, it is necessary to ensure that the walls and bridge piers are vertical. To this end, methods such as leveling instruments, total stations, or manual plumb bobs can be used to detect verticality.

[0003] However, methods such as levels, total stations, or manual plumb bobs can only acquire data from a limited number of points, failing to cover the entire object. Furthermore, the measurement results rely on manual operation, resulting in significant subjective errors and repeatability discrepancies. As the height and structural complexity of manufactured objects increase, manual measurement is not only time-consuming and labor-intensive but also struggles to detect local deviations in a timely manner, making it difficult to meet the requirements of high-precision and high-efficiency quality inspection.

[0004] In conclusion, improving the accuracy and efficiency of verticality detection is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a deep learning-based intelligent detection method for bridge pier verticality, which can, to some extent, solve the technical problem of how to improve the accuracy and efficiency of verticality detection. This application also provides a deep learning-based intelligent detection system for bridge pier verticality.

[0006] To achieve the above objectives, this application provides the following technical solution:

[0007] A deep learning-based intelligent method for detecting the verticality of bridge piers includes:

[0008] Acquire initial point cloud data after scanning the object to be detected, the object to be detected including bridge piers;

[0009] The target point cloud data belonging to the object to be detected in the initial point cloud data is identified by a pre-trained point cloud segmentation model.

[0010] Edge error points in the target point cloud data are identified and removed to obtain processed point cloud data.

[0011] The object to be detected is fitted with a model based on the processed point cloud data, and the central axis of the object to be detected is determined based on the model fitting result.

[0012] The perpendicularity index of the object to be tested is generated based on the central axis.

[0013] The point cloud segmentation model includes a neighborhood aggregation module, a key point conversion module, and a connection module. The neighborhood aggregation module is used to represent multi-scale context and local details, the key point conversion module is used to capture remote dependency information based on local details, and the connection module is used to connect the neighborhood aggregation module and the key point conversion module.

[0014] Preferably, the initial point cloud data obtained after scanning the object to be detected includes:

[0015] Obtain the raw point cloud data of the object to be inspected after scanning it with a 3D laser scanner;

[0016] With the principal axis as the vertical direction, the original point cloud data is normalized to obtain normalized point cloud data;

[0017] The normalized point cloud data is denoised to obtain denoised point cloud data.

[0018] The point cloud resolution of the denoised point cloud data is uniformly adjusted to the target spacing to obtain the initial point cloud data.

[0019] Preferably, edge error points in the target point cloud data are identified and removed to obtain processed point cloud data, including:

[0020] The target point cloud data is unfolded into a two-dimensional image along the main axis to obtain two-dimensional projected point cloud data;

[0021] The two-dimensional projected point cloud data is used to construct a two-dimensional raster image;

[0022] Identify the boundary regions of the two-dimensional raster image;

[0023] In the two-dimensional projected point cloud data, points whose distance value from the boundary region is less than a set distance value are selected as edge error points;

[0024] The edge error points are removed from the two-dimensional projected point cloud data to obtain processed point cloud data.

[0025] Preferably, the process of fitting a model to the object to be detected based on the processed point cloud data, and determining the central axis of the object to be detected based on the model fitting result, includes:

[0026] The cross-section of the object to be detected is identified;

[0027] In response to the fact that the cross-section of the object to be detected is cylindrical, the cylindrical fitting algorithm is used to calculate the fitting cylindrical model parameters of the processed point cloud data;

[0028] The processed point cloud data is sampled to obtain sampling points;

[0029] Based on the parameters of the fitted cylinder model, the fitting distance from the sampling point to the surface of the fitted cylinder is generated;

[0030] Based on the least squares fit and the fit distance, a target function for the column model parameters is constructed.

[0031] Generate the current column model parameters based on the objective function of the column model parameters;

[0032] Determine whether to end the iteration;

[0033] If the iteration continues, return to the step of sampling the processed point cloud data to obtain sample points;

[0034] If the iteration ends, the center axis is obtained based on the current column model parameters.

[0035] Preferably, the point cloud segmentation model architecture consists of an encoder and a decoder; the encoder consists of a downsampling layer and a network base block, and the network base block consists of the neighborhood aggregation module, the key point conversion module and the connection module; the decoder consists of an upsampling layer and a multilayer perceptron.

[0036] Preferably, the data processing procedure of the neighborhood aggregation module includes:

[0037] ;

[0038] ;

[0039] ;

[0040] in, This represents the output of the neighborhood aggregation module; Indicates the NA header number; Indicates the total number of NA headers; Indicates adjacent aggregation characteristics; Indicates adjacent features; This indicates point-by-point generation; This represents a learnable reduction function; This represents the spatial coordinates of the local neighborhood centered at point i. This represents the semantic feature set corresponding to the local neighborhood centered at point i; Indicates a join operation; 'k' represents the delimiter; This represents a multilayer perceptron; This represents the normalized exponential function.

[0041] Preferably, the data processing procedure of the connection module includes:

[0042] ;

[0043] in, This indicates the connection result of the connection module; Indicates learnable parameters; Represents the sigmoid function; This represents the output result of the neighborhood aggregation module; This indicates the output result of the key point conversion module.

[0044] A deep learning-based intelligent detection system for bridge pier verticality includes:

[0045] The first acquisition module is used to acquire the initial point cloud data obtained after scanning the object to be detected, the object to be detected including bridge piers;

[0046] The first identification module is used to identify target point cloud data belonging to the object to be detected in the initial point cloud data through a pre-trained point cloud segmentation model;

[0047] The first processing module is used to identify and remove edge error points in the target point cloud data to obtain processed point cloud data.

[0048] The first fitting module is used to perform model fitting on the object to be detected based on the processed point cloud data, and to determine the central axis of the object to be detected based on the model fitting result.

[0049] The first generation module is used to generate the verticality index of the object to be detected based on the central axis.

[0050] The point cloud segmentation model includes a neighborhood aggregation module, a key point conversion module, and a connection module. The neighborhood aggregation module is used to represent multi-scale context and local details, the key point conversion module is used to capture remote dependency information based on local details, and the connection module is used to connect the neighborhood aggregation module and the key point conversion module.

[0051] Preferably, the first acquisition module includes:

[0052] The first acquisition unit is used to acquire the original point cloud data obtained by the 3D laser scanner after scanning the object to be inspected;

[0053] The first processing unit is used to perform coordinate normalization on the original point cloud data with the principal axis as the vertical direction to obtain normalized point cloud data.

[0054] The first denoising unit is used to denoise the normalized point cloud data to obtain denoised point cloud data.

[0055] The first adjustment unit is used to uniformly adjust the point cloud resolution of the denoised point cloud data to the target spacing to obtain the initial point cloud data.

[0056] Preferably, the first processing module includes:

[0057] The first projection unit is used to unfold the target point cloud data into a two-dimensional image along the main axis direction to obtain two-dimensional projected point cloud data.

[0058] The first construction unit is used to construct a two-dimensional raster image from the two-dimensional projected point cloud data;

[0059] The first recognition unit is used to identify the boundary region of the two-dimensional raster image;

[0060] The first filtering unit is used to filter points in the two-dimensional projected point cloud data whose distance value from the boundary region is less than a set distance value as edge error points.

[0061] The first elimination unit is used to eliminate the edge error points in the two-dimensional projected point cloud data to obtain processed point cloud data.

[0062] This application provides a deep learning-based intelligent detection method for bridge pier verticality. The method involves acquiring initial point cloud data after scanning an object to be detected, including bridge piers; identifying target point cloud data belonging to the object to be detected within the initial point cloud data using a pre-trained point cloud segmentation model; identifying and removing edge error points in the target point cloud data to obtain processed point cloud data; fitting a model to the object to be detected based on the processed point cloud data, and determining the central axis of the object based on the model fitting results; and generating a verticality index for the object based on the central axis. The point cloud segmentation model includes a neighborhood aggregation module, a keypoint transformation module, and a connection module. The neighborhood aggregation module represents multi-scale context and local details, the keypoint transformation module captures long-range dependency information based on local details, and the connection module connects the neighborhood aggregation module and the keypoint transformation module. In this application, because the neighborhood aggregation module is used to represent multi-scale context and local details, the key point conversion module is used to capture long-range dependency information based on local details, and the connection module is used to connect the neighborhood aggregation module and the key point conversion module, the point cloud segmentation model can achieve high-precision automatic segmentation of objects to be detected at different scales under complex background noise, providing a reliable target point cloud for verticality detection and enhancing the applicability of verticality detection. Furthermore, the identification and removal of edge error points in the target point cloud data can eliminate the influence of error points, retaining only stable point cloud data of the core area for verticality detection, thus improving detection accuracy. The entire process is automatic, requiring no manual intervention, resulting in high detection efficiency. The deep learning-based intelligent detection system for bridge pier verticality provided in this application also solves the corresponding technical problems. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0064] Figure 1 A flowchart illustrating a deep learning-based intelligent detection method for bridge pier verticality, provided as an embodiment of this application;

[0065] Figure 2 This is a schematic diagram of the point cloud segmentation model;

[0066] Figure 3 This is a schematic diagram of the operation of the transformer at a key point;

[0067] Figure 4 Point cloud diagram of circular bridge piers;

[0068] Figure 5 Point cloud map of rectangular bridge piers;

[0069] Figure 6 A schematic diagram of a deep learning-based intelligent detection system for bridge pier verticality provided in an embodiment of this application;

[0070] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0071] Figure 8 This is another structural schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0073] Please see Figure 1 , Figure 1 A flowchart of a deep learning-based intelligent detection method for bridge pier verticality provided in an embodiment of this application.

[0074] This application provides a method for intelligent detection of bridge pier verticality based on deep learning, which may include the following steps:

[0075] Step S101: Obtain the initial point cloud data after scanning the object to be detected.

[0076] In practical applications, verticality detection of the object to be detected can be performed through image processing. This requires obtaining the initial point cloud data after scanning the object. The type of the object to be detected can be determined according to the application scenario. For example, the object to be detected can be a wall, bridge pier, table, chair, etc.

[0077] In an exemplary embodiment, initial point cloud data can be acquired using a 3D laser scanner. Specifically, during the process of acquiring the initial point cloud data obtained after scanning the object to be inspected, the original point cloud data obtained after scanning the object using the 3D laser scanner can be acquired. The original point cloud data is then normalized using the principal axis as the vertical direction, for example, the Z-axis, to obtain normalized point cloud data, which facilitates subsequent verticality calculations. The normalized point cloud data is then denoised, for example, using a Statistical Outlier Removal (SOR) algorithm to filter out isolated points and measurement errors, resulting in denoised point cloud data. The point cloud resolution of the denoised point cloud data is then uniformly adjusted to the target spacing, for example, by using a voxel grid filter to unify the point cloud resolution to a 1cm spacing, thus obtaining initial point cloud data. This reduces computational load while preserving the geometric features of the object to be inspected.

[0078] In specific application scenarios, in the process of acquiring the original point cloud data obtained after scanning the object to be inspected by a 3D laser scanner, in order to collect as much point cloud data as possible, a high-precision 3D laser scanner can be used to scan the object to be inspected from all directions to obtain its complete surface point cloud data. During the scanning process, multiple stations can be set up to ensure coverage without blind spots. Each point has millimeter-level positioning accuracy, so that the original point cloud data can fully reflect the shape and tilt of the object to be inspected, providing basic data for subsequent verticality detection.

[0079] Step S102: Identify target point cloud data belonging to the object to be detected in the initial point cloud data through a pre-trained point cloud segmentation model; the point cloud segmentation model includes a neighborhood aggregation module, a key point transformation module, and a connection module; the neighborhood aggregation module is used to represent multi-scale context and local details, the key point transformation module is used to capture remote dependency information based on local details, and the connection module is used to connect the neighborhood aggregation module and the key point transformation module.

[0080] In practical applications, the initial point cloud data obtained after scanning the object to be detected may contain background noise and other point clouds that affect verticality detection. To avoid these point clouds affecting subsequent verticality detection, a pre-trained point cloud segmentation model can be used to identify target point cloud data belonging to the object to be detected within the initial point cloud data. In other words, the point cloud segmentation model extracts point cloud data that can be used for verticality detection of the object. Furthermore, the point cloud segmentation model enables automatic identification of target point cloud data, avoiding errors caused by manually selecting the region of the object to be detected, and improving the reliability and efficiency of the processing.

[0081] In an exemplary embodiment, the point cloud segmentation model of this application may include a neighborhood aggregations module (NAM) for local perception, a transformer module (PTFM) for global modeling, and a connector module (CM) for connecting NAM and PTFM. NAM efficiently learns complex local features at different scales through improved NA (Normalized Attention) operations and a multi-head mechanism, representing multi-scale context and local details. PTFM efficiently performs global attention using a set of learnable keypoints and captures long-range dependency information based on local details. CM, acting as a fusion of high-frequency and low-frequency signals, connects NAM and PTFM, thereby combining the advantages of both blocks by adaptively assigning mixed weights to the local and global environments.

[0082] In specific application scenarios, the architecture of a point cloud segmentation model can consist of an encoder and a decoder, such as... Figure 2 As shown, the encoder consists of a downsampling layer and a network base block, which comprises NAM, PTFM, and CM; the decoder consists of an upsampling layer and a multilayer perceptron. Features are transmitted through skip connections between the encoder and decoder. Finally, three fully connected layers with softmax are used to predict the classification score for each point, where the segmentation result is determined by the class with the highest score.

[0083] like Figure 2 As shown, NAM consists of a multihead neighbor aggregation (MNA) and a multilayer perceptron (MLP). Details of the MNA are as follows... Figure 2As shown. In MNA, this application adopts the set abstraction layer of the PointNet++ network as the basic model for the neighborhood aggregation (NA) operation because it is simple and effective. As described in Equation (1), the set abstraction layer of PointNet++ consists of two steps: 1) local feature extraction and 2) local feature aggregation.

[0084] (1)

[0085] In the formula F N For adjacent features; F NA Adjacent aggregation features; and represents the spatial coordinates and semantic feature set corresponding to the local neighborhood centered at point i, respectively; ⊕ represents the connection operation.

[0086] This application, while retaining its main structure, incorporates simple and intuitive enhancements to address the following issues: 1) In the local feature extraction step, positional embeddings represent the relative position of the centroid within its neighborhood. This asymmetry, combined with the absolute features of neighboring points, can lead to semantic gaps. 2) In the local feature aggregation step, max pooling may result in the loss of local details.

[0087] Therefore, this application first introduces a symmetric feature extractor, where both the positional embedding and semantic features are represented based on the relative information of the centroids within the local neighborhood. This symmetric extractor can bridge the semantic gap between the positional embedding and semantic features. The improvement of this application can be described as follows:

[0088] (2)

[0089] Subsequently, this application enhances local features by amplifying more important features. Specifically, this application captures the importance of features using a softmax function. Then, this application uses a learnable reduction function φ to aggregate local neighborhood information to the centroid as a pooling operation. The reduction function φ is a linear mapping that reduces the neighbor dimension k to 1. This method not only allows the neural network to learn the most salient local features but also covers the entire local neighborhood, thereby mitigating the loss of local details caused by using only max pooling. The improvement of this application can be stated as follows:

[0090] (3)

[0091] In the formula, ⊗ represents point-by-point generation, and φ represents a learnable reduction function.

[0092] Finally, based on the designed NA operation, a multi-head neighborhood aggregation is constructed. Compared to typical NA operations that focus on features at a finite scale, the method in this application includes more NA heads, and expects each head to focus on local features at different scales. In the implementation, each head F... Nai The neighbor search range is the previous head F Nai−1 Twice as of now. The improvements in this application can be summarized as follows:

[0093] (4)

[0094] Model ablation experiments revealed that embedding the aforementioned MNA block into the basic point cloud segmentation network PointNet++ can improve the model segmentation accuracy (mIoU) from 67.6% to 83.1%, an improvement of 15.5%.

[0095] In other words, the NAM data processing procedure includes:

[0096] ;

[0097] ;

[0098] ;

[0099] in, This represents the output of the NAM; Indicates the NA header number; Indicates the total number of NA headers; Indicates adjacent aggregation characteristics; Indicates adjacent features; This indicates point-by-point generation; This represents a learnable reduction function; This represents the spatial coordinates of the local neighborhood centered at point i. This represents the semantic feature set corresponding to the local neighborhood centered at point i; Indicates a join operation; 'k' represents the delimiter; This represents a multilayer perceptron, which consists of BN (batch normalization layer), linear (linear mapping), GELU (activation function), and linear (linear mapping). This represents the normalized exponential function.

[0100] In specific application scenarios, such as Figure 2 As shown, the keypoint transformation module (PTFM) consists of keypoint transformer (KPT) operations and multilayer perceptrons (MLP), see... Figure 2Directly below. This module is used to capture remote dependency information based on local details. See the diagram below for a schematic of how this module works. Figure 3 , Figure 3 The central dot represents the centroid of the neighborhood, and the other dots represent keypoints. The module's operation consists of two phases: the first phase involves keypoint search and neighborhood aggregation, and the second phase involves aggregating global attention on the keypoints. The specific workflow is as follows: First, the point cloud shape of the entire input scene is depicted by only a few keypoints, so these keypoints must be able to fully represent local details. In solving the challenge of representing local details, the neighborhood aggregation (NA) operation can effectively gather information from the local neighborhood of the centroid. Therefore, before searching for keypoints, we perform an NA operation to ensure that the keypoints can fully represent local details. Thus, in summary, the role of PTFM can be described as how to represent the entire shape using only a few keypoints. Model ablation experiments revealed that embedding the aforementioned PTFM module into the basic point cloud segmentation network PointNet++ can improve the model segmentation accuracy (mIoU) from 67.6% to 79.3%, an improvement of 11.7%. Embedding both the aforementioned MNA and PTFM modules into the basic point cloud segmentation network PointNet++ can improve the model segmentation accuracy (mIoU) from 67.6% to 85.7%, an improvement of 18.1%.

[0101] In specific application scenarios, such as Figure 2 As shown, the connector module (CM) consists of a connection strategy and an MLP. Initial features are input into NAM to obtain local features, and then input into PTFM to obtain global features. By combining the local and global features within the connector module, a comprehensive representation is obtained, which can be described as: In other words, the data processing procedure of CM includes: ;

[0102] in, This indicates the connection result of CM; Indicates learnable parameters; Represents the sigmoid function; This represents the output of NAM; This indicates the output of PTFM.

[0103] Model ablation experiments revealed that embedding the MNA, PTFM, and CM modules simultaneously into the basic point cloud segmentation network PointNet++ can improve the model segmentation accuracy (mIoU) from 67.6% to 87.3%, an improvement of 19.7%.

[0104] In this way, the point cloud segmentation model automatically focuses on the geometric patterns and spatial structures of the point cloud of the object to be detected during feature learning, effectively filtering out redundant information such as the ground, supports, and obstacles, and retaining only the region of the object to be detected. Compared with manual cropping based on Return on Investment (ROI) or extraction based on rule conditions, this method can adapt to objects of different cross-sections and heights; it has good robustness to interference from construction sites such as shadows, occlusions, and cluttered backgrounds; it achieves intelligent extraction without manual interaction or setting heuristic rules; and it supports batch recognition of a large number of point clouds of objects to be detected.

[0105] It should be noted that the training process of the point cloud segmentation model can be flexibly determined according to actual needs. For example, training point cloud data of the object to be detected can be obtained, and preprocessing such as coordinate transformation, denoising, and downsampling can be performed on the training point cloud data to obtain the training dataset. The preprocessed training dataset can then be manually segmented for semantic annotation. Assuming the object to be detected is a bridge pier, the semantic annotation results can include three categories: bridge pier, ground, and other background noise. The segmentation data of different categories can be represented by different labels, such as... Figure 4 and Figure 5 As shown, Figure 4 and Figure 5 In the diagram, k1 represents the ground or other background, k2 represents the bridge pier, and k3 represents the cap beam. The training and validation sets are partitioned in an 8:2 ratio, and data augmentation is performed using rotation, scaling, noise addition, and reordering. During network training, a single device with 24GB of memory was used for data training. The following parameters were set during training: the network structure consisted of a 4-layer X-Conv convolutional module + upsampling module, the optimizer was Adam, the initial learning rate was 0.001, the decay rate was 0.8 (every 5000 iterations), the batch size was 8, the number of label categories was 3 (the object to be detected, the ground, and other backgrounds), the weights were initialized uniformly, and the loss function was the cross-entropy loss function (with class weights). A fixed batch size of 24,000 points was input into the network, with 100 iterations per batch. CrossEntropy loss with label smoothing, the AdamW optimizer, an initial learning rate of 0.01, and a weight decay rate of 10⁻⁴ (with cosine decay) were used to optimize all models. The training rounds were set to 100 rounds, and the overall accuracy on the validation set was over 92%, with the IoU (Intersection over Union) metric reaching over 87% in the pier category.

[0106] Step S103: Identify and remove edge error points in the target point cloud data to obtain processed point cloud data.

[0107] In practical applications, after obtaining the target point cloud data using a point cloud segmentation model, although all the target point cloud data belong to the objects to be detected, there may be errors. In order to further remove error points, edge error points in the target point cloud data can be identified and removed to obtain processed point cloud data, so that subsequent applications can process the point cloud data for verticality detection.

[0108] In an exemplary embodiment, edge error points in the target point cloud data are identified and removed. During the process of processing the point cloud data, image processing methods can be used to detect the boundary contours of the extracted target point cloud data, and points too close to the boundary can be removed. That is, the target point cloud data can be unfolded into a two-dimensional image along the principal axis. For example, a rectangular cross-section can be projected onto the principal plane, and a circular cross-section can be unfolded into a circumferentially flattened image, thus obtaining two-dimensional projected point cloud data. Assuming the target point cloud data is... If the principal axis is the Z-axis, then the two-dimensional projected point cloud data is: The two-dimensional projected point cloud data is used to construct a two-dimensional raster image I(x,y). Since the area near the edges in the point cloud image often exhibits significant deviations or noise due to laser incident angles or object edges, this embodiment excludes edge point data within a certain range before fitting, retaining only the points of the main body of the object to be detected for model fitting. This avoids interference from irregular boundary points on the fitting results, making the central axis of the fitting more accurate and reliable, i.e., identifying the boundary region B of the two-dimensional raster image. For example, using the Canny edge detection method to identify boundary regions, in two-dimensional projected point cloud data, filtering regions whose distance from the boundary is less than a set distance value. The points are identified as edge error points. These edge error points are removed from the 2D projected point cloud data to obtain the processed point cloud data. Set distance value A value between 5 and 10 cm can be used.

[0109] Step S104: Perform model fitting on the object to be detected based on the processed point cloud data, and determine the central axis of the object to be detected based on the model fitting results.

[0110] Step S105: Generate the perpendicularity index of the object to be inspected based on the central axis.

[0111] In practical applications, after obtaining accurate processed point cloud data, a model can be fitted to the object to be detected based on the processed point cloud data, and the central axis of the object to be detected can be determined based on the model fitting results; finally, the verticality index of the object to be detected is generated based on the central axis.

[0112] In an exemplary embodiment, during the process of fitting a model to the object to be detected based on the processed point cloud data and determining the central axis of the object to be detected based on the model fitting result, the cross-section of the object to be detected can be identified. In response to the cross-section of the object to be detected being cylindrical, a cylindrical fitting algorithm is used to calculate the fitting cylindrical model parameters of the processed point cloud data. For example, the RANSAC (Random Sample Consensus) cylindrical fitting algorithm can be used for calculation. Taking cylindrical fitting as an example, the minimum number of samples is 3, and the fitting cylindrical model parameters are (r, c, v), where r is the radius of the cylinder; c is any point on the axis; and v is the unit vector in the direction of the axis. The processed point cloud data is sampled to obtain sampling points; sampling points are generated based on the fitting cylindrical model parameters. Fitting distance to the surface of the fitted cylinder , Two vertical lines indicate the absolute value, and one vertical line represents a dividing mark. (λ refers to the maximum allowable error for a point to be close to the cylinder), then it is considered that... These are the interior points of the model. For laser point cloud data with millimeter-level precision, λ = 5~10mm. Based on least squares fitting and the fitting distance, the objective function for the cylindrical model parameters is constructed. The algorithm generates the current column model parameters based on the objective function of the column model parameters; it then determines whether to end the iteration; if it continues, it returns to the step of sampling the processed point cloud data to obtain sampling points; if it ends the iteration, it obtains the central axis based on the current column model parameters. For example, in the RANSAC algorithm, the number of iterations is set to 1000, and the distance tolerance is 1cm, finally obtaining the spatial vector of the fitted central axis. In this way, the robustness of the RANSAC model fitting is enhanced by the multi-point sampling and multiple fitting strategy, reducing the impact of single random sampling errors on the final axis determination. The combination of boundary avoidance and multi-point fitting significantly improves the accuracy and stability of vertical axis extraction. In addition, in response to the rectangular cross-section of the object to be detected, a vertical principal plane can be fitted based on the processed point cloud data, and the intersection of the vertical principal planes can be used as the central axis.

[0113] In an exemplary embodiment, the verticality index can be the vertical deviation angle or the top offset of the object to be detected. Assuming the central axis direction vector is v, and the unit vector z=(0,0,1) of the Z-axis direction, the formula for calculating the angle between them is: ; with the vertical deviation angle (90°-θ) or top offset As a verticality indicator, H represents the height of the object being inspected.

[0114] It should be noted that after obtaining the verticality index, it is also possible to visualize and report on the verticality index. For example, a 3D color deviation map can be generated, which means that the local offset of each point is represented by color, with blue representing verticality and red representing large deviation. For example, a PDF inspection report can be output, including fields such as the object number to be inspected, measurement time, deviation angle, offset, and whether it is qualified. Furthermore, it can be uploaded to the construction quality acceptance system and displayed in conjunction with the BIM model.

[0115] This application provides a deep learning-based intelligent detection method for bridge pier verticality. The method involves acquiring initial point cloud data after scanning an object to be detected, including bridge piers; identifying target point cloud data belonging to the object to be detected within the initial point cloud data using a pre-trained point cloud segmentation model; identifying and removing edge error points in the target point cloud data to obtain processed point cloud data; fitting a model to the object to be detected based on the processed point cloud data, and determining the central axis of the object based on the model fitting results; and generating a verticality index for the object based on the central axis. The point cloud segmentation model includes a neighborhood aggregation module, a keypoint transformation module, and a connection module. The neighborhood aggregation module represents multi-scale context and local details, the keypoint transformation module captures long-range dependency information based on local details, and the connection module connects the neighborhood aggregation module and the keypoint transformation module. In this application, since the neighborhood aggregation module is used to represent multi-scale context and local details, the key point conversion module is used to capture remote dependency information based on local details, and the connection module is used to connect the neighborhood aggregation module and the key point conversion module, the point cloud segmentation model can achieve high-precision automatic segmentation of objects to be detected at different scales under complex background noise, providing a reliable target point cloud for verticality detection and enhancing the applicability of verticality detection. Furthermore, it is necessary to identify and remove edge error points in the target point cloud data, which can eliminate the influence of error points and retain only the stable point cloud data of the core area for verticality detection, thereby improving the detection accuracy. Moreover, the entire process is automatic and requires no manual intervention, resulting in high detection efficiency.

[0116] Please see Figure 6 , Figure 6 This is a schematic diagram of a deep learning-based intelligent detection system for bridge pier verticality, provided as an embodiment of this application.

[0117] This application provides an intelligent bridge pier verticality detection system based on deep learning, which may include:

[0118] The first acquisition module 101 is used to acquire the initial point cloud data obtained after scanning the object to be detected;

[0119] The first identification module 102 is used to identify target point cloud data belonging to the object to be detected in the initial point cloud data through a pre-trained point cloud segmentation model;

[0120] The first processing module 103 is used to identify and remove edge error points in the target point cloud data to obtain processed point cloud data.

[0121] The first fitting module 104 is used to perform model fitting on the object to be detected based on the processed point cloud data, and to determine the central axis of the object to be detected based on the model fitting result.

[0122] The first generation module 105 is used to generate the verticality index of the object to be detected based on the central axis.

[0123] The point cloud segmentation model includes NAM, PTFM, and CM; NAM is used to represent multi-scale context and local details, PTFM is used to capture remote dependency information based on local details, and CM is used to connect NAM and PTFM.

[0124] This application provides an intelligent bridge pier verticality detection system based on deep learning, wherein the first acquisition module may include:

[0125] The first acquisition unit is used to acquire the original point cloud data obtained by the 3D laser scanner after scanning the object to be inspected;

[0126] The first processing unit is used to normalize the coordinates of the original point cloud data with the principal axis as the vertical direction to obtain normalized point cloud data.

[0127] The first denoising unit is used to denoise the normalized point cloud data to obtain denoised point cloud data.

[0128] The first adjustment unit is used to uniformly adjust the point cloud resolution of the denoised point cloud data to the target spacing to obtain the initial point cloud data.

[0129] This application provides an intelligent bridge pier verticality detection system based on deep learning, wherein the first processing module may include:

[0130] The first projection unit is used to unfold the target point cloud data into a two-dimensional image along the main axis direction to obtain two-dimensional projected point cloud data.

[0131] The first building unit is used to construct a two-dimensional raster image from two-dimensional projected point cloud data;

[0132] The first recognition unit is used to identify the boundary region of the two-dimensional raster image;

[0133] The first filtering unit is used to filter points in the two-dimensional projected point cloud data whose distance value from the boundary area is less than a set distance value as edge error points;

[0134] The first elimination unit is used to eliminate edge error points in the two-dimensional projected point cloud data to obtain processed point cloud data.

[0135] This application provides an intelligent bridge pier verticality detection system based on deep learning, wherein the first fitting module may include:

[0136] The first identification unit is used to identify the cross-section of the object to be detected;

[0137] The first fitting unit is used to calculate the fitting cylindrical model parameters of the processed point cloud data in response to the fact that the cross-section of the object to be detected is cylindrical; sample the processed point cloud data to obtain sampling points; generate the fitting distance from the sampling points to the surface of the fitted cylinder based on the fitting cylindrical model parameters; construct the objective function of the cylindrical model parameters based on the least squares fitting and the fitting distance; generate the current cylindrical model parameters based on the objective function of the cylindrical model parameters; determine whether to end the iteration; if the iteration continues, return to the step of sampling the processed point cloud data to obtain sampling points; if the iteration ends, obtain the central axis based on the current cylindrical model parameters.

[0138] This application provides a deep learning-based intelligent detection system for bridge pier verticality. The point cloud segmentation model architecture consists of an encoder and a decoder. The encoder consists of a downsampling layer and a network base block, which consists of NAM, PTFM, and CM. The decoder consists of an upsampling layer and a multilayer perceptron.

[0139] This application provides an embodiment of a deep learning-based intelligent detection system for bridge pier verticality. The NAM data processing procedure includes:

[0140] ;

[0141] ;

[0142] ;

[0143] in, This represents the output of the NAM; Indicates the NA header number; Indicates the total number of NA headers; Indicates adjacent aggregation characteristics; Indicates adjacent features; This indicates point-by-point generation; This represents a learnable reduction function; This represents the spatial coordinates of the local neighborhood centered at point i. This represents the semantic feature set corresponding to the local neighborhood centered at point i; Indicates a join operation; 'k' represents the delimiter; This represents a multilayer perceptron; This represents the normalized exponential function.

[0144] This application provides an embodiment of a deep learning-based intelligent pier verticality detection system. The data processing procedure for the pier verticality includes:

[0145] ;

[0146] in, This indicates the connection result of the CM; Indicates learnable parameters; Represents the sigmoid function; This represents the output result of the NAM; This indicates the output result of the PTFM.

[0147] This application also provides an electronic device and a computer-readable storage medium, both of which have the corresponding effects of the deep learning-based intelligent detection method for bridge pier verticality provided in the embodiments of this application. Please refer to... Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0148] An electronic device provided in this application includes a memory 201 and a processor 202. The memory 201 stores a computer program, and the processor 202 executes the computer program to implement the steps of the intelligent detection method for bridge pier verticality based on deep learning as described in any of the above embodiments.

[0149] Please see Figure 8Another electronic device provided in this application embodiment may further include: an input port 203 connected to the processor 202 for transmitting commands input from the outside to the processor 202; a display unit 204 connected to the processor 202 for displaying the processing results of the processor 202 to the outside; and a communication module 205 connected to the processor 202 for enabling communication between the electronic device and the outside. The display unit 204 may be a display panel, a laser scanner, or the like; the communication method used by the communication module 205 includes, but is not limited to, Mobile High-Definition Link (MHL), Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), wireless connectivity: Wireless Fidelity (WiFi), Bluetooth communication technology, Bluetooth Low Energy communication technology, and communication technology based on IEEE 802.11s.

[0150] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the deep learning-based intelligent detection method for bridge pier verticality as described in any of the above embodiments.

[0151] The computer-readable storage media involved in this application include random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs (compact disc read-only memory), or any other form of storage media known in the art.

[0152] For descriptions of relevant parts of the intelligent bridge pier verticality detection system, electronic device, and computer-readable storage medium based on deep learning provided in this application, please refer to the detailed description of the corresponding parts in the intelligent bridge pier verticality detection method based on deep learning provided in this application, which will not be repeated here. Furthermore, parts of the technical solutions provided in this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0153] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0154] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A deep learning-based intelligent detection method for bridge pier verticality, characterized in that, include: Acquire initial point cloud data after scanning the object to be detected, the object to be detected including bridge piers; The target point cloud data belonging to the object to be detected in the initial point cloud data is identified by a pre-trained point cloud segmentation model. Edge error points in the target point cloud data are identified and removed to obtain processed point cloud data. The object to be detected is fitted with a model based on the processed point cloud data, and the central axis of the object to be detected is determined based on the model fitting result. The perpendicularity index of the object to be tested is generated based on the central axis. The point cloud segmentation model includes a neighborhood aggregation module, a key point conversion module, and a connection module. The neighborhood aggregation module is used to represent multi-scale context and local details, the key point conversion module is used to capture remote dependency information based on local details, and the connection module is used to connect the neighborhood aggregation module and the key point conversion module.

2. The method according to claim 1, characterized in that, Acquire the initial point cloud data obtained after scanning the object to be detected, including: Obtain the raw point cloud data of the object to be inspected after scanning it with a 3D laser scanner; With the principal axis as the vertical direction, the original point cloud data is normalized to obtain normalized point cloud data; The normalized point cloud data is denoised to obtain denoised point cloud data. The point cloud resolution of the denoised point cloud data is uniformly adjusted to the target spacing to obtain the initial point cloud data.

3. The method according to claim 1, characterized in that, Edge error points in the target point cloud data are identified and removed to obtain processed point cloud data, including: The target point cloud data is unfolded into a two-dimensional image along the main axis to obtain two-dimensional projected point cloud data; The two-dimensional projected point cloud data is used to construct a two-dimensional raster image; Identify the boundary regions of the two-dimensional raster image; In the two-dimensional projected point cloud data, points whose distance value from the boundary region is less than a set distance value are selected as edge error points; The edge error points are removed from the two-dimensional projected point cloud data to obtain processed point cloud data.

4. The method according to claim 1, characterized in that, The process involves fitting a model to the object to be detected based on the processed point cloud data, and determining the central axis of the object to be detected based on the model fitting results, including: The cross-section of the object to be detected is identified; In response to the fact that the cross-section of the object to be detected is cylindrical, the cylindrical fitting algorithm is used to calculate the fitting cylindrical model parameters of the processed point cloud data; The processed point cloud data is sampled to obtain sampling points; Based on the parameters of the fitted cylinder model, the fitting distance from the sampling point to the surface of the fitted cylinder is generated; Based on the least squares fit and the fit distance, a target function for the column model parameters is constructed. Generate the current column model parameters based on the objective function of the column model parameters; Determine whether to end the iteration; If the iteration continues, return to the step of sampling the processed point cloud data to obtain sample points; If the iteration ends, the center axis is obtained based on the current column model parameters.

5. The method according to any one of claims 1 to 4, characterized in that, The point cloud segmentation model architecture consists of an encoder and a decoder; the encoder consists of a downsampling layer and a network base block, and the network base block consists of the neighborhood aggregation module, the key point conversion module and the connection module; the decoder consists of an upsampling layer and a multilayer perceptron.

6. The method according to claim 5, characterized in that, The data processing procedure of the neighborhood aggregation module includes: ； ； ； in, This represents the output of the neighborhood aggregation module; Indicates the NA header number; Indicates the total number of NA headers; Indicates adjacent aggregation characteristics; Indicates adjacent features; This indicates point-by-point generation; This represents a learnable reduction function; This represents the spatial coordinates of the local neighborhood centered at point i. This represents the semantic feature set corresponding to the local neighborhood centered at point i; Indicates a join operation; 'k' represents the delimiter; This represents a multilayer perceptron; This represents the normalized exponential function.

7. The method according to claim 5, characterized in that, The data processing procedure of the connection module includes: ； in, This indicates the connection result of the connection module; Indicates learnable parameters; Represents the sigmoid function; This represents the output result of the neighborhood aggregation module; This indicates the output result of the key point conversion module.

8. A deep learning-based intelligent detection system for bridge pier verticality, characterized in that, include: The first acquisition module is used to acquire the initial point cloud data obtained after scanning the object to be detected, the object to be detected including bridge piers; The first identification module is used to identify target point cloud data belonging to the object to be detected in the initial point cloud data through a pre-trained point cloud segmentation model; The first processing module is used to identify and remove edge error points in the target point cloud data to obtain processed point cloud data. The first fitting module is used to perform model fitting on the object to be detected based on the processed point cloud data, and to determine the central axis of the object to be detected based on the model fitting result. The first generation module is used to generate the verticality index of the object to be detected based on the central axis. The point cloud segmentation model includes a neighborhood aggregation module, a key point conversion module, and a connection module. The neighborhood aggregation module is used to represent multi-scale context and local details, the key point conversion module is used to capture remote dependency information based on local details, and the connection module is used to connect the neighborhood aggregation module and the key point conversion module.

9. The system according to claim 8, characterized in that, The first acquisition module includes: The first acquisition unit is used to acquire the original point cloud data obtained by the 3D laser scanner after scanning the object to be inspected; The first processing unit is used to perform coordinate normalization on the original point cloud data with the principal axis as the vertical direction to obtain normalized point cloud data. The first denoising unit is used to denoise the normalized point cloud data to obtain denoised point cloud data. The first adjustment unit is used to uniformly adjust the point cloud resolution of the denoised point cloud data to the target spacing to obtain the initial point cloud data.

10. The system according to claim 8, characterized in that, The first processing module includes: The first projection unit is used to unfold the target point cloud data into a two-dimensional image along the main axis direction to obtain two-dimensional projected point cloud data. The first construction unit is used to construct a two-dimensional raster image from the two-dimensional projected point cloud data; The first recognition unit is used to identify the boundary region of the two-dimensional raster image; The first filtering unit is used to filter points in the two-dimensional projected point cloud data whose distance value from the boundary region is less than a set distance value as edge error points. The first elimination unit is used to eliminate the edge error points in the two-dimensional projected point cloud data to obtain processed point cloud data.

Images