A single-Ping sonar point cloud real-time detection method, device and medium for bridge underwater structure diseases
By processing single-Ping sonar point clouds using the HAE-PointNet++ network, the problems of defect error and sample imbalance in underwater bridge structure inspection are solved, achieving high-precision real-time defect identification. This method is suitable for real-time inspection of underwater bridge structures and inspection carried out by unmanned vessels.
Patent Information
- Application Number
- CN202611587174.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-14
- Publication Date
- 2026-11-10
AI Technical Summary
Existing three-dimensional sonar detection methods are easily affected by errors in the underwater structure detection of bridges, leading to missed or false detections of defects. Furthermore, the sample size is limited and the category distribution is uneven, making it difficult to achieve real-time and high-precision defect identification.
The HAE-PointNet++ single-ping sonar point cloud segmentation network is adopted. Multi-scale features are extracted through the PointNet++ backbone network and a local anomaly enhancement module is introduced. Combined with coarse and fine segmentation strategies, a hierarchical segmentation head is constructed for disease identification. Diverse datasets are constructed using measured, semi-synthetic, and fully synthetic point clouds for training.
It improves the real-time detection accuracy and robustness of defects in single-Ping sonar point clouds, is suitable for real-time detection of underwater bridge structures, alleviates the class imbalance problem, and enhances the ability to perceive small-scale defects with weak features.
Smart Images

Figure CN123069129A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the technical field of underwater bridge inspection, and particularly relates to a method, equipment and medium for real-time detection of underwater bridge structural defects using single-Ping sonar point clouds. Background Technology
[0002] Underwater bridge structures are constantly exposed to complex hydrodynamic conditions, silt erosion, corrosion, ship impacts, and biofouling, making them prone to defects such as spalling, exposed reinforcement, cracks, diameter reduction, and localized defects. If these defects are not detected in a timely manner, they may further affect the durability and service safety of the underwater bridge foundation. Due to the low illumination, turbid water, and poor accessibility of the underwater environment, conventional optical inspection methods are insufficient to reliably obtain the appearance of underwater bridge structures.
[0003] Three-dimensional imaging sonar can acquire three-dimensional spatial information of targets in low-visibility water using sound waves, thus becoming an important technical means for the inspection of underwater bridge structures. Existing three-dimensional sonar inspection methods mostly construct a relatively complete point cloud of the underwater bridge structure by registering and stitching data from multiple frames or multiple stations, and then perform structural identification and defect detection on the global point cloud model. However, multi-frame point cloud stitching is easily affected by factors such as sonar platform positioning errors, attitude estimation errors, time synchronization errors, and point cloud registration errors, resulting in local misalignment, boundary ghosting, and geometric distortion in the stitched point cloud. For weak feature defects such as local spalling and diameter reduction, these errors may mask their true geometric characteristics, leading to missed or false detections. Furthermore, the multi-frame registration, stitching, and global model reconstruction processes typically have high computational costs, making it difficult to meet the needs of real-time on-site inspection.
[0004] A single-ping sonar point cloud is a local 3D point cloud acquired by a 3D imaging sonar through a single acoustic pulse. It eliminates the need for cross-ping registration and multi-frame stitching, preserving the original geometric and acoustic intensity information at the current observation moment. It features high data fidelity, short processing time, and strong real-time performance. However, limited by the sonar beam coverage, a single-ping point cloud only contains local terrain and structures within the current field of view, and cannot form a complete underwater structure or riverbed model. Furthermore, single-ping sonar point clouds typically suffer from low point density, significant local data gaps, sparse geometric information, high acoustic noise, and a small number of defects, causing underwater structural defects to often appear as a few, weak, and discontinuous local anomalies in the point cloud.
[0005] In single-ping sonar point clouds, the number of terrain points and normal structure points is usually far greater than that of disease points. Directly performing three-class classification can easily lead to the disease category being overwhelmed by the dominant category. At the same time, the differences between terrain and structure are mainly manifested as regional scale differences, while the differences between normal structure and disease are mainly manifested as local geometric anomalies and sound intensity anomalies. The discriminant features required for the two are different. Therefore, existing segmentation networks are difficult to effectively solve the problems of class imbalance and weak disease feature recognition in single-ping sonar point clouds.
[0006] Furthermore, existing deep learning-based point cloud segmentation methods typically rely on a large number of accurately labeled training samples. However, it is difficult to obtain a large number of measured single-Ping sonar point clouds of underwater bridge structures containing real defects, and accurate labeling is costly. This results in limited sample size and uneven class distribution, which can easily lead to overfitting and insufficient generalization ability of the segmentation model. Therefore, it is difficult to provide a stable data foundation for the training, validation, and performance evaluation of single-Ping sonar defect detection networks. Summary of the Invention
[0007] Purpose of the invention: The purpose of this invention is to provide a method, device and medium for real-time detection of defects in underwater bridge structures using single-Ping sonar point clouds, thereby improving the real-time detection accuracy, robustness and engineering applicability of defects in single-Ping sonar point clouds.
[0008] Technical Solution: To achieve the above objectives, the present invention provides a real-time detection method for single-Ping sonar point clouds of underwater bridge structural defects, comprising the following steps: S1. Obtain multiple sets of measured single-Ping sonar point clouds, multiple sets of semi-synthetic single-Ping sonar point clouds, and multiple sets of fully synthesized single-Ping sonar point clouds, merge them to form dataset D, and label the single-Ping sonar points in each set of single-Ping sonar point cloud samples in dataset D to form real labels. ; S2. Construct a single-ping sonar point cloud segmentation network HAE-PointNet++, with PointNet++ as the backbone network. Multi-scale feature encoding and hierarchical feature propagation are performed through the PointNet++ backbone network to obtain multi-scale point-level features. A local anomaly enhancement module is introduced to extract high-dimensional anomaly embedding features of the diseased area relative to the normal structure in the neighborhood. The multi-scale point-level features and high-dimensional anomaly embedding features are residually fused to obtain anomaly enhancement features. A hierarchical segmentation head including a coarse classification head and a fine classification head is constructed. The anomaly enhancement features are input into the hierarchical segmentation head for classification, and finally, three types of point-level predicted labels are obtained: terrain, normal structure, and disease. S3. Combine dataset D with the real labels. The dataset is divided into training and validation sets, and coarse and fine segmentation supervision labels are set. The coarse segmentation loss adopts the weighted cross-entropy loss function, and the fine segmentation loss adopts the Focal loss function. Together, they constitute the hierarchical training loss of HAE-PointNet++. HAE-PointNet++ is trained, and the network parameters are updated by minimizing the hierarchical training loss to obtain the trained HAE-PointNet++. S4. Based on the trained HAE-PointNet++, perform disease detection and output disease point cloud, disease label and number of disease points.
[0009] Optionally, step S1 specifically includes the following steps: S1.1 For the underwater structure of the bridge, a three-dimensional real-time sonar device was used to acquire the original measured single-Ping sonar point cloud in a surround-scan manner. After noise reduction preprocessing, multiple sets of measured single-Ping sonar point clouds were obtained. ; Among them, the measured single-Ping sonar point cloud , A set of measured single-ping sonar point clouds The number of points in; the first The three-dimensional coordinates of each point , Indicates the first The sonar echo intensity corresponding to each point; S1.2, Segmentation of measured single-ping sonar point cloud The system uses terrain point clouds and structure point clouds. Randomly selected point clusters from the terrain point clouds are then subjected to coordinate offset operations. The offset point clusters are defined as defect points, ultimately generating multiple sets of semi-synthesized single-Ping sonar point clouds. ,in , A set of semi-synthesized single-Ping sonar point clouds The number of points in the middle; S1.3 Constructing a fully synthetic single-Ping sonar point cloud based on sonar imaging principles ,in, , A set of fully synthesized single-Ping sonar point clouds The number of points in the middle; Specifically, the first step is to construct a three-dimensional model that includes the underwater structure of the bridge, the riverbed topography, and the affected areas. ; Let the horizontal field of view of the sonar be... The vertical field of view of the sonar is The number of horizontal beams is and vertical beam number The number of points in the fully synthesized single Ping sonar point cloud , No. Horizontal angle of a sound wave and vertical angle They are respectively: , , Then the first in the sonar coordinate system The direction of the center beam of the sound wave for: , In the formula, This is the matrix transpose. Let the sonar pose be: , In the formula, This is a 3×3 rotation matrix from the sonar coordinate system to the world coordinate system; This is the translation vector of the sonar center in the world coordinate system; Then the first The direction of the center beam of the sound wave, when converted to the world coordinate system, is: , The origin of the sonar launch is: , No. Beamform and 3D Model of a Sound Wave The distance between the first intersection point and the first intersection point is: , in The distance between all intersection points of the beam and the 3D model; No. The point of impact of the sound wave is: , Let the normal vector of the hit face be... The direction of sound wave propagation is Then the surface reflection term Represented as: , in, The angular response index; Normal vector of the hit face The transpose of ; No. Synthesis echo intensity of bar sound waves for: , in, The initial sound intensity, The sound absorption coefficient of water body. Echo intensity noise; Considering sonar radial resolution and propagation distance error , No. The measurement distance of the sound wave is: , in The function is a rounding function, and the propagation distance error is... Using a normal distribution, i.e. , The standard deviation is the normal distribution. No. The coordinates of the fully synthesized sonar point of the sound wave are: , Introducing the first Sound wave blocking mask Then we get the first The three-dimensional coordinates of the target object collected by the sound wave and the echo intensity of a sound wave ; , , In the formula, , It follows the Bernoulli distribution. The probability of water obscuring the target object; Obtain a fully synthesized single-ping sonar point cloud : , in, For the first The three-dimensional coordinates of the target object were collected by the sound waves. For the first The echo intensity of a sound wave; S1.4, Analyzing multiple sets of measured single-ping sonar point clouds Semi-synthesized single-ping sonar point cloud and fully synthesized single-ping sonar point clouds The samples are merged to form dataset D. For each group of samples in dataset D, individual Ping sonar points are labeled to form real labels. Then the first Sample Represented as: ,in, 0 represents terrain, 1 represents normal structure, and 2 represents disease; The three-dimensional coordinates of each point , Indicates the first The sonar echo intensity corresponding to each point Indicates the first Sample The number of points in the middle.
[0010] Optionally, S2 specifically includes the following steps: S2.1. Sample the input single-Ping sonar point cloud sample with a fixed number of points, perform normalization preprocessing, and obtain the preprocessed single-Ping sonar point cloud. for: , in, For single-ping sonar point clouds The number of points in the middle, For the first Normalized coordinates of each point, For the first Normalized sound intensity at individual points; For all points, , , Output the coordinates of a single Ping sonar point cloud. and reflection intensity attribute characteristics ; S2.2. Extracting preprocessed single-ping sonar point clouds using the PointNet++ backbone network. Multi-scale point-level features Specifically, this involves pinging the coordinates of a single sonar point cloud. Input the PointNet++ backbone network, which includes L-level set abstraction modules. and L-level feature propagation module ,in The L-level ensemble abstraction module progressively reduces the spatial resolution of the point cloud and increases the feature dimension, and then the L-level feature propagation module progressively restores it to the original point-level resolution and fuses the reflection intensity attribute features. Multi-scale point-level features are obtained. ,in For the first Multi-scale point-level features of individual points; S2.3 Build a local anomaly enhancement module to enhance the coordinates of single-ping sonar point clouds. Input the local anomaly enhancement module to calculate 5D local anomaly features for each point. ; 5-dimensional local anomaly feature vectors of all points Mapped to high-dimensional anomaly embedding features ; S2.4 Embedding high-dimensional anomalies into features Multi-scale point-level features Residual fusion is performed to form anomaly enhancement features. , For the first Anomaly enhancement features at individual points; S2.5, Based on anomaly enhancement features After the hierarchical segmentation head performs both coarse and fine segmentation, the coarse segmentation head outputs the terrain probability and structure probability, while the fine segmentation head outputs the normal structure probability and defect probability, resulting in the... The probability of three classifications at each point , ; S2.6, Introduce a disease probability threshold United Nations The output of the three-class probability of the nth point. Point-level prediction label for each point : , in, ; Finally, obtain the point-level predicted labels for all points: .
[0011] Optionally, S2.1 specifically includes the following steps: A fixed number of points are sampled from the input single-Ping sonar point cloud sample to obtain... From each point, we obtain the sampled single-Ping sonar point cloud. : , Point cloud coordinate center for: , Scale normalization factor for: , No. Normalized coordinates of points for: , in To ensure that the fraction is a very small number, preventing the fraction from having a denominator of 0; Mean sound intensity and standard deviation They are respectively: , , No. Normalized sound intensity at individual points for: , Finally, the preprocessed single-ping sonar point cloud is obtained. for: , Output the coordinates of a single Ping sonar point cloud. and reflection intensity attribute characteristics ,in , .
[0012] Optionally, in S2.2, the PointNet++ backbone network is set to... Level Collection Abstraction Module The Middle The center point is Its radius The next neighborhood is The corresponding local features are represented as: , in, For neighboring points, These are the local coordinates of the neighboring points relative to the center point. For the first Level point features, preset It is an empty set; For the first Level 1 Shared multilayer perceptron at a given radius scale For channel-by-channel max pooling; For multiple scale radii Multi-scale local features are obtained by channel splicing: , In the formula, This is a feature channel splicing operation; This represents the total number of scale radii; No. Level Collection Abstraction Module Output features are , , For the first Level Collection Abstraction Module The number of points in the middle, ; when At that time, using the first Level feature propagation module For the first Output features of the hierarchical set abstraction module and the Output features of the hierarchical set abstraction module Perform fusion processing to obtain the first Level fusion features ; when At that time, using the first Level feature propagation module For the first Level fusion features and the Output features of the hierarchical set abstraction module Perform fusion processing to obtain the first Level fusion features ; when At that time, the first-level feature propagation module is used. For the second level fusion feature and reflection intensity attribute characteristics The fusion process is performed to obtain multi-scale point-level features. .
[0013] Optionally, step S2.3 specifically includes the following steps: A local anomaly enhancement module was built to enhance the coordinates of a single Ping sonar point cloud. Input the local anomaly enhancement module to calculate 5D local anomaly features for each point. ; For point First search its Nearest neighbor points: , in, Represented as point cloud Nearest neighbor algorithm; Neighborhood Center for: , in For point of The first in the nearest neighbor domain The three-dimensional coordinates of each point; Center offset for: , in, These are the three coordinate components offset from the center; Distance residual for: , The neighborhood covariance matrix is: , Perform eigenvalue decomposition on the covariance matrix: , In the formula, For the first The local neighborhood covariance matrix corresponding to each point The m-th eigenvalue; For eigenvalues The corresponding unit eigenvector; m is the index of the eigenvalue and its corresponding eigenvector, and ; Take the eigenvector corresponding to the smallest eigenvalue as the local plane normal vector. : , The local plane residual is: , Construct the first 5D local anomaly feature vector of a point : , And obtain the 5D local anomaly feature matrix of all points. ; 5-dimensional local anomaly feature vectors of all points Mapped to high-dimensional anomaly embedding features First, the first 5D local anomaly feature vector of a point Input small multilayer perceptron , mapped to the first High-dimensional anomaly embedding features of points: , And obtain the high-dimensional anomaly embedding features of all points. , .
[0014] Optionally, step S2.5 specifically includes the following steps: For the The coarse segmentation head first outputs two raw scores for each point, indicating whether it belongs to the terrain region or the structure region. : , in, ; This represents the learnable weight matrix in the coarse segmentation head; It is the classification bias vector of the coarse segmentation head. This is the original classification score vector output by the coarse segmentation head, with its two components corresponding to the terrain region and the structure region, respectively. Then through The activation function normalizes the two original classification scores into a probability distribution, thus obtaining the first... The probability that a point belongs to terrain or structure, coarsely segmented probability vector. Right now: ,in, ; The probability of predicting a feature as terrain during the coarse segmentation stage. The probability of predicting a structure during the coarse segmentation stage; For the At each point, the fine-grained segmentation head first outputs two raw scores: one for the normal structural region and one for the diseased region. : , in, This represents the learnable weight matrix in the fine-grained head. It is the classification bias vector of fine segmentation. This is the original classification score vector output by the fine segmentation head, with two vectors corresponding to normal structure and disease, respectively; Then through The function normalizes the two original classification scores into a probability distribution, thus obtaining the first... The probability that a point belongs to a normal structure or a disease; finely segment the probability vector. Right now: ,in: , Predicting the first stage of fine segmentation The probability that a point has a normal structure; Predicting the first stage of fine segmentation The probability that each point belongs to a disease; Obtain the The probability of three-class classification for the i-th point, the i-th Terrain probability at each point for: , No. The probability of normal structure at each point for: , No. The probability of disease at each point for: , No. The three-class probability vector of each point : .
[0015] Optionally, step S3 specifically includes the following steps: Coarse segmentation supervision label Defined as: , in, Indicates terrain, This refers to structures, which include normal structures and defects; The coarse segmentation loss uses the weighted cross-entropy loss function: , In the formula, Coarse segmentation probability vector Medium and coarse segmentation supervision labels The corresponding predicted probability; For coarse segmentation of category weights; fine-segmented supervision labels Defined as: , in, Indicates a normal structure. Indicates disease; The fine-segmentation loss uses the Focal loss function: , in The number of point clouds involved in the fine segmentation; For focusing parameters; To further refine the category weights; For fine-segmented probability vectors Medium and fine segmentation supervision labels The corresponding predicted probability; Coarse segmentation loss and fine segmentation loss constitute the hierarchical training loss of the HAE-PointNet++ network: ,
[0016] In the formula, These are the weighting coefficients for the coarse segmentation loss. The weight coefficients for the fine-grained segmentation loss are determined by minimizing the hierarchical training loss. Update the network parameters to obtain the trained HAE-PointNet++ network.
[0017] Based on the same inventive concept, the present invention provides an electronic device, including a processor and a storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the single-Ping sonar point cloud real-time detection method for underwater bridge structural defects as described above.
[0018] Based on the same inventive concept, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the single-Ping sonar point cloud real-time detection method for underwater bridge structural defects as described above.
[0019] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: (1) This invention directly processes single-ping three-dimensional sonar point clouds without relying on multi-ping (frame) stitching, which can avoid the impact of multi-frame registration error and platform positioning error on disease identification; (2) The present invention uses measured single Ping sonar point clouds, semi-synthetic single Ping sonar point clouds and fully synthetic single Ping sonar point clouds as the basis to form a dataset, which can provide diverse disease samples and accurate point-level labels for network training. (3) The present invention adopts a hierarchical segmentation strategy that combines coarse segmentation and fine segmentation, which makes the identification of defects more consistent with the semantic relationship of defects of underwater bridge structures embedded on the surface of the structure. This can reduce the risk of defect points being submerged by terrain and normal structure points and alleviate the problem of class imbalance. (4) This invention proposes a local anomaly enhancement module, which improves the network's ability to perceive small-scale weak feature diseases by using explicit anomaly features such as center offset, distance residual, and local plane residual; (5) This invention improves the real-time detection accuracy, robustness and engineering applicability of defects in single Ping sonar point clouds, and is applicable to engineering application scenarios such as real-time detection of underwater bridge structures, detection carried by unmanned vessels, detection of underwater robots and bridge health monitoring. Attached Figure Description
[0020] Figure 1 This is a schematic diagram of the process of the present invention; Figure 2 This is a schematic diagram of HAE-PointNet++ of the present invention; Figure 3 This is a rendering of the fully synthesized single-Ping sonar point cloud generated in this invention; Figure 4 This is a comparison of the segmentation results of HAE-PointNet++ on the measured single-Ping sonar point cloud in this invention. Detailed Implementation
[0021] The technical solution of the present invention will be further described below with reference to the accompanying drawings.
[0022] Example 1, such as Figure 1As shown, the present invention provides a real-time detection method for single-Ping sonar point clouds of underwater bridge structural defects, comprising the following steps: S1. Obtain multiple sets of measured single-Ping sonar point clouds. Multiple sets of semi-synthesized single-ping sonar point clouds and multiple sets of fully synthesized single-Ping sonar point clouds The samples are merged to form dataset D. For each group of single-Ping sonar point cloud samples in dataset D, the single-Ping sonar points are labeled to form real labels. .
[0023] Among them, the first Sample Represented as: , 0 represents terrain, 1 represents normal structure, and 2 represents disease; The three-dimensional coordinates of each point , Indicates the first The sonar echo intensity corresponding to each point Indicates the first Sample The number of points in the middle.
[0024] Step S1 specifically includes the following steps: S1.1 For the underwater structure of the bridge, a three-dimensional real-time sonar device was used to acquire the original measured single-Ping sonar point cloud in a surround-scan manner. After noise reduction preprocessing, multiple sets of measured single-Ping sonar point clouds were obtained. .
[0025] Among them, the measured single-Ping sonar point cloud , A set of measured single-ping sonar point clouds The number of points in; the first The three-dimensional coordinates of each point , Indicates the first The sonar echo intensity corresponding to each point; since there are outlier interference points in the original measured single-Ping sonar point cloud, SOR statistical filtering is used to remove noise points in the measured single-Ping sonar point cloud.
[0026] S1.2, Segmentation of measured single-ping sonar point cloud The system uses terrain point clouds and structure point clouds. Randomly selected point clusters from the terrain point clouds are then subjected to coordinate offset operations. The offset point clusters are defined as defect points, ultimately generating multiple sets of semi-synthesized single-Ping sonar point clouds. ,in , A set of semi-synthesized single-Ping sonar point clouds The number of points in the data. This invention can expand the number of disease samples while maintaining the distribution characteristics of measured sonar data.
[0027] S1.3 Constructing a fully synthetic single-Ping sonar point cloud based on sonar imaging principles ,in, , A set of fully synthesized single-Ping sonar point clouds The number of points in the middle.
[0028] like Figure 3 As shown, specifically, a three-dimensional model is first constructed, including the underwater structure of the bridge, the riverbed topography, and the affected areas. ; Let the horizontal field of view of the sonar be... The vertical field of view of the sonar is The number of horizontal beams is and vertical beam number The number of points in the fully synthesized single Ping sonar point cloud , No. Horizontal angle of a sound wave and vertical angle They are respectively: , , Then the first in the sonar coordinate system The direction of the center beam of a sound wave for: , In the formula, This is the matrix transpose.
[0029] Let the sonar pose be: , In the formula, This is a 3×3 rotation matrix from the sonar coordinate system to the world coordinate system; is the translation vector of the sonar center in the world coordinate system.
[0030] Then the first The direction of the center beam of the sound wave, when converted to the world coordinate system, is: , The origin of the sonar launch is: , No. Beamform and 3D Model of a Sound Wave The distance between the first intersection point and the first intersection point is: , in This represents the distance between all intersection points of the beam and the 3D model.
[0031] No. The point of impact of the sound wave is: , Let the normal vector of the hit face be... The direction of sound wave propagation is Then the surface reflection term Represented as: , in The angle response index is used to control the sensitivity of the target surface to changes in the incident angle. Normal vector of the hit face The transpose of .
[0032] No. Synthesis echo intensity of bar sound waves for: , in, The initial sound intensity, The sound absorption coefficient of water body. This refers to echo intensity noise.
[0033] Considering sonar radial resolution and propagation distance error , No. The measurement distance of the sound wave is: , in The function is a rounding function, and the propagation distance error is... Using a normal distribution, i.e. , is the standard deviation of a normal distribution.
[0034] No. The coordinates of the fully synthesized sonar point of the sound wave are: , To simulate data gaps caused by suspended matter or clutter in water, the first... Sound wave blocking mask Then we get the first The three-dimensional coordinates of the target object collected by the sound wave and the echo intensity of a sound wave .
[0035] , , In the formula, For the first A masking layer for blocking sound waves. , It follows the Bernoulli distribution. The probability of water obscuring the target object is 0.1.
[0036] This yields a fully synthesized single-Ping sonar point cloud. : , in, For the first The three-dimensional coordinates of the target object were collected by the sound waves. For the first The echo intensity of a sound wave.
[0037] S1.4, Analyzing multiple sets of measured single-ping sonar point clouds Semi-synthesized single-ping sonar point cloud and fully synthesized single-ping sonar point clouds The samples are merged to form dataset D. For each group of samples in dataset D, individual Ping sonar points are labeled to form real labels. Then the first Sample Represented as: ,in, 0 represents terrain, 1 represents normal structure, and 2 represents disease; The three-dimensional coordinates of each point , Indicates the first The sonar echo intensity corresponding to each point Indicates the first Sample The number of points in the middle.
[0038] S2, such as Figure 2As shown, to address the issues of sparse point clouds, weak local geometric features, and low proportion of diseased points in single-Ping sonar point clouds, a hierarchical local anomaly enhancement single-Ping sonar point cloud segmentation network, HAE-PointNet++, is constructed. HAE-PointNet++ uses PointNet++ as its backbone network. Multi-scale point-level features are obtained through multi-scale feature encoding and hierarchical feature propagation of the PointNet++ backbone network. A local anomaly enhancement module is introduced to extract high-dimensional anomaly embedding features of diseased areas relative to the normal structure of their neighbors. The multi-scale point-level features and high-dimensional anomaly embedding features are residually fused to obtain anomaly enhancement features. A hierarchical segmentation head, including a coarse classification head and a fine classification head, is constructed. The anomaly enhancement features are input into the hierarchical segmentation head for classification, ultimately obtaining three types of point-level predicted labels: terrain, normal structure, and diseased areas. The single-Ping sonar point cloud segmentation network HAE-PointNet++ of this invention can adapt to the characteristics of uneven point density and sparse local geometric information in single-Ping sonar point clouds. The multi-scale point-level features of this invention simultaneously contain local geometric information, sound intensity information, and contextual semantic information. The anomaly enhancement feature of this invention can introduce local geometric anomaly information while preserving the semantic expressive power of the PointNet++ backbone, thus avoiding the damage of local anomaly features to the backbone semantic features. The hierarchical segmentation head of this invention can first distinguish between terrain and structural regions, and then further identify defect points within the structural regions, thereby alleviating the class imbalance problem among terrain points, normal structural points, and defect points.
[0039] S2 specifically includes the following steps: S2.1. Sample the input single-Ping sonar point cloud sample with a fixed number of points, perform normalization preprocessing, and obtain the preprocessed single-Ping sonar point cloud. for: , in, For single-ping sonar point clouds The number of points in the middle, For the first Normalized coordinates of each point, For the first The intensity of the sound is normalized to a single point.
[0040] For all points, , , Output the coordinates of a single Ping sonar point cloud. and reflection intensity attribute characteristics .
[0041] S2.1 specifically includes the following steps: For the input of the first A single Ping sonar point cloud sample Perform sampling at a fixed number of points to obtain One point, The value is set to 4096, resulting in the sampled single-ping sonar point cloud. : , Point cloud coordinate center for: , Scale normalization factor for: , No. Normalized coordinates of points for: , in To prevent fractions from having a denominator of 0, the denominator must be a very small number.
[0042] Mean sound intensity and standard deviation They are respectively: , , No. Normalized sound intensity at individual points for: , Finally, the preprocessed single-ping sonar point cloud is obtained. for: , Output the coordinates of a single Ping sonar point cloud. and reflection intensity attribute characteristics ,in , .
[0043] S2.2. Extracting preprocessed single-ping sonar point clouds using the PointNet++ backbone network. Multi-scale point-level features Specifically, this involves pinging the coordinates of a single sonar point cloud. Input the PointNet++ backbone network, which includes L-level set abstraction modules. and L-level feature propagation module ,in The L-level ensemble abstraction module progressively reduces the spatial resolution of the point cloud and increases the feature dimension, and then the L-level feature propagation module progressively restores it to the original point-level resolution and fuses the reflection intensity attribute features. Multi-scale point-level features are obtained. ,in For the first Multi-scale point-level features of individual points.
[0044] The PointNet++ backbone network includes PointNet++ multi-scale local feature encoding and PointNet++ decoding, and an L-level set abstraction module. Complete PointNet++ multi-scale local feature encoding and L-level feature propagation module. Complete PointNet++ decoding.
[0045] The data flow representation of the PointNet++ backbone network is as follows: , The changes in the feature channels are as follows: , Let the PointNet++ backbone network be... Level Collection Abstraction Module The Middle The center point is Its radius The next neighborhood is The corresponding local features are represented as: , in, For neighboring points, These are the local coordinates of the neighboring points relative to the center point. For the first Level point features, preset It is an empty set; For the first Level 1 Shared multilayer perceptron at a given radius scale This is for channel-by-channel max pooling.
[0046] For multiple scale radii Multi-scale local features are obtained by channel splicing: , In the formula, This is a feature channel splicing operation; This represents the total number of scale radii.
[0047] No. Level Collection Abstraction Module Output features are , , For the first Level Collection Abstraction Module The number of points in the middle, .
[0048] Level 1 Collection Abstraction Module Lieutenant General Original Downsampling from one point to 1024 center points: The MLPs at the two scales are set as follows: The output features from the two scales are concatenated, and the first-level set abstraction module is used. The number of output channels is 32 + 64 = 96; Level 1 Collection Abstraction Module Output features are .
[0049] Specifically, , , , , Preset It is an empty set.
[0050] Level 2 Collection Abstraction Module The 1024 points were downsampled to 256 center points: The MLPs at the two scales are set as follows: The output features from the two scales are concatenated, and the second-level set abstraction module is used. The number of output channels is 128 + 128 = 256, which is the first level of the abstract module. Output features are .
[0051] , , , , Level 3 Collection Abstraction Module The 256 points were downsampled to 64 center points: The MLPs at the two scales are set as follows: The output features from the two scales are concatenated, and the third-level set abstraction module is used. The number of output channels is 256 + 256 = 512, which is the third level of the set abstraction module. Output features are .
[0052] , , , , Level 4 Collection Abstraction Module The original 64 points were downsampled to 16 center points: The MLPs at the two scales are set as follows: The output features from the two scales are concatenated, and the fourth-level set abstraction module is used. The number of output channels is 512 + 512 = 1024, which is the fourth level of the set abstraction module. Output features are .
[0053] , , , , Therefore, the number of points in the encoding stage changes sequentially as follows: The corresponding number of output feature channels are as follows: .
[0054] After L-level set abstraction module After multi-scale local feature encoding, low-resolution point sets and low-resolution semantic features are obtained, which are then propagated through an L-level feature propagation module. The low-resolution semantic features are gradually restored to the original point-level resolution.
[0055] when At that time, using the first Level feature propagation module For the first Output features of the hierarchical set abstraction module and the Output features of the hierarchical set abstraction module Perform fusion processing to obtain the first Level fusion features ; when At that time, using the first Level feature propagation module For the first Level fusion features and the Output features of the hierarchical set abstraction module Perform fusion processing to obtain the first Level fusion features ; when At that time, the first-level feature propagation module is used. For the second level fusion feature and reflection intensity attribute characteristics The fusion process is performed to obtain multi-scale point-level features. .
[0056] Specifically, the fourth-level set abstraction module Output features and Level 3 Collection Abstraction Module Output features are Common input Level 4 feature propagation module Processing is performed to output the fourth level of fused features. ; The fourth level of fusion features and Level 2 Collection Abstraction Module Output features are Common input Level 3 feature propagation module Processing is performed to output the third-level fusion feature. ; The third level of fusion features and Level 1 Collection Abstraction Module Output features are Common input level 2 feature propagation module Process the data and output the second-level fusion features. ; The second level of fusion features and reflection intensity attribute characteristics Same input Level 1 feature propagation module Processing is performed to output multi-scale point-level features. .
[0057] Level 4 Feature Propagation Module Lieutenant General Level 4 Collection Abstraction Module The 16 features are propagated to the third-level set abstraction module. The corresponding 64-point resolution, the interpolated 1024-dimensional feature and the level 3 set abstraction module 512-dimensional skip connection feature splicing: splicing features Complete feature fusion to obtain .
[0058] , , , in, To represent the fourth-level feature propagation module In the middle, through the 4th level set abstraction module After performing neighborhood interpolation on the low-resolution features, the resulting... Upsampled feature vectors of the points to be recovered; This is the set of upsampled features for all 64 points to be recovered; For low-resolution point sets The Middle The high-dimensional semantic feature vector carried by each neighboring point, i.e. ; For low-resolution point set and points to be recovered The set of nearest neighbors; To prevent fractions from having a denominator of 0, the amount must be extremely small. Neighboring points interpolation points Interpolation weights: , in For point , The three-dimensional coordinates.
[0059] Level 3 Feature Propagation Module The 64-point feature is propagated to a resolution of 256 points and then combined with the second-level set abstraction module. 256-dimensional skip connection feature splicing: splicing features Complete feature fusion to obtain .
[0060] , , , in To represent the third-level feature propagation module In the middle, through the fourth-level feature propagation module After performing neighborhood interpolation on the low-resolution features, the resulting... Upsampled feature vectors of the points to be recovered; This is the set of upsampled features for all 256 points to be recovered; For Level 4 fusion features The high-dimensional semantic feature vector in, i.e. .
[0061] Level 2 Feature Propagation Module The 256-point feature is propagated to a resolution of 1024 points and then combined with the first-level set abstraction module. 96-dimensional skip connection feature splicing: splicing features After completing feature fusion, we obtain: .
[0062] , , , in To represent the second-level feature propagation module In the middle, through the third-level feature propagation module After performing neighborhood interpolation on the low-resolution features, the resulting... Upsampled feature vectors of the points to be recovered; This is the set of upsampled features for all 1024 points to be recovered; For Level 3 fusion features The high-dimensional semantic feature vector in, i.e. .
[0063] Level 2 Feature Propagation Module The 1024 features of the lieutenant general were further propagated to the original source. Point resolution combines 128-dimensional upsampled features with the original point's one-dimensional input reflection intensity attribute features. Splicing: 128 + 1 = 129; splicing feature... After completing the feature mapping, we finally obtain: This yields 128-dimensional multi-scale point-level features that correspond one-to-one with each measurement point in the original single-Ping sonar point cloud. : , , , in To represent the first-level feature propagation module In the middle, through the second-level feature propagation module After performing neighborhood interpolation on the low-resolution features, the resulting... Upsampled feature vectors of the points to be recovered; For all An upsampled feature set of points to be recovered; For Level 2 fusion features The high-dimensional semantic feature vector in, i.e. .
[0064] S2.3 Build a local anomaly enhancement module to enhance the coordinates of single-ping sonar point clouds. Input the local anomaly enhancement module to calculate 5D local anomaly features for each point. ; 5-dimensional local anomaly feature vectors of all points Mapped to high-dimensional anomaly embedding features .
[0065] S2.3 specifically includes the following steps: A local anomaly enhancement module was built to enhance the coordinates of a single Ping sonar point cloud. Input the local anomaly enhancement module to calculate 5D local anomaly features for each point. ; For point First search its Nearest neighbor points: , in, Represented as the k-nearest neighbor algorithm for point clouds; Neighborhood Center for: , in For point The k-nearest neighbor in the neighborhood The three-dimensional coordinates of each point; .
[0066] Center offset for: , in, These are the three coordinate components offset from the center; Distance residual for: , The neighborhood covariance matrix is: , Perform eigenvalue decomposition on the covariance matrix: , In the formula, For the first The local neighborhood covariance matrix corresponding to each point The m-th eigenvalue; For eigenvalues The corresponding unit eigenvector; m is the index of the eigenvalue and its corresponding eigenvector, and m=1,2,3.
[0067] Take the eigenvector corresponding to the smallest eigenvalue as the local plane normal vector. : , The local plane residual is: , Therefore, the first... 5D local anomaly feature vector of a point : , And obtain the 5D local anomaly feature matrix of all points. .
[0068] 5-dimensional local anomaly feature vectors of all points Mapped to high-dimensional anomaly embedding features First, the first 5D local anomaly feature vector of a point Input small multilayer perceptron , mapped to the first High-dimensional anomaly embedding features of points: , Among them, small multilayer sensing machine for ; Indicates the first High-dimensional anomaly embedding features of points; And obtain the high-dimensional anomaly embedding features of all points. , .
[0069] S2.4 Embedding high-dimensional anomalies into features Multi-scale point-level features Residual fusion is performed to form anomaly enhancement features. , For the first The abnormal enhancement features of each point.
[0070] Specifically, feature fusion is performed using feature concatenation, one-dimensional convolutional feature fusion, and residual feature enhancement operations to obtain the first feature. Point anomaly enhancement features The formula is: , , , in, For the first The splicing features of individual points This is a tensor concatenation function; The first output of the PointNet++ decoder Features of each point; For convolution kernel; For the first High-dimensional anomaly embedding features at each point.
[0071] All points are represented as: , in, This represents the point-level features after anomalous enhancement; this residual fusion method preserves the output features of the PointNet++ backbone network. At the same time, high-dimensional anomaly embedding features are introduced. This avoids local abnormal features from damaging the semantic expression of the backbone network.
[0072] S2.5, Based on anomaly enhancement features After the hierarchical segmentation head performs both coarse and fine segmentation, the coarse segmentation head outputs the terrain probability and structure probability, while the fine segmentation head outputs the normal structure probability and defect probability, resulting in the... The probability of three classifications at each point , .
[0073] S2.5 specifically includes the following steps: For the The coarse segmentation head first outputs two raw scores for each point, indicating whether it belongs to the terrain region or the structure region. : , in, ; This represents the learnable weight matrix in the coarse segmentation head; It is the classification bias vector of the coarse segmentation head. The learnable weight matrix and classification bias vector are parameters that are automatically learned during network training. They are used to convert the abnormal augmentation point-level features into the original classification scores of the two classes: terrain and structure. This is the original classification score vector output by the coarse segmentation head, with its two components corresponding to the terrain region and the structure region, respectively.
[0074] Then through The activation function normalizes the two original classification scores into a probability distribution, thus obtaining the first... The probability that a point belongs to terrain or structure, coarsely segmented probability vector. Right now: , in, ; The probability of predicting a feature as terrain during the coarse segmentation stage. The probability of predicting a structure during the coarse segmentation stage.
[0075] For the At each point, the fine-grained segmentation head first outputs two raw scores: one for the normal structural region and one for the diseased region. : , in, This represents the learnable weight matrix in the fine-grained head. It is a finely segmented classification bias vector. The learnable weight matrix and classification bias vector are parameters that are automatically learned during network training. They are used to convert the abnormal enhancement point-level features into the original classification scores of two classes: structure and defect. This is the original classification score vector output by the fine segmentation head, with two vectors corresponding to normal structure and disease, respectively.
[0076] Then through The function normalizes the two original classification scores into a probability distribution, thus obtaining the first... The probability that a point belongs to a normal structure or a disease; finely segment the probability vector. Right now: , in: , Predicting the first stage of fine segmentation The probability that a point has a normal structure; Predicting the first stage of fine segmentation The probability that each point belongs to a disease.
[0077] Obtain the The probability of three-class classification for the i-th point, the i-th Terrain probability at each point for: , No. The probability of normal structure at each point for: , No. The probability of disease at each point for: , No. The three-class probability vector of each point : .
[0078] S2.6 To improve the disease recall rate, a disease probability threshold is introduced. United Nations The output of the three-class probability of the nth point. Point-level prediction label for each point : , in, .
[0079] Finally, obtain the point-level predicted labels for all points: .
[0080] S3. Combine dataset D with the real labels. The dataset is divided into training and validation sets, and coarse and fine segmentation supervision labels are set. The coarse segmentation loss adopts the weighted cross-entropy loss function, and the fine segmentation loss adopts the Focal loss function. Together, they constitute the hierarchical training loss of the HAE-PointNet++ network. The HAE-PointNet++ network is trained, and the network parameters are updated by minimizing the hierarchical training loss to obtain the trained HAE-PointNet++ network.
[0081] S3 specifically includes the following steps: Coarse segmentation supervision label Defined as: , in, Indicates terrain, This refers to structures, which include normal structures and defects; The coarse segmentation loss of the coarse segmentation head uses the weighted cross-entropy loss function: , In the formula, Coarse segmentation probability vector Medium and coarse segmentation supervision labels The corresponding predicted probability; The weights for coarse segmentation categories are set as follows: 1.0 for terrain regions and 1.5 for structural regions.
[0082] Fine-grained segmentation only monitors points where the original, true labels belong to normal structures or defects; fine-grained segmentation monitors labels. Defined as: , in, Indicates a normal structure. It indicates disease or illness.
[0083] To enhance the network's ability to learn about a small number of disease points, the fine-segmentation loss of the fine-segmentation head adopts the Focal loss function: , in The number of point clouds involved in the fine segmentation; To focus parameters, ; To assign weights to the subcategories, the weight for the subcategories corresponding to normal structures is 1.5, and the weight for the subcategories corresponding to diseases is 3.0. For fine-segmented probability vectors Medium and fine segmentation supervision labels The corresponding predicted probability.
[0084] The coarse segmentation loss and the fine segmentation loss together constitute the hierarchical training loss of the HAE-PointNet++ network: , In the formula, These are the weighting coefficients for the coarse segmentation loss. These are the weighting coefficients for the fine-segmentation loss. , By minimizing the training loss of each layer. Update the network parameters to obtain the trained HAE-PointNet++ network.
[0085] S4. Based on the trained HAE-PointNet++ network, perform disease detection and output disease point cloud, disease label and number of disease points.
[0086] The denoised measured single-Ping sonar point cloud data to be tested The input to the trained HAE-PointNet++ network is represented as the output prediction result: , Extracting disease point clouds : , Final output: Disease point cloud Disease Labels and the number of disease points .
[0087] This invention uses single-ping sonar point clouds acquired by 3D real-time imaging sonar as the detection object. By constructing a dataset composed of measured single-ping point clouds, semi-synthetic single-ping point clouds, and fully synthetic single-ping point clouds, a hierarchically enhanced HAE-PointNet++ point cloud segmentation network is trained. The trained network is then used to perform point-level semantic segmentation and defect region clustering on the single-ping sonar point clouds to be detected, thereby outputting the detection results of underwater bridge structure defects.
[0088] like Figure 4 As shown, five different samples are input into the HAE-PointNet++ network, the PointNet++ network, and PointNet for prediction, respectively. Figure 4It can be seen that the trained HAE-PointNet++ network can perform point-level segmentation on the measured single-Ping sonar point cloud, distinguishing between terrain, normal structure, and diseased areas, and generating effective detection results for local diseased areas. This invention can reduce the impact of multi-measurement point cloud stitching errors on underwater disease identification, improve the real-time detection capability of weak feature diseases in sparse single-Ping sonar point clouds, and is suitable for automated detection of underwater structural diseases of bridges.
[0089] Example 2: An electronic device of the present invention includes a processor and a storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the single-Ping sonar point cloud real-time detection method for underwater bridge structural defects as described above.
[0090] Example 3: A computer-readable storage medium of the present invention stores a computer program thereon, which, when executed by a processor, implements the steps of the single-Ping sonar point cloud real-time detection method for underwater bridge structural defects as described above.
Claims
1. A method for real-time detection of underwater bridge structural defects using single-Ping sonar point clouds, characterized in that, Includes the following steps: S1. Obtain multiple sets of measured single-Ping sonar point clouds, multiple sets of semi-synthetic single-Ping sonar point clouds, and multiple sets of fully synthesized single-Ping sonar point clouds, merge them to form dataset D, and label the single-Ping sonar points in each set of single-Ping sonar point cloud samples in dataset D to form real labels. ; S2. Construct a single-ping sonar point cloud segmentation network HAE-PointNet++, with PointNet++ as the backbone network. Multi-scale feature encoding and hierarchical feature propagation are performed through the PointNet++ backbone network to obtain multi-scale point-level features. A local anomaly enhancement module is introduced to extract high-dimensional anomaly embedding features of the diseased area relative to the normal structure of the neighborhood. Anomaly enhancement features are obtained by residual fusion of multi-scale point-level features and high-dimensional anomaly embedding features; a hierarchical segmentation head including a coarse classification head and a fine classification head is constructed, and the anomaly enhancement features are input into the hierarchical segmentation head for classification, finally obtaining three types of point-level prediction labels: terrain, normal structure, and disease. S3. Combine dataset D with the real labels. The dataset is divided into training and validation sets, and coarse and fine segmentation supervision labels are set. The coarse segmentation loss adopts the weighted cross-entropy loss function, and the fine segmentation loss adopts the Focal loss function. Together, they constitute the hierarchical training loss of HAE-PointNet++. HAE-PointNet++ is trained, and the network parameters are updated by minimizing the hierarchical training loss to obtain the trained HAE-PointNet++. S4. Based on the trained HAE-PointNet++, perform disease detection and output disease point cloud, disease label and number of disease points.
2. The method for real-time detection of single-Ping sonar point clouds for underwater bridge structural defects according to claim 1, characterized in that, Step S1 specifically includes the following steps: S1.1 For the underwater structure of the bridge, a three-dimensional real-time sonar device was used to acquire the original measured single-Ping sonar point cloud in a surround-scan manner. After noise reduction preprocessing, multiple sets of measured single-Ping sonar point clouds were obtained. ; Among them, the measured single-Ping sonar point cloud , A set of measured single-ping sonar point clouds The number of points in; the first The three-dimensional coordinates of each point , Indicates the first The sonar echo intensity corresponding to each point; S1.2, Segmentation of measured single-ping sonar point cloud The system uses terrain point clouds and structure point clouds. Randomly selected point clusters from the terrain point clouds are then subjected to coordinate offset operations. The offset point clusters are defined as defect points, ultimately generating multiple sets of semi-synthesized single-Ping sonar point clouds. ,in , A set of semi-synthesized single-Ping sonar point clouds The number of points in the middle; S1.3 Constructing a fully synthetic single-Ping sonar point cloud based on sonar imaging principles ,in, , A set of fully synthesized single-ping sonar point clouds The number of points in the middle; Specifically, the first step is to construct a three-dimensional model that includes the underwater structure of the bridge, the riverbed topography, and the affected areas. ; Let the horizontal field of view of the sonar be... The vertical field of view of the sonar is The number of horizontal beams is and vertical beam number The number of points in the fully synthesized single Ping sonar point cloud , No. Horizontal angle of a sound wave and vertical angle They are respectively: , , Then the first in the sonar coordinate system The direction of the center beam of the sound wave for: , In the formula, This is the matrix transpose. Let the sonar pose be: , In the formula, This is a 3×3 rotation matrix from the sonar coordinate system to the world coordinate system; This is the translation vector of the sonar center in the world coordinate system; Then the first The direction of the center beam of the sound wave, when converted to the world coordinate system, is: , The origin of the sonar launch is: , No. Beamform and 3D Model of a Sound Wave The distance between the first intersection point and the first intersection point is: , in The distance between all intersection points of the beam and the 3D model; No. The point of impact of the sound wave is: , Let the normal vector of the hit face be... The direction of sound wave propagation is Then the surface reflection term Represented as: , in The angular response index; Normal vector of the hit face The transpose of ; No. Synthesis echo intensity of bar sound waves for: , in, The initial sound intensity, The sound absorption coefficient of water body. Echo intensity noise; Considering sonar radial resolution and propagation distance error , No. The measurement distance of the sound wave is: , in The function is a rounding function, and the propagation distance error is... Using a normal distribution, i.e. , The standard deviation is the normal distribution. No. The coordinates of the fully synthesized sonar point of the sound wave are: , Introducing the first Sound wave blocking mask Then we get the first The three-dimensional coordinates of the target object collected by the sound wave and the echo intensity of a sound wave ; , , In the formula, , It follows the Bernoulli distribution. The probability of water obscuring the target object; Obtain a fully synthesized single-ping sonar point cloud : , in, For the first The three-dimensional coordinates of the target object were collected by the sound waves. For the first The echo intensity of a sound wave; S1.4, Analyzing multiple sets of measured single-ping sonar point clouds Semi-synthesized single-ping sonar point cloud and fully synthesized single-ping sonar point clouds The samples are merged to form dataset D. For each group of samples in dataset D, individual Ping sonar points are labeled to form real labels. Then the first Sample Represented as: ,in, 0 represents terrain, 1 represents normal structure, and 2 represents disease; The three-dimensional coordinates of each point , Indicates the first The sonar echo intensity corresponding to each point Indicates the first Sample The number of points in the middle.
3. The method for real-time detection of single-Ping sonar point clouds for underwater bridge structural defects according to claim 1, characterized in that, S2 specifically includes the following steps: S2.
1. Sample the input single-Ping sonar point cloud sample with a fixed number of points, perform normalization preprocessing, and obtain the preprocessed single-Ping sonar point cloud. for: , in, Point clouds of a single Ping sonar The number of points in the middle, For the first Normalized coordinates of each point, For the first Normalized sound intensity at a single point; For all points, , , Output the coordinates of a single Ping sonar point cloud. and reflection intensity attribute characteristics ; S2.
2. Extracting preprocessed single-ping sonar point clouds using the PointNet++ backbone network. Multi-scale point-level features Specifically, this involves pinging the coordinates of a single sonar point cloud. Input the PointNet++ backbone network, which includes L-level set abstraction modules. and L-level feature propagation module ,in The L-level ensemble abstraction module progressively reduces the spatial resolution of the point cloud and increases the feature dimension, and then the L-level feature propagation module progressively restores it to the original point-level resolution and fuses the reflection intensity attribute features. Multi-scale point-level features are obtained. ,in For the first Multi-scale point-level features of individual points; S2.3 Build a local anomaly enhancement module to enhance the coordinates of single-ping sonar point clouds. Input the local anomaly enhancement module to calculate 5D local anomaly features for each point. ; 5-dimensional local anomaly feature vectors of all points Mapped to high-dimensional anomaly embedding features ; S2.4 Embedding high-dimensional anomalies into features Multi-scale point-level features Residual fusion is performed to form anomaly enhancement features , For the first Anomaly enhancement features at individual points; S2.5, Based on anomaly enhancement features After the hierarchical segmentation head performs both coarse and fine segmentation, the coarse segmentation head outputs the terrain probability and structure probability, while the fine segmentation head outputs the normal structure probability and defect probability, resulting in the... The probability of three classifications at each point , ; S2.6, Introduce a disease probability threshold United Nations The output of the three-class probability of the nth point. Point-level prediction label for each point : , in, ; Finally, obtain the point-level predicted labels for all points: .
4. The method for real-time detection of single-Ping sonar point clouds for underwater bridge structural defects according to claim 3, characterized in that, S2.1 specifically includes the following steps: A fixed number of points are sampled from the input single-Ping sonar point cloud sample to obtain... From each point, we obtain the sampled single-Ping sonar point cloud. : , Point cloud coordinate center for: , Scale normalization factor for: , No. Normalized coordinates of points for: , in To ensure that the fraction is a very small number, preventing the fraction from having a denominator of 0; Mean sound intensity and standard deviation They are respectively: , , No. Normalized sound intensity at individual points for: , Finally, the preprocessed single-ping sonar point cloud is obtained. for: , Output the coordinates of a single Ping sonar point cloud. and reflection intensity attribute characteristics ,in , .
5. The method for real-time detection of single-Ping sonar point clouds for underwater bridge structural defects according to claim 3, characterized in that, In S2.2, the PointNet++ backbone network is set to... Level Collection Abstraction Module The Middle The center point is Its radius The next neighborhood is The corresponding local features are represented as: , in, For neighboring points, These are the local coordinates of the neighboring points relative to the center point. For the first Level point features, preset It is an empty set; For the first Level 1 Shared multilayer perceptron at a given radius scale For channel-by-channel max pooling; For multiple scale radii Multi-scale local features are obtained by channel splicing: , In the formula, This is a feature channel splicing operation; This represents the total number of scale radii; No. Level Collection Abstraction Module Output features are , , For the first Level Collection Abstraction Module The number of points in the middle, ; when At that time, using the first Level feature propagation module For the first Output features of the hierarchical set abstraction module and the Output features of the hierarchical set abstraction module Perform fusion processing to obtain the first Level fusion features ; when At that time, using the first Level feature propagation module For the first Level fusion features and the Output features of the hierarchical set abstraction module Perform fusion processing to obtain the first Level fusion features ; when At that time, the first-level feature propagation module is used. For the second level of fusion features and reflection intensity attribute characteristics The fusion process is performed to obtain multi-scale point-level features. .
6. The method for real-time detection of single-Ping sonar point clouds for underwater bridge structural defects according to claim 3, characterized in that, S2.3 specifically includes the following steps: A local anomaly enhancement module was built to enhance the coordinates of a single Ping sonar point cloud. Input the local anomaly enhancement module to calculate 5D local anomaly features for each point. ; For point First search its Nearest neighbor points: , in, Represented as the k-nearest neighbor algorithm for point clouds; Neighborhood Center for: , in For point The k-nearest neighbor in the neighborhood of The three-dimensional coordinates of each point; Center offset for: , in, These are the three coordinate components offset from the center; Distance residual for: , The neighborhood covariance matrix is: , Perform eigenvalue decomposition on the covariance matrix: , In the formula, For the first The local neighborhood covariance matrix corresponding to each point The One eigenvalue; For eigenvalues The corresponding unit eigenvector; Let be the index of the eigenvalue and its corresponding eigenvector, and ; Take the eigenvector corresponding to the smallest eigenvalue as the local plane normal vector. : , The local plane residual is: , Construct the first 5D local anomaly feature vector of a point : , And obtain the 5D local anomaly feature matrix of all points. ; 5-dimensional local anomaly feature vectors of all points Mapped to high-dimensional anomaly embedding features First, the first 5D local anomaly feature vector of a point Input small multilayer perceptron , mapped to the first High-dimensional anomaly embedding features of points: , And obtain the high-dimensional anomaly embedding features of all points. , .
7. The method for real-time detection of single-Ping sonar point clouds for underwater bridge structural defects according to claim 3, characterized in that, S2.5 specifically includes the following steps: For the The coarse segmentation head first outputs two raw scores for each point, indicating whether it belongs to the terrain region or the structure region. : , in, ; This represents the learnable weight matrix in the coarse segmentation head; It is the classification bias vector of the coarse segmentation head. This is the original classification score vector output by the coarse segmentation head, with its two components corresponding to the terrain region and the structure region, respectively. Then through The activation function normalizes the two original classification scores into a probability distribution, thus obtaining the first... The probability that a point belongs to terrain or structure, coarsely segmented probability vector. Right now: , in, ; The probability of predicting a feature as terrain during the coarse segmentation stage. The probability of predicting a structure during the coarse segmentation stage; For the At each point, the fine-grained segmentation head first outputs two raw scores: one for the normal structural region and one for the diseased region. : , in, This represents the learnable weight matrix in the fine-grained head. It is the classification bias vector of fine segmentation. This is the original classification score vector output by the fine segmentation head, with two vectors corresponding to normal structure and disease, respectively; Then through The function normalizes the two original classification scores into a probability distribution, thus obtaining the first... The probability that a point belongs to a normal structure or a disease; finely segment the probability vector. Right now: , in: , Predicting the first stage of fine segmentation The probability that a point has a normal structure; Predicting the first stage of fine segmentation The probability that each point belongs to a disease; Obtain the The probability of three-class classification for the i-th point, the i-th Terrain probability at each point for: , No. The probability of normal structure at each point for: , No. The probability of disease at each point for: , No. The three-class probability vector of each point : 。 8. The method for real-time detection of single-Ping sonar point clouds for underwater bridge structural defects according to claim 7, characterized in that, S3 specifically includes the following steps: Coarse segmentation supervision label Defined as: , in, Indicates terrain, This refers to structures, which include normal structures and defects; The coarse segmentation loss uses the weighted cross-entropy loss function: , In the formula, Coarse segmentation probability vector Medium and coarse segmentation supervision labels The corresponding predicted probability; For coarse segmentation of category weights; fine-segmented supervision labels Defined as: , in, Indicates a normal structure. Indicates disease; The fine-segmentation loss uses the Focal loss function: , in The number of point clouds involved in the fine segmentation; For focusing parameters; To further refine the category weights; For fine-segment probability vectors Medium and fine segmentation supervision labels The corresponding predicted probability; Coarse segmentation loss and fine segmentation loss constitute the hierarchical training loss of the HAE-PointNet++ network: , In the formula, These are the weighting coefficients for the coarse segmentation loss. The weight coefficients for the fine-grained segmentation loss are determined by minimizing the hierarchical training loss. Update the network parameters to obtain the trained HAE-PointNet++ network.
9. An electronic device, characterized in that, Including processor and storage media; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the single-Ping sonar point cloud real-time detection method for underwater structural defects of bridges as described in any one of claims 1-8.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the steps of the single-Ping sonar point cloud real-time detection method for underwater structural defects of bridges as described in any one of claims 1-8.
