Bridge underwater three-dimensional sonar point cloud denoising method, system and device based on deep learning and medium

Abstract

The invention discloses a bridge underwater three-dimensional sonar point cloud denoising method, system and device based on deep learning and a medium, and the method comprises the following steps: constructing a three-dimensional sonar point cloud denoising network, the three-dimensional sonar point cloud de-noising network comprises a de-noising device module composed of a point cloud feature extraction module, a noise removal module, a manifold up-sampling module and a perception consistency loss function module. The de-noising device module is used as a generator and forms an adversarial generation type network main body structure with a de-noising effect discriminator module. A bridge underwater sonar point cloud denoising training set is generated, the training set is composed of a plurality of actually measured point clouds, artificial denoising and unit sphere normalization processing are carried out, and Gaussian noise is added to simulate noise of the bridge underwater sonar point clouds; and training the three-dimensional sonar point cloud denoising network, inputting the noise-containing point cloud into the trained three-dimensional sonar point cloud denoising network, and outputting the denoised point cloud. According to the method, the noise of the underwater sonar point cloud of the bridge can be effectively removed, and the quality of the sonar point cloud is improved.

Description

Technical Field

[0001] This invention belongs to the field of point cloud processing technology in the bridge industry, and particularly relates to a method, system, device and medium for denoising underwater three-dimensional sonar point clouds of bridges based on deep learning. Background Technology

[0002] The underwater foundations and scour of bridges need to be inspected regularly. Using three-dimensional sonar to regularly inspect the underwater foundations of bridges and the surrounding riverbed, and to obtain three-dimensional point clouds to construct a digital morphological model of the bridge foundations and scour pits, can effectively monitor the development of related defects and take corresponding maintenance measures in a timely manner.

[0003] However, due to the complex underwater environment and the mobile sonar scanning mode, a significant amount of noise is unavoidable in sonar point clouds. Based on their morphology, noise can be categorized into outlier noise and fuzzy noise. Outlier noise is typically caused by floating debris and aquatic organisms, manifesting as anomalous point clouds far removed from the actual surface of the point cloud—i.e., outlier values. Fuzzy noise is usually caused by factors such as insufficient stitching accuracy of mobile sonar scanning data, appearing as point clouds of a certain thickness surrounding the actual surface of the structure. This noise severely disrupts the underlying geometry of the sonar point cloud, increasing the uncertainty of sonar point cloud measurements and consequently interfering with subsequent downstream tasks such as scour morphology analysis and basic disease observation.

[0004] Conventional sonar acquisition software typically uses filtering-based denoising methods, which essentially remove outliers from sonar point clouds based on statistical features. While these methods can remove outlier noise, they struggle with blurry noise within sonar point clouds. Existing traditional point cloud denoising algorithms exhibit poor generalization and are inadequate for tasks involving significant noise removal from sonar point clouds. Current deep learning-based denoising methods are generally developed for high-quality laser point clouds. However, sonar point clouds differ from laser point clouds; their quality is lower and their point density is sparse, making it difficult to directly transfer relevant deep learning networks to denoise bridge sonar point clouds. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to provide a deep learning-based method for denoising underwater 3D sonar point clouds of bridges, which can effectively remove noise from underwater sonar point clouds of bridges and improve the quality of sonar point clouds.

[0006] The second objective of this invention is to provide a deep learning-based underwater three-dimensional sonar point cloud denoising system for bridges.

[0007] A third objective of this invention is to provide an electronic device.

[0008] A fourth objective of this invention is to provide a computer-readable storage medium.

[0009] Technical Solution: To achieve the above objectives, this invention discloses a method for denoising underwater 3D sonar point clouds of bridges based on deep learning, comprising the following steps: (1) Construct a three-dimensional sonar point cloud denoising network. The three-dimensional sonar point cloud denoising network includes a denoiser module consisting of a point cloud feature extraction module, a noise removal module, a manifold upsampling module, and a perceptual consistency loss function module. The denoiser module is used as a generator and a denoising effect discriminator module to form the main structure of the adversarial generative network. The denoising module extracts features from the sonar point cloud, denoises, and samples to generate a clean point cloud. It forms an adversarial generative network (GAN) with a denoising effect discriminator module based on fully connected layers. The denoising module acts as the generator in the GAN, and during network training, the denoising effect discriminator module references the clean point cloud to distinguish point cloud noise. The point cloud feature extraction module extracts sonar point cloud features, and the noise removal module removes outlier and blurry noise from the sonar point cloud. The manifold upsampling module upsamples the point cloud to avoid a reduction in the number of sonar point clouds and feature loss due to noise removal. The perceptual consistency loss function module evaluates the difference between the denoised point cloud and the clean point cloud. (2) Generate a bridge underwater sonar point cloud denoising training set. The bridge underwater sonar point cloud denoising training set consists of multiple measured point clouds. Each point cloud is artificially denoised and normalized to a unit sphere. Then, Gaussian noise is added to simulate the noise of the bridge underwater sonar point cloud. (3) Train the three-dimensional sonar point cloud denoising network. Input the noisy point cloud into the trained three-dimensional sonar point cloud denoising network and output the denoised point cloud result.

[0010] Optionally, the point cloud feature extraction module first inputs a noisy n-point cloud containing n points. The features output from the first dynamic graph convolution module are input into the first 3D graph pooling module. After pooling, the features are sequentially input into the second and third dynamic graph convolution modules. The resulting features are then input into the second 3D graph pooling module. After pooling, the features are input into the fourth dynamic graph convolution module. The output features are input into the first upsampling module. The sampled features are input into the fifth dynamic graph convolution module. The output fifth feature is input into the sixth dynamic graph convolution module. The output sixth feature is sequentially input into the seventh, eighth, and ninth dynamic graph convolution modules. The output ninth feature and the sixth feature are added together and then input into the tenth dynamic graph convolution module. The output tenth feature and the fifth feature are added together and then input into the eleventh dynamic graph convolution module. The output features are then input into the second upsampling module to obtain the final extracted features.

[0011] Optionally, each dynamic graph convolutional module in the point cloud feature extraction module uses the same structure. The dynamic graph convolutional module first inputs sonar point clouds or features. The sonar point clouds or features are concatenated with the features extracted by the first dynamic graph convolutional layer to output concatenated features. The concatenated features are input to the first fully connected layer, the concatenated features are input to the CBAM attention layer and the second dynamic graph convolutional layer to output enhanced features. The enhanced features are concatenated with the output features of the first fully connected layer to obtain the output features of the dynamic graph convolutional module. The CBAM attention layer is used to enhance the network's local morphological perception ability. The first and second dynamic graph convolutional layers in the dynamic graph convolutional module are both used to learn the features of noisy sonar point clouds, assuming The F-dimensional features of a noisy sonar point cloud are represented. Points originating from sonar point clouds The size of the neighborhood is M, and Centered on learnable dynamic graph convolution kernel For the central point cloud The expression for the dynamic graph convolution calculation process is as follows: , , in Represents the eigenvector. Represents the weight vector. This represents the direction vector of the convolution kernel. Representing the eigenvector and convolution weight vector The inner product, This represents the direction vector of the point cloud neighborhood, which is formed by the point cloud within the neighborhood of the convolution kernel. With the center point Calculate the vector difference; Represents the neighborhood direction vector of the point cloud and convolution kernel direction vector The inner product; Represents the neighborhood direction vector of the point cloud The length of the mold, Represents the direction vector of the convolution kernel The length of the module.

[0012] Optionally, in the noise removal module, the output features of the point cloud feature extraction module are first input into the neighbor attention layer, and then the noise features are scored based on the fully connected layer. The higher the score, the greater the probability of it being outlier noise. Based on the score, 50% of the point cloud is removed to obtain the preliminary denoised point cloud. The preliminary denoised point cloud is input into the second fully connected layer to predict the corrected displacement value. The corrected displacement value is added to the preliminary denoised point cloud to obtain the fully denoised point cloud. The neighbor attention layer in the noise removal module improves the reliability of point cloud noise evaluation scores based on the correlation between the point cloud and its local neighborhood. This is applied to the features of the i-th point cloud output by the point cloud feature extraction module. , For querying the matrix, keywords This represents the feature difference between the i-th point cloud and its neighboring point clouds. Then the neighbor attention score The calculation formula is: .

[0013] Optionally, the manifold upsampling module copies and stitches together the initial denoised point cloud to obtain the final point cloud. As input to the manifold upsampling layer, the manifold upsampling layer upsamples the high-density point cloud by learning the local manifold representation of the sonar point cloud; in the manifold upsampling layer, for the input point cloud set Each point cloud Local manifold using parameters Multilayer perceptron representation From the eigenvector and two-dimensional coordinate pairs sampled from the normalized parameter domain Parameterization; then the point cloud is upsampled from the manifold. The calculation formula is: , The manifold-upsampled point cloud is input into the third fully connected layer. The output is added to the features obtained after copying and stitching the fully denoised point cloud to obtain the final output upsampled denoised point cloud set. .

[0014] Optionally, the perceptual consistency loss function module comprises a weighted average of the chamfer distance and the top-k approximate Hausdorff distance. The calculation formula is: , in Indicates the chamfer distance. This indicates that the top-k approximate Hausdorff distance is... and Let represent the weighting coefficients of the perceived consistency loss function, and satisfy . ; where the chamfer distance The calculation formula is: , in This represents the upsampled, denoised point cloud set. This represents the i-th point in the upsampled, denoised point cloud set; This indicates a clean sonar point cloud and point set. Let i represent the i-th point in the clean sonar point cloud set; The perceptual consistency loss function uses the top-k approximation of Hausdorff distance. The calculation formula is: , , , in This represents the distance calculated from the clean sonar point cloud to the denoised point cloud. The distance from the denoised point cloud to the clean sonar point cloud The former The average of the maximum distances.

[0015] Optionally, the input to the denoising effect discriminator module is the upsampled denoised point cloud. The denoising effect discriminator module includes a fourth fully connected layer, a fifth fully connected layer, and a sixth fully connected layer. The output of each fully connected layer is pooled and concatenated as the output feature of the denoising effect discriminator module. The loss function of the denoising effect discriminator module is calculated using the following formula: , This is the output of the noise reduction effect discriminator module; Adversarial loss function The loss function of the denoising effect discriminator module and the perceptual consistency loss function are weighted and calculated using the following formula: , In the formula and These are weighted parameters, and they satisfy... .

[0016] Based on the same inventive concept, this invention discloses a deep learning-based underwater 3D sonar point cloud denoising system for bridges, comprising: The denoising network construction module is used to construct a 3D sonar point cloud denoising network. The 3D sonar point cloud denoising network includes a denoiser module consisting of a point cloud feature extraction module, a noise removal module, a manifold upsampling module, and a perceptual consistency loss function module. The denoiser module is used as a generator and a denoising effect discriminator module to form an adversarial generative network structure. The denoising module extracts features from the sonar point cloud, denoises, and samples to generate a clean point cloud. It forms an adversarial generative network (GAN) with a denoising effect discriminator module based on fully connected layers. The denoising module acts as the generator in the GAN, and during network training, the denoising effect discriminator module references the clean point cloud to distinguish point cloud noise. The point cloud feature extraction module extracts sonar point cloud features, and the noise removal module removes outlier and blurry noise from the sonar point cloud. The manifold upsampling module upsamples the point cloud to avoid a reduction in the number of sonar point clouds and feature loss due to noise removal. The perceptual consistency loss function module evaluates the difference between the denoised point cloud and the clean point cloud. The training data generation module is used to generate a training set for denoising underwater sonar point clouds of bridges. The training set consists of multiple measured point clouds. Each point cloud has undergone manual denoising and unit sphere normalization, and then Gaussian noise is added to simulate the noise of the underwater sonar point clouds of bridges. The denoising point cloud output module is used to train a 3D sonar point cloud denoising network. It takes a noisy point cloud as input to the trained 3D sonar point cloud denoising network and outputs the denoised point cloud result.

[0017] Based on the same inventive concept, the present invention provides an electronic device including a processor and a storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method described above.

[0018] Based on the same inventive concept, the computer-readable storage medium of the present invention stores a computer program thereon, characterized in that the program, when executed by a processor, implements the steps of the method described above.

[0019] Beneficial Effects: Compared with the prior art, the present invention has the following significant advantages: The three-dimensional sonar point cloud denoising network of the present invention has good feature extraction performance for sparse and high-noise sonar point clouds, and the noise removal module can effectively perform targeted denoising for different types of noise in sonar point clouds. The use of the manifold upsampling module solves the filtering problem that may lead to a reduction in the number of point clouds due to denoising. The perceptual consistency loss function module accurately evaluates the difference between the denoised point cloud and the clean point cloud, making the denoised point cloud closer to the clean point cloud. The denoising effect discriminator module and the denoiser module form an adversarial generative network structure to enhance the denoising performance of sonar point clouds. This network overcomes the problem of difficult removal of noise from sonar point clouds, improves the quality of sonar point clouds, and reduces the uncertainty of sonar point cloud measurements. Attached Figure Description

[0020] Figure 1 This is a flowchart of the present invention; Figure 2This is a diagram of the three-dimensional sonar point cloud denoising network structure in this invention; Figure 3 This is a schematic diagram of the dynamic graph convolution module in the feature extraction module of this invention; Figure 4 This is a schematic diagram of the feature extraction module in this invention; Figure 5 This is a schematic diagram of the noise removal module in this invention; Figure 6 This is a schematic diagram of the manifold upsampling module in this invention; Figure 7 This is a schematic diagram of the manifold denoising effect discriminator module in this invention; Figure 8 This is a schematic diagram of the underwater sonar point cloud dataset after denoising and normalization processing in this invention. Figure 9 This is a qualitative result diagram of the underwater sonar scour pit topographic point cloud denoising in this invention; Figure 10 This is a qualitative result diagram of the point cloud denoising of underwater sonar pile foundation in this invention; Figure 11 This is a point cloud profile result of underwater sonar point cloud denoising in this invention. Detailed Implementation

[0021] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0022] like Figure 1 As shown, this invention discloses a method for denoising underwater 3D sonar point clouds of bridges based on deep learning, comprising the following steps: (1) Construct a 3D sonar point cloud denoising network. The 3D sonar point cloud denoising network includes a denoiser module consisting of a point cloud feature extraction module, a noise removal module, a manifold upsampling module, and a perceptual consistency loss function module. The denoiser module is used as a generator and a denoising effect discriminator module to form the main structure of the adversarial generative network, such as Figure 2 As shown.

[0023] The point cloud feature extraction module is used to extract sonar point cloud features, and the noise removal module is used to remove outlier noise and blur noise from the sonar point cloud. The manifold upsampling module is used to upsample the point cloud to avoid the reduction in the number of sonar point clouds and feature loss caused by noise removal. The perceptual consistency loss function module is used to evaluate the difference between the denoised point cloud and the clean point cloud and enhance the denoising performance of the network. The denoiser module is used to extract features from the sonar point cloud, denoise, and sample to generate a clean point cloud. The denoiser module is the generator in the adversarial generative network. It forms an adversarial generative network with the denoising effect discriminator module based on fully connected layers. During the network training process, the denoising effect discriminator module refers to the clean point cloud to distinguish point cloud noise, thereby enhancing the network's noise removal effect on the sonar point cloud.

[0024] like Figure 4 As shown, the point cloud feature extraction module first inputs a noisy n-point cloud containing n points. The features output from the first dynamic graph convolution module are input into the first 3D graph pooling module. After pooling, the features are sequentially input into the second and third dynamic graph convolution modules. The resulting features are then input into the second 3D graph pooling module. After pooling, the features are input into the fourth dynamic graph convolution module. The output features are input into the first upsampling module. The sampled features are input into the fifth dynamic graph convolution module. The output fifth feature is input into the sixth dynamic graph convolution module. The output sixth feature is sequentially input into the seventh, eighth, and ninth dynamic graph convolution modules. The output ninth feature and the sixth feature are added together and then input into the tenth dynamic graph convolution module. The output tenth feature and the fifth feature are added together and then input into the eleventh dynamic graph convolution module. The output features are then input into the second upsampling module to obtain the final extracted features.

[0025] The parameters for each layer in the point cloud feature extraction module are set as follows: The first dynamic graph convolutional layer has an input feature map feature dimension of 3 and an output feature map feature dimension of 32; the second dynamic graph convolutional layer has an input feature map feature dimension of 32 and an output feature map feature dimension of 64; the third dynamic graph convolutional layer has an input feature map feature dimension of 64 and an output feature map feature dimension of 128; the fourth dynamic graph convolutional layer has an input feature map feature dimension of 128 and an output feature map feature dimension of 320; the fifth dynamic graph convolutional layer has an input feature map feature dimension of 320 and an output feature map feature dimension of 160; the sixth dynamic graph convolutional layer has an input feature map feature dimension of 160 and an output feature map feature dimension of 80; the seventh dynamic graph convolutional layer has an input feature map feature dimension of 80 and an output feature map feature dimension of 6. 4; The eighth dynamic graph convolutional layer has an input feature map feature dimension of 64 and an output feature map feature dimension of 64; the ninth dynamic graph convolutional layer has an input feature map feature dimension of 64 and an output feature map feature dimension of 80; the tenth dynamic graph convolutional layer has an input feature map feature dimension of 80 and an output feature map feature dimension of 160; the eleventh dynamic graph convolutional layer has an input feature map feature dimension of 160 and an output feature map feature dimension of 320; the first and second 3D graph pooling modules both include max pooling of 3D graph point cloud features and random downsampling of the point cloud, used to aggregate features at various scales to enhance the robustness of the point cloud feature extraction module and reduce noise and density inhomogeneity interference, wherein the pooling point cloud convolutional kernel size is 4 and the downsampling rate is 0.9; the first and second upsampling modules have an upsampling rate of 1.11.

[0026] like Figure 3 As shown, each dynamic graph convolutional module uses the same structure. The dynamic graph convolutional module first inputs sonar point clouds or features. The sonar point clouds or features are concatenated with the features extracted by the first dynamic graph convolutional layer to output concatenated features. The concatenated features are input to the first fully connected layer, the concatenated features are input to the CBAM attention layer and the second dynamic graph convolutional layer to output enhanced features. The enhanced features are concatenated with the output features of the first fully connected layer to obtain the output features of the dynamic graph convolutional module. The CBAM attention layer is used to enhance the network's local morphological perception ability.

[0027] The first and second dynamic graph convolutional layers in each dynamic graph convolutional module are used to learn features of noisy sonar point clouds, assuming The F-dimensional features of a noisy sonar point cloud are represented. Points originating from sonar point clouds The size of the neighborhood is M, and Centered on learnable dynamic graph convolution kernel For the central point cloud The expression for the dynamic graph convolution calculation process is as follows: , , in Represents the eigenvector. Represents the weight vector. This represents the direction vector of the convolution kernel. Representing the eigenvector and convolution weight vector The inner product, This represents the direction vector of the point cloud neighborhood, which is formed by the point cloud within the neighborhood of the convolution kernel. With the center point Calculate the vector difference; Represents the neighborhood direction vector of the point cloud and convolution kernel direction vector The inner product; Represents the neighborhood direction vector of the point cloud The length of the mold, Represents the direction vector of the convolution kernel The length of the module.

[0028] like Figure 5 As shown, the noise removal module is used to remove outlier noise and blur noise from the sonar point cloud. In the noise removal module, the output features of the point cloud feature extraction module are first input into the neighbor attention layer. Then, the noise features are scored based on the fully connected layer. The higher the score, the greater the probability of it being outlier noise. 50% of the point cloud is removed according to the score to obtain the preliminary denoised point cloud. The preliminary denoised point cloud is input into the second fully connected layer to predict the corrected displacement value. The corrected displacement value is added to the preliminary denoised point cloud to obtain the fully denoised point cloud.

[0029] The neighbor attention layer in the noise removal module improves the reliability of point cloud noise evaluation scores based on the correlation between the point cloud and its local neighborhood. This is applied to the features of the i-th point cloud output by the point cloud feature extraction module. , For querying the matrix, keywords This represents the feature difference between the i-th point cloud and its neighboring point clouds. Then the neighbor attention score The calculation formula is: .

[0030] like Figure 6 As shown, the manifold upsampling module is used to upsample the point cloud to avoid the reduction in the number of sonar point clouds and the loss of features due to noise removal. In the manifold upsampling module, the preliminary denoised point cloud is copied and stitched together to obtain the final point cloud. As input to the manifold upsampling layer, the manifold upsampling layer upsamples the high-density point cloud by learning the local manifold representation of the sonar point cloud; in the manifold upsampling layer, for the input point cloud set Each point cloud Local manifold using parameters Multilayer perceptron representation From the eigenvector and two-dimensional coordinate pairs sampled from the normalized parameter domain Parameterization; then the point cloud is upsampled from the manifold. The calculation formula is: , The manifold-upsampled point cloud is input into the third fully connected layer. The output is added to the features obtained after copying and stitching the fully denoised point cloud to obtain the final output upsampled denoised point cloud set. .

[0031] The perceptual consistency loss function module is used to evaluate the difference between the denoised point cloud and the clean point cloud, making the denoised point cloud closer to the clean point cloud. To effectively constrain the training of the network for denoising and suppress blurry noise distributed around the real surface, the perceptual consistency loss function is composed of a weighted sum of chamfer distance and top-k approximate Hausdorff distance. The calculation formula is: , in Indicates the chamfer distance. This indicates that the top-k approximate Hausdorff distance is... and Let represent the weighting coefficients of the perceived consistency loss function, and satisfy . ; where the chamfer distance The calculation formula is: , in This represents the upsampled, denoised point cloud set. This represents the i-th point in the upsampled, denoised point cloud set; This indicates a clean sonar point cloud and point set. Let i represent the i-th point in the clean sonar point cloud set.

[0032] Standard Hausdorff distance is extremely sensitive to outliers and its lack of differentiability negatively impacts deep learning model training. To improve robustness, the perceptual consistency loss function uses a top-k approximation of Hausdorff distance. The calculation formula is: , , , in This represents the distance calculated from the clean sonar point cloud to the denoised point cloud. The distance from the denoised point cloud to the clean sonar point cloud The former The average of the maximum distances.

[0033] like Figure 7 As shown, the input to the denoising effect discriminator module is the upsampled denoised point cloud. The denoising effect discriminator module includes a fourth fully connected layer, a fifth fully connected layer, and a sixth fully connected layer. The output of each fully connected layer is pooled and concatenated as the output feature of the denoising effect discriminator module. The loss function of the denoising effect discriminator module is calculated as follows: , This is the output of the noise reduction effect discriminator module.

[0034] Adversarial loss function The loss function of the denoising effect discriminator module and the perceptual consistency loss function are weighted and calculated using the following formula: , In the formula and These are weighted parameters, and they satisfy... .

[0035] (2) Generate a bridge underwater sonar point cloud denoising training set. The bridge underwater sonar point cloud denoising training set consists of multiple measured point clouds. Each point cloud has undergone manual denoising and unit sphere normalization. The underwater sonar point cloud data obtained after processing is as follows: Figure 8 As shown, Gaussian noise is then added to simulate the noise of the underwater sonar point cloud of the bridge.

[0036] (2.1) The bridge underwater sonar point cloud denoising training set consists of multiple measured point clouds. Each point cloud has undergone manual denoising and unit sphere normalization. Manual denoising involves using CloudCompare software to manually remove noise from each point cloud by comparing it with bridge design drawings and related data, so as to meet the quality requirements of actual bridge underwater inspection. Then, unit sphere normalization is performed by shifting the center of each point cloud to the origin of the coordinate system, and then calculating the maximum value of all points in the point cloud to the origin. The coordinates of each point are divided by the maximum value to obtain the normalized point cloud. (2.2) Based on the normalized point cloud, Gaussian noise passing through the diagonal of the bounding box with a standard deviation of 1%~6% is added to simulate the noise of the underwater sonar point cloud of the bridge.

[0037] (3) The three-dimensional sonar point cloud denoising network was trained. The network weights were optimized based on the gradient descent method. The Adam optimizer was used with a learning rate of 0.0005. During the training process, the learning rate was dynamically adjusted according to the network loss using the ReduceLROnPlateau regulator. The learning rate decay factor was set to 0.5, the patience period was set to 10, and the minimum learning rate was set to 0.000001. The training was considered complete when the number of training rounds reached 200 or the loss stabilized without significant fluctuations.

[0038] (4) Input the noisy point cloud into the trained 3D sonar point cloud denoising network and output the denoised point cloud result.

[0039] like Figure 9 and Figure 10 As shown, this is a qualitative result diagram of underwater sonar point cloud denoising achieved by the present invention. Figure 11 The image shown is a point cloud profile resulting from the underwater sonar point cloud denoising method implemented according to this invention. From... Figure 9 , Figure 10 and Figure 11 As can be seen from this, the present invention can effectively remove noise from underwater sonar point clouds of bridges, reduce the thickness of the point cloud profile, reduce the uncertainty of point cloud measurement during the scanning process, and improve the accuracy of sonar point clouds.

[0040] Example 2: This invention discloses a deep learning-based underwater 3D sonar point cloud denoising system for bridges, comprising: The denoising network construction module is used to construct a 3D sonar point cloud denoising network. The 3D sonar point cloud denoising network includes a denoiser module consisting of a point cloud feature extraction module, a noise removal module, a manifold upsampling module, and a perceptual consistency loss function module. The denoiser module is used as a generator and a denoising effect discriminator module to form the main structure of the adversarial generative network.

[0041] The denoising module extracts features from the sonar point cloud, denoises, and samples to generate a clean point cloud. It forms an adversarial generative network (GAN) with a denoising effect discriminator module based on fully connected layers. The denoising module acts as the generator in the GAN, and during network training, the denoising effect discriminator module references the clean point cloud to distinguish point cloud noise. The point cloud feature extraction module extracts sonar point cloud features, and the noise removal module removes outlier and blurry noise from the sonar point cloud. The manifold upsampling module upsamples the point cloud to avoid a reduction in the number of sonar point clouds and feature loss due to noise removal. The perceptual consistency loss function module evaluates the difference between the denoised point cloud and the clean point cloud.

[0042] The point cloud feature extraction module first inputs a noisy n-point cloud containing n points. The features output from the first dynamic graph convolution module are input into the first 3D graph pooling module. After pooling, the features are sequentially input into the second and third dynamic graph convolution modules. The resulting features are then input into the second 3D graph pooling module. After pooling, the features are input into the fourth dynamic graph convolution module. The output features are input into the first upsampling module. The sampled features are input into the fifth dynamic graph convolution module. The output fifth feature is input into the sixth dynamic graph convolution module. The output sixth feature is sequentially input into the seventh, eighth, and ninth dynamic graph convolution modules. The output ninth feature and the sixth feature are added together and then input into the tenth dynamic graph convolution module. The output tenth feature and the fifth feature are added together and then input into the eleventh dynamic graph convolution module. The output features are then input into the second upsampling module to obtain the final extracted features.

[0043] The parameters for each layer in the point cloud feature extraction module are set as follows: The first dynamic graph convolutional layer has an input feature map feature dimension of 3 and an output feature map feature dimension of 32; the second dynamic graph convolutional layer has an input feature map feature dimension of 32 and an output feature map feature dimension of 64; the third dynamic graph convolutional layer has an input feature map feature dimension of 64 and an output feature map feature dimension of 128; the fourth dynamic graph convolutional layer has an input feature map feature dimension of 128 and an output feature map feature dimension of 320; the fifth dynamic graph convolutional layer has an input feature map feature dimension of 320 and an output feature map feature dimension of 160; the sixth dynamic graph convolutional layer has an input feature map feature dimension of 160 and an output feature map feature dimension of 80; the seventh dynamic graph convolutional layer has an input feature map feature dimension of 80 and an output feature map feature dimension of 6. 4; The eighth dynamic graph convolutional layer has an input feature map feature dimension of 64 and an output feature map feature dimension of 64; the ninth dynamic graph convolutional layer has an input feature map feature dimension of 64 and an output feature map feature dimension of 80; the tenth dynamic graph convolutional layer has an input feature map feature dimension of 80 and an output feature map feature dimension of 160; the eleventh dynamic graph convolutional layer has an input feature map feature dimension of 160 and an output feature map feature dimension of 320; the first and second 3D graph pooling modules both include max pooling of 3D graph point cloud features and random downsampling of the point cloud, used to aggregate features at various scales to enhance the robustness of the point cloud feature extraction module and reduce noise and density inhomogeneity interference, wherein the pooling point cloud convolutional kernel size is 4 and the downsampling rate is 0.9; the first and second upsampling modules have an upsampling rate of 1.11.

[0044] Each dynamic graph convolutional module uses the same structure. The dynamic graph convolutional module first inputs sonar point clouds or features. The sonar point clouds or features are concatenated with the features extracted by the first dynamic graph convolutional layer to output concatenated features. The concatenated features are input to the first fully connected layer, the concatenated features are input to the CBAM attention layer and the second dynamic graph convolutional layer to output enhanced features. The enhanced features are concatenated with the output features of the first fully connected layer to obtain the output features of the dynamic graph convolutional module. The CBAM attention layer is used to enhance the network's local morphological perception ability.

[0045] The first and second dynamic graph convolutional layers in each dynamic graph convolutional module are used to learn features of noisy sonar point clouds, assuming The F-dimensional features of a noisy sonar point cloud are represented. Points originating from sonar point clouds The size of the neighborhood is M, and Centered on learnable dynamic graph convolution kernel For the central point cloud The expression for the dynamic graph convolution calculation process is as follows: , , in Represents the eigenvector. Represents the weight vector. This represents the direction vector of the convolution kernel. Representing the eigenvector and convolution weight vector The inner product, This represents the direction vector of the point cloud neighborhood, which is formed by the point cloud within the neighborhood of the convolution kernel. With the center point Calculate the vector difference; Represents the neighborhood direction vector of the point cloud and convolution kernel direction vector The inner product; Represents the neighborhood direction vector of the point cloud The length of the mold, Represents the direction vector of the convolution kernel The length of the module.

[0046] like Figure 5 As shown, the noise removal module is used to remove outlier noise and blur noise from the sonar point cloud. In the noise removal module, the output features of the point cloud feature extraction module are first input into the neighbor attention layer. Then, the noise features are scored based on the fully connected layer. The higher the score, the greater the probability of it being outlier noise. 50% of the point cloud is removed according to the score to obtain the preliminary denoised point cloud. The preliminary denoised point cloud is input into the second fully connected layer to predict the corrected displacement value. The corrected displacement value is added to the preliminary denoised point cloud to obtain the fully denoised point cloud.

[0047] The neighbor attention layer in the noise removal module improves the reliability of point cloud noise evaluation scores based on the correlation between the point cloud and its local neighborhood. This is applied to the features of the i-th point cloud output by the point cloud feature extraction module. , For querying the matrix, keywords This represents the feature difference between the i-th point cloud and its neighboring point clouds. Then the neighbor attention score The calculation formula is: , like Figure 6 As shown, the manifold upsampling module is used to upsample the point cloud to avoid the reduction in the number of sonar point clouds and the loss of features due to noise removal. In the manifold upsampling module, the preliminary denoised point cloud is copied and stitched together to obtain the final point cloud. As input to the manifold upsampling layer, the manifold upsampling layer upsamples the high-density point cloud by learning the local manifold representation of the sonar point cloud; in the manifold upsampling layer, for the input point cloud set Each point cloud Local manifold using parameters Multilayer perceptron representation From the eigenvector and two-dimensional coordinate pairs sampled from the normalized parameter domain Parameterization; then the point cloud is upsampled from the manifold. The calculation formula is: , The manifold-upsampled point cloud is input into the third fully connected layer. The output is added to the features obtained after copying and stitching the fully denoised point cloud to obtain the final output upsampled denoised point cloud set. .

[0048] The perceptual consistency loss function module is used to evaluate the difference between the denoised point cloud and the clean point cloud, making the denoised point cloud closer to the clean point cloud. To effectively constrain the training of the network for denoising and suppress blurry noise distributed around the real surface, the perceptual consistency loss function is composed of a weighted sum of chamfer distance and top-k approximate Hausdorff distance. The calculation formula is: , in Indicates the chamfer distance. This indicates that the top-k approximate Hausdorff distance is... and Let represent the weighting coefficients of the perceived consistency loss function, and satisfy . ; where the chamfer distance The calculation formula is: , in This represents the upsampled, denoised point cloud set. This represents the i-th point in the upsampled, denoised point cloud set; This indicates a clean sonar point cloud and point set. Let i represent the i-th point in the clean sonar point cloud set.

[0049] Standard Hausdorff distance is extremely sensitive to outliers and its lack of differentiability negatively impacts deep learning model training. To improve robustness, the perceptual consistency loss function uses a top-k approximation of Hausdorff distance. The calculation formula is: , , , in This represents the distance calculated from the clean sonar point cloud to the denoised point cloud. The distance from the denoised point cloud to the clean sonar point cloud The former The average of the maximum distances.

[0050] The input to the denoising effect discriminator module is the upsampled denoised point cloud. The module includes a fourth, fifth, and sixth fully connected layer. The outputs of each fully connected layer are pooled and concatenated to form the output features of the denoising effect discriminator module. The loss function of the denoising effect discriminator module is calculated using the following formula: , This is the output of the noise reduction effect discriminator module.

[0051] Adversarial loss function The loss function of the denoising effect discriminator module and the perceptual consistency loss function are weighted and calculated using the following formula: , In the formula and These are weighted parameters, and they satisfy... .

[0052] The training data generation module is used to generate a bridge underwater sonar point cloud denoising training set. The bridge underwater sonar point cloud denoising training set consists of multiple measured point clouds. Each point cloud has been manually denoised and normalized to a unit sphere, and then Gaussian noise is added to simulate the noise of the bridge underwater sonar point cloud.

[0053] The bridge underwater sonar point cloud denoising training set consists of multiple measured point clouds. Each point cloud underwent manual denoising and unit sphere normalization. Manual denoising involved using CloudCompare software to manually remove noise from each point cloud by comparing it with bridge design drawings and related data, achieving the quality requirements for actual underwater bridge inspection. Unit sphere normalization was then performed by shifting the center of each point cloud to the origin, calculating the maximum value of all points in the point cloud relative to the origin, and dividing the coordinates of each point by the maximum value to obtain the normalized point cloud.

[0054] Based on the normalized point cloud, Gaussian noise with a standard deviation of 1% to 6% is added to simulate the noise of the underwater sonar point cloud of the bridge.

[0055] The denoised point cloud output module is used to train the 3D sonar point cloud denoising network. It inputs noisy point clouds into the trained network and outputs the denoised point cloud results. The network weights are optimized using gradient descent with the Adam optimizer and a learning rate of 0.0005. During training, the learning rate is dynamically adjusted based on the network loss using the ReduceLROnPlateau regulator, with a learning rate decay factor of 0.5, a patience period of 10, and a minimum learning rate of 0.000001. Training is considered complete when the number of training epochs reaches 200 or the loss stabilizes without significant fluctuations.

[0056] Example 3: An electronic device according to the present invention includes a processor and a storage medium; Storage media are used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method described above.

[0057] Example 4: The computer-readable storage medium of the present invention stores a computer program thereon, which, when executed by a processor, implements the steps of the method described above.

Claims

1. A method for denoising a bridge underwater three-dimensional sonar point cloud based on deep learning, characterized in that, The method comprises the following steps: (1) constructing a three-dimensional sonar point cloud denoising network, the three-dimensional sonar point cloud denoising network comprising a point cloud feature extraction module, a noise removal module, a manifold upsampling module, and a perceptual consistency loss function module to form a denoiser module, the denoiser module being used as a generator and a denoising effect discriminator module to form an adversarial generative network main structure; The denoiser module is used to extract the features of the sonar point cloud, remove noise, and sample to generate a clean point cloud. The denoiser module and the denoising effect discriminator module based on the full connection layer form an adversarial generative network. The denoiser module is the generator in the adversarial generative network. During the network training process, the denoising effect discriminator module refers to the clean point cloud to distinguish the point cloud noise. The point cloud feature extraction module is used to extract the features of the sonar point cloud, and the noise removal module is used to remove the outlier noise and fuzzy noise in the sonar point cloud. The manifold upsampling module is used to upsample the point cloud to avoid the reduction of the number of sonar point cloud points and the loss of features caused by noise removal. The perceptual consistency loss function module is used to evaluate the gap between the denoised point cloud and the clean point cloud. (2) generating a bridge underwater sonar point cloud denoising training set, the bridge underwater sonar point cloud denoising training set being composed of multiple measured point clouds, each point cloud being artificially denoised and unit sphere normalized, and then being added with Gaussian noise to simulate the noise of the bridge underwater sonar point cloud; (3) training the three-dimensional sonar point cloud denoising network, inputting the point cloud containing noise into the trained three-dimensional sonar point cloud denoising network, and outputting the denoised point cloud result.

2. The bridge underwater three-dimensional sonar point cloud denoising method based on deep learning according to claim 1, characterized in that: The point cloud feature extraction module first inputs a noisy sonar point cloud containing n points , and outputs a feature to a first 3D graph pooling module. The pooled feature is sequentially input to a second dynamic graph convolution module and a third dynamic graph convolution module. The obtained feature is input to a second 3D graph pooling module. The pooled feature is input to a fourth dynamic graph convolution module. The output feature is input to a first upsampling module. The sampled feature is input to a fifth dynamic graph convolution module. The fifth feature is input to a sixth dynamic graph convolution module. The output sixth feature is sequentially input to a seventh dynamic graph convolution module, an eighth dynamic graph convolution module, and a ninth dynamic graph convolution module. The ninth feature and the sixth feature are added and input to a tenth dynamic graph convolution module. The tenth feature and the fifth feature are added and input to an eleventh dynamic graph convolution module. The output feature is input to a second upsampling module to obtain the final extracted feature.

3. The bridge underwater three-dimensional sonar point cloud denoising method based on deep learning according to claim 2, characterized in that: Each dynamic graph convolution module in the point cloud feature extraction module uses the same structure. In the dynamic graph convolution module, the sonar point cloud or the feature is first inputted. The sonar point cloud or the feature is spliced with the feature extracted through the first dynamic graph convolution layer to output the spliced feature. The spliced feature is inputted into the first full connection layer. The spliced feature is inputted into the CBAM attention layer and the second dynamic graph convolution layer to output the enhanced feature. The enhanced feature is spliced with the output feature of the first full connection layer to obtain the output feature of the dynamic graph convolution module. The CBAM attention layer is used to enhance the local shape perception ability of the network. The first dynamic graph convolution layer and the second dynamic graph convolution layer in the dynamic graph convolution module are both used to learn features of the noisy sonar point cloud, assuming F-dimensional features representing the noisy sonar point cloud, representing a point from the sonar point cloud , a learnable dynamic graph convolution kernel with size M fields centered at the center point cloud The expression of the dynamic graph convolution calculation process is: ， ， wherein denotes a feature vector, denotes a weight vector, denotes a kernel direction vector, denotes a feature vector and the inner product of the convolution weight vector denotes a point cloud neighborhood direction vector, computed from the vector difference of the point cloud and the center point within the kernel neighborhood; denotes the inner product of the point cloud neighborhood direction vector and the kernel direction vector denotes the length of the point cloud neighborhood direction vector denotes the length of the kernel direction vector .​​​ 4. The bridge underwater three-dimensional sonar point cloud denoising method based on deep learning according to claim 1, characterized in that: In the noise removal module, the output feature of the point cloud feature extraction module is first inputted into the neighbor point attention layer, and then the noise feature score is calculated based on the full connection layer. The higher the score, the greater the probability of outlier noise. According to the score, 50% of the point cloud is removed to obtain the preliminary denoised point cloud. The preliminary denoised point cloud is inputted into the second full connection layer to predict the correction displacement value. The correction displacement value is added to the preliminary denoised point cloud to obtain the completely denoised point cloud. The neighbor point attention layer in the noise removal module improves the reliability of the point cloud noise evaluation score according to the correlation between the point cloud and its local neighborhood, and the feature of the i th point cloud output by the point cloud feature extraction module , is a query matrix, and the keyword represents the feature difference between the i th point cloud and its adjacent point cloud , then the neighbor point attention score The calculation formula is: 。 5. The bridge underwater three-dimensional sonar point cloud denoising method based on deep learning according to claim 4, characterized in that: The manifold upsampling module copies and stitches together the preliminary denoised point cloud to obtain the final point cloud. As input to the manifold upsampling layer, the manifold upsampling layer upsamples the high-density point cloud by learning the local manifold representation of the sonar point cloud; in the manifold upsampling layer, for the input point cloud set Each point cloud Local manifold using parameters Multilayer perceptron representation From the eigenvector and two-dimensional coordinate pairs sampled from the normalized parameter domain Parameterization; then the point cloud is upsampled from the manifold. The calculation formula is: ， The point cloud after upsampling on the manifold inputs a third fully connected layer, and the output obtained is added to the features obtained after copying and splicing the completely denoised point cloud to obtain the final output of the upsampled denoised point cloud point set .

6. The bridge underwater three-dimensional sonar point cloud denoising method based on deep learning according to claim 5, characterized in that: The perceptual consistency loss function in the perceptual consistency loss function module is composed of a chamfer distance and a top-k approximate Hausdorff distance weighted, and a calculation formula of the perceptual consistency loss function is: ​ ， wherein denotes a chamfer distance, denotes a top-k approximate Hausdorff distance, and denotes a weighting coefficient of the perceptual consistency loss function, and satisfies ; wherein the chamfer distance is calculated by the following formula: ， wherein denotes the set of up-sampled denoised point cloud points, denotes the i-th point in the set of up-sampled denoised point cloud points; denotes the set of clean sonar point cloud points, denotes the i-th point in the set of clean sonar point cloud points; The perceptual consistency loss function adopts a top-k approximate Hausdorff distance The calculation formula is: ， ， ， wherein represents the distance of a clean sonar point cloud to a denoised point cloud and the distance of a denoised point cloud to a clean sonar point cloud the front average of the maximum distances.

7. The bridge underwater three-dimensional sonar point cloud denoising method based on deep learning according to claim 6, characterized in that: The input of the denoising effect discriminator module is the upsampled denoised point cloud. The denoising effect discriminator module comprises a fourth full connection layer, a fifth full connection layer, and a sixth full connection layer. The output of each full connection layer is pooled and spliced to serve as the output feature of the denoising effect discriminator module. The loss function of the denoising effect discriminator module is calculated according to the following formula: ， is the output of the denoising effect discriminator module; Adversarial loss function consisting of a loss function of the denoising effect discriminator module and a perceptual consistency loss function, and the calculation formula is: ， wherein and are weighting parameters, and satisfy .

8. A deep learning-based bridge underwater three-dimensional sonar point cloud denoising system, characterized in that, It comprises: The denoising network construction module is configured to construct a three-dimensional sonar point cloud denoising network, and the three-dimensional sonar point cloud denoising network comprises a point cloud feature extraction module, a noise removal module, a manifold upsampling module, and a perceptual consistency loss function module. The denoiser module is configured to extract features of the sonar point cloud, denoise, and sample to generate a clean point cloud. The denoiser module and the denoising effect discriminator module form an adversarial generative network. The point cloud feature extraction module is configured to extract features of the sonar point cloud.

9. An electronic device, comprising: The noise removal module is configured to remove outlier noise and fuzzy noise in the sonar point cloud. The manifold upsampling module is configured to upsample the point cloud to avoid a reduction in the number of points in the sonar point cloud and feature loss caused by noise removal. The perceptual consistency loss function module is configured to evaluate the gap between the denoised point cloud and the clean point cloud.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The training data generation module is configured to generate a bridge underwater sonar point cloud denoising training set. The denoised point cloud output module is configured to train the three-dimensional sonar point cloud denoising network. The storage medium is configured to store instructions. The processor is configured to operate according to the instructions to perform the steps of the method of any one of claims 1-7. The program is executed by the processor to implement the steps of the method of any one of claims 1-7.

Images