Underwater three-dimensional scene data completion method and system based on deep learning

Abstract

The invention discloses an underwater three-dimensional scene data completion method and system based on deep learning, relates to the technical field of computer graphics, and solves the technical problem of data missing caused by sound shadow shielding, strong filtering or semantic segmentation in submarine topography surveying and mapping data. According to the technical scheme, the method is characterized in that a synthetic training set based on programmed terrain modeling is constructed, and high-resolution and large-scale training data are provided; a double-discriminator generative adversarial network comprising a completion network, a context discriminator and a boundary quality discriminator is designed, stable convergence of the network is achieved through a three-stage progressive training strategy, and meanwhile self-adaptive processing facing different missing scales is achieved. Wherein the boundary quality discriminator is used for enhancing the boundary continuity of the complementation area. According to the method, rapid and high-quality completion of the multi-scale underwater three-dimensional data is realized, the structural consistency of the completion area is remarkably improved while the calculation efficiency is maintained, and an effective solution is provided for real-time underwater surveying and mapping application.

Description

Technical Field

[0001] This application relates to the field of computer graphics technology, and in particular to a method and system for completing underwater 3D scene data based on deep learning. Background Technology

[0002] With the development of marine science, current ocean exploration is gradually moving towards the deep sea. Underwater 3D scene mapping is an important task in marine science, and some instruments, such as multibeam echo sounders (MBES), are widely used to acquire high-resolution underwater 3D data. However, due to limitations in their measurement principles and experimental conditions, the acquired underwater 3D data often suffers from missing data. Many current studies focus on solving this problem. Traditional mathematical interpolation methods, such as Kriging interpolation and spline interpolation, can provide reasonable results, but they often fail to capture complex topographic features and tend to over-smooth key topographic structures, a situation particularly pronounced when the missing scale is large. While existing deep learning methods offer better results, they suffer from the following problems: First, there is a lack of sufficient high-quality, high-resolution, and large-scale publicly available underwater 3D datasets for supervised network training; second, elevation data is well-suited for efficient processing by converting it into 2D data, but existing 2D data completion methods are primarily designed for natural images and are difficult to apply directly to single-channel, unbounded underwater 3D data. Furthermore, some methods exhibit poor continuity of the completed region boundaries, affecting the structural rationality of the terrain; finally, learning-based methods typically limit the efficiency of data processing.

[0003] To address the aforementioned issues, there is an urgent need to design a modular approach to achieve rapid and high-quality completion of underwater 3D scene data. Summary of the Invention

[0004] This application provides a method and system for completing underwater 3D scene data based on deep learning, the technical purpose of which is to achieve fast and high-quality completion of underwater 3D data.

[0005] The above-mentioned technical objective of this application is achieved through the following technical solution:

[0006] A deep learning-based underwater 3D scene data completion method includes:

[0007] Underwater global terrain point cloud data is generated by procedural terrain modeling. The real data missing patterns of the underwater global terrain point cloud data are simulated by mask generation technology to obtain a synthetic training set.

[0008] The dual-discriminator generative adversarial network is trained using a synthetic training set to obtain the final completion network;

[0009] Inference is performed based on the final completion network to complete the areas to be completed in the underwater 3D scene, resulting in the final underwater 3D scene data;

[0010] The dual-discriminator generative adversarial network includes a completion network, a context discriminator, and a boundary quality discriminator.

[0011] The training of the dual-discriminator generative adversarial network includes:

[0012] The completion network and the boundary quality discriminator are jointly trained to obtain the first completion network and the first boundary quality discriminator.

[0013] The context discriminator is trained independently to obtain the first context discriminator;

[0014] The first completion network, the first boundary quality discriminator, and the first context discriminator are subjected to collaborative adversarial training to obtain the final completion network.

[0015] Preferably, the construction of the synthetic training set includes:

[0016] The ridge noise and berth noise are mixed and then smoothed to generate a terrain model;

[0017] Point cloud sampling is performed on the terrain model to obtain rasterized underwater global terrain point cloud data; the density of the point cloud sampling is less than the density of the terrain model.

[0018] The three-dimensional underwater global terrain point cloud data is converted into a two-dimensional matrix on a horizontal plane to obtain single-channel image format data; in the two-dimensional matrix, the element index corresponds to the point cloud position, and the element value represents the point cloud elevation.

[0019] A global mask is generated by erosion simulation to identify non-missing data in the underwater global terrain point cloud data; the size of the global mask is the same as the size of the underwater global terrain point cloud data.

[0020] Local masks are generated by erosion simulation and polygon creation to identify missing data in underwater global terrain point cloud data that needs to be completed; the size of the local mask is smaller than the size of the underwater global terrain point cloud data.

[0021] Subtracting the missing data to be completed and the non-missing data from the single-channel image format data yields the synthesized training set, i.e., the first global 3D data.

[0022] Preferably, the completion network includes a global encoder, a local encoder, a decoder, and a residual module. First local 3D data and a local mask are extracted from the first global 3D data and a global mask to fill in the missing region. The local encoder is then trained based on the first local 3D data and local masks of different sizes to obtain a trained local encoder. The first global 3D data is encoded into global features by the global encoder, and the first local 3D data is encoded into local features by the trained local encoder. The global and local features are fused to obtain fused features. The fused features are decoded by the decoder to obtain second local 3D data of the same size as the first local 3D data. The second local 3D data is input to the residual module and superimposed with the output of the residual module for completion.

[0023] The residual module is implemented based on gradient data. It calculates the x- and y-direction gradient data of the input second local 3D data using the Sobel operator, concatenates the second local 3D data, local mask, x-direction gradient data, and y-direction gradient data into 4-channel data, and then calculates the residual. Finally, the calculated residual is superimposed with the second local 3D data to obtain the output I of the residual module. net The final locally completed data output by the completion network is represented as follows:

[0024] I out =I local ⊙M local +I net ⊙(1-M local );

[0025] Among them, I out I represents the final partially completed data output by the completion network. local M represents the first local 3D data input to the complete network. local ⊙ represents the local mask of the input network completion, and ⊙ represents element-wise matrix multiplication.

[0026] Preferably, the completion network completes the loss L. comp Conduct training to compensate for the loss L comp Represented as:

[0027]

[0028] Among them, L null L represents the elevation data error of the area to be supplemented. sc This represents the spatial consistency loss, i.e., the gradient error of the region to be filled; w n w c Both represent loss weights, w comp For w n and w c sum;

[0029] By compensating for the loss L comp When training the completion network, the ranges of the first global 3D data and the first local 3D data are first normalized to [-1,1] based on the original range of the underwater global terrain point cloud data. After training, the final local completion data I is then used. out The range is inversely normalized to the original range of the underwater global terrain point cloud data.

[0030] Preferably, the context discriminator includes a global discriminator, a local discriminator, and a fusion discriminator; after embedding the final local completion data into the first global 3D data, a second global 3D data is generated. The global discriminator simultaneously receives the second global 3D data and the global mask, the local discriminator receives the final local completion data, and the global discriminator and the local discriminator encode the information they receive into deep features respectively. The fusion discriminator then discriminates the deep features and outputs the final discrimination result.

[0031] The context discriminator determines the loss L. cd Training, L cd Represented as:

[0032]

[0033] Among them, L real L represents the loss determined by real data. fake Indicates loss assessment based on false data; L fm L represents the feature matching loss, which is the reciprocal of the distance between the output features of the global and local discriminators; lsb L represents the scale equilibrium loss; R1 Represents the R1 regularization term; w c w f w l w R1 w represents the weight of each loss term. cd This represents the sum of all weighted data.

[0034] Preferably, the boundary quality discriminator includes a truth boundary extractor and a boundary extraction network, and compares the final local completion data generated by the completion network through the truth boundary extractor and the boundary extraction network;

[0035] The truth boundary extractor extracts the boundary M at the intersection of 0 and 1 values ​​in the local mask corresponding to the first local 3D data through a convolution kernel K. b Extraction is performed, boundary M b The boundary width is 2 pixels, and the convolution kernel K is represented as:

[0036]

[0037] The boundary extraction network extracts the boundaries M of the final local completion data through multiple dilated convolutional layers. be Perform identification;

[0038] Based on boundary M b and boundary M be The differences are used to achieve joint training of the boundary quality discriminator and the completion network;

[0039] The loss of the boundary quality discriminator includes the identification loss L. recog And smoothing loss L smooth The recognition loss L recog Represented as:

[0040] L recog =BCE(M b M be );

[0041] Where BCE(·) represents the cross-entropy loss;

[0042] The smoothing loss L smooth Represented as:

[0043] L smooth =BCE(M be ⊙M b ,0);

[0044] Where BCE(·) represents cross-entropy loss and ⊙ represents element-wise matrix multiplication.

[0045] Preferably, training the dual-discriminator generative adversarial network specifically includes:

[0046] The completion network and the boundary quality discriminator are pre-trained to obtain a first completion network and a first boundary quality discriminator with fixed parameters; wherein, the completion loss L comp and the smoothing loss L smooth The recognition loss L is used to improve the performance of the completion network. recog Used to improve the performance of the boundary quality discriminator;

[0047] The first context discriminator is obtained by training the first completion network with fixed parameters.

[0048] After removing the parameter fixation, the first completion network, the first boundary quality discriminator, and the first context discriminator are subjected to collaborative adversarial training to obtain the final completion network.

[0049] A deep learning-based underwater 3D scene data completion system is provided. This system implements an underwater 3D scene data completion method and includes:

[0050] The data generation module generates underwater global terrain point cloud data through procedural terrain modeling. It simulates the real data missing patterns of the underwater global terrain point cloud data using mask generation technology to obtain a synthetic training set.

[0051] The training control module trains the dual-discriminator generative adversarial network using a synthesized training set to obtain the final completion network;

[0052] The inference output module performs inference based on the final completion network to complete the areas to be completed in the underwater 3D scene, thus obtaining the final underwater 3D scene data.

[0053] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a method for completing underwater three-dimensional scene data.

[0054] A computer storage medium storing a computer program, which, when executed by a processor, implements a method for completing underwater three-dimensional scene data.

[0055] The above technical solution can achieve at least some of the following technical effects:

[0056] (1) Solving the problem of supervised training of underwater 3D data by synthesizing training sets. A high-resolution synthetic training set is constructed based on procedural terrain modeling, which effectively solves the problem of scarcity of high-quality underwater 3D data and provides rich training samples for deep learning networks.

[0057] (2) Ensure the geometric continuity of the completed region through a boundary quality discriminator. To address the technical challenge of poor boundary continuity in learning-based completion methods, a boundary quality discriminator is introduced. Multi-layer dilated convolution is used to accurately extract the boundary features of the completed region. An inverse optimization mechanism between boundary recognition success rate and output smoothness is established, which effectively improves the data completion quality.

[0058] (3) Enhance the structural rationality of terrain data processing through gradient data. By using the gradient-enhanced residual learning mechanism, the underwater 3D data completion results can better maintain the slope changes and structural features of the original terrain, thereby improving the authenticity and structural continuity of the completion results.

[0059] (4) Rapid completion of underwater 3D scene data is achieved through a lightweight generative network structure. Millisecond-level completion of 3D data is achieved through the combined effect of a lightweight generative network and an efficient discriminator. Attached Figure Description

[0060] Figure 1This is a schematic diagram of the network structure and process of a deep learning-based underwater 3D scene data completion method in an embodiment of this application;

[0061] Figure 2 This is a schematic diagram of the completed network structure in an embodiment of the present invention;

[0062] Figure 3 This is a schematic diagram of the context discriminator network structure in an embodiment of the present invention;

[0063] Figure 4 This is a schematic diagram of the boundary quality discriminator network structure in an embodiment of the present invention;

[0064] Figure 5 This is a schematic diagram showing the visualization results of relevant data during the training process of the boundary quality discriminator in an embodiment of the present invention. Detailed Implementation

[0065] The technical solution of this application will be described in detail below with reference to the accompanying drawings. The following embodiments are only used to explain the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0066] like Figure 1 As shown, the underwater 3D scene data completion method based on deep learning described in this application includes:

[0067] 100: High-resolution underwater global terrain point cloud data is generated through procedural terrain modeling. The real data missing patterns of the underwater global terrain point cloud data are simulated using mask generation technology to obtain a synthetic training set.

[0068] Preferably, the construction of the synthetic training set includes:

[0069] 1001: Mix ridge noise and berth noise and then smooth them to generate a terrain model.

[0070] 1002: Point cloud sampling is performed on the terrain model to obtain rasterized underwater global terrain point cloud data; wherein, the density of the point cloud sampling is less than the density of the terrain model.

[0071] 1003: Convert 3D underwater global terrain point cloud data into a 2D matrix on a horizontal plane to obtain single-channel image format data; where the element index in the 2D matrix corresponds to the point cloud position, and the element value represents the point cloud elevation;

[0072] 1004: A global mask is generated through erosion simulation to identify non-missing data in the underwater global terrain point cloud data; the size of the global mask is the same as the size of the underwater global terrain point cloud data.

[0073] 1005: Local masks are generated through erosion simulation and polygon creation to identify missing data in underwater global terrain point cloud data that needs to be completed; the size of the local mask is smaller than the size of the underwater global terrain point cloud data.

[0074] In this embodiment, the size of the underwater global terrain point cloud data and the global mask is 600×600. The size of the local mask is set to three types to accommodate different missing scales, namely 257×257, 129×129, and 65×65. Of course, other numbers or sizes of local masks can also be set, and there is no limitation here.

[0075] 1006: Subtract the missing data to be completed and the non-missing data from the single-channel image format data to obtain the synthetic training set, i.e., the first global 3D data.

[0076] 101: The dual discriminator generative adversarial network is trained using a synthetic training set to obtain the final completion network.

[0077] Preferably, the dual-discriminator generative adversarial network includes a completion network, a context discriminator, and a boundary quality discriminator.

[0078] Preferably, such as Figure 2 As shown, the completion network uses a dual encoder structure to process local and global inputs and integrates a gradient-enhanced residual module.

[0079] Preferably, the completion network includes a global encoder, a local encoder, a decoder, and a residual module. First local 3D data and a local mask are extracted from the first global 3D data and a global mask to fill in the missing region. The local encoder is then trained based on the first local 3D data and local masks of different sizes to obtain a trained local encoder. The first global 3D data is encoded into global features by the global encoder, and the first local 3D data is encoded into local features by the trained local encoder. The global and local features are fused to obtain fused features. The fused features are decoded by the decoder to obtain second local 3D data of the same size as the first local 3D data. The second local 3D data is input to the residual module and superimposed with the output of the residual module for completion. Skip connections are used between the decoder and the local encoder to retain some original features.

[0080] The residual module is implemented based on gradient data. It calculates the x- and y-direction gradient data of the input second local 3D data using the Sobel operator, concatenates the second local 3D data, local mask, x-direction gradient data, and y-direction gradient data into 4-channel data, and then calculates the residual. Finally, the calculated residual is superimposed with the second local 3D data to obtain the output I of the residual module. netThe final locally completed data output by the completion network is represented as follows:

[0081] I out =I local ⊙M local +I net ⊙(1-M local );

[0082] Among them, I out I represents the final partially completed data output by the completion network. local M represents the first local 3D data input to the complete network. local ⊙ represents the local mask of the input network completion, and ⊙ represents element-wise matrix multiplication.

[0083] Preferably, the completion network completes the loss L. comp Conduct training to compensate for the loss L comp Represented as:

[0084]

[0085] Among them, L null L represents the elevation data error of the area to be supplemented. sc This represents the spatial consistency loss, i.e., the gradient error of the region to be filled; w n w c Both represent loss weights, w comp For w n and w c sum;

[0086] By compensating for the loss L comp When training the completion network, the ranges of the first global 3D data and the first local 3D data are first normalized to [-1,1] based on the original range of the underwater global terrain point cloud data. After training, the final local output I is then processed. out The range is inversely normalized to the original range of the underwater global terrain point cloud data.

[0087] Preferably, the context discriminator maintains the consistency between global and local features through a global discriminator and a local discriminator.

[0088] Preferably, such as Figure 3 As shown, the context discriminator includes a global discriminator, a local discriminator, and a fusion discriminator. After embedding the final local completion data into the first global 3D data, a second global 3D data is generated. The global discriminator simultaneously receives the second global 3D data and the global mask, while the local discriminator receives the final local completion data. The global and local discriminators are enhanced by adding gradient data channels.

[0089] The global discriminator and the local discriminator encode the information they receive into deep features, and the fusion discriminator then discriminates the deep features and outputs the final discriminant result in a linear layer.

[0090] The context discriminator determines the loss L. cd Training, L cd Represented as:

[0091]

[0092] Among them, L real L represents the loss determined by real data. fake Indicates loss assessment based on false data; L fm L represents the feature matching loss, which is the reciprocal of the distance between the output features of the global and local discriminators, used to increase the difference between real and fake features; lsb L represents the scale-balancing loss, used to balance the orders of magnitude of the loss from judging real data and the loss from judging fake data, preventing the discriminator from classifying all data into the same outcome; R1 Represents the R1 regularization term, which stabilizes the training process through gradient penalty; w c w f w l w R1 w represents the weight of each loss term. cd This represents the sum of all weighted data.

[0093] Preferably, the boundary quality discriminator improves the continuity of the boundary of the completed region by establishing an inverse relationship between the boundary recognition success rate and the output smoothness.

[0094] Preferably, such as Figure 4 As shown, the boundary quality discriminator includes a truth boundary extractor and a boundary extraction network, which compare the final local completion data generated by the completion network.

[0095] The truth boundary extractor extracts the boundary M at the intersection of 0 and 1 values ​​in the local mask corresponding to the first local 3D data through a convolution kernel K. b Extraction is performed, boundary M b The boundary width is 2 pixels, and the convolution kernel K is represented as:

[0096]

[0097] The boundary extraction network extracts the boundaries M of the final local completion data through multiple dilated convolutional layers. be The quality of the completion network's output is assessed by identifying the boundary extraction network's output. The closer the boundary extraction network's output is to the ground truth, the worse the completion network's result is.

[0098] Based on boundary Mb and boundary M be The differences are used to achieve joint training of the boundary quality discriminator and the completion network.

[0099] Preferably, the loss of the boundary quality discriminator includes an identification loss L. recog And smoothing loss L smooth The recognition loss L recog Represented as:

[0100] L recog =BCE(M b M be );

[0101] Where BCE(·) represents the cross-entropy loss.

[0102] The smoothing loss L smooth Represented as:

[0103] L smooth =BCE(M be ⊙M b ,0);

[0104] Where BCE(·) represents cross-entropy loss and ⊙ represents element-wise matrix multiplication.

[0105] Preferably, training the dual-discriminator generative adversarial network includes:

[0106] 1011: Pre-train the completion network and the boundary quality discriminator to establish basic completion capabilities and boundary extraction capabilities, resulting in a first completion network and a first boundary quality discriminator with fixed parameters; wherein, the completion loss L comp and the smoothing loss L smooth The recognition loss L is used to improve the performance of the completion network. recog Used to improve the performance of the boundary quality discriminator;

[0107] 1012: The context discriminator is trained by a first completion network with fixed parameters to obtain a first context discriminator, thereby forming a fine-grained discrimination capability;

[0108] In this embodiment, a fixed 600×600 global elevation data is used, and multi-scale network pre-training is performed for different missing scales of 257×257, 129×129, and 65×65. During inference, the network is adaptively selected to complete the network based on the missing scale, forming a more refined inference model. Figure 1 The inference results under different missing scales are presented.

[0109] 1013: Remove parameter fixation, perform collaborative adversarial training on the first completion network, the first boundary quality discriminator, and the first context discriminator to achieve overall optimization and obtain the final completion network.

[0110] 102: Based on the final completion network, inference is performed to complete the areas to be completed in the underwater 3D scene, and the final underwater 3D scene data is obtained.

[0111] The underwater 3D scene data completion system based on deep learning described in this application includes a data generation module, a training control module, and an inference output module.

[0112] The data generation module is used to generate underwater global terrain point cloud data through procedural terrain modeling. It simulates the real data missing patterns of the underwater global terrain point cloud data based on mask generation technology to obtain a synthetic training set.

[0113] The training control module is used to train the bi-discriminator generative adversarial network by synthesizing a training set to obtain the final completed network.

[0114] The inference output module is used to perform inference based on the final completion network, complete the areas to be completed in the underwater 3D scene, and obtain the final underwater 3D scene data.

[0115] Figure 5 The data visualization results of the boundary quality discriminator training process in the embodiments of this application are shown. In the local mask, the black position represents the region to be completed, and the white position in the ground truth boundary and the extracted boundary represents the boundary region.

[0116] In summary, the underwater 3D scene data completion method and system based on deep learning achieves rapid and efficient completion of missing areas in underwater 3D data. It also considers the structural consistency and contextual coherence between the completed area and the surrounding area, thus achieving efficient 3D data enhancement.

[0117] The above specific embodiments are only for illustrating the technical concept and structural features of the present invention, and are intended to enable those skilled in the art to implement them. However, the above content does not limit the scope of protection of the present invention. Any equivalent changes or modifications made in accordance with the spirit and essence of the present invention should fall within the scope of protection of the present invention.

Claims

1. A method for completing underwater 3D scene data based on deep learning, characterized in that, include: Underwater global terrain point cloud data is generated by procedural terrain modeling. The real data missing patterns of the underwater global terrain point cloud data are simulated by mask generation technology to obtain a synthetic training set. The dual-discriminator generative adversarial network is trained using a synthetic training set to obtain the final completion network; Inference is performed based on the final completion network to complete the areas to be completed in the underwater 3D scene, resulting in the final underwater 3D scene data; The dual-discriminator generative adversarial network includes a completion network, a context discriminator, and a boundary quality discriminator. The training of the dual-discriminator generative adversarial network includes: The completion network and the boundary quality discriminator are jointly trained to obtain the first completion network and the first boundary quality discriminator. The context discriminator is trained independently to obtain the first context discriminator; The first completion network, the first boundary quality discriminator, and the first context discriminator are subjected to collaborative adversarial training to obtain the final completion network.

2. The underwater three-dimensional scene data completion as described in claim 1, characterized in that, The construction of the synthetic training set includes: The ridge noise and berth noise are mixed and then smoothed to generate a terrain model; Point cloud sampling is performed on the terrain model to obtain rasterized underwater global terrain point cloud data; the density of the point cloud sampling is less than the density of the terrain model. The three-dimensional underwater global terrain point cloud data is converted into a two-dimensional matrix on a horizontal plane to obtain single-channel image format data; in the two-dimensional matrix, the element index corresponds to the point cloud position, and the element value represents the point cloud elevation. A global mask is generated by erosion simulation to identify non-missing data in the underwater global terrain point cloud data; the size of the global mask is the same as the size of the underwater global terrain point cloud data. Local masks are generated by erosion simulation and polygon creation to identify missing data in underwater global terrain point cloud data that needs to be completed; the size of the local mask is smaller than the size of the underwater global terrain point cloud data. Subtracting the missing data to be completed and the non-missing data from the single-channel image format data yields the synthetic training set, i.e., the first global 3D data.

3. The underwater three-dimensional scene data completion method as described in claim 2, characterized in that, The completion network includes a global encoder, a local encoder, a decoder, and a residual module. First local 3D data and a local mask are extracted from the first global 3D data and a global mask to fill in the missing region. The local encoder is then trained based on the first local 3D data and local masks of different sizes to obtain the trained local encoder. The first global 3D data is encoded into global features by the global encoder, and the first local 3D data is encoded into local features by the trained local encoder. The global and local features are fused to obtain fused features. The fused features are decoded by the decoder to obtain second local 3D data of the same size as the first local 3D data. The second local 3D data is input to the residual module and superimposed with the output of the residual module for completion. The residual module is implemented based on gradient data. It calculates the x- and y-direction gradient data of the input second local 3D data using the Sobel operator, concatenates the second local 3D data, local mask, x-direction gradient data, and y-direction gradient data into 4-channel data, and then calculates the residual. Finally, the calculated residual is superimposed with the second local 3D data to obtain the output I of the residual module. net The final locally completed data output by the completion network is represented as follows: I out =I local ⊙M local +I net ⊙(1-M local ); Among them, I out I represents the final partially completed data output by the completion network. local M represents the first local 3D data input to the complete network. local ⊙ represents the local mask of the input network completion, and ⊙ represents element-wise matrix multiplication.

4. The underwater three-dimensional scene data completion as described in claim 3, characterized in that, The completion network completes the loss L. comp Conduct training to compensate for the loss L comp Represented as: Among them, L null L represents the elevation data error of the area to be supplemented. sc This represents the spatial consistency loss, i.e., the gradient error of the region to be filled in; w n w c Both represent loss weights, w comp For w n and w c sum; By compensating for the loss L comp When training the completion network, the ranges of the first global 3D data and the first local 3D data are first normalized to [-1,1] based on the original range of the underwater global terrain point cloud data. After training, the final local completion data I is then used. out The range is inversely normalized to the original range of the underwater global terrain point cloud data.

5. The underwater three-dimensional scene data completion as described in claim 1, characterized in that, The context discriminator includes a global discriminator, a local discriminator, and a fusion discriminator. After embedding the final local completion data into the first global 3D data, a second global 3D data is generated. The global discriminator simultaneously receives the second global 3D data and the global mask. The local discriminator receives the final local completion data. The global discriminator and the local discriminator encode the information they receive into deep features. The fusion discriminator then discriminates the deep features and outputs the final discrimination result. The context discriminator determines the loss L. cd Training, L cd Represented as: Among them, L real L represents the loss determined by real data. fake Indicates loss assessment based on false data; L fm L represents the feature matching loss, which is the reciprocal of the distance between the output features of the global and local discriminators; lsb L represents the scale equilibrium loss; R1 Represents the R1 regularization term; w c w f w l w R1 w represents the weight of each loss term. cd This represents the sum of all weighted data.

6. The underwater three-dimensional scene data completion as described in claim 1, characterized in that, The boundary quality discriminator includes a truth boundary extractor and a boundary extraction network, which compare the final local completion data generated by the completion network through the truth boundary extractor and the boundary extraction network. The truth boundary extractor extracts the boundary M at the intersection of 0 and 1 values ​​in the local mask corresponding to the first local 3D data through a convolution kernel K. b Extraction is performed, boundary M b The boundary width is 2 pixels, and the convolution kernel K is represented as: The boundary extraction network extracts the boundaries M of the final local completion data through multiple dilated convolutional layers. be Perform identification; Based on boundary M b and boundary M be The differences are used to achieve joint training of the boundary quality discriminator and the completion network; The loss of the boundary quality discriminator includes the identification loss L. recog and smoothing loss L smooth The recognition loss L recog Represented as: L recog =BCE(M b ,M be ); Where BCE(·) represents the cross-entropy loss; The smoothing loss L smooth Represented as: L smooth =BCE(M be ⊙M b ,0); Where BCE(·) represents cross-entropy loss and ⊙ represents element-wise matrix multiplication.

7. The underwater three-dimensional scene data completion as described in claim 6, characterized in that, The training of the dual-discriminator generative adversarial network specifically includes: The completion network and the boundary quality discriminator are pre-trained to obtain a first completion network and a first boundary quality discriminator with fixed parameters; wherein, the completion loss L comp and the smoothing loss L smooth The recognition loss L is used to improve the performance of the completion network. recog Used to improve the performance of the boundary quality discriminator; The first context discriminator is obtained by training the first completion network with fixed parameters. After removing the parameter fixation, the first completion network, the first boundary quality discriminator, and the first context discriminator are subjected to collaborative adversarial training to obtain the final completion network.

8. A deep learning-based underwater 3D scene data completion system, characterized in that, The underwater 3D scene data completion system is used to implement the underwater 3D scene data completion method according to any one of claims 1-7, and the underwater 3D scene data completion system includes: The data generation module generates underwater global terrain point cloud data through procedural terrain modeling. It simulates the real data missing patterns of the underwater global terrain point cloud data using mask generation technology to obtain a synthetic training set. The training control module trains the dual-discriminator generative adversarial network using a synthesized training set to obtain the final completion network; The inference output module performs inference based on the final completion network to complete the areas to be completed in the underwater 3D scene, thus obtaining the final underwater 3D scene data.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the underwater three-dimensional scene data completion method as described in any one of claims 1 to 8.

10. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the underwater three-dimensional scene data completion method as described in any one of claims 1 to 8.

Images