A method and system for underwater three-dimensional panoramic reconstruction based on binocular optical images

Abstract

The present invention provides a method and system for underwater three-dimensional panoramic reconstruction based on binocular optical images, which can effectively cope with the problem of optical image degradation in complex seabed environments, restore and enhance images through a deep learning algorithm, and improve the quality of underwater image reconstruction; at the same time, by optimizing the stereo vision matching algorithm, high-precision underwater point cloud data can be generated; in addition, through environmental adaptive point cloud correction, the deviation caused by optical distortion is corrected, and the accuracy of underwater panoramic reconstruction is further improved; the present invention can effectively improve the accuracy and efficiency of underwater three-dimensional reconstruction, solve the problems of low image quality and optical distortion in traditional methods, and is suitable for marine biological surveys, underwater archaeology, marine ranch management and other fields, and provides reliable data support for the research and utilization of underwater environments.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning and image processing, and more specifically, to an underwater three-dimensional panoramic reconstruction method and system based on binocular optical images. Background Art

[0002] With the increasing efforts in the development of marine resources in my country, as well as the implementation of projects such as the construction of smart ocean ranches, offshore oil extraction, and offshore wind power, my country's demand for the perception and monitoring of the seabed environment is growing. Underwater three-dimensional panoramic reconstruction technology can provide accurate seabed environmental data for these marine projects and has become one of the key technologies in modern marine engineering. At present, underwater three-dimensional reconstruction technology is mainly used in scenarios such as marine biological surveys, underwater archaeology, marine ranch management, seabed monitoring, and seabed mapping. However, due to the complexity of the underwater environment, underwater optical images are often affected by factors such as water absorption, scattering, and refraction during the reconstruction process, resulting in a decrease in image quality, thereby affecting the reconstruction accuracy.

[0003] In practical applications, underwater 3D reconstruction faces many challenges, such as the rapid attenuation and distortion of light when it propagates in water, which leads to a decrease in image contrast, brightness and other qualities, thus affecting the reconstruction effect. In addition, the special underwater environment, such as turbidity and changes in lighting conditions, also increase the difficulty of 3D reconstruction.

[0004] An existing patent document discloses a method for super-resolution reconstruction of underwater images based on a generative adversarial network, comprising: inputting a high-resolution image A into a designed and constructed underwater image degradation model to generate a real underwater low-resolution image B; inputting the underwater low-resolution image B into a designed and constructed generator model to output a super-resolution image C; inputting the high-resolution image A and the super-resolution image C into a designed and constructed discriminator model, and outputting probabilities for judging true and false; the generative network and the discriminative network learn and confront each other until the generator and the discriminator reach a balance; using quantitative indicators to evaluate the generator model, and using the evaluated generator model to perform underwater image super-resolution reconstruction; this scheme only improves the quality of the reconstructed image based on the generative adversarial network, but the underwater environment is complex, and the scheme cannot correct the deviation caused by optical distortion, so the quality of the reconstructed image still cannot meet high accuracy requirements. Summary of the invention

[0005] In order to overcome the defect of the above-mentioned prior art that it is difficult to obtain high-precision and high-quality underwater panoramic reconstruction images, the present invention provides an underwater three-dimensional panoramic reconstruction method and system based on binocular optical images, which can effectively deal with the problem of optical image degradation in complex seabed environments. The present invention restores and enhances images through a deep learning algorithm, thereby improving the quality of underwater image reconstruction; at the same time, by optimizing the stereo vision matching algorithm, high-precision underwater point cloud data can be generated; in addition, through environmental adaptive point cloud correction, the deviation caused by optical distortion is corrected, thereby further improving the accuracy of underwater panoramic reconstruction.

[0006] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0007] A method for underwater three-dimensional panoramic reconstruction based on binocular optical images comprises the following steps:

[0008] S1: Use the calibrated binocular camera to continuously collect several underwater binocular optical images from different positions and angles;

[0009] S2: preprocessing each underwater binocular optical image using a preset deep learning algorithm, wherein the preprocessing includes image denoising, image enhancement and restoration, and image defogging;

[0010] S3: Use a stereo vision matching algorithm based on dense matching to perform pixel-by-pixel matching on the preprocessed underwater binocular optical image to obtain a dense depth map of the underwater scene, and generate underwater three-dimensional point cloud data based on the dense depth map of the underwater scene;

[0011] S4: performing environment-adaptive point cloud correction on the underwater three-dimensional point cloud data according to Snell's law and environmental factors to obtain corrected underwater three-dimensional point cloud data;

[0012] S5: Based on the ORB-SLAM3 framework, the corrected underwater three-dimensional point cloud data is subjected to real-time point cloud posture estimation and stitching to construct an underwater three-dimensional panoramic image.

[0013] Preferably, the step S1 comprises:

[0014] S1.1: Camera calibration: Place a calibration plate in an underwater environment, use a binocular camera to capture multi-angle images of the calibration plate, use a calibration algorithm to calculate the intrinsic and extrinsic parameters of the binocular camera and save them, and obtain the calibrated binocular camera;

[0015] S1.2: Image acquisition: using the calibrated binocular camera to continuously acquire a number of underwater binocular optical images from different positions and angles;

[0016] S1.3: Image storage: All collected underwater binocular optical images and their corresponding timestamp information are stored as basic data for underwater 3D reconstruction.

[0017] Preferably, in step S2, image denoising includes:

[0018] Gaussian filtering is used to smooth the input underwater binocular optical image to remove high-frequency noise, which can be expressed as:

[0019] J(x,y)=I(x,y)*G(x,y)

[0020]

[0021] Wherein, J(x,y) is the underwater binocular optical image after Gaussian filtering; I(x,y) is the input underwater binocular optical image; G(x,y) is a two-dimensional Gaussian kernel function, σ represents the standard deviation of the Gaussian kernel; (x,y) represents the pixel coordinates in the underwater binocular optical image; * represents the convolution operation;

[0022] The non-local mean denoising algorithm is used to perform secondary denoising on the Gaussian filtered underwater binocular optical image J(x,y), which can be expressed as:

[0023]

[0024] Among them, J′(x,y) is the underwater binocular optical image after secondary denoising; w(x,x ′ ,y,y ′ ) represents the pixel (x,y) and the pixel (x ′ ,y ′ ), Ω is the similarity weight around the pixel (x ′ ,y ′ ) neighborhood, usually a region with (x ′ ,y ′ ) is a square or rectangular window centered on the image, containing the set of pixels on the image that participate in the similarity calculation; h is the smoothing parameter; C(x,y) is the normalization constant;

[0025] Establish a deep convolutional neural network CNN for image denoising, using pre-labeled noisy images I noise (x, y) trains the parameters of the deep convolutional neural network CNN, constructs the first loss function L1 for parameter optimization, and obtains the trained deep convolutional neural network CNN; the formula of the first loss function L1 is:

[0026]

[0027] Among them, S(x i ,y i) is an ideal noise-free image, J cnn (x i ,y i ) is the output image of the deep convolutional neural network CNN; N is the number of pixels;

[0028] The trained deep convolutional neural network CNN is used to perform tertiary denoising on the underwater binocular optical image J′(x, y) after secondary denoising to complete the image denoising.

[0029] Preferably, in step S2, image enhancement and restoration includes:

[0030] A super-resolution reconstruction method based on CycleGAN is adopted to establish the first generative adversarial network GAN and construct the second loss function L2 for supervised training. The trained first generative adversarial network GAN is used to perform super-resolution reconstruction and color correction on underwater binocular optical images to restore the details and color characteristics of the image.

[0031] The second loss function L2 formula is:

[0032] L2=λ1L content +λ2L texture +λ3L TV

[0033]

[0034]

[0035] Wherein, λ1, λ2 and λ3 are the first, second and third balance coefficients respectively, satisfying λ1+λ2+λ3=1; L content , L texture and L TV are content loss, texture loss and total variation loss respectively; φ is the feature extraction function; Y and are the real image and the generated image of the first generative adversarial network GAN respectively; D is the texture feature extraction function; is the pixel value of the generated image at pixel position (i, j) of the first generative adversarial network GAN.

[0036] Preferably, in step S2, image defogging includes:

[0037] A second generative adversarial network GAN is established and a third loss function L3 is constructed for supervised training. The trained second generative adversarial network GAN is used to perform image dehazing on underwater binocular optical images.

[0038] The formula of the third loss function L3 is:

[0039] L3=λ4L content +λ5Ladversarial

[0040]

[0041]

[0042] Wherein, λ4 and λ5 are the fourth and fifth balance coefficients respectively, satisfying λ4+λ5=1; L content and L adversarial are content loss and adversarial loss respectively; φ is the feature extraction function; Y and are the real image and the generated image of the second generative adversarial network GAN respectively; D and G are the discriminator and generator of the second generative adversarial network GAN respectively; X is the input image of the second generative adversarial network GAN.

[0043] Preferably, in step S3, using a dense matching-based stereo vision matching algorithm to perform pixel-by-pixel matching on the preprocessed underwater binocular optical image to obtain a dense depth map of the underwater scene includes:

[0044] S3.1: Perform histogram equalization and image correction on the preprocessed underwater binocular optical image to improve the contrast and consistency of the image;

[0045] S3.2: Feature matching: extract the image grayscale information in the left and right views and match each pixel in the left and right views based on the matching function to obtain a disparity map; the matching function is a matching cost function based on the image grayscale difference, and the calculation formula is:

[0046]

[0047] Where C(x,y,d) is the matching cost of the pixel point (x,y) under disparity d; L and I R are the pixel grayscale values ​​of the left and right views respectively, and N1 is the window neighborhood centered on the pixel point (x, y); the matching function determines the disparity value of each pixel and constructs a disparity map by finding the minimum matching cost between the images of the left and right views;

[0048] S3.3: Disparity optimization: The disparity map is optimized by bidirectional consistency detection and sub-pixel interpolation, and a global optimization energy function is constructed to minimize the discontinuity error to obtain an optimized disparity map; the formula of the global optimization energy function is:

[0049]

[0050] Where E(d) is the energy of the disparity map; p and q are two pixel points in the image after feature matching, and C(p, d p) is the parallax d at point p p The matching cost is λ, λ is the smoothness weight, and T is the disparity threshold;

[0051] S3.4: Depth calculation: Obtain the calibration parameters of the binocular camera, calculate the depth information in the underwater scene using the parallax inversion formula according to the optimized disparity map, and construct a dense depth map of the underwater scene; the parallax inversion formula is:

[0052]

[0053] Among them, Z(x,y) is the depth value at the pixel point (x,y), D(x,y) is the optimized disparity value at the pixel point (x,y); f and B are the focal length and baseline distance in the binocular camera calibration parameters respectively.

[0054] Preferably, in step S4, performing environment-adaptive point cloud correction on the underwater three-dimensional point cloud data according to Snell's law and environmental factors, and obtaining the corrected underwater three-dimensional point cloud data comprises:

[0055] Refraction correction: Snell's law is used to calculate the refraction of light on the water surface and correct the geometric deviation caused by the refraction of light. The formula of Snell's law is:

[0056]

[0057] Among them, θ1 and θ2 are the angle of incidence and the angle of refraction respectively, and n1 and n2 are the refractive indices of water and air respectively;

[0058] Scattering and environment adaptive correction: Use machine learning algorithms to establish an environment adaptive correction model, dynamically adjust the parameters of the correction model according to scattering characteristics, ambient light intensity, underwater depth and temperature conditions, use the environment adaptive correction model with adjusted parameters to perform point cloud correction, and obtain corrected underwater three-dimensional point cloud data.

[0059] Preferably, in step S5, based on the ORB-SLAM3 framework, performing real-time attitude estimation and stitching of the corrected underwater three-dimensional point cloud data includes:

[0060] Based on the ORB-SLAM3 framework, ORB features in underwater binocular optical images are extracted to track the position and posture of the binocular camera in the underwater environment in real time.

[0061] According to the changes in the binocular camera's position, the spatial position of the corrected underwater 3D point cloud data is updated in real time, so that the point cloud data collected at different time points can be accurately superimposed;

[0062] The underwater 3D point cloud data after multiple frames of correction are stitched together through the point cloud stitching algorithm to construct an underwater 3D panoramic image.

[0063] Preferably, after step S5, the step further includes:

[0064] The chamfer distance is used to evaluate the stitching accuracy of the overlapping parts of the corrected underwater 3D point cloud data, and the chamfer distance between different stitched point cloud data is calculated and minimized to improve the stitching accuracy and achieve high-precision underwater 3D panoramic reconstruction.

[0065] The present invention also provides an underwater three-dimensional panoramic reconstruction system based on binocular optical images, which uses the above-mentioned underwater three-dimensional panoramic reconstruction method based on binocular optical images, including:

[0066] Image acquisition unit: used to continuously acquire a number of underwater binocular optical images from different positions and angles using a calibrated binocular camera;

[0067] An image preprocessing unit is used to preprocess each underwater binocular optical image using a preset deep learning algorithm, wherein the preprocessing includes image denoising, image enhancement and restoration, and image defogging;

[0068] Image matching unit: used to perform pixel-by-pixel matching on the preprocessed underwater binocular optical image using a stereo vision matching algorithm based on dense matching, obtain a dense depth map of the underwater scene, and generate underwater three-dimensional point cloud data according to the dense depth map of the underwater scene;

[0069] A correction unit: used for performing environment-adaptive point cloud correction on the underwater three-dimensional point cloud data according to Snell's law and environmental factors, and obtaining corrected underwater three-dimensional point cloud data;

[0070] Panoramic reconstruction unit: used for performing real-time pose estimation and stitching of the corrected underwater three-dimensional point cloud data based on the ORB-SLAM3 framework to construct an underwater three-dimensional panoramic image.

[0071] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0072] The present invention provides an underwater three-dimensional panoramic reconstruction method and system based on binocular optical images. First, a calibrated binocular camera is used to continuously collect a number of underwater binocular optical images from different positions and angles; then, a preset deep learning algorithm is used to perform image denoising, image enhancement and restoration, and image defogging on each underwater binocular optical image; then, a stereo vision matching algorithm based on dense matching is used to perform pixel-by-pixel matching on the pre-processed underwater binocular optical images to obtain a dense depth map of the underwater scene, and underwater three-dimensional point cloud data is generated according to the dense depth map of the underwater scene; then, according to Snell's law and environmental factors, the underwater three-dimensional point cloud data is subjected to environment-adaptive point cloud correction to obtain the corrected underwater three-dimensional point cloud data; finally, based on the ORB-LAM3 framework, the corrected underwater three-dimensional point cloud data is subjected to real-time point cloud posture estimation and splicing to construct an underwater three-dimensional panoramic image;

[0073] The present invention can effectively deal with the problem of optical image degradation in complex seabed environments, restore and enhance images through deep learning algorithms, and improve the quality of underwater image reconstruction; at the same time, by optimizing the stereo vision matching algorithm, it can generate high-precision underwater point cloud data; in addition, through environmental adaptive point cloud correction, the deviation caused by optical distortion is corrected, and the accuracy of underwater panoramic reconstruction is further improved; the present invention can effectively improve the accuracy and efficiency of underwater three-dimensional reconstruction, solve the problems of low image quality and optical distortion in traditional methods, and is suitable for marine biological surveys, underwater archaeology, marine ranch management and other fields, and provides reliable data support for the research and utilization of underwater environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 This is a flow chart of an underwater three-dimensional panoramic reconstruction method based on binocular optical images provided in Example 1.

[0075] Figure 2 Schematic diagram of the binocular camera parameter calibration board provided in Example 2.

[0076] Figure 3 This is a schematic diagram of an underwater binocular optical image collected by the binocular camera provided in Example 2.

[0077] Figure 4 This is a partial depth map generated using the stereo matching algorithm provided in Example 2.

[0078] Figure 5 The partial point cloud image generated using the partial depth image provided in Example 2.

[0079] Figure 6 This is the image feature extraction diagram provided in Example 2.

[0080] Figure 7The reconstructed underwater panoramic point cloud map provided in Example 2 is displayed using open3D.

[0081] Figure 8 This is a structural diagram of an underwater three-dimensional panoramic reconstruction system based on binocular optical images provided in Example 3. DETAILED DESCRIPTION

[0082] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0083] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0084] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0085] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0086] Example 1

[0087] like Figure 1 As shown, this embodiment provides an underwater three-dimensional panoramic reconstruction method based on binocular optical images, comprising the following steps:

[0088] S1: Use the calibrated binocular camera to continuously collect several underwater binocular optical images from different positions and angles;

[0089] S2: preprocessing each underwater binocular optical image using a preset deep learning algorithm, wherein the preprocessing includes image denoising, image enhancement and restoration, and image defogging;

[0090] S3: Use a stereo vision matching algorithm based on dense matching to perform pixel-by-pixel matching on the preprocessed underwater binocular optical image to obtain a dense depth map of the underwater scene, and generate underwater three-dimensional point cloud data based on the dense depth map of the underwater scene;

[0091] S4: performing environment-adaptive point cloud correction on the underwater three-dimensional point cloud data according to Snell's law and environmental factors to obtain corrected underwater three-dimensional point cloud data;

[0092] S5: Based on the ORB-SLAM3 framework, the corrected underwater three-dimensional point cloud data is subjected to real-time point cloud posture estimation and stitching to construct an underwater three-dimensional panoramic image.

[0093] In the specific implementation process, firstly, a calibrated binocular camera is used to continuously collect several underwater binocular optical images from different positions and angles;

[0094] Then, the preset deep learning algorithm is used to perform image denoising, image enhancement and restoration, and image defogging on each underwater binocular optical image.

[0095] Then, the preprocessed underwater binocular optical images are matched pixel by pixel using a stereo vision matching algorithm based on dense matching to obtain a dense depth map of the underwater scene, and then underwater three-dimensional point cloud data is generated according to the dense depth map of the underwater scene;

[0096] Then, according to Snell's law and environmental factors, the underwater three-dimensional point cloud data is subjected to environment-adaptive point cloud correction to obtain the corrected underwater three-dimensional point cloud data;

[0097] Finally, based on the ORB-LAM3 framework, the corrected underwater 3D point cloud data is subjected to real-time pose estimation and stitching to construct an underwater 3D panoramic image.

[0098] This method can effectively deal with the problem of optical image degradation in complex seabed environments. It restores and enhances images through deep learning algorithms, thereby improving the quality of underwater image reconstruction. At the same time, by optimizing the stereo vision matching algorithm, it can generate high-precision underwater point cloud data. In addition, through environmental adaptive point cloud correction, the deviation caused by optical distortion is corrected, further improving the accuracy of underwater panoramic reconstruction. This method can effectively improve the accuracy and efficiency of underwater three-dimensional reconstruction, solve the problems of low image quality and optical distortion in traditional methods, and is suitable for marine biological surveys, underwater archaeology, marine ranch management and other fields, and provides reliable data support for the research and utilization of underwater environments.

[0099] Example 2

[0100] This embodiment provides an underwater 3D panoramic reconstruction method based on binocular optical images, comprising the following steps:

[0101] S1: Use the calibrated binocular camera to continuously collect several underwater binocular optical images from different positions and angles;

[0102] S2: preprocessing each underwater binocular optical image using a preset deep learning algorithm, wherein the preprocessing includes image denoising, image enhancement and restoration, and image defogging;

[0103] S3: Use a stereo vision matching algorithm based on dense matching to perform pixel-by-pixel matching on the preprocessed underwater binocular optical image to obtain a dense depth map of the underwater scene, and generate underwater three-dimensional point cloud data based on the dense depth map of the underwater scene;

[0104] S4: performing environment-adaptive point cloud correction on the underwater three-dimensional point cloud data according to Snell's law and environmental factors to obtain corrected underwater three-dimensional point cloud data;

[0105] S5: Based on the ORB-SLAM3 framework, real-time pose estimation and stitching of the corrected underwater three-dimensional point cloud data are performed to construct an underwater three-dimensional panoramic image;

[0106] The step S1 comprises:

[0107] S1.1: Camera calibration: Place a calibration plate in an underwater environment, use a binocular camera to capture multi-angle images of the calibration plate, use a calibration algorithm to calculate the intrinsic and extrinsic parameters of the binocular camera and save them, and obtain the calibrated binocular camera;

[0108] S1.2: Image acquisition: using the calibrated binocular camera to continuously acquire a number of underwater binocular optical images from different positions and angles;

[0109] S1.3: Image storage: All collected underwater binocular optical images and their corresponding timestamp information are stored as basic data for underwater 3D reconstruction;

[0110] In step S2, image denoising includes:

[0111] Gaussian filtering is used to smooth the input underwater binocular optical image to remove high-frequency noise, which can be expressed as:

[0112] J(x,y)=I(x,y)*G(x,y)

[0113]

[0114] Wherein, J(x,y) is the underwater binocular optical image after Gaussian filtering; I(x,y) is the input underwater binocular optical image; G(x,y) is a two-dimensional Gaussian kernel function, σ represents the standard deviation of the Gaussian kernel; (x,y) represents the pixel coordinates in the underwater binocular optical image; * represents the convolution operation;

[0115] The non-local mean denoising algorithm is used to perform secondary denoising on the Gaussian filtered underwater binocular optical image J(x,y), which can be expressed as:

[0116]

[0117] Among them, J′(x,y) is the underwater binocular optical image after secondary denoising; w(x,x ′ ,y,y ′ ) represents the pixel (x,y) and the pixel (x ′ ,y ′ ), Ω is the similarity weight around the pixel (x ′ ,y ′ ) neighborhood; h is the smoothing parameter; C(x,y) is the normalization constant;

[0118] Establish a deep convolutional neural network CNN for image denoising, using pre-labeled noisy images I noise (x, y) trains the parameters of the deep convolutional neural network CNN, constructs the first loss function L1 for parameter optimization, and obtains the trained deep convolutional neural network CNN; the formula of the first loss function L1 is:

[0119]

[0120] Among them, S(x i ,y i ) is an ideal noise-free image, J cnn (x i ,y i ) is the output image of the deep convolutional neural network CNN; N is the number of pixels;

[0121] Using a trained deep convolutional neural network (CNN) to perform a third denoising on the underwater binocular optical image J′(x, y) after the second denoising, the image denoising is completed;

[0122] In step S2, image enhancement and restoration include:

[0123] A super-resolution reconstruction method based on CycleGAN is adopted to establish the first generative adversarial network GAN and construct the second loss function L2 for supervised training. The trained first generative adversarial network GAN is used to perform super-resolution reconstruction and color correction on underwater binocular optical images to restore the details and color characteristics of the image.

[0124] The second loss function L2 formula is:

[0125] L2=λ1L content +λ2L texture +λ3L TV

[0126]

[0127]

[0128] Wherein, λ1, λ2 and λ3 are the first, second and third balance coefficients respectively, satisfying λ1+λ2+λ3=1; L content , L texture and L TV are content loss, texture loss and total variation loss respectively; φ is the feature extraction function; Y and are the real image and the generated image of the first generative adversarial network GAN respectively; D is the texture feature extraction function; is the pixel value of the generated image at pixel position (i, j) of the first generative adversarial network GAN;

[0129] In step S2, image defogging includes:

[0130] A second generative adversarial network GAN is established and a third loss function L3 is constructed for supervised training. The trained second generative adversarial network GAN is used to perform image dehazing on underwater binocular optical images.

[0131] The formula of the third loss function L3 is:

[0132] L3=λ4L content +λ5L adversarial

[0133]

[0134]

[0135] Wherein, λ4 and λ5 are the fourth and fifth balance coefficients respectively, satisfying λ4+λ5=1; L content and L adversarial are content loss and adversarial loss respectively; φ is the feature extraction function; Y and are the real image and the generated image of the second generation adversarial network GAN respectively; D and G are the discriminator and generator of the second generation adversarial network GAN respectively; X is the input image of the second generation adversarial network GAN;

[0136] In step S3, the pre-processed underwater binocular optical image is matched pixel by pixel using a stereo vision matching algorithm based on dense matching to obtain a dense depth map of the underwater scene, including:

[0137] S3.1: Perform histogram equalization and image correction on the preprocessed underwater binocular optical image to improve the contrast and consistency of the image;

[0138] S3.2: Feature matching: extract the image grayscale information in the left and right views and match each pixel in the left and right views based on the matching function to obtain a disparity map; the matching function is a matching cost function based on the image grayscale difference, and the calculation formula is:

[0139]

[0140] Where C(x,y,d) is the matching cost of the pixel point (x,y) under disparity d; L and I R are the pixel grayscale values ​​of the left and right views respectively, and N1 is the window neighborhood centered on the pixel point (x, y); the matching function determines the disparity value of each pixel and constructs a disparity map by finding the minimum matching cost between the images of the left and right views;

[0141] S3.3: Disparity optimization: The disparity map is optimized by bidirectional consistency detection and sub-pixel interpolation, and a global optimization energy function is constructed to minimize the discontinuity error to obtain an optimized disparity map; the formula of the global optimization energy function is:

[0142]

[0143] Where E(d) is the energy of the disparity map; p and q are two pixel points in the image after feature matching, and C(p, d p ) is the parallax d at point p p The matching cost is λ, λ is the smoothness weight, and T is the disparity threshold;

[0144] S3.4: Depth calculation: Obtain the calibration parameters of the binocular camera, calculate the depth information in the underwater scene using the parallax inversion formula according to the optimized disparity map, and construct a dense depth map of the underwater scene; the parallax inversion formula is:

[0145]

[0146] Where Z(x,y) is the depth value at the pixel point (x,y), D(x,y) is the optimized disparity value at the pixel point (x,y); f and B are the focal length and baseline distance in the binocular camera calibration parameters respectively;

[0147] In step S4, according to Snell's law and environmental factors, the underwater three-dimensional point cloud data is subjected to environment-adaptive point cloud correction, and the corrected underwater three-dimensional point cloud data is obtained, which includes:

[0148] Refraction correction: Snell's law is used to calculate the refraction of light on the water surface and correct the geometric deviation caused by the refraction of light. The formula of Snell's law is:

[0149]

[0150] Among them, θ1 and θ2 are the angle of incidence and the angle of refraction respectively, and n1 and n2 are the refractive indices of water and air respectively;

[0151] Scattering and environment adaptive correction: Use machine learning algorithms to establish an environment adaptive correction model, dynamically adjust the parameters of the correction model according to scattering characteristics, ambient light intensity, underwater depth and temperature conditions, use the environment adaptive correction model with adjusted parameters to perform point cloud correction, and obtain corrected underwater 3D point cloud data;

[0152] In the step S5, based on the ORB-SLAM3 framework, real-time attitude estimation and splicing of the corrected underwater three-dimensional point cloud data include:

[0153] Based on the ORB-SLAM3 framework, ORB features in underwater binocular optical images are extracted to track the position and posture of the binocular camera in the underwater environment in real time.

[0154] According to the changes in the binocular camera's position, the spatial position of the corrected underwater 3D point cloud data is updated in real time, so that the point cloud data collected at different time points can be accurately superimposed;

[0155] The underwater 3D point cloud data after multiple frames of correction are stitched together through a point cloud stitching algorithm to construct an underwater 3D panoramic image;

[0156] The chamfer distance is used to evaluate the stitching accuracy of the overlapping parts of the corrected underwater 3D point cloud data, and the chamfer distance between different stitched point cloud data is calculated and minimized to improve the stitching accuracy and achieve high-precision underwater 3D panoramic reconstruction.

[0157] In the specific implementation process, first use the calibrated binocular camera to continuously collect several underwater binocular optical images from different positions and angles; the specific steps are as follows:

[0158] S1.1: Camera calibration: Place a calibration plate in an underwater environment, use a binocular camera to capture multi-angle images of the calibration plate, use a calibration algorithm to calculate the intrinsic and extrinsic parameters of the binocular camera and save them, and obtain the calibrated binocular camera; the calibration diagram is shown in Figure 2 As shown;

[0159] S1.2: Image acquisition: Use the calibrated binocular camera to continuously acquire a number of underwater binocular optical images from different positions and angles; the acquired binocular images are as follows: Figure 3 As shown;

[0160] S1.3: Image storage: All collected underwater binocular optical images and their corresponding timestamp information are stored as basic data for underwater 3D reconstruction;

[0161] In order to solve the degradation and distortion of underwater images, the preset deep learning algorithm is then used to perform image denoising, image enhancement and restoration, and image defogging on each underwater binocular optical image.

[0162] 1) Image denoising includes:

[0163] In underwater environments, due to the scattering and absorption of light, images are often affected by various noises, resulting in reduced image quality. In order to improve the accuracy of subsequent reconstruction, the captured binocular images need to be effectively denoised. Gaussian filtering is used to smooth the input underwater binocular optical images to remove high-frequency noise, thereby reducing small interference in the image, which can be expressed as:

[0164] J(x,y)=I(x,y)*G(x,y)

[0165]

[0166] Wherein, J(x,y) is the underwater binocular optical image after Gaussian filtering; I(x,y) is the input underwater binocular optical image; G(x,y) is a two-dimensional Gaussian kernel function, σ represents the standard deviation of the Gaussian kernel; (x,y) represents the pixel coordinates in the underwater binocular optical image; * represents the convolution operation;

[0167] In order to further improve the denoising effect, the non-local mean (NLM) denoising algorithm is used to perform secondary denoising on the underwater binocular optical image J(x, y) after Gaussian filtering. This method can effectively remove noise and retain image details by calculating the similarity weight and considering the similarity of all pixels in the image, which is expressed as:

[0168]

[0169] Among them, J′(x,y) is the underwater binocular optical image after secondary denoising; w(x,x ′ ,y,y ′ ) represents the pixel (x,y) and the pixel (x ′ ,y ′ ), Ω is the similarity weight around the pixel (x ′ ,y ′ ) neighborhood; k is a smoothing parameter used to control the sensitivity of the similarity measure; C(x,y) is a normalization constant used to ensure that the mean of the weighted result is appropriate and that the denoised image remains realistic;

[0170] Finally, deep learning technology is used to further improve the denoising effect. A deep convolutional neural network (CNN) is established for image denoising. noise (x, y) trains the parameters of the deep convolutional neural network CNN, constructs the first loss function L1 for parameter optimization, and obtains the trained deep convolutional neural network CNN; the formula of the first loss function L1 is:

[0171]

[0172] Among them, S(x i ,y i ) is an ideal noise-free image, J cnn (x i ,y i ) is the output image of the deep convolutional neural network CNN; N is the number of pixels; the loss function gradually reduces the difference between the denoising result and the ideal image by optimizing the parameters of the CNN network, thereby generating a clear denoised image;

[0173] Using a trained deep convolutional neural network CNN, the underwater binocular optical image J′(x, y) after the secondary denoising is denoised three times to complete the image denoising;

[0174] 2) Image enhancement and restoration include:

[0175] Various factors in the underwater environment, such as suspended particles in the water and light scattering, lead to the generally low quality of underwater images, which are manifested as low contrast and color distortion. In order to improve the clarity and color performance of underwater images, a super-resolution reconstruction method based on CycleGAN (SR-CycleGAN) is adopted. The first generative adversarial network GAN is established and the second loss function L2 is constructed for supervised training. The trained first generative adversarial network GAN is used to perform super-resolution reconstruction and color correction on underwater binocular optical images to restore the details and color characteristics of the image.

[0176] In order to further improve the image quality, the loss function combines content loss, texture loss and total variation loss. Content loss is used to measure the difference between the generated image and the real image features. Texture loss is used to calculate the similarity between the generated image and the real image in texture features. Total variation loss is used to improve image smoothness by suppressing noise and unnecessary details.

[0177] The second loss function L2 formula is:

[0178] L2=λ1L content +λ2L texture +λ3L TV

[0179]

[0180]

[0181] Among them, λ1, λ2 and λ3 are the first, second and third balance coefficients respectively, satisfying λ1+λ2+λ c =1;L content , L texture and L TV are content loss, texture loss and total variation loss respectively; φ is the feature extraction function; Y and are the real image and the generated image of the first generative adversarial network GAN respectively; D is the texture feature extraction function; is the pixel value of the generated image at pixel position (i, j) of the first generative adversarial network GAN;

[0182] Through the above steps, the clarity and contrast of underwater images are significantly improved, so that they can better present details and provide high-quality image data for subsequent 3D panoramic reconstruction;

[0183] 3) Image dehazing includes:

[0184] The haze phenomenon in the underwater environment is mainly caused by the scattering and absorption of light when it propagates in the water, resulting in blurred imaging and loss of details; compared with images on land foggy days, the haze effect on underwater images is more serious, resulting in loss of visual information; this embodiment establishes a second generative adversarial network GAN, and optimizes the generator to restore a clear image from the haze image through adversarial training of the generator and the discriminator; this embodiment constructs a third loss function L3 for supervised training, and uses the trained second generative adversarial network GAN to perform image defogging on underwater binocular optical images, so as to effectively remove the haze effect of underwater images and improve image quality;

[0185] The formula of the third loss function L3 is:

[0186] L3=λ4L content +λ5L adversarial

[0187]

[0188]

[0189] Wherein, λ4 and λ5 are the fourth and fifth balance coefficients respectively, satisfying λ4+λ5=1; L content and L adversarial are content loss and adversarial loss respectively. The content loss is used to ensure that the content of the generated image is consistent with the real image, and the adversarial loss is used to optimize the visual effect of the generated image to make it as close to the real image as possible. The discriminator evaluates the generated image and the real image. φ is the feature extraction function (such as the features extracted by the convolutional neural network); Y and are the real clear image and the dehazed image generated by the second generation adversarial network GAN; D and G are the discriminator and generator of the second generation adversarial network GAN; X is the hazy image input by the second generation adversarial network GAN;

[0190] Through the above steps, the clarity and contrast of underwater images are effectively improved, providing high-quality image data for subsequent 3D panoramic reconstruction;

[0191] Then, the preprocessed underwater binocular optical image is matched pixel by pixel using a stereo vision matching algorithm based on dense matching to obtain a dense depth map of the underwater scene.

[0192] In the image matching stage, in order to obtain high-precision underwater 3D panoramic reconstruction data, this method adopts a stereo vision algorithm based on dense matching; the algorithm calculates the disparity map by matching the images obtained by the left and right binocular cameras pixel by pixel, thereby restoring the depth information of the scene; the specific steps of stereo vision matching are as follows:

[0193] S3.1: Perform histogram equalization and image correction on the pre-processed underwater binocular optical image to improve the contrast and consistency of the image and ensure the accuracy in the subsequent matching process;

[0194] S3.2: Feature matching: extract the image grayscale information in the left and right views and match each pixel in the left and right views based on the matching function to obtain a disparity map; the matching function is a matching cost function based on the image grayscale difference, and the calculation formula is:

[0195]

[0196] Where C(x,y,d) is the matching cost of the pixel point (x,y) under disparity d; L and I R are the pixel grayscale values ​​of the left and right views respectively, and N1 is the window neighborhood centered on the pixel point (x, y); the matching function determines the disparity value of each pixel and constructs a disparity map by finding the minimum matching cost between the images of the left and right views;

[0197] S3.3: Disparity optimization: The disparity map is optimized by bidirectional consistency detection and sub-pixel interpolation, and a global optimization energy function is constructed to minimize the discontinuity error to obtain an optimized disparity map; the formula of the global optimization energy function is:

[0198]

[0199] Where E(d) is the energy of the disparity map; p and q are two pixel points in the image after feature matching, and C(p, d p ) is the parallax d at point p p The matching cost is λ, λ is the smoothness weight, and T is the disparity threshold. This optimization process can effectively maintain the smoothness of the disparity map while retaining edge details.

[0200] S3.4: Depth calculation: Obtain the calibration parameters of the binocular camera, calculate the depth information in the underwater scene using the parallax inversion formula according to the optimized disparity map, and construct a dense depth map of the underwater scene; the parallax inversion formula is:

[0201]

[0202] Where Z(x,y) is the depth value at the pixel point (x,y), D(x,y) is the optimized disparity value at the pixel point (x,y); f and B are the focal length and baseline distance in the binocular camera calibration parameters respectively;

[0203] Through the above steps, a high-precision dense depth map can be obtained, which provides accurate geometric information for the subsequent underwater 3D panoramic reconstruction. The generated depth map is as follows: Figure 4 As shown; then generate high-precision underwater three-dimensional point cloud data based on the dense depth map of the underwater scene;

[0204] Then, according to Snell's law and environmental factors, the underwater 3D point cloud data is subjected to environmental adaptive point cloud correction to correct the deviation caused by optical distortion and obtain the corrected underwater 3D point cloud data;

[0205] The optical properties in underwater environments are significantly different from those in air. When light propagates in water, it will produce deviations due to changes in the refractive index, resulting in a certain degree of distortion in the spatial position of the point cloud data. In order to ensure that the generated point cloud data accurately reflects the three-dimensional geometric structure of the object, this method corrects the point cloud data based on Snell's law and the environment adaptive algorithm:

[0206] Refraction correction: Snell's law is used to calculate the refraction of light on the water surface and correct the geometric deviation caused by the refraction of light. The formula of Snell's law is:

[0207]

[0208] Where θ1 and θ2 are the angle of incidence and angle of refraction, respectively, and n1 and n2 are the refractive indices of water and air, respectively. By calculating the deviation of each ray when it enters and leaves the water, the position of the point cloud data is adjusted to match the actual object geometry.

[0209] Scattering and environmental adaptive correction: The scattering of light by underwater suspended matter and particles will further affect the imaging quality. A machine learning algorithm is used to establish an environmental adaptive correction model. The parameters of the correction model are dynamically adjusted according to the scattering characteristics, ambient light intensity, underwater depth and temperature conditions. The environmental adaptive correction model with adjusted parameters is used to perform point cloud correction, making the point cloud correction more accurate and adaptable to various underwater scenes. The corrected underwater 3D point cloud data is obtained, and some of the generated point cloud images are shown below. Figure 5 As shown;

[0210] Finally, based on the ORB-LAM3 framework, the corrected underwater 3D point cloud data is subjected to real-time pose estimation and stitching to construct an underwater 3D panoramic image.

[0211] After point cloud data is generated and corrected, in order to construct a complete underwater 3D panorama, this method performs point cloud posture estimation and stitching correction based on the ORB-SLAM3 framework; the ORB-SLAM3 algorithm uses visual odometer and keyframe technology to track the camera's posture changes in real time, and uses loop detection to achieve position closure, thereby ensuring the global consistency of the 3D point cloud in the scene;

[0212] Based on the ORB-SLAM3 framework, ORB features in underwater binocular optical images are extracted to track the position and posture of the binocular camera in the underwater environment in real time.

[0213] According to the changes in the binocular camera posture, the spatial position of the corrected underwater 3D point cloud data is updated in real time, so that the point cloud data collected at different time points can be accurately superimposed; feature extraction such as Figure 6 As shown;

[0214] Based on the posture estimation, the underwater 3D point cloud data after multiple frames of correction are stitched together through the point cloud stitching algorithm to construct an underwater 3D panoramic image.

[0215] In order to evaluate the stitching accuracy of the overlapping parts of the point cloud, this method uses the chamfer distance to evaluate the stitching accuracy of the overlapping parts of the corrected underwater 3D point cloud data; the chamfer distance evaluates the overlapping error of the stitching part by calculating the average distance between the nearest points of the two point cloud sets. The formula is as follows:

[0216]

[0217] Among them, P and Q are two point cloud sets, respectively, and ‖pq‖ is the distance between two points;

[0218] By calculating and minimizing the chamfer distance between different point cloud data, the stitching accuracy of the point cloud in the overlapping part can be ensured, and high-precision underwater 3D panoramic reconstruction can be achieved. Figure 7 As shown;

[0219] After stitching is completed, the generated point cloud data, camera pose and stitching evaluation results are saved to the specified storage location for further processing or application analysis; each generated point cloud file should be timestamped to ensure the uniqueness and traceability of the data; at the same time, the camera pose file is recorded for subsequent pose correction or accuracy evaluation, and the stitching evaluation results are stored in the log file to facilitate subsequent inspection of the stitching quality;

[0220] This method can effectively deal with the problem of optical image degradation in complex seabed environments. It restores and enhances images through deep learning algorithms, thereby improving the quality of underwater image reconstruction. At the same time, by optimizing the stereo vision matching algorithm, it can generate high-precision underwater point cloud data. In addition, through environmental adaptive point cloud correction, the deviation caused by optical distortion is corrected, further improving the accuracy of underwater panoramic reconstruction. This method can effectively improve the accuracy and efficiency of underwater three-dimensional reconstruction, solve the problems of low image quality and optical distortion in traditional methods, and is suitable for marine biological surveys, underwater archaeology, marine ranch management and other fields, and provides reliable data support for the research and utilization of underwater environments.

[0221] Example 3

[0222] like Figure 8 As shown, this embodiment provides an underwater three-dimensional panoramic reconstruction system based on binocular optical images, applying an underwater three-dimensional panoramic reconstruction method based on binocular optical images described in Embodiment 1 or 2, including:

[0223] Image acquisition unit 301: used to continuously acquire a number of underwater binocular optical images from different positions and angles using a calibrated binocular camera;

[0224] An image preprocessing unit 302 is used to preprocess each underwater binocular optical image using a preset deep learning algorithm, wherein the preprocessing includes image denoising, image enhancement and restoration, and image defogging;

[0225] Image matching unit 303: used to perform pixel-by-pixel matching on the preprocessed underwater binocular optical image using a stereo vision matching algorithm based on dense matching, obtain a dense depth map of the underwater scene, and generate underwater three-dimensional point cloud data according to the dense depth map of the underwater scene;

[0226] Correction unit 304: used for performing environment-adaptive point cloud correction on the underwater three-dimensional point cloud data according to Snell's law and environmental factors, and obtaining corrected underwater three-dimensional point cloud data;

[0227] Panoramic reconstruction unit 305: used to perform real-time pose estimation and stitching of the corrected underwater three-dimensional point cloud data based on the ORB-SLAM3 framework to construct an underwater three-dimensional panoramic image.

[0228] In the specific implementation process, first, the image acquisition unit 301 uses a calibrated binocular camera to continuously acquire a number of underwater binocular optical images from different positions and angles;

[0229] Then the image preprocessing unit 302 uses a preset deep learning algorithm to perform image denoising, image enhancement and restoration, and image defogging on each underwater binocular optical image;

[0230] Then, the image matching unit 303 uses a stereo vision matching algorithm based on dense matching to perform pixel-by-pixel matching on the preprocessed underwater binocular optical image to obtain a dense depth map of the underwater scene, and generates underwater three-dimensional point cloud data according to the dense depth map of the underwater scene;

[0231] Then, the correction unit 304 performs environment-adaptive point cloud correction on the underwater three-dimensional point cloud data according to Snell's law and environmental factors, and obtains corrected underwater three-dimensional point cloud data;

[0232] Finally, the panoramic reconstruction unit 305 performs real-time pose estimation and splicing of the corrected underwater 3D point cloud data based on the ORB-LAM3 framework to construct an underwater 3D panoramic image.

[0233] This system can effectively deal with the problem of optical image degradation in complex seabed environments. It restores and enhances images through deep learning algorithms, thereby improving the quality of underwater image reconstruction. At the same time, by optimizing the stereo vision matching algorithm, it can generate high-precision underwater point cloud data. In addition, through environmental adaptive point cloud correction, the deviation caused by optical distortion is corrected, further improving the accuracy of underwater panoramic reconstruction. This system can effectively improve the accuracy and efficiency of underwater three-dimensional reconstruction, solve the problems of low image quality and optical distortion in traditional methods, and is suitable for marine biological surveys, underwater archaeology, marine ranch management and other fields, and provide reliable data support for the research and utilization of underwater environments.

[0234] The same or similar reference numerals correspond to the same or similar components;

[0235] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;

[0236] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. An underwater three-dimensional panoramic reconstruction method based on binocular optical images, characterized in that: The following steps are involved: S1: Use the calibrated binocular camera to continuously collect several underwater binocular optical images from different positions and angles; S2: preprocessing each underwater binocular optical image using a preset deep learning algorithm, wherein the preprocessing includes image denoising, image enhancement and restoration, and image defogging; S3: Use the dense matching-based stereo vision matching algorithm to perform pixel-by-pixel matching on the preprocessed underwater binocular optical image to obtain a dense depth map of the underwater scene, including: S3.1: Perform histogram equalization and image correction on the preprocessed underwater binocular optical image to improve the contrast and consistency of the image; S3.2: Feature matching: extract the image grayscale information in the left and right views and match each pixel in the left and right views based on the matching function to obtain a disparity map; the matching function is a matching cost function based on the image grayscale difference, and the calculation formula is: Where C(x,y,d) is the matching cost of the pixel point (x,y) under disparity d; L and I R are the pixel grayscale values ​​of the left and right views respectively, and N1 is the window neighborhood centered on the pixel point (x, y); the matching function determines the disparity value of each pixel and constructs a disparity map by finding the minimum matching cost between the images of the left and right views; S3.3: Disparity optimization: The disparity map is optimized by bidirectional consistency detection and sub-pixel interpolation, and a global optimization energy function is constructed to minimize the discontinuity error to obtain an optimized disparity map; the formula of the global optimization energy function is: Where E(d) is the energy of the disparity map; p and q are two pixel points in the image after feature matching, and C(p, d p ) is the parallax d at point p p The matching cost, λ is the smoothness weight, and T is the disparity threshold; S3.4: Depth calculation: Obtain the calibration parameters of the binocular camera, calculate the depth information in the underwater scene using the parallax inversion formula according to the optimized disparity map, and construct a dense depth map of the underwater scene; the parallax inversion formula is: Where Z(x,y) is the depth value at the pixel point (x,y), D(x,y) is the optimized disparity value at the pixel point (x,y); f and B are the focal length and baseline distance in the binocular camera calibration parameters respectively; Generate underwater three-dimensional point cloud data based on a dense depth map of the underwater scene; S4: performing environment-adaptive point cloud correction on the underwater three-dimensional point cloud data according to Snell's law and environmental factors to obtain corrected underwater three-dimensional point cloud data; S5: Based on the ORB-SLAM3 framework, the corrected underwater three-dimensional point cloud data is subjected to real-time point cloud posture estimation and stitching to construct an underwater three-dimensional panoramic image.

2. The underwater three-dimensional panoramic reconstruction method based on binocular optical images according to claim 1 is characterized in that: The step S1 comprises: S1.1: Camera calibration: Place a calibration plate in an underwater environment, use a binocular camera to capture multi-angle images of the calibration plate, use a calibration algorithm to calculate the intrinsic and extrinsic parameters of the binocular camera and save them, and obtain the calibrated binocular camera; S1.2: Image acquisition: using the calibrated binocular camera to continuously acquire a number of underwater binocular optical images from different positions and angles; S1.3: Image storage: All collected underwater binocular optical images and their corresponding timestamp information are stored as basic data for underwater 3D reconstruction.

3. The underwater three-dimensional panoramic reconstruction method based on binocular optical images according to claim 1 is characterized in that: In step S2, image denoising includes: Gaussian filtering is used to smooth the input underwater binocular optical image to remove high-frequency noise, which can be expressed as: J(x,y)=I(x,y)*G(x,y) Wherein, J(x,y) is the underwater binocular optical image after Gaussian filtering; I(x,y) is the input underwater binocular optical image; G(x,y) is a two-dimensional Gaussian kernel function, σ represents the standard deviation of the Gaussian kernel; (x,y) represents the pixel coordinates in the underwater binocular optical image; * represents the convolution operation; The non-local mean denoising algorithm is used to perform secondary denoising on the Gaussian filtered underwater binocular optical image J(x,y), which can be expressed as: Among them, J′(x,y) is the underwater binocular optical image after secondary denoising; w(x,x ′ ,y,y ′ ) represents the pixel (x,y) and the pixel (x ′ ,y ′ ), Ω is the similarity weight around the pixel (x ′ ,y ′ ) neighborhood; h is the smoothing parameter; C(x,y) is the normalization constant; Establish a deep convolutional neural network CNN for image denoising, using pre-labeled noisy images I noise (x, y) trains the parameters of the deep convolutional neural network CNN, constructs the first loss function L1 for parameter optimization, and obtains the trained deep convolutional neural network CNN; the formula of the first loss function L1 is: Among them, S(x i ,y i ) is an ideal noise-free image, J cnn (x i ,y i ) is the output image of the deep convolutional neural network CNN; N is the number of pixels; The trained deep convolutional neural network CNN is used to perform tertiary denoising on the underwater binocular optical image J′(x, y) after secondary denoising to complete the image denoising.

4. The underwater three-dimensional panoramic reconstruction method based on binocular optical images according to claim 1, characterized in that: In step S2, image enhancement and restoration include: A super-resolution reconstruction method based on CycleGAN is adopted to establish the first generative adversarial network GAN and construct the second loss function L2 for supervised training. The trained first generative adversarial network GAN is used to perform super-resolution reconstruction and color correction on underwater binocular optical images to restore the details and color characteristics of the image. The second loss function L2 formula is: <h2 style=";text-align:left;direction:ltr">L2 = λ1L<h2 style=";text-align:left;direction:ltr"> content <h2 style=";text-align:left;direction:ltr"> +λ2L<h2 style=";text-align:left;direction:ltr"> texture <h2 style=";text-align:left;direction:ltr"> +λ3L<h2 style=";text-align:left;direction:ltr"> TV Wherein, λ1, λ2 and λ3 are the first, second and third balance coefficients respectively, satisfying λ1+λ2+λ3=1; L content , L texture and L TV are content loss, texture loss and total variation loss respectively; φ is the feature extraction function; Y and are the real image and the generated image of the first generative adversarial network GAN respectively; D is the texture feature extraction function; is the pixel value of the generated image at pixel position (i, j) of the first generative adversarial network GAN.

5. The underwater three-dimensional panoramic reconstruction method based on binocular optical images according to claim 1, characterized in that: In step S2, image defogging includes: A second generative adversarial network GAN is established and a third loss function L3 is constructed for supervised training. The trained second generative adversarial network GAN is used to perform image dehazing on underwater binocular optical images. The formula of the third loss function L3 is: L3=λ4L content +λ5L adversarial Wherein, λ4 and λ5 are the fourth and fifth balance coefficients respectively, satisfying λ4+λ5=1; L content and L adversarial are content loss and adversarial loss respectively; φ is the feature extraction function; Y and are the real image and the generated image of the second generative adversarial network GAN respectively; D and G are the discriminator and generator of the second generative adversarial network GAN respectively; X is the input image of the second generative adversarial network GAN.

6. The underwater three-dimensional panoramic reconstruction method based on binocular optical images according to claim 1, characterized in that: In step S4, according to Snell's law and environmental factors, the underwater three-dimensional point cloud data is subjected to environment-adaptive point cloud correction, and the corrected underwater three-dimensional point cloud data is obtained, which includes: Refraction correction: Snell's law is used to calculate the refraction of light on the water surface and correct the geometric deviation caused by the refraction of light. The formula of Snell's law is: Among them, θ1 and θ2 are the angle of incidence and the angle of refraction respectively, and n1 and n2 are the refractive indices of water and air respectively; Scattering and environment adaptive correction: Use machine learning algorithms to establish an environment adaptive correction model, dynamically adjust the parameters of the correction model according to scattering characteristics, ambient light intensity, underwater depth and temperature conditions, use the environment adaptive correction model with adjusted parameters to perform point cloud correction, and obtain corrected underwater three-dimensional point cloud data.

7. The underwater three-dimensional panoramic reconstruction method based on binocular optical images according to claim 1, characterized in that: In the step S5, based on the ORB-SLAM3 framework, real-time attitude estimation and splicing of the corrected underwater three-dimensional point cloud data include: Based on the ORB-SLAM3 framework, ORB features in underwater binocular optical images are extracted to track the position and posture of the binocular camera in the underwater environment in real time. According to the changes in the binocular camera's position, the spatial position of the corrected underwater 3D point cloud data is updated in real time, so that the point cloud data collected at different time points can be accurately superimposed; The underwater 3D point cloud data after multiple frames of correction are stitched together through the point cloud stitching algorithm to construct an underwater 3D panoramic image.

8. The method for underwater three-dimensional panoramic reconstruction based on binocular optical images according to claim 7, characterized in that: After step S5, the following steps are also included: The chamfer distance is used to evaluate the stitching accuracy of the overlapping parts of the corrected underwater 3D point cloud data, and the chamfer distance between different stitched point cloud data is calculated and minimized to improve the stitching accuracy and achieve high-precision underwater 3D panoramic reconstruction.

9. An underwater three-dimensional panoramic reconstruction system based on binocular optical images, using an underwater three-dimensional panoramic reconstruction method based on binocular optical images as claimed in any one of claims 1 to 8, characterized in that: include: Image acquisition unit: used to continuously acquire a number of underwater binocular optical images from different positions and angles using a calibrated binocular camera; An image preprocessing unit is used to preprocess each underwater binocular optical image using a preset deep learning algorithm, wherein the preprocessing includes image denoising, image enhancement and restoration, and image defogging; Image matching unit: used to perform pixel-by-pixel matching on the preprocessed underwater binocular optical image using a stereo vision matching algorithm based on dense matching, obtain a dense depth map of the underwater scene, and generate underwater three-dimensional point cloud data according to the dense depth map of the underwater scene; A correction unit: used for performing environment-adaptive point cloud correction on the underwater three-dimensional point cloud data according to Snell's law and environmental factors, and obtaining corrected underwater three-dimensional point cloud data; Panoramic reconstruction unit: used for performing real-time pose estimation and stitching of the corrected underwater three-dimensional point cloud data based on the ORB-SLAM3 framework to construct an underwater three-dimensional panoramic image.

Images