Deep sea optical image reconstruction method based on vision and point cloud

Abstract

The invention discloses a deep sea optical image reconstruction method based on vision and point cloud. The method comprises the following steps: S10, obtaining an original RGB image, an RGB true value image and a depth map; s20, carrying out pixel alignment on the three images to obtain an RGBD data set; and S30, the RGB-D data set is used for training and testing a deep sea image reconstruction network, and a reconstructed image is generated. Rebuilt deep sea images and video pictures are used as application backgrounds, depth information graphs corresponding to the deep sea images are obtained through a line laser scanning method, corresponding original deep sea degraded images are obtained through a high-definition RGB camera, and the deep sea degraded images are obtained through estimation of a polarization-related physical model in a mode of additionally installing a polaroid. And obtaining a picture true value picture of the reconstructed deep sea image.

Description

Technical Field

[0001] The invention belongs to the technical field of reconstructing deep-sea images, and relates to a deep-sea optical image reconstruction method based on vision and point cloud. Background Art

[0002] The ocean contains many unknown organisms and potential resources. The development of marine resources and the development of marine economy have gradually become an indispensable and important part of the history of human scientific and technological development. Computer vision technology plays an increasingly important role in carrying out marine-related activities, especially in various underwater platforms, such as unmanned underwater vehicles (ROVs). The main content of marine computer vision technology includes underwater image processing. However, the complexity of the deep-sea environment often leads to a serious degradation of image quality, which brings great challenges to underwater image processing. This serious degradation of image quality is multifaceted, including image color distortion, fogging, uneven brightness, etc.

[0003] Due to the influence of deep sea water, light of different wavelengths has different attenuation rates, and long-wavelength light attenuates faster, which causes red light to attenuate faster in the deep sea, and the image is bluer or greener. In addition, microorganisms and suspended particles in the water absorb most of the light and cause scattering, resulting in a decrease in image contrast and blurring. These factors interfere with the work of the underwater vision system, so it is necessary to apply image reconstruction technology to reverse the process of underwater image degradation, eliminate color cast and blur, and effectively improve the visual quality of the underwater vision system input image, bringing convenience to subsequent advanced tasks such as target detection, recognition and classification.

[0004] The attenuation of light underwater is mainly caused by absorption and scattering effects, which leads to image degradation such as reduced contrast and blurring. In addition, different wavelengths of light have different attenuation rates when propagating underwater, resulting in image color distortion. In clear water, red light at a depth of 5 meters disappears first, followed by orange light at a depth of 10 meters. Blue light, which has the shortest wavelength of visible light, travels the farthest in water, which causes underwater images to present an undesirable blue-green hue. Small particles, plankton, and dissolved organic matter present in the water often cause serious noise problems for underwater imaging and exacerbate the effects of backscattering.

[0005] In the study of deep-sea image restoration methods, the core issue is the composition of the light source in the underwater imaging process. Due to the lack of natural light in the deep sea, artificial light sources are required in the deep-sea imaging process. Unlike natural light, the characteristics of artificial light sources themselves will lead to uneven illumination, with obvious bright spots appearing in the center of the light source illumination point and dark spots forming at the edge of the image. In addition, since artificial light sources will produce absorption and scattering during the propagation of light sources, there will be inherent image quality degradation problems. In order to improve the quality of underwater images, a lot of research has been carried out in academia. Most studies focus on directly designing image enhancement techniques or networks without considering the physical model of underwater imaging.

[0006] Although underwater image reconstruction technology based on traditional visual processing algorithms and deep learning has achieved certain results, there are still many problems to be solved:

[0007] (1) Combining underwater physical models with deep learning technology has great potential in the field of deep-sea image restoration. However, the major challenge that this technical route must address is that existing image reconstruction methods cannot perform targeted restoration of objects of different distances and sizes in the deep sea. The image reconstruction process often uses global physical model parameter estimation, resulting in a huge room for improvement in reconstruction effect.

[0008] (2) In order to adapt to the real environment, deep-sea image reconstruction methods need to be optimized to achieve real-time performance. However, most existing underwater image restoration methods have a long processing time and are difficult to meet the needs of real-time applications;

[0009] (3) Although most of the research on underwater image restoration focuses on single images, the practical application of underwater images has also been extended to videos. There is often a temporal and spatial correlation between video frames, which allows the reconstruction of real-time underwater video images to appropriately utilize the information of previous and next frames, but this information has not been fully utilized in existing research;

[0010] (4) At present, deep-sea image reconstruction algorithms mainly rely on pure visual methods and do not fully utilize point cloud data for multimodal fusion, resulting in limited reconstruction accuracy and detail expression. For example, although the three-dimensional reconstruction method based on a monocular camera can obtain seabed topography information, it is insufficient in handling complex environments and improving accuracy. In addition, the application of modern image processing technology in deep-sea high-definition image analysis is also mainly focused on image preprocessing and feature extraction, lacking the integration of point cloud data. Therefore, the application effect of existing algorithms in deep-sea environments is limited, and it is urgent to introduce multimodal information such as point cloud data to improve the reconstruction quality. Summary of the invention

[0011] To solve the above problems, the present invention takes the reconstruction of deep-sea images and video screens as the application background, obtains the depth information map of the corresponding deep-sea image by the line laser scanning method, and obtains the corresponding original deep-sea degraded image by the high-definition RGB camera, and then obtains the reconstructed deep-sea image screen true value photo by adding a polarizer and estimating the polarization-related physical model. Through pixel-level alignment of images, the present invention establishes a set of deep-sea image reconstruction data sets, in which each set of data simultaneously contains the original deep-sea image, the corresponding depth map and the deep-sea image reconstruction true value photo. Finally, the present invention conducts in-depth research on the physical model of deep-sea optical degradation, and designs a multimodal deep-sea image reconstruction network that can reconstruct deep-sea image screens, so that it can fully combine depth information, appropriately correct the color deviation and blur of deep-sea images, and use the network for appropriate optimization, so that it can be applied to the real-time reconstruction scene of deep-sea video.

[0012] The present invention provides a deep-sea optical image reconstruction method based on vision and point cloud, comprising the following steps:

[0013] S10, obtaining the original RGB image, the RGB true value image and the depth map;

[0014] S20, align the pixels of the three images to obtain an RGBD data set;

[0015] S30, uses the RGB-D dataset for training and testing the deep-sea image reconstruction network to generate reconstructed images.

[0016] Preferably, the S10 comprises the following steps:

[0017] S11, using an RGB camera to obtain an original RGB image;

[0018] S12, assembling a polarizing plate on the RGB camera to obtain an RGB true value image;

[0019] S13, RGB camera combined with line beam laser to obtain point cloud information and depth map.

[0020] Preferably, after the polarizer is installed on the RGB camera in S12, each time data collection starts, the RGB camera is aimed at infinity in the water, and the brightest picture is captured by adjusting the polarizer. And the darkest picture Since there is no object in the picture at this time, it is considered that the light in the picture at this time is all scattered light from the water, that is:

[0021] in, and are the scattered light intensities corresponding to when the polarizer is rotated to the brightest and darkest positions of the image; if no polarizer is defined, the scattered light intensity obtained by directly shooting with the RGB camera is Then there is Then define the elimination coefficient This elimination coefficient is fixed in the same water body, and the true value image is obtained in subsequent object shooting;

[0022] In each subsequent shooting process of the object, the polarizer is rotated to obtain the brightest and darkest images, which are recorded as I and max and I min , the picture at this time contains both transmitted light and scattered light, and the transmitted light is isotropic, so there is

[0023] Where S is the transmitted light intensity;

[0024] And by

[0025] The intensity of light transmitted by the object can be obtained

[0026] The image obtained after this calculation can be used as the true value map of the reconstructed image required in the RGBD dataset.

[0027] Preferably, when the laser line emitted by the line beam laser in S13 is projected onto the surface of an object, if the surface of the object is higher than the reference plane, the laser line will be distorted and thus captured by an RGB camera. Through a calibrated laser data acquisition system and laser triangulation, a corresponding depth true value map can be obtained.

[0028] Preferably, each set of data in the RGBD data set in S20 includes an original RGB image, a reconstructed RGB image, i.e., an RGB true value image, and a corresponding depth map.

[0029] Preferably, the step S30 specifically includes the following steps:

[0030] S31, the encoding structure extracts image features of different scales through multiple downsampling and convolution to form an image pyramid;

[0031] S32, the encoder encodes the position of the smallest scale feature map and sends it to a Transformer structure with a multi-head attention mechanism to extract and reconstruct the features;

[0032] S33, the image pyramid is also fed into a residual network with additional positional encoding embedding to extract features from multi-scale information;

[0033] S34, in the decoding structure, the decoder includes several upsampling convolution modules, which upsample the output feature map of the encoder multiple times, thereby reconstructing the image pyramid into a corresponding reconstructed image.

[0034] Preferably, the encoder and decoder are regarded as generators, and a discriminator is added to form a generative adversarial network structure. The discriminator performs multiple downsampling and convolution on the RGB reconstructed true value image in the dataset, and compares the obtained feature map with the image pyramid output by the generator, and calculates the loss in the color LAB space.

[0035] Preferably, the residual network includes a convolution module, a pooling module, an average pooling module and a fully connected module.

[0036] Preferably, a flattening module, a long short-term memory network and an upsampling module are sequentially arranged after the residual network.

[0037] The beneficial effects of the present invention include at least:

[0038] (1) Improved the effect of deep-sea image reconstruction Through the application of multimodal deep learning models, this method can better match and utilize visual and point cloud data for image reconstruction, greatly improving the effect of deep-sea image reconstruction compared to existing methods. Especially in deep-sea environments with turbid water, it can effectively deal with adverse factors such as water scattering and particulate matter absorbing light, and achieve reconstruction of deep-sea turbid scene images or distant background images.

[0039] (2) It is conducive to the development of new deep-sea visual aids. Through targeted optimization of the method, the present invention optimizes the processing delay of the algorithm and improves the efficiency of image reconstruction. The application of deep learning models also improves end-to-end processing capabilities, simplifies the reconstruction process, and lowers the application threshold.

[0040] (3) Promote the development of deep-sea resources and support marine scientific research. High-definition, detailed and information-rich deep-sea images are of great significance for the exploration and development of deep-sea resources. The deep-sea image reconstruction technology provided by the present invention can provide reliable data support for deep-sea scientific research and economic activities, and promote the effective development and utilization of deep-sea resources. At the same time, high-quality deep-sea images are of great value for deep-sea navigation, diving, understanding biodiversity, and paying attention to global environmental changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flowchart of the steps of the deep-sea optical image reconstruction method based on vision and point cloud of the present invention;

[0042] Figure 2 A schematic diagram of a laser data acquisition system and acquisition screen of a deep-sea optical image reconstruction method based on vision and point cloud according to a specific embodiment of the present invention;

[0043] Figure 3 A schematic diagram of the network structure of a deep-sea optical image reconstruction method based on vision and point cloud according to a specific embodiment of the present invention;

[0044] Figure 4 The overall network structure diagram of the deep-sea optical image reconstruction method based on vision and point cloud according to a specific embodiment of the present invention;

[0045] Figure 5 This is a network structure diagram for improving the real-time reconstruction scene of the deep-sea optical image reconstruction method based on vision and point cloud according to a specific embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0047] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.

[0048] The main process of the present invention is as follows Figure 1 As shown, S10, the original RGB image, the RGB true value image and the depth map are obtained;

[0049] S20, align the pixels of the three images to obtain an RGBD data set;

[0050] S30, uses the RGB-D dataset for training and testing the deep-sea image reconstruction network to generate reconstructed images.

[0051] The technical solution uses a high-definition RGB camera that can be equipped with a polarizer to shoot the original RGB (red, green, and blue) images and RGB true value images in the data set, and obtains high-precision point cloud information and depth map information through a laser data acquisition system including a green beam laser. After the images are corrected and affine transformed by a series of internal and external parameter matrices, these images achieve pixel-level alignment in terms of viewing angle, thereby obtaining a complete set of RGBD data sets, each of which contains high-definition original RGB images, ideal reconstructed RGB images (i.e., RGB true value images) and corresponding depth maps. Finally, the present invention uses the data set obtained by the above method for training and testing deep-sea image reconstruction networks. Since the objects in deep-sea images are usually of different sizes and distances, and the contour information loss of distant objects is relatively serious, the network of the present invention combines the attention structure of Vision Transformer that combines position information with the residual connection structure of ResNet, adds the mechanism of image pyramid, and designs a depth map information extraction branch and a multimodal information fusion module. In order to better restore information of different scales in deep-sea images, the present invention uses an image pyramid to perform convolution and feature extraction at different scales and fuse them at the highest feature scale; and the present invention uses a position embedding design to enable information of objects at different positions and close distances to be shared and optimized together. In response to the loss of contour information of distant objects, the present invention calculates the depth map through point cloud information, and fuses the RGB image and depth features through a depth map feature extraction branch and fusion module, thereby better restoring the missing object visual information in the RGB image.

[0052] S10 includes the following steps:

[0053] S11, using an RGB camera to obtain an original RGB image;

[0054] S12, assembling a polarizing plate on the RGB camera to obtain an RGB true value image;

[0055] S13, RGB camera combined with line beam laser to obtain point cloud information and depth map.

[0056] In order to obtain the desired reconstructed image while capturing the original RGB image, physical means are needed to filter the color cast and uneven brightness caused by the absorption and scattering of light by deep sea water. Considering that the scattered light caused by deep sea water is usually anisotropic, while the transmitted light reflected by the object itself is isotropic, it is possible to consider installing a polarizer on the RGB camera lens to capture the corresponding polarized image.

[0057] After the polarizer is installed on the RGB camera in S12, each time you start collecting data, aim the RGB camera at infinity in the water and adjust the polarizer to capture the brightest picture. And the darkest picture Since there is no object in the picture at this time, it is considered that the light in the picture at this time is all scattered light from the water, that is:

[0058] in, and are the scattered light intensities corresponding to when the polarizer is rotated to the brightest and darkest positions of the image; if no polarizer is defined, the scattered light intensity obtained by directly shooting with the RGB camera is Then there is Then define the elimination coefficient This elimination coefficient is fixed in the same water body, and the true value image is obtained in subsequent object shooting;

[0059] In each subsequent shooting process of the object, the polarizer is rotated to obtain the brightest and darkest images, which are recorded as I and max and I min , the picture at this time contains both transmitted light and scattered light, and the transmitted light is isotropic, so there is

[0060] Where S is the transmitted light intensity;

[0061] And by

[0062] The intensity of light transmitted by the object can be obtained

[0063] The image obtained after this calculation can be used as the true value map of the reconstructed image required in the RGBD dataset.

[0064] Considering that deep sea water has different absorption coefficients and scattering coefficients for light of different wavelengths, the transmission attenuation coefficient of long-wavelength light in water is relatively large. This invention uses green laser light to scan the seabed topography, and cooperates with a high-definition RGB camera to form a laser data acquisition system. For system design and captured images, see Figure 2 When the laser line is projected onto the surface of an object, if the surface of the object is higher than the reference plane, the laser line will be distorted and thus captured by a high-definition camera. Figure 3 As shown. Through a calibrated laser data acquisition system, the corresponding depth truth map can be obtained through laser triangulation.

[0065] When the laser line emitted by the line beam laser in S13 is projected onto the surface of an object, if the surface of the object is higher than the reference plane, the laser line will be distorted and thus captured by the RGB camera. Through a calibrated laser data acquisition system and laser triangulation, the corresponding depth truth map can be obtained.

[0066] Each set of data in the RGBD dataset in S20 contains the original RGB image, the reconstructed RGB image, i.e., the RGB true value image, and the corresponding depth map.

[0067] Since deep-sea images usually contain objects of different scales and distances, and the overall picture brightness is uneven, this paper adds a multi-head attention mechanism and a position embedding mechanism to address this problem, so that it can extract and reconstruct the features of objects of different scales, and make full use of the features of different positions to complement each other, thereby improving the local quality of image reconstruction. The network structure of the pure vision branch can be found in Figure 3 .

[0068] The innovation of this method lies in the ability to collect depth truth and reconstructed image truth in complex deep-sea environments through a calibrated system, providing a high-quality data set for the deep-sea image reconstruction system. The specific steps and system design of the deep-sea reconstruction image truth collection method are protected, including how to collect true value images through a polarization camera and the subsequent data processing process.

[0069] S30 specifically includes the following steps:

[0070] S31, the encoding structure extracts image features of different scales through multiple downsampling and convolution to form an image pyramid;

[0071] S32, the encoder encodes the smallest scale feature map by position and feeds it into a Transformer structure with a multi-head attention mechanism ( Figure 3 ViT deep feature extraction module) to extract and reconstruct features;

[0072] S33, the image pyramid is also fed into a residual network with an additional positional encoding embedding ( Figure 3 The residual multi-scale feature extraction network is used to extract features from multi-scale information;

[0073] S34, in the decoding structure, the decoder includes several upsampling convolution modules, which upsample the output feature map of the encoder multiple times, thereby reconstructing the image pyramid into a corresponding reconstructed image.

[0074] The network structure designed by the present invention includes encoding and decoding structures.

[0075] The encoding structure extracts image features of different scales through multiple downsampling and convolution to form an image pyramid. Subsequently, the encoder position-encodes the feature map of the smallest scale and feeds it into a Transformer structure with a multi-head attention mechanism to extract and reconstruct the features. The Transformer structure contains modules such as LinearProjection, Position Embedding, Layer Norm, Multi-Head Attention, and Multi-Layer Perception. These modules divide the input feature map into several small blocks, embed position information, and calculate the correlation between small feature blocks one by one. Finally, several regularizations and linear transformations are performed to obtain the final processed feature map, which can be used to complete some features of the image pyramid. At the same time, the image pyramid is also fed into a residual network with additional position encoding embedding to extract features of multi-scale information. The residual network contains multiple modules, namely Conv - Convolution, Pool - Pooling, Avg Pool - Average Pooling, fc - full connection, etc.

[0076] In the decoding structure, the decoder contains several upsampling convolution modules, which upsample the output feature map of the encoder multiple times to reconstruct the image pyramid into the corresponding reconstructed image.

[0077] The innovation of this reconstruction method lies in the fact that it uses a multi-scale image pyramid structure and a multi-modal information input design to reconstruct deep-sea objects of different sizes and blurred images in the distance, achieving relatively accurate and efficient deep-sea image reconstruction. The core deep learning network structure design used for image reconstruction in this method is protected.

[0078] In order to enable the reconstruction network to effectively perform supervised training, the present invention regards the encoder and decoder described above as generators, and adds a discriminator to form a GAN-Generative Adversarial Networks generative adversarial network structure. The discriminator will perform multiple downsampling and convolution on the RGB reconstructed true value photos in the data set, and compare the obtained feature map with the image pyramid output by the generator, and calculate the loss in the color LAB space. By calculating the loss through the image pyramid, the gradient vanishing problem can be effectively avoided, and the convergence of the deep network can be better promoted.

[0079] Because the contour information of distant objects in the image is often severely lost, it is extremely difficult to reconstruct it through a pure vision deep learning network. In order to use the depth map information to complete the information of the distant object contour in the image, the present invention adds a depth map feature extraction and fusion structure to the deep learning network as a supplement to the pure vision network backbone. The network structure with deep feature fusion designed by the present invention is shown in Figure 1. Figure 5 .

[0080] This branch uses the structure of the convolutional network and is divided into an encoder and a decoder. Through dual-stream input, the encoder extracts features from the depth map in multiple layers and integrates them into the visual image pyramid in sequence. The decoder decodes the image pyramid that combines both visual information and depth information to obtain a reconstructed image that combines depth map information.

[0081] For deep-sea video reconstruction scenarios, the deep-sea video images are reconstructed in real time by streamlining the algorithm and taking into account the previous and next frames of the video. In order to apply the deep learning network described in the present invention to scenarios with high real-time requirements, such as underwater visual assistance systems, the present invention simplifies and optimizes the aforementioned network so that it can be applied to scenarios with high real-time requirements, and even real-time image reconstruction can be achieved in some scenarios. In addition, the improved network model can make full use of the previous and next frame information to reconstruct the current picture frame through the design of long and short-term memory structures. At the same time, the design of long and short-term memory makes the weight of newer picture frames relatively high, while older picture frames are gradually forgotten, which also meets the objective requirements of the usage scenarios of the present invention. See the improved network model for details. Figure 5 , the flattening module Flatten, the long short-term memory network LSTM and the upsampling module Upsampling are set up in sequence after the residual network. The improved structure takes into account the actual application scenarios, retains only the pure visual branch, and prunes the pure visual branch to ensure that the overall network latency is low. At the same time, the improved network structure adds LSTM-Long Short Term Memory long short-term memory network and corresponding supporting modules, so that the network retains the corresponding previous and next frame information when processing video stream input. By using the previous and next frame information, it can greatly reduce the complexity of the algorithm while ensuring the reconstruction effect.

[0082] This improvement simplifies the network structure to a certain extent, retains appropriate weights for the trained deep learning network, removes inefficient modules, and adopts a composite structure design of convolution-long short-term memory. It fully utilizes the correlation between previous and next frames of the video, reduces the complexity of the algorithm, and realizes real-time system.

[0083] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A deep-sea optical image reconstruction method based on vision and point cloud, characterized in that: The following steps are involved: S10, obtaining the original RGB image, the RGB true value image and the depth map; S20, align the pixels of the three images to obtain an RGBD data set; S30, uses the RGB-D dataset for training and testing the deep-sea image reconstruction network to generate reconstructed images.

2. The deep-sea optical image reconstruction method based on vision and point cloud according to claim 1, characterized in that: The S10 comprises the following steps: S11, using an RGB camera to obtain an original RGB image; S12, assembling a polarizing plate on the RGB camera to obtain an RGB true value image; S13, RGB camera combined with line beam laser to obtain point cloud information and depth map.

3. The deep-sea optical image reconstruction method based on vision and point cloud according to claim 2, characterized in that: After the polarizer is installed on the RGB camera in the S12, each time the data is collected, the RGB camera is aimed at infinity in the water, and the brightest picture is captured by adjusting the polarizer. And the darkest picture Since there is no object in the picture at this time, it is considered that the light in the picture at this time is all scattered light from the water, that is: in, and are the scattered light intensities corresponding to when the polarizer is rotated to the brightest and darkest positions of the image; if no polarizer is defined, the scattered light intensity obtained by directly shooting with the RGB camera is Then there is Then define the elimination coefficient This elimination coefficient is fixed in the same water body, and the true value image is obtained in subsequent object shooting; In each subsequent shooting process of the object, the polarizer is rotated to obtain the brightest and darkest images, which are recorded as I and max and I min , the picture at this time contains both transmitted light and scattered light, and the transmitted light is isotropic, so there is Where S is the transmitted light intensity; And by The intensity of light transmitted by the object can be obtained The image obtained after this calculation can be used as the true value map of the reconstructed image required in the RGBD dataset.

4. The deep-sea optical image reconstruction method based on vision and point cloud according to claim 2, characterized in that: When the laser line emitted by the line beam laser in S13 is projected onto the surface of an object, if the surface of the object is higher than the reference plane, the laser line will be distorted and thus captured by the RGB camera. Through a calibrated laser data acquisition system and laser triangulation, the corresponding depth true value map can be obtained.

5. The deep-sea optical image reconstruction method based on vision and point cloud according to claim 1, characterized in that: Each set of data in the RGBD data set in S20 includes an original RGB image, a reconstructed RGB image, i.e., an RGB true value image, and a corresponding depth map.

6. The deep-sea optical image reconstruction method based on vision and point cloud according to claim 1, characterized in that: The S30 specifically includes the following steps: S31, the encoding structure extracts image features of different scales through multiple downsampling and convolution to form an image pyramid; S32, the encoder encodes the position of the smallest scale feature map and sends it to a Transformer structure with a multi-head attention mechanism to extract and reconstruct the features; S33, the image pyramid is also fed into a residual network with additional positional encoding embedding to extract features from multi-scale information; S34, in the decoding structure, the decoder includes several upsampling convolution modules, which upsample the output feature map of the encoder multiple times, thereby reconstructing the image pyramid into a corresponding reconstructed image.

7. The deep-sea optical image reconstruction method based on vision and point cloud according to claim 6, characterized in that: The encoder and decoder are regarded as generators, and a discriminator is added to form a generative adversarial network structure. The discriminator will perform multiple downsampling and convolution on the RGB reconstructed true value images in the dataset, and compare the obtained feature maps with the image pyramid output by the generator, and calculate the loss in the color LAB space.

8. The deep-sea optical image reconstruction method based on vision and point cloud according to claim 6, characterized in that: The residual network includes a convolution module, a pooling module, an average pooling module and a fully connected module.

9. The deep-sea optical image reconstruction method based on vision and point cloud according to claim 6, characterized in that: A flattening module, a long short-term memory network and an upsampling module are sequentially arranged after the residual network.

Images