Water body interference removal method for terrain reconstruction of deep sea binocular camera

Abstract

The invention belongs to the technical field of deep sea image processing, and particularly relates to a water body interference removal method for deep sea binocular camera terrain reconstruction, which comprises the following steps of: firstly, constraining an image acquired by a binocular camera by adopting two different deep learning networks, and outputting a parallax matrix constrained by the deep learning networks; removing a seawater area in the Lab color space by using a threshold method to obtain an initial effective area, and sequentially performing expansion, hole filling and corrosion morphological operation on the initial effective area to obtain a final effective area and an ineffective area; and mapping the final effective region into the constrained parallax matrix, and setting a parallax value corresponding to the invalid region as an invalid value, thereby realizing removal of water interference. According to the scheme, interference caused by the pure seawater region in the binocular image is removed through the processes of double deep learning network constraint, color space conversion and threshold method operation, morphological operation and the like, the continuity and integrity of the image region are ensured, and the segmentation precision is effectively improved.

Description

Technical Field

[0001] The invention belongs to the technical field of deep-sea image processing, and in particular relates to a method for removing water interference for deep-sea binocular camera terrain reconstruction. Background Art

[0002] With the continuous deepening of deep-sea scientific research and marine resource development, the application of underwater visual sensing technology in topographic surveys, deep-sea environmental monitoring, marine life observation and other fields has become increasingly important. As one of the core devices of underwater visual sensing, binocular cameras can achieve three-dimensional reconstruction through stereo vision technology, providing scientists and engineers with accurate deep-sea topographic information.

[0003] Ideally, the 3D reconstruction of a stereo camera should focus on the actual surface of objects, but due to the optical properties of water bodies, pure seawater areas are sometimes mistaken as sources of reconstructible depth information, resulting in unnecessary 3D reconstruction. This phenomenon stems from the transparency, reflection properties, or light refraction effects of water bodies, which interfere with the camera's depth estimation and introduce unnecessary "false depth information" into the 3D model. Pure seawater areas that are mistakenly reconstructed as objects with depth will affect the accuracy of reconstruction and scene understanding. For example, planes or areas that should not exist may appear in the 3D model, making the shape and position of the actual object unclear, which has an adverse effect on deep-sea exploration missions.

[0004] Therefore, there is an urgent need for a technology that can effectively distinguish between pure seawater areas and actual object surfaces, reduce or remove the impact of pure seawater areas in three-dimensional terrain reconstruction, improve the three-dimensional reconstruction accuracy of binocular cameras, and thereby improve the accuracy of deep-sea exploration and research. Summary of the invention

[0005] In view of the problem that pure seawater areas existing in images taken by binocular cameras in the prior art introduce a large amount of noise, which affects the accuracy of three-dimensional reconstruction, the present invention proposes a water interference removal method for deep-sea binocular camera terrain reconstruction, which is used to segment the pure seawater area and the effective information area in the image. It can not only accurately identify the pure seawater area in the image, but also effectively reduce the noise caused by deep-sea water interference, thereby greatly improving the accuracy of three-dimensional terrain reconstruction.

[0006] The present invention is implemented by adopting the following technical scheme: a method for removing water interference for deep-sea binocular camera terrain reconstruction, comprising the following steps:

[0007] Step A: using two different deep learning networks to constrain the images collected by the binocular camera, and outputting the disparity matrix D constrained by the deep learning network;

[0008] Step B: for the image captured by the binocular camera, the seawater area is removed by using the threshold method in the Lab color space to obtain the initial effective area BW;

[0009] Step B1: Convert the image collected by the binocular camera from the RGB color space to the Lab color space, and use the threshold method to preliminarily constrain the images of the three channels, L channel, a channel, and b channel. Assume that the distribution of most of the pure seawater areas in the three channels is as follows:

[0010] Step B2: According to different application scenarios and requirements, the methods for obtaining the initial effective area include the following:

[0011] (1) In situations where the requirements are high, the brightness channel and the color channel are constrained, and the segmentation method that retains the effective area to the greatest extent is:

[0012]

[0013] (2) For applications with low requirements, the operation to remove seawater areas to the greatest extent is:

[0014]

[0015] (3) For the first two cases, since the image is sensitive to brightness during segmentation, a dynamic threshold based on statistical principles is used. replace As shown below:

[0016]

[0017] in, Indicates the minimum threshold of the L channel, which is used to determine the lower limit of the brightness of the seawater area. represents the maximum threshold of the L channel, which is used to determine the upper limit of the brightness of the seawater area. L(i,j) represents the brightness value of the pixel in the Lab color space. a(i,j) represents the a channel value of the pixel in the Lab color space, representing the range from green to red. Indicates the minimum threshold of channel a, which is used to determine the lower limit of the green to red range in the seawater area. represents the maximum threshold of the a channel, which is used to determine the upper limit of the green to red range in the seawater area. b(i,j) represents the b channel value of the pixel in the Lab color space, representing the range from blue to yellow. Indicates the minimum threshold of the b channel, which is used to determine the lower limit of the blue to yellow range of the seawater area. represents the maximum threshold of the b channel, which is used to determine the upper limit of the blue to yellow range in the seawater area, μ 海水统计 With σ 海水统计is the mean and standard deviation of the pixel values ​​of the L channel of the pure seawater area in more than ten pictures of different scenes, μ 全图 With σ 全图 is the mean and standard deviation of all pixel values ​​of the L channel of the image to be segmented, W1 and W2 are weights, k is a coefficient, M is the binary mask of the segmented seawater area, M(i,j) is the element in the binary mask of the segmented seawater area, and the initial effective area is the opposite part, which is expressed as:

[0018]

[0019] Where BW is the initial valid area, and BW(i,j) is the element in the initial valid area;

[0020] Step C, performing dilation, hole filling and corrosion morphological operations on the initial valid area BW in sequence to obtain the final valid area BW3 and the invalid area;

[0021] Step D: Map the final valid area BW3 to the constrained disparity matrix D obtained in step A, and set the disparity value corresponding to the invalid area to an invalid value, thereby achieving the removal of water interference.

[0022] Furthermore, step C is specifically implemented in the following manner:

[0023] (1) First, the initial effective area is expanded, and the mathematical description is as follows:

[0024]

[0025] Among them, BW1 is the area after the dilation operation, and B1 is the structural element;

[0026] (2) Then fill the holes in the area after the dilation operation. If the photographed target contains holes, skip this step and assume that the output after filling the holes in the BW1 area is BW2;

[0027] (3) Finally, the output after filling the holes is corroded to restore the image to its original size before expansion:

[0028]

[0029] Among them, BW3 is the final valid area after the corrosion operation, BW2 is the area after the hole filling operation, and B2 is the structural element.

[0030] Furthermore, in step D, the specific mapping formula is expressed as:

[0031]

[0032] Among them, D NoWaterD is the parallax matrix that finally removes the interference of pure seawater area. NoWater (i, j) is the final disparity value after removing the interference of pure seawater area, D(i, j) is the disparity value constrained by the dual deep learning network, BW3(i, j) is the element in the final effective area, and the final 3D terrain point cloud is obtained by combining the internal parameters of the camera. The point cloud restored by multiple frames of images is stitched together to restore the deep-sea 3D terrain map without the interference of pure seawater area. Similarly, the segmented area can be mapped to the image to obtain the image without the seawater area.

[0033] Furthermore, step A is specifically implemented in the following manner:

[0034] Assume that D1 and D2 are disparity matrices generated by two deep learning networks based on the same pair of binocular camera stereo images, then their disparity difference ΔD(i,j) is expressed as follows;

[0035] ΔD(i,j)=|D1(i,j)-D2(i,j)|

[0036] if ΔD(i,j)>T, mark D1(i,j) and D2(i,j) as invalid values

[0037] Among them, T is a fixed threshold set, D1(i, j) and D2(i, j) are the disparity values ​​generated by the two deep learning networks at the image pixel point (i, j), and any one of the marked D1 or D2 is used as the disparity matrix D constrained by the dual deep learning networks.

[0038] Furthermore, in step A, the original disparity matrix is ​​obtained through a single deep learning network to replace the constrained disparity matrix. After the threshold method and morphological operation of the Lab color space, BW3 is directly mapped to the original disparity matrix to also achieve the removal of water interference.

[0039] Compared with the prior art, the advantages and positive effects of the present invention are:

[0040] This solution can remove some water interference in seawater through dual neural network constraints, and can also remove most of the random noise and incorrectly matched points, eliminating the subsequent point cloud denoising process; through the image segmentation algorithm, combined with the brightness and color channel characteristics of the Lab color space, it can accurately identify and segment the pure seawater area in the image, thereby greatly reducing the noise interference caused by these areas. The algorithm of this solution is simple and has a very high running speed; by performing morphological operations on the effective area, the continuity and integrity of the image area are ensured, thereby improving the segmentation accuracy; it provides a reliable and efficient solution for removing water interference for binocular camera 3D reconstruction in deep-sea environments, and has broad application prospects and practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flow chart of the interference removal method according to an embodiment of the present invention;

[0042] Figure 2 Schematic diagram of the statistical results of the seawater area according to an embodiment of the present invention, (a) is the brightness distribution of the L channel, (b) is the color distribution of the a channel, and (c) is the color distribution of the b channel;

[0043] Figure 3 This is a schematic diagram of global brightness statistics results according to an embodiment of the present invention;

[0044] Figure 4 Schematic diagram of an original image and an original point cloud according to an embodiment of the present invention, (a) is an original image, and (b) is an original point cloud image;

[0045] Figure 5 Schematic diagram of an image and a point cloud after removing seawater interference according to an embodiment of the present invention, (a) is an image after removing interference, and (b) is a point cloud image after removing interference;

[0046] Figure 6 Schematic diagram of the effect of image segmentation according to an embodiment of the present invention; (a) is an unsegmented image, and (b) is a segmented image;

[0047] Figure 7 The three-dimensional topographic map without removing seawater interference in the embodiment of the present invention;

[0048] Figure 8 This is a three-dimensional topographic map with seawater interference removed according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described below in conjunction with the accompanying drawings and embodiments. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can also be implemented in other ways different from those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0050] A water interference removal method for deep-sea binocular camera terrain reconstruction, combined with Figure 1 As shown, the following steps are included:

[0051] Step 1: Use two different deep learning networks to constrain the images collected by the binocular camera and output the disparity matrix constrained by the deep learning network;

[0052] Pure seawater areas usually have no obvious texture features, which can easily lead to noise and errors in disparity estimation. When a single network performs depth prediction in these areas, false depth points may be generated, thus affecting the overall structure of the 3D reconstruction. This example uses two different deep learning networks for constraints. False depth points will be filtered out due to inconsistent predictions of the two networks, which can effectively reduce the generation of false depth points and avoid the appearance of unnecessary surfaces or structures in the 3D model. The disparity difference ΔD(i,j) is described by the following formula:

[0053] ΔD(i,j)=|D1(i,j)-D2(i,j)|

[0054] if ΔD(i,j)>T, mark D1(i,j) and D2(i,j) as invalid values

[0055] Among them, D1(i,j) and D2(i,j) are the disparity values ​​generated by the two deep learning networks at the image pixel point (i,j); T is a fixed threshold, which is usually set to T<5, and the smaller the value, the smaller the difference allowed between the two disparity matrices. The disparity matrices obtained by the two deep learning networks should be the same, and the points with disparity differences greater than the threshold are set to invalid values. Finally, any of the marked matrices D1 or D2 is used as the result D of the dual neural network constraint output.

[0056] Step 2: For the image captured by the binocular camera, the threshold method is used in the Lab (CIE Lab) color space to remove the seawater area and obtain the initial effective area;

[0057] The image captured by the binocular camera is converted from RGB color space to Lab (CIE Lab) color space, and the image is initially constrained by using the threshold method of three channels (L channel, a channel and b channel), and the image is roughly divided into a pure seawater area and an initial effective area;

[0058] Through a large number of statistics on deep-sea images, it is found that the differences between pure seawater areas and normal target areas in deep-sea images are mainly the following two points: 1) The brightness of pure seawater areas is usually low because they lack the reflected light of effective targets; 2) Due to the properties of seawater itself, pure seawater areas often appear blue-green. These two properties are most obvious in the Lab (CIELab) color space, so using the threshold method in the Lab color space is the best choice.

[0059] Assume that most (but not all) of the pure seawater area is distributed among the three channels as follows:

[0060]

[0061] Constrain the brightness channel and color channel separately, and the segmentation method that retains the valid area to the greatest extent is:

[0062]

[0063] In some occasions where the requirements are not high, using only the L (brightness) channel and the a (green-red) channel or the L (brightness) channel and the b (blue-yellow) channel can also achieve a better segmentation effect. The operation to remove the seawater area to the greatest extent is:

[0064]

[0065] Since the image is more sensitive to brightness during the segmentation process, a dynamic threshold based on statistical principles can be used in situations with higher requirements. replace As shown below:

[0066]

[0067] Among them, μ 海水统计 With σ 海水统计 is the mean and standard deviation of the pixel values ​​of the L channel of the pure seawater area in more than ten pictures of different scenes, μ 全图 With σ 全图 is the mean and standard deviation of all pixel values ​​in the L channel of the image to be segmented. The formula represents the dynamic threshold The bright part in the seawater area (μ 海水统计 +2σ 海水统计 ) and the darker part (μ 全图 -kσ 全图 ) are jointly restricted. The coefficient k in the formula is generally set to 1.28 (the darker 10% in the normal distribution), and can also be adjusted according to actual conditions. W1 and W2 are weights. The smaller the value of W1, the higher the dynamic threshold. The greater the influence of the brightness of the image itself, the greater the influence of the brightness of the image itself. Usually W1 is 0.7, and the solution formula of W2 is as follows:

[0068] W1(μ 海水统计 +2σ 海水统计 )+W2(μ 全图统计 -1.28σ 全图统计 )=(μ 海水统计 +2σ 海水统计 )

[0069] where μ 全图统计 With σ 全图统计 is the cumulative statistical mean and standard deviation of all pixel values ​​of the L channel in more than ten pictures of different scenes, so that In (μ 海水统计 +2σ海水统计 ) fluctuates around.

[0070] The M(i,j) in the formula is the binary mask of the segmented seawater area, and the initial effective area should be the opposite part, that is:

[0071]

[0072] The BW(i,j) part is the initial effective area.

[0073] Step 3: Perform morphological operations on the initial valid area BW, including dilation, hole filling, and corrosion operations;

[0074] (1) First, dilate the initial valid area: In a binary image, some small gaps or broken areas in the foreground may cause the segmentation result to be incoherent. These discontinuous areas can be connected through the dilation operation. Dilation can fill the small discontinuous areas to make the entire area more continuous. The mathematical description is as follows:

[0075]

[0076] Wherein BW1 is the area after the dilation operation, B1 is the structural element, and a disk is selected in the present invention. The radius is adjusted according to the image resolution, and the radius is usually set to 5 at a resolution of 1920x1080.

[0077] (2) Then, the hole filling is performed on the area after the dilation operation. The purpose of hole filling is to eliminate some holes inside the BW1 area. Usually, there are no holes in the seabed terrain in deep-sea images. Filling holes can make the entire area in the image more continuous. However, in some special cases, the target itself contains holes, so this step should be skipped. Assume that the output after the holes in the BW1 area are filled is BW2.

[0078] (3) Finally, the output after filling the holes is corroded to return the image to its original size before dilation, ensuring that the image does not over-expand after filling the gaps and holes. Finally, the segmented area can be output:

[0079]

[0080] Among them, BW3 is the final effective area after the corrosion operation, BW2 is the area after the hole filling operation, and B2 is a structural element. In the present invention, a disk is selected, and the radius is the same as or slightly larger than B1.

[0081] Step 4: Map the final valid area obtained by segmentation in step 3 to the disparity matrix D constrained by the dual deep learning network obtained in step 1, and set the disparity value corresponding to the invalid area (the area where BW3(i,j)=0) to an invalid value, thereby effectively removing most of the noise interference caused by deep sea water or distant dark areas.

[0082] Since the segmentation based on the threshold method and morphological operation is not in the disparity matrix, the segmentation result is mapped to the disparity matrix, and the disparity values ​​of the invalid areas in the segmented area are all set to invalid values. The specific mapping formula is as follows:

[0083]

[0084] By combining the disparity matrix D that finally removes the interference of the pure seawater area NoWater The final 3D terrain point cloud (the point cloud is formed by three-dimensional reconstruction of the disparity matrix) can be obtained by combining the point clouds restored from multiple frames of images. The deep-sea 3D terrain map without interference from pure seawater areas can be restored. Similarly, the segmented area can be mapped to the image to obtain an image without the seawater area.

[0085] In addition, it should be noted that in the solution of the present invention, step 1 does not need to use two different deep learning networks to constrain the images collected by the binocular camera, but instead obtains the original disparity matrix through a single deep learning network, and uses the threshold method and morphological operations of the Lab color space to directly map BW3 to the original disparity matrix, which can also achieve better results.

[0086] In order to more clearly understand the solution of the present invention, the advantages of this solution are described in detail below using a practical application case as an example:

[0087] Example 1: Use the Lab color space threshold method and morphological operations to remove seawater from deep-sea hydrothermal vent images:

[0088] Step 1: Dynamic threshold preparatory work: Count the pixel values ​​of Lab space of deep sea pictures taken at the same time. This embodiment counts the seawater area of ​​25 pictures and obtains (μ 海水统计 +2σ 海水统计 ) is about 35. At the same time, the L channel of 12 deep-sea images of different scenes taken during the same period was statistically analyzed, and (μ 全图统计 -1.28σ 全图统计 ) is approximately 9.55.

[0089] Taking W1 as 0.7, we can calculate that W2 is about 1.1, so

[0090] Step 2: Take the left camera image and convert it to Lab color space for segmentation using the threshold method. The values ​​in this example are:

[0091]

[0092] Pick as the initial effective area.

[0093] Step 3: Perform morphological operations of dilation, hole filling, and erosion on the initial effective area BW. In this example, the disk radius of B1 in the dilation operation is 5, and the disk radius of B2 in the erosion operation is 5, to obtain the final effective area BW3 after segmentation.

[0094] Step 4: Map the final valid area to the image and disparity matrix to obtain the image and point cloud without the seawater area. The final result is as follows: Figure 4 , Figure 5 shown.

[0095] Example 2: 3D terrain reconstruction using binocular cameras in the Hippocampus cold spring area:

[0096] Preparatory work: Complete the underwater system calibration, distortion correction and stereo correction of the deep-sea binocular camera, obtain the internal parameters of the camera and the corrected left and right images of the binocular camera to improve the accuracy of 3D reconstruction.

[0097] Step 1: Pass the left and right images of the binocular camera into two deep learning networks with different architectures to obtain two different disparity matrices (based on the left camera). This example uses two public networks MC-Stereo and RAFT-Stereo. The formula ΔD(i,j)=|D1(i,j)-D2(i,j)| is used to determine the difference in the disparity values ​​of the two disparity matrices. The value of D1(i,j) where ΔD(i,j)>T is set to an invalid value, and then the marked D1 is output as D, where T=2.

[0098] Step 2: Take the left camera image and convert it to Lab color space for segmentation using the threshold method. The values ​​in this example are:

[0099] M(i,j)=(0≤L(i,j)≤38)∧((-25≤a(i,j)≤0)∧(-60≤b(i,j)≤0))

[0100] Thereby, pure seawater is roughly separated from the dark area. as the initial effective area.

[0101] Step 3: Perform morphological operations of dilation, hole filling, and erosion on the initial effective area BW in sequence. In this example, the disk radius of B1 in the dilation operation is 5, and the disk radius of B2 in the erosion operation is 5, to obtain the final effective area BW3 after segmentation.

[0102] like Figure 6 As shown, by directly applying BW3 to the left camera image, we can see that the segmented image not only removes the pure seawater area and the too dark area, but also retains the cold spring bubbles, which is very necessary in the observation of cold springs.

[0103] Step 4: Set all the parallax values ​​of the invalid areas in the segmented area to invalid values, and stitch the point clouds restored from multiple frames of images to restore the deep sea 3D topographic map without the interference of pure seawater areas. Figure 7 As shown in the figure, the topographic map after removing the interference of seawater is as follows Figure 8 shown.

[0104] The above description is only a preferred embodiment of the present invention and does not limit the present invention in other forms. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A method for removing water interference for deep-sea binocular camera terrain reconstruction, characterized in that: The following steps are involved: Step A: using two different deep learning networks to constrain the images collected by the binocular camera, and outputting the disparity matrix D constrained by the deep learning network; Step B: for the image captured by the binocular camera, the threshold method is used in the Lab color space to remove the seawater area to obtain the initial effective area BW; Step C, performing dilation, hole filling and corrosion morphological operations on the initial valid area BW in sequence to obtain the final valid area BW3 and the invalid area; Step D: Map the final valid area BW3 to the constrained disparity matrix D obtained in step A, and set the disparity value corresponding to the invalid area to an invalid value, thereby achieving the removal of water interference.

2. The method for removing water interference for deep-sea binocular camera terrain reconstruction according to claim 1, characterized in that: The step B is specifically implemented in the following manner: Step B1: Convert the image collected by the binocular camera from the RGB color space to the Lab color space, and use the threshold method to preliminarily constrain the images of the three channels, namely, the L channel, the a channel, and the b channel. Assume that the distribution of the pure seawater area in the three channels is as follows: Step B2: According to different application scenarios and requirements, the methods for obtaining the initial effective area include the following: (1) In situations where the requirements are high, the brightness channel and the color channel are constrained, and the segmentation method that retains the effective area to the greatest extent is: (2) For applications with low requirements, the operation to remove seawater areas to the greatest extent is: (3) For the first two cases, since the image is more sensitive to brightness during the segmentation process, a dynamic threshold based on statistical principles is used. replace As shown below: in, Indicates the minimum threshold of the L channel, represents the maximum threshold of the L channel, L(i,j) represents the brightness value of the pixel in the Lab color space, a(i,j) represents the a channel value of the pixel in the Lab color space, Indicates the minimum threshold of channel a, represents the maximum threshold of the a channel, b(i,j) represents the b channel value of the pixel in the Lab color space, Indicates the minimum threshold of the b channel, Indicates the maximum threshold of channel b, μ 海水统计 With σ 海水统计 is the mean and standard deviation of the pixel values ​​of the L channel of the pure seawater area in more than ten pictures of different scenes, μ 全图 With σ 全图 is the mean and standard deviation of all pixel values ​​of the L channel of the image to be segmented, W1 and W2 are weights, k is a coefficient, M is the binary mask of the segmented seawater area, M(i,j) is the element in the binary mask of the segmented seawater area, and the initial effective area is the opposite part, which is expressed as: Among them, BW is the initial valid area, and BW(i,j) is the element in the initial valid area.

3. The method for removing water interference for deep-sea binocular camera terrain reconstruction according to claim 1, characterized in that: The step C is specifically implemented in the following manner: (1) First, the initial effective area is expanded, and the mathematical description is as follows: Among them, BW1 is the area after the dilation operation, and B1 is the structural element; (2) Then fill the holes in the area after the dilation operation. If the photographed target contains holes, skip this step and assume that the output after filling the holes in the BW1 area is BW2; (3) Finally, the output after filling the holes is corroded to restore the image to its original size before expansion: Among them, BW3 is the final valid area after the corrosion operation, BW2 is the area after the hole filling operation, and B2 is the structural element.

4. The method for removing water interference for deep-sea binocular camera terrain reconstruction according to claim 1, characterized in that: In step D, the specific mapping formula is expressed as: Among them, D NoWater D is the parallax matrix that finally removes the interference of pure seawater area. NoWater (i, j) is the final disparity value after removing the interference of pure seawater area, D(i, j) is the disparity value constrained by the dual deep learning network, BW3(i, j) is the element in the final effective area, and the final 3D terrain point cloud is obtained by combining the internal parameters of the camera. The point cloud restored by multiple frames of images is stitched together to restore the deep-sea 3D terrain map without the interference of pure seawater area. Similarly, the segmented area can be mapped to the image to obtain the image without the seawater area.

5. The method for removing water interference for deep-sea binocular camera terrain reconstruction according to claim 1, characterized in that: The step A is specifically implemented in the following manner: Assume that D1 and D2 are disparity matrices generated by two deep learning networks based on the same pair of binocular camera stereo images, then their disparity difference ΔD(i,j) is expressed as follows; ΔD(i,j)=|D1(i,j)-D2(i,j)| if ΔD(i,j)>T, mark D1(i,j) and D2(i,j) as invalid values Among them, T is a fixed threshold set, D1(i, j) and D2(i, j) are the disparity values ​​generated by the two deep learning networks at the image pixel point (i, j), and any one of the marked D1 or D2 is used as the disparity matrix D constrained by the dual deep learning networks.

6. The method for removing water interference for deep-sea binocular camera terrain reconstruction according to claim 1, characterized in that: In step A, the original disparity matrix is ​​obtained through a single deep learning network to replace the constrained disparity matrix. After the threshold method and morphological operation of the Lab color space, BW3 is directly mapped to the original disparity matrix to also achieve the removal of water interference.

Images