Underwater variable baseline stereo imaging system, noise removal method and image enhancement method

Abstract

The invention relates to the technical field of underwater three-dimensional imaging and image processing, in particular to an underwater variable baseline three-dimensional imaging system, a noise removal method and an image enhancement method. The invention discloses an underwater variable-baseline stereo imaging system based on four cameras. The underwater variable-baseline stereo imaging system comprises a control cabin, a sliding rail and two binocular camera cabins, the two binocular camera cabins are assembled in parallel and collinear, one binocular camera cabin is fixed on the sliding rail, and the other binocular camera cabin is movably arranged on the sliding rail; and the distance between the two binocular camera cabins is adjustable. Through flexible calling of the four cameras, the problems of limited imaging visual field, accuracy change of fixed baseline length under different imaging distances, random noise interference under an underwater weak illumination condition and the like of a conventional binocular camera in practical application are solved, and estimation of water optical attenuation parameters is realized based on the system. And color compensation is carried out on the underwater image by using the parameter, so that the underwater imaging quality is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater three-dimensional imaging and image processing, and in particular to an underwater variable baseline stereoscopic imaging system using four cameras, a noise removal method and an image enhancement method. Background Art

[0002] At present, underwater binocular stereo imaging technology has been widely used in many fields, especially in ocean exploration, shipwreck archaeology, seabed topography mapping, and underwater robot positioning and navigation. Although binocular stereo imaging technology has a wide range of application scenarios, the existing underwater binocular detection technology still has many problems, the most prominent of which are the following:

[0003] (1) The effective range of the binocular stereo imaging system is the overlapping area of ​​the two cameras’ fields of view, resulting in a smaller field of view;

[0004] (2) Current underwater binocular stereo imaging systems all have fixed baselines, which will affect the spatial accuracy of such systems when imaging objects at different distances. In particular, the error of short-baseline systems in estimating the depth of distant objects will increase significantly;

[0005] (3) The seawater environment is different from the land environment. Due to poor lighting conditions and interference from seawater, a large amount of random noise will be introduced during the binocular image parallax matching. This random noise is extremely difficult to remove using only the later image processing algorithm.

[0006] (4) The attenuation characteristics of water bodies lead to large color deviations in underwater imaging. Since the attenuation coefficient of the current water body cannot be accurately obtained, the underwater image processing effect is poor. Summary of the invention

[0007] The purpose of the present invention is to overcome the above-mentioned defects of the prior art, and propose an underwater variable baseline stereo imaging system, a noise removal method and an image enhancement method, which solve the problems faced by existing binocular cameras in practical applications, such as limited imaging field of view, accuracy variation of fixed baseline length at different imaging distances, random noise interference under weak underwater light conditions, etc., and based on the system, the optical attenuation parameters of water bodies are estimated, and the parameters are used to perform color compensation on underwater images to improve the underwater imaging quality.

[0008] The technical solution of the present invention is: an underwater variable baseline stereoscopic imaging system, which includes a control cabin, a slide rail, and two binocular camera cabins;

[0009] Two binocular camera cabins are parallelly collinearly assembled, one of which is fixed on a slide rail, and a first camera and a second camera are arranged in the binocular camera cabin; the other binocular camera cabin is movably arranged on the slide rail, and a third camera and a fourth camera are arranged in the binocular camera cabin;

[0010] The distance between the two binocular camera cabins is adjustable.

[0011] In the present invention, the system can realize a large range of combined imaging modes:

[0012] The position of the movable binocular camera cabin on the slide rail is adjusted so that the effective imaging range formed by the first camera and the second camera, the effective imaging range formed by the third camera and the fourth camera, and the effective imaging range formed by the second camera and the third camera are completely connected to obtain a point cloud file with a large viewing angle.

[0013] The system can realize a small-range high-precision redundant imaging mode: an overlapping area is formed between the effective imaging range formed by the first camera and the third camera, and the effective imaging range formed by the second camera and the fourth camera;

[0014] Adjust the position of the movable binocular camera cabin on the slide rail to maximize the overlapping area.

[0015] The present invention also discloses a method for removing underwater random noise based on the underwater variable baseline stereo imaging system, which comprises the following steps:

[0016] S1, calculating the disparity of the binocular cameras for the third camera and the fourth camera, and obtaining a disparity map of the two cameras;

[0017] S2. According to the disparity maps of the third camera and the fourth camera, corresponding points are found in the binocular stereo camera composed of the first camera and the third camera, and in the binocular stereo camera composed of the second camera and the fourth camera, and then the disparity consistency is compared to achieve denoising.

[0018] In the above step S1, the disparity of the third camera and the fourth camera is calculated to obtain the disparity map G3 of the third camera and the fourth camera.

[0019] G3(i,j)=a;

[0020] Wherein, i represents the horizontal coordinate of the corresponding pixel point, j represents the vertical coordinate of the corresponding pixel point, and a represents the disparity value between the third camera and the fourth camera;

[0021] In step S2, according to the coordinates of the corresponding pixel points in the pictures of the third camera and the fourth camera, the corresponding points are found in the binocular stereo camera composed of the first camera and the third camera, and the pixel disparity value D of the corresponding point in the disparity overlap area of ​​the first camera and the third camera is obtained. 13 =G1(i,j+a)=c;

[0022] Similarly, find the corresponding point in the binocular stereo camera composed of the second camera and the fourth camera, and obtain the pixel disparity value D of the corresponding point in the disparity overlap area of ​​the second camera and the fourth camera.24 =G2(i,j)=b;

[0023] The disparity value D of the same target point 13 and D 24 For comparison, if |D 13 -D 24 |=|cb|≤δ, the target point is retained; if |D 13 -D 24 |=|cb|>δ, the target point is discarded, where δ is a manually set threshold;

[0024] After removing the noise, point cloud stitching can be performed according to the calibration results to achieve the generation of high-precision dense point cloud.

[0025] The present application also discloses a method for realizing image enhancement by using the above-mentioned underwater variable baseline stereoscopic imaging system, which comprises the following steps:

[0026] A. Estimation of water attenuation coefficient;

[0027] B. Based on the water attenuation coefficient estimation result obtained in step A, the color channel of the image is compensated.

[0028] The specific implementation process of step A is as follows:

[0029] When shooting in the deep sea, all the illumination comes from artificial light sources. According to Lambert-Beer's law, the attenuation of light in water can be approximately expressed using the following formula:

[0030] I(d)=I0·e -kd ,

[0031] ΔI(d)=I0-I(d)=I0·(1-e -kd ),

[0032] Where I0 is the intensity of the incident light; I(d) is the intensity of the light received by the camera; ΔI(d) is the attenuation of the light; d is the distance the light travels in the water; k is the water attenuation coefficient, k is a function related to the wavelength, corresponding to the three color channels of the image R, G, and B, k includes k r , k g , k b Three servings;

[0033] Assume that the vector from the illumination source to the underwater target is A, the vector from the target to the left binocular camera is B, the vector from the underwater target to the right binocular camera is C, the vector from the right binocular camera to the left binocular camera is D, and the vector from the right binocular camera to the illumination source is E. The optical path d2 of the illumination light emitted by the illumination source entering the right binocular camera is expressed as:

[0034] d2=|A|+|C|,

[0035] The optical path d3 of the illumination light emitted by the illumination light source entering the left binocular camera is expressed as:

[0036] d3=|A|+|B|,

[0037] A=C+E,

[0038] The optical path difference Δd between optical path d2 and d3 is expressed as:

[0039] Δd=d3-d2=|B|-|C|,

[0040] The distance between the underwater target and the right binocular camera is expressed as:

[0041] C=(-X2,-Y2,-Z2),

[0042]

[0043] Among them, (X2, Y2, Z2) is the spatial position of the target object stored in the corresponding pixel point of the target object in the depth matrix of the right binocular camera;

[0044] The distance between the underwater target and the left binocular camera is expressed as:

[0045] B=(-X2,-Y2,-Z2),

[0046]

[0047] Among them, (X3, Y3, Z3) is the spatial position of the target object stored in the corresponding pixel point of the target object in the depth matrix of the left binocular camera;

[0048] Substitute the obtained |B| and |C| into the expression of the optical path difference Δd, then the optical path difference Δd is:

[0049]

[0050] The depth matrix of the binocular camera is calculated using the intrinsic parameter matrix and pixel coordinates. The specific form is as follows:

[0051]

[0052] Where m and n are the coordinates of the image pixel coordinate system, X, Y, and Z are the coordinates of the camera coordinate system stored at the position (m, n) in the depth matrix; c x Indicates the pixel coordinates of the center point of the image in the horizontal direction; c y Indicates the pixel coordinate of the center point of the image in the vertical direction; f x , f y is the focal length expressed in pixels, fx is the focal length in the x direction, f y is the focal length in the y direction; q represents the horizontal distance between the two cameras constituting the binocular camera, and p is the disparity value at the pixel position (m, n) in the binocular camera disparity map;

[0053] The target point light intensity I received by the camera is written in the following form:

[0054]

[0055] Among them, I T is the initial light intensity reflected by the target point, d T is the distance from the target point to the camera, s is the aperture area of ​​the camera lens;

[0056] Substituting the pixel values ​​I2, I3 and distances |C|, |B| corresponding to the same target point in the right and left binocular cameras into the attenuation formula, we can obtain:

[0057]

[0058] Substituting the Δd value, the water body attenuation coefficient k is calculated;

[0059] Since I2 and I3 contain three channel values ​​of R, G, and B, the water attenuation coefficients k of the three color channels can be obtained by calculating them separately. r , k g , k b :

[0060]

[0061] The specific implementation process of step B is as follows:

[0062] Color compensation is divided into two parts. One part is to compensate for the light attenuation at the distance |A| between the light emitted by the illumination source and the underwater target, and the other part is to compensate for the light attenuation at the distance between the underwater target and the binocular camera.

[0063] The compensation of the R, G, and B channel values ​​of the pixels corresponding to the target point in the right binocular camera image can be written as:

[0064]

[0065] Among them, α, β, and γ are the compensation coefficients of the red, green, and blue components at the distance |A| between the light emitted by the illumination light source and the target point; α'', β'', and γ'' are the compensation coefficients of the red, green, and blue components at the distance |C| between the reflected light from the target point and the right binocular camera; is the value of the red channel of the pixel before restoration in the RGB format image matrix of the right binocular camera; The green channel value of the pixel before restoration in the RGB format image matrix of the right binocular camera; is the value of the blue channel of the pixel before restoration in the RGB format image matrix of the right binocular camera; I'' r is the value of the red channel pixel in the restored right binocular camera; I″ g is the value of the green channel pixel in the restored right binocular camera; I″ b is the value of the blue channel pixel in the restored right binocular camera;

[0066] The compensation of the R, G, and B channel values ​​of the pixels corresponding to the target point in the left binocular camera image can be written as:

[0067]

[0068] Among them, α ′ , β ′ , γ ′ are the compensation coefficients of the red, green and blue components at the distance |B| between the reflected light of the underwater target point and the left binocular camera; is the value of the red channel of the pixel before restoration in the RGB format image matrix of the left binocular camera; The green channel value of the pixel before restoration in the RGB format image matrix of the left binocular camera; I′ is the value of the blue channel of the pixel before restoration in the RGB format image matrix of the left binocular camera; r is the value of the red channel pixel in the restored left binocular camera; I′ g is the value of the green channel pixel in the restored left binocular camera; I′ b It is the value of the blue channel pixel in the restored left binocular camera.

[0069] The beneficial effects of the present invention are:

[0070] (1) The underwater variable baseline stereoscopic imaging system based on four cameras proposed in this application can realize a large-range combined imaging mode, through which a point cloud file with a large viewing angle can be obtained;

[0071] (2) The underwater variable baseline stereo imaging system based on four cameras proposed in this application can also realize a small-scale high-precision redundant imaging mode. The overlapping area can be detected by two sets of binocular cameras at the same time, and has rich redundant information. This redundant information can be used to remove random noise caused by seawater disturbance and dim light, further enhancing the imaging accuracy. In addition, this application realizes adjustable baseline, so the maximum overlapping area can be obtained by adjusting the baseline distance; in addition, the imaging accuracy of the binocular cameras at different distances can be guaranteed by adjusting the baseline distance;

[0072] (3) The method for removing underwater random noise proposed in this application can effectively remove a large amount of random noise introduced in the process of binocular image disparity matching;

[0073] (4) This application uses two sets of binocular cameras under a long baseline to achieve imaging of the same target at different distances. The water attenuation coefficient is further calculated based on the color channel difference of the same target in the two sets of images at different distances, and is used to compensate the color of the underwater image, thereby achieving color compensation for the entire image.

[0074] (5) The four cameras in this application can obtain more video streams, and the robustness of the system is significantly improved, which is very beneficial for complex and changeable underwater environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 It is a structural schematic diagram of the underwater variable baseline stereoscopic imaging system of the present invention;

[0076] Figure 2 It is a working schematic diagram of the large-scale combined imaging mode;

[0077] Figure 3 It is a working schematic diagram of a small-scale high-precision redundant imaging mode;

[0078] Figure 4 It is a diagram of the method of removing underwater random noise;

[0079] Figure 5 It is a flow chart for removing underwater random noise;

[0080] Figure 6 It is a structural diagram of the water attenuation coefficient estimation model based on four phases.

[0081] In the figure: 1 control cabin; 2 slide rail; 3 left binocular camera cabin; 4 right binocular camera cabin; 5 right camera power supply and communication composite cable; 6 slide rail motor control cable; 7 left camera power supply and communication composite cable; 8 first camera; 9 second camera; 10 third camera; 11 fourth camera; 12 first effective imaging range; 13 second effective imaging range; 14 third effective imaging range; 15 fourth effective imaging range; 16 fifth effective imaging range; 17 overlapping area; 18 illumination light source; 19 underwater target; 20 left binocular camera; 21 right binocular camera. DETAILED DESCRIPTION

[0082] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0083] In the following description, specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in a variety of other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0084] Example 1

[0085] like Figure 1 As shown, an underwater variable baseline stereoscopic imaging system using four cameras described in the present application includes a control cabin 1, a slide rail 2 and four high-definition cameras. The four high-definition cameras are of the same model and have the same optical parameters and imaging parameters. Among the four high-definition cameras, two high-definition cameras are packaged in a watertight cabin as a group, and the watertight cabin realizes the waterproof and pressure resistance of the camera, thereby forming two groups of binocular cameras and binocular camera cabins with binocular cameras installed respectively. The two binocular camera cabins in this embodiment include a left binocular camera cabin 3 and a right binocular camera cabin 4. The two binocular camera cabins are assembled in parallel and in line, and the two binocular camera cabins are connected by a waterproof cable.

[0086] One of the binocular camera cabins is fixed on the slide rail, and the other binocular camera cabin can move along the slide rail, and during the movement of the binocular camera cabin along the slide rail, the two binocular camera cabins always remain parallel and collinear. In this embodiment, the right binocular camera cabin 4 is fixed on the slide rail 2, and the left binocular camera cabin 3 is slidably arranged on the slide rail 2. The movement of the left binocular camera cabin 3 can change the distance between the two groups of binocular cameras.

[0087] The control cabin 1 is responsible for the overall communication control of the system and the processing of the video stream. The binocular cameras in the left binocular camera cabin 3 and the right binocular camera cabin 4 are used to shoot video content. The left binocular camera cabin 3 is connected to the control cabin 1 through the left camera power supply communication composite cable 7, the right binocular camera cabin 4 is connected to the control cabin 1 through the right camera power supply communication composite cable 5, and the slide rail 2 is connected to the control cabin 1 through the slide rail motor control cable 6. The right camera power supply communication composite cable 5 and the left camera power supply communication composite cable 7 provide power and communication for the binocular camera cabins on both sides respectively; the slide rail motor control cable 6 provides driving power and signal control for the motor in the slide rail.

[0088] By adjusting the baseline distance between the left binocular camera cabin 3 and the right binocular camera cabin 4, the system has two working modes: large-scale combined imaging mode and small-scale high-precision redundant imaging mode. No matter which working mode is used, underwater camera calibration, underwater distortion correction, and preparatory work for stereo correction are required in advance.

[0089] The working diagram of the large-scale combined imaging mode is as follows: Figure 2As shown. The left binocular camera cabin 3 is provided with a first camera 8 and a second camera 9, and the right binocular camera cabin 4 is provided with a third camera 10 and a fourth camera 11. In this mode, the first camera 8 and the second camera 9 form a binocular stereo camera, whose effective imaging range is the first effective imaging range 12; the third camera 10 and the fourth camera 11 form a binocular stereo camera, whose effective imaging range is the second effective imaging range 13; the second camera 9 and the third camera 10 form a binocular stereo camera, whose effective imaging range is the third effective imaging range 14.

[0090] Adjust the position of the left binocular camera cabin 3 on the slide rail 2 so that the first effective imaging range 12, the second effective imaging range 13, and the third effective imaging range 14 formed by the three binocular stereo cameras are completely connected to obtain the largest possible three-dimensional imaging area.

[0091] In the preparation stage, it is necessary to calibrate the stereo cameras of the first camera 8 and the second camera 9, the third camera 10 and the fourth camera 11, the second camera 9 and the third camera 10, and the second camera 9 and the fourth camera 11. In the formal working stage, the calibration results of the first camera 8 and the second camera 9, the calibration results of the second camera 9 and the third camera 10, and the calibration results of the third camera 10 and the fourth camera 11 are used to reconstruct the three-dimensional point cloud files of the three binocular stereo cameras respectively, and the calibration results of the second camera 9 and the third camera 10, and the calibration results of the second camera 9 and the fourth camera 11 are used for the registration and alignment between the above three three-dimensional point cloud files, so that the final imaging result forms a point cloud file with a large viewing angle.

[0092] The working diagram of the small-scale high-precision redundant imaging mode is as follows: Figure 3 As shown. The first camera 8 and the third camera 10 form a binocular stereo camera 1-3, and the effective imaging range of the binocular stereo camera 1-3 is the IV effective imaging range 15. The second camera 9 and the fourth camera 11 form a binocular stereo camera 2-4, and the effective imaging range of the binocular stereo camera 2-4 is the V effective imaging range 16. The overlapping area 17 between the IV effective imaging range 15 and the V effective imaging range 16 is. By adjusting the position of the left binocular camera cabin 3 on the slide rail 2, the overlapping area 17 between the IV effective imaging range 15 and the V effective imaging range 16 is maximized. The overlapping area 17 can be detected by two groups of binocular stereo cameras at the same time, and has rich redundant information. This redundant information can be used to remove random noise caused by seawater disturbance and dark light, and further enhance the accuracy of imaging.

[0093] Since the two binocular cameras are located very close to each other, their viewing angles are very close in actual use, and the overlapping area of ​​the images is also very large. Based on this, we propose a denoising method to remove underwater random noise.

[0094] Since the positions of the two binocular cameras are almost the same and the overlapping area shoots the same content, the disparity values ​​of the corresponding pixels in the overlapping area should be roughly the same. The matching of corresponding pixels also uses the binocular matching algorithm. Usually, the calculation of the disparity map is relative to the left camera, that is, the binocular stereo camera 1-3 composed of the first camera 8 and the third camera 10 in the figure is based on the third camera 10, and the binocular stereo camera 2-4 composed of the second camera 9 and the fourth camera 11 is based on the fourth camera 11. The disparity calculation of the binocular camera is performed on the third camera 10 and the fourth camera 11, and the disparity map G3 of the third camera 10 and the fourth camera 11 can be obtained, that is:

[0095] G3(i,j)=a

[0096] Among them, i represents the horizontal coordinate of the corresponding pixel point, j represents the vertical coordinate of the corresponding pixel point, and a represents the disparity value between the third camera and the fourth camera.

[0097] The coordinates of the corresponding pixels in the pictures of the third camera 10 and the fourth camera 11 are consistent with the coordinates of the corresponding points in the disparity map G1 of the binocular stereo camera 1-3 and the disparity map G2 of the binocular stereo camera 2-4. After finding the corresponding points in the binocular stereo camera 1-3 and the binocular stereo camera 2-4 according to the coordinates of the corresponding pixels in the pictures of the third camera 10 and the fourth camera 11, the consistency comparison of the disparity can be performed. G2(i,j) corresponds to the same target point as G1(i,j+a).

[0098] Assume that the disparity value of the corresponding point in the disparity overlap area of ​​binocular stereo cameras 1-3 is c, that is, D 13 =G1(i,j+a)=c; the disparity value of the corresponding point in the disparity overlap area of ​​binocular stereo cameras 2-4 is b, that is, D 24 = G2(i,h) = b. The disparity value D of the same target point in the binocular stereo cameras 1-3 13 , and the disparity value D in the binocular stereo camera 2-4 24 For comparison, if |D 13 -D 24 |=|cb|≤δ, the corresponding target point is retained; if |D 13 -D 24 |=|cb|>δ, the corresponding target point will be discarded. δ is a manually set threshold and needs to be adjusted according to the actual situation. After removing the noise, the point cloud can be spliced ​​according to the calibration results to achieve the generation of high-precision point cloud. The specific process is as follows Figure 4 and Figure 5 shown.

[0099] In addition to the two main working modes mentioned above, any two cameras of the present invention can form a normal binocular camera that can be used normally, which greatly enhances the fault tolerance of the system, which is very beneficial for deep-sea work. At the same time, the two working modes mentioned above do not conflict within a specific imaging range and can be used simultaneously.

[0100] Example 2

[0101] Based on the underwater variable baseline stereoscopic imaging system using four cameras proposed in Example 1, the present application also proposes a method for underwater image color enhancement and restoration.

[0102] Due to water attenuation, underwater images mostly present a blue-green hue, resulting in loss of scene color information and poor underwater imaging effects; existing underwater image enhancement and processing methods mostly adjust the image color channels based on general water attenuation parameters, which results in the limited application of underwater image color enhancement and restoration methods in extreme environment scenes (hydrothermal areas / cold spring areas). The present application utilizes two sets of binocular cameras under the long baseline proposed in Example 1 to achieve imaging of the same target under different distance conditions. The water attenuation coefficient is further calculated based on the color channel difference of the same target in the two sets of images at different distances and used to compensate for the color of the underwater image.

[0103] The underwater image color enhancement and restoration method in this embodiment includes the following steps.

[0104] In the first step, based on the underwater variable baseline stereoscopic imaging system using four cameras proposed in Example 1, the water body attenuation coefficient is estimated.

[0105] Due to the attenuation effect of water, there is almost no natural light in the deep sea. When shooting in the deep sea, all the lighting comes from artificial light sources. In this case, according to Lambert-Beer's law, the attenuation of light in water can be approximately expressed by the following formula:

[0106] I(d)=I0·e -kd ,

[0107] ΔI(d)=I0-I(d)=I0·(1-e -kd ),

[0108] Where I0 is the intensity of the incident light; I(d) is the intensity of the light received by the camera; ΔI(d) is the attenuation of the light; d is the distance the light travels in the water; k is the water attenuation coefficient, k is a function related to the wavelength, corresponding to the three color channels of the image R, G, and B, k includes k r , k g , k b Three servings.

[0109] This scheme uses two sets of binocular cameras under a long baseline to estimate the attenuation coefficient of the water body according to the color channel difference of the same target point in the two sets of cameras. The water body attenuation coefficient estimation model based on four cameras is as follows: Figure 6 shown.

[0110] The vector from the illumination light source 18 to the underwater target 19 is A, the vector from the target 19 to the left binocular camera 20 is B, the vector from the underwater target 19 to the right binocular camera 21 is C, the vector from the right binocular camera 21 to the left binocular camera 20 is D, and the vector from the right binocular camera 21 to the illumination light source 18 is E. The value of the vector E depends on the installation position of the light source and the camera, and can be measured after the installation is completed. The optical path d2 of the illumination light emitted by the illumination light source 18 entering the right binocular camera 21 can be expressed as:

[0111] d2=|A|+|C|,

[0112] The optical path d3 of the illumination light emitted by the illumination light source 18 entering the left binocular camera 20 can be expressed as:

[0113] d3=|A|+|B|,

[0114] A=C+E,

[0115] The optical path difference Δd between optical path d2 and d3 can be expressed as:

[0116] Δd=d3-d2=|B|-|C|,

[0117] The distance between the underwater target 19 and the binocular camera can be calculated using the spatial coordinates stored in the binocular camera depth matrix corresponding to the pixel position of the target. Therefore, the distance between the underwater target 19 and the right binocular camera 21 can be expressed as:

[0118] C=(-X2,-Y2,-Z2),

[0119]

[0120] Among them, (X2, Y2, Z2) is the spatial position of the target object stored in the corresponding pixel point of the target object in the depth matrix of the right binocular camera.

[0121] The distance between the underwater target 19 and the left binocular camera 20 can be expressed as:

[0122] B=(-X2,-Y2,-Z2),

[0123]

[0124] Among them, (X3, Y3, Z3) is the spatial position of the target object stored in the corresponding pixel point of the target object in the depth matrix of the left binocular camera.

[0125] At this time, the above-obtained |B| and |C| are substituted into the expression of the optical path difference Δd, and the optical path difference Δd can be written as follows:

[0126]

[0127] The depth matrix of the binocular camera can be calculated using the intrinsic parameter matrix and pixel coordinates. The specific form is as follows:

[0128]

[0129] Where m and n are the coordinates of the image pixel coordinate system, X, Y, Z are the coordinates of the camera coordinate system stored at the position (m, n) in the depth matrix; c x Indicates the pixel coordinates of the center point of the image in the horizontal direction; c y Indicates the pixel coordinate of the center point of the image in the vertical direction; f x , f y is the focal length expressed in pixels, f x is the focal length in the x direction, f y is the focal length in the y direction; q represents the horizontal distance between the two cameras that make up the binocular camera, and p is the disparity value at the (m, n) pixel position in the binocular camera disparity map.

[0130] When the angles between the two groups of cameras and the light source are not much different, the reflection differences at different angles on the surface of the object can be ignored, and the target point can be regarded as an isotropic reflected light source. The light intensity I of the target point received by the camera can be written as follows:

[0131]

[0132] Among them, I T is the initial light intensity reflected by the target point, d T is the distance from the target point to the camera, and s is the aperture area of ​​the camera lens.

[0133] Substituting the pixel values ​​I2, I3 and the distances |C|, |B| corresponding to the same target point in the right binocular camera 21 and the left binocular camera 20 into the attenuation formula, we can obtain:

[0134]

[0135] Using the elimination method, we can divide the two equations to get:

[0136]

[0137] Substituting the Δd value, the water attenuation coefficient k can be calculated.

[0138] Since I2 and I3 contain three channel values ​​of R, G, and B, the water attenuation coefficients k of the three color channels can be obtained by calculating them separately. r , k g , k b .

[0139]

[0140] In order to reduce the calculation error of k, the value of Δd should be as large as possible. According to the relationship between the three sides of the triangle, Δd<|D|, so the distance between the left binocular camera 20 and the right binocular camera 21 should be as large as possible.

[0141] In the second step, the color channel of the image is compensated based on the estimated water attenuation coefficient obtained in the first step.

[0142] Taking the image captured by the right binocular camera 21 as an example, color compensation is divided into two parts. One part is to compensate for the light attenuation at the distance |A| between the light emitted by the illumination light source 18 and the underwater target 19, and the other part is to compensate for the light attenuation at the distance |C| between the underwater target 19 and the right binocular camera 21. The red, green and blue component compensation coefficients α, β, γ at the distance |A| between the light emitted by the illumination light source and the target point are as follows:

[0143]

[0144] Compensation coefficient α of the red, green and blue components at the distance |C| between the target point reflected light and the right binocular camera ‘’ , β ‘’ , γ ‘’ They are:

[0145]

[0146] The compensation coefficient is used to compensate the R, G, and B channel values ​​of the pixels corresponding to the target point in the image of the right binocular camera 21, which can be written as:

[0147]

[0148] is the value of the red channel of the pixel before restoration in the RGB format image matrix of the right binocular camera; The green channel value of the pixel before restoration in the RGB format image matrix of the right binocular camera; is the value of the blue channel of the pixel before restoration in the RGB format image matrix of the right binocular camera; I'' r is the value of the red channel pixel in the restored right binocular camera; I″ g is the value of the green channel pixel in the restored right binocular camera; I″ bIt is the value of the blue channel pixel in the restored right binocular camera.

[0149] Similarly, using the compensation coefficient to compensate the R, G, and B channel values ​​of the pixels corresponding to the target point in the image of the left binocular camera 20 can be written as:

[0150]

[0151]

[0152] is the value of the red channel of the pixel before restoration in the RGB format image matrix of the left binocular camera; The green channel value of the pixel before restoration in the RGB format image matrix of the left binocular camera; is the value of the blue channel of the pixel before restoration in the image matrix of the RGB format in the left binocular camera; I ′ is the value of the red channel pixel in the restored left binocular camera; I ′ is the value of the green channel pixel in the restored left binocular camera; I ′ is the value of the blue channel pixel in the restored left binocular camera. The compensation coefficients of the red, green and blue components at the distance |B| between the underwater target point reflected light and the left binocular camera 4 are α ′ , β ′ , γ ′ .

[0153] By performing the above compensation operation on all pixels in the image, color compensation for the entire image can be achieved. Since this method compensates each channel based on the existing value, the compensated pixel value may exceed the grayscale value range of the image display. In this case, the image color channel can be scaled proportionally to ensure normal display of the image.

[0154] The underwater variable baseline stereoscopic imaging system, image processing method and image enhancement method provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined in this article can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown in this article, but will conform to the widest range consistent with the principles and novel features disclosed in this article.

Claims

1. An underwater variable baseline stereoscopic imaging system, characterized in that: Includes a control cabin, slide rails, and two binocular camera cabins; Two binocular camera cabins are parallelly collinearly assembled, one of which is fixed on a slide rail, and a first camera and a second camera are arranged in the binocular camera cabin; the other binocular camera cabin is movably arranged on the slide rail, and a third camera and a fourth camera are arranged in the binocular camera cabin; The distance between the two binocular camera cabins is adjustable.

2. The underwater variable baseline stereoscopic imaging system according to claim 1, characterized in that: The system is capable of a wide range of combined imaging modes: The position of the movable binocular camera cabin on the slide rail is adjusted so that the effective imaging range formed by the first camera and the second camera, the effective imaging range formed by the third camera and the fourth camera, and the effective imaging range formed by the second camera and the third camera are completely connected to obtain a point cloud file with a large viewing angle.

3. The underwater variable baseline stereoscopic imaging system according to claim 1, characterized in that: The system can realize a small-range high-precision redundant imaging mode: an overlapping area is formed between the effective imaging range formed by the first camera and the third camera, and the effective imaging range formed by the second camera and the fourth camera; Adjust the position of the movable binocular camera cabin on the slide rail to maximize the overlapping area.

4. A method for removing underwater random noise based on the underwater variable baseline stereoscopic imaging system according to any one of claims 1 to 3, characterized in that: The following steps are involved: S1, calculating the disparity of the binocular cameras for the third camera and the fourth camera to obtain a disparity map of the two cameras; S2. According to the disparity maps of the third camera and the fourth camera, corresponding points are found in the binocular stereo camera composed of the first camera and the third camera, and in the binocular stereo camera composed of the second camera and the fourth camera, and then the disparity consistency is compared to achieve denoising.

5. The method for removing underwater random noise according to claim 4, characterized in that: In step S1, the disparity of the third camera and the fourth camera is calculated to obtain a disparity map G3 of the third camera and the fourth camera. G3(i,j)=a; Wherein, i represents the horizontal coordinate of the corresponding pixel point, j represents the vertical coordinate of the corresponding pixel point, and a represents the disparity value between the third camera and the fourth camera; In step S2, according to the coordinates of the corresponding pixel points in the pictures of the third camera and the fourth camera, the corresponding points are found in the binocular stereo camera composed of the first camera and the third camera, and the pixel disparity value D of the corresponding point in the disparity overlap area of ​​the first camera and the third camera is obtained. 13 =G1(i,j+a)=c; Similarly, find the corresponding point in the binocular stereo camera composed of the second camera and the fourth camera, and obtain the pixel disparity value D of the corresponding point in the disparity overlap area of ​​the second camera and the fourth camera. 24 =G2(i,j)=b; The disparity value D of the same target point 13 and D 24 For comparison, if |D 13 -D 24 |=|cb|≤δ, the target point is retained; if |D 13 -D 24 |=|cb|>δ, the target point is discarded, where δ is a manually set threshold; After removing the noise, point cloud stitching can be performed according to the calibration results to achieve the generation of high-precision dense point cloud.

6. A method for realizing image enhancement using the underwater variable baseline stereoscopic imaging system according to claim 1, characterized in that: The following steps are involved: A. Estimation of water attenuation coefficient; B. Based on the water attenuation coefficient estimation result obtained in step A, the color channel of the image is compensated.

7. The image enhancement method according to claim 6, characterized in that: The specific implementation process of step A is as follows: When shooting in the deep sea, all the illumination comes from artificial light sources. According to Lambert-Beer's law, the attenuation of light in water can be approximately expressed using the following formula: I(d)=I0·e -kd , ΔI(d)=I0-I(d)=I0·(1-e -kd ), Where I0 is the intensity of the incident light; I(d) is the intensity of the light received by the camera; ΔI(d) is the attenuation of the light; d is the distance the light travels in the water; k is the water attenuation coefficient, k is a function related to the wavelength, corresponding to the three color channels of the image R, G, and B, k includes k r , k g , k b Three servings; Assume that the vector from the illumination source to the underwater target is A, the vector from the target to the left binocular camera is B, the vector from the underwater target to the right binocular camera is C, the vector from the right binocular camera to the left binocular camera is D, and the vector from the right binocular camera to the illumination source is E. The optical path d2 of the illumination light emitted by the illumination source entering the right binocular camera is expressed as: d2=|A|+|C|, The optical path d3 of the illumination light emitted by the illumination light source entering the left binocular camera is expressed as: d3=|A|+|B|, A=C+E, The optical path difference Δd between optical path d2 and d3 is expressed as: Δd=d3-d2=|B|-|C|, The distance between the underwater target and the right binocular camera is expressed as: C=(-X2,-Y2,-Z2), Among them, (X2, Y2, Z2) is the spatial position of the target object stored in the corresponding pixel point of the target object in the depth matrix of the right binocular camera; The distance between the underwater target and the left binocular camera is expressed as: B=(-X2,-Y2,-Z2), Among them, (X3, Y3, Z3) is the spatial position of the target object stored in the corresponding pixel point of the target object in the depth matrix of the left binocular camera; Substitute the obtained |B| and |C| into the expression of the optical path difference Δd, then the optical path difference Δd is: The depth matrix of the binocular camera is calculated using the intrinsic parameter matrix and pixel coordinates. The specific form is as follows: Where m and n are the coordinates of the image pixel coordinate system, X, Y, and Z are the coordinates of the camera coordinate system stored at the position (m, n) in the depth matrix; c x Indicates the pixel coordinates of the center point of the image in the horizontal direction; c y Indicates the pixel coordinate of the center point of the image in the vertical direction; f x , f y is the focal length expressed in pixels, f x is the focal length in the x direction, f y is the focal length in the y direction; q represents the horizontal distance between the two cameras constituting the binocular camera, and p is the disparity value at the pixel position (m, n) in the binocular camera disparity map; The target point light intensity I received by the camera is written in the following form: Among them, I T is the initial light intensity reflected by the target point, d T is the distance from the target point to the camera, s is the aperture area of ​​the camera lens; Substituting the pixel values ​​I2, I3 and distances |C|, |B| corresponding to the same target point in the right and left binocular cameras into the attenuation formula, we can obtain: Substituting the Δd value, the water body attenuation coefficient k is calculated; Since I2 and I3 contain three channel values ​​of R, G, and B, the water attenuation coefficients k of the three color channels can be obtained by calculating them separately. r , k g , k b :

8. The image enhancement method according to claim 6, characterized in that: The specific implementation process of step B is as follows: Color compensation is divided into two parts. One part is to compensate for the light attenuation at the distance |A| between the light emitted by the illumination source and the underwater target, and the other part is to compensate for the light attenuation at the distance between the underwater target and the binocular camera. The compensation of the R, G, and B channel values ​​of the pixels corresponding to the target point in the right binocular camera image can be written as: Among them, α, β, and γ are the compensation coefficients of the red, green, and blue components at the distance |A| between the light emitted by the illumination light source and the target point; α'', β'', and γ'' are the compensation coefficients of the red, green, and blue components at the distance |C| between the reflected light from the target point and the right binocular camera; is the value of the red channel of the pixel before restoration in the RGB format image matrix of the right binocular camera; The green channel value of the pixel before restoration in the RGB format image matrix of the right binocular camera; is the value of the blue channel of the pixel before restoration in the RGB format image matrix of the right binocular camera; I'' r is the value of the red channel pixel in the restored right binocular camera; I" g is the value of the green channel pixel in the restored right binocular camera; I" b is the value of the blue channel pixel in the restored right binocular camera; The compensation of the R, G, and B channel values ​​of the pixels corresponding to the target point in the left binocular camera image can be written as: Among them, α′, β′, and γ′ are the compensation coefficients of the red, green, and blue components at the distance |B| between the reflected light of the underwater target point and the left binocular camera; is the value of the red channel of the pixel before restoration in the RGB format image matrix of the left binocular camera; The green channel value of the pixel before restoration in the RGB format image matrix of the left binocular camera; I′ is the value of the blue channel of the pixel before restoration in the RGB format image matrix of the left binocular camera; r is the value of the red channel pixel in the restored left binocular camera; I′ g is the value of the green channel pixel in the restored left binocular camera; I′ b It is the value of the blue channel pixel in the restored left binocular camera.

Images