A method for underwater absolute distance estimation based on neural network and a small number of point measurements

Abstract

The present invention relates to the field of computer vision technology, and specifically discloses an underwater absolute distance estimation method based on a neural network and a small number of point measurements. A fine-grained depth map is obtained by constructing, training and testing a self-supervised underwater depth estimation network. Since the depth map only contains relative distance information, i.e., depth value, and there is a problem of inconsistent scale, the present invention further measures the absolute distance of a small number of points and uses the distance as the true value to train a polynomial function fitting network, thereby being able to accurately fit a polynomial function between the depth value and the absolute distance, and facilitating the acquisition of the global absolute distance of the original underwater image. The distance between the surrounding environment and the operating subject is one of the most important information required for the autonomous operation of an underwater robot. Accurate underwater depth estimation is the cornerstone for realizing autonomous underwater exploration, and provides services for obstacle avoidance and refined operations of underwater robots.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an underwater absolute distance estimation method based on neural network and a small number of point measurements. Background Art

[0002] The ocean is rich in biological resources, marine space resources, mineral resources and other materials and energy that can be used by humans. In today's global situation where the contradiction between tight food, resources and energy supply and rapid population growth is becoming increasingly prominent, the development of marine resources is an inevitable historical development. However, the underwater environment is complex and diverse, which adds many difficulties for humans to carry out autonomous underwater operations. Underwater robots are intelligent agents that perform complex tasks. One of the main challenges at present is underwater visual perception. Accurate underwater distance estimation can enhance the situational awareness of underwater robots and achieve autonomy. However, due to the inherent physical properties of water bodies such as absorption and scattering of light, accurate distance measurement underwater is particularly difficult, and traditional underwater distance measurement methods cannot meet the needs of refined operations. Summary of the invention

[0003] The present invention provides an underwater absolute distance estimation method based on a neural network and a small number of point measurements, and solves the technical problem of how to estimate the absolute distance of an underwater global environment relatively quickly and accurately.

[0004] In order to solve the above technical problems, the present invention provides an underwater absolute distance estimation method based on a neural network and a small number of point measurements, comprising the steps of:

[0005] S1: Use a monocular camera to collect multiple original images of different underwater scenes to obtain an underwater original image dataset;

[0006] S2: Using the underwater original image dataset to train a self-supervised underwater depth estimation network;

[0007] S3: using a monocular camera to collect a measured image when actually measuring an underwater environment, and using the trained self-supervised underwater depth estimation network to test the measured image to obtain a depth image with a depth value;

[0008] S4: Select a specific number of image coordinate points in the depth image to obtain a depth value of each image coordinate point;

[0009] S5: using a laser distance measuring sensor to measure the absolute distance value of the actual position corresponding to each of the image coordinate points;

[0010] S6: Use a neural network to fit the function between the depth value selected in step S4 and the absolute distance value measured in step S5, and calculate the absolute distance values ​​corresponding to other depth values ​​in the depth image based on the fitted function.

[0011] Furthermore, the step S2 specifically includes the steps of:

[0012] S21: Building a self-supervised underwater depth estimation network, the self-supervised underwater depth estimation network includes a deep network and a pre-trained optical flow network, the deep network is based on the MonoDepth2 network framework, and the optical flow network adopts the FlowNet2.0 architecture;

[0013] S22: inputting the underwater original image data set into the optical flow network to obtain a corresponding optical flow map, and further generating a mask map of the non-occluded area in the optical flow map;

[0014] S23: performing CLAHE image enhancement and Sobel operator operation on the underwater original image data set to obtain corresponding enhancement images and first-order gradient images;

[0015] S24: Inputting the underwater original image data set into the deep network for training. During the training process, the parameters of the deep network itself are updated according to gradient back propagation based on a first loss function. The first loss function is constructed based on the depth image generated by the deep network, the reconstructed image generated by the deep network, the optical flow image, the mask image, the enhancement image, and the first-order gradient image.

[0016] Further, in step S24, the first loss function is expressed as:

[0017] L=L p +λ s L s +λ c L c +λ g L g

[0018] Among them, L p represents the photometric loss between the enhanced image, the reconstructed image and the mask image, L s represents the smoothness loss between the depth image and the enhancement map, L c represents the consistency loss between the depth information of the depth image, the optical flow information of the optical flow map and the mask map, L g represents the gradient loss between the first-order gradient map, the first-order gradient map of the reconstructed image and the mask image, λ s , c , g is the weight coefficient corresponding to the loss.

[0019] Furthermore, the luminosity loss L p It is expressed as:

[0020]

[0021] Among them, I′ t , They represent the enhanced image and the reconstructed image respectively, parameter α=0.85, SSIM(,) represents the structural similarity index, ||,||1 represents the L1 loss, and M represents the mask image;

[0022] In order to avoid the influence of occluded areas on self-supervised depth estimation, the consistency check of optical flow and image color recognition is used to generate a mask map M of the non-occluded area. The mask map M is expressed as:

[0023] M=[w f (x)+w b (x+w f (x))] <Th∩[I1(x)-I2(x+w f (x))] <Th

[0024] Among them, I1 represents the current frame, i.e., the picture of frame 0, I2 represents the picture of the previous frame, i.e., the picture of frame -1, x represents the position on the pixel coordinates, and w f represents the optical flow from I1 to I2, w b represents the optical flow with opposite directions, Th represents the threshold value, which is set to 1;

[0025] Smoothing loss L s It is expressed as:

[0026]

[0027] Among them, D(p t ) represents the pixel point p in the depth image t The estimated distance of |·| represents the absolute value of the element. is a vector differential operator, T represents the weighted transpose of the image gradient, The enhancement map is also shown.

[0028] Further,

[0029] Consistency loss L c is the pixel space consistency loss L ci and camera space consistency loss L cc The sum of these two components;

[0030] Pixel space consistency loss L ci It is expressed as:

[0031]

[0032] Among them, p t is the two-dimensional coordinate of a pixel point in the current frame, i.e. frame 0, and are obtained through depth information and optical flow information and p t The source image corresponding to the point is the two-dimensional coordinate in the +1 frame or -1 frame, where ||.|| represents the two-dimensional distance;

[0033] Camera space consistency loss L cc It is expressed as:

[0034]

[0035] in, and Indicates that in the source camera coordinate space, the depth information and optical flow information are obtained respectively and p t The three-dimensional coordinates corresponding to the point, where ||.|| represents the three-dimensional space distance;

[0036] Gradient loss L g It is expressed as:

[0037]

[0038] Among them, g t Represents the first-order gradient map of the current frame, i.e., frame 0. represents the first-order gradient of the reconstructed map.

[0039] Furthermore, the step S4 specifically includes the steps of:

[0040] S41: normalizing the depth image pixels to obtain a corresponding normalized depth image;

[0041] S42: In the normalized depth image, at least one pixel coordinate point is selected from each of the four intervals A1, A2, A3, and A4 where the normalized pixel value is within the range A1, A2, A3, and A4. <A2<A3<A4;

[0042] S43: Determine the depth value of each selected pixel coordinate point.

[0043] Preferably, in step S42, the four intervals A1, A2, A3, and A4 are 0.01-0.05, 0.38-0.42, 0.68-0.72, and 0.88-0.92, respectively.

[0044] Furthermore, the step S5 specifically includes the steps of:

[0045] S51: Fix the camera and the laser distance sensor adjacent to each other, and make the direction of the laser distance sensor consistent with the direction of the optical axis of the camera;

[0046] S52: Calibrate the camera to obtain internal and external parameters of the camera;

[0047] S53: according to the internal and external parameters of the camera, convert the pixel coordinate point selected in step S4 from the image coordinate system to the coordinate value in the camera coordinate system and the world coordinate system;

[0048] S54: adjusting the angle of the laser ranging sensor according to the coordinate value of step S53, and measuring the absolute distance value of the current laser ranging point.

[0049] Furthermore, the step S6 specifically includes the steps of:

[0050] S61: using the depth value obtained in step S4 as a data set and the absolute distance value measured in step S5 as a true value to train the polynomial function fitting neural network, and in the training process, updating the parameters of the polynomial function fitting neural network itself according to the gradient back propagation based on the second loss function;

[0051] S62: predicting absolute distance values ​​corresponding to other depth values ​​in the depth image according to the polynomial function obtained by fitting the polynomial function fitting neural network after the training is completed.

[0052] Preferably, the polynomial function fitting neural network includes a fully connected layer, and the second loss function adopts the MSELoss loss function.

[0053] The present invention provides an underwater absolute distance estimation method based on a neural network and a small number of point measurements, and obtains a fine-grained depth map by constructing, training and testing a self-supervised underwater depth estimation network. Since the depth map only contains relative distance information, i.e., depth values, and there is a problem of scale inconsistency, the present invention further measures the absolute distance of a small number of points and uses the distance as the true value to train the polynomial function fitting network, so that the polynomial function between the depth value and the absolute distance can be accurately fitted, which is convenient for obtaining the global absolute distance of the original underwater image. The distance between the surrounding environment and the operating subject is one of the most important information required for the autonomous operation of underwater robots. Accurate underwater depth estimation is the cornerstone for realizing underwater autonomous exploration, and provides services for underwater robots to avoid obstacles and perform refined operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flowchart of the steps of an underwater absolute distance estimation method based on a neural network and a small number of point measurements provided by an embodiment of the present invention;

[0055] Figure 2 is an architecture diagram of a self-supervised underwater depth estimation network provided by an embodiment of the present invention;

[0056] Figure 3 This is an example diagram of an underwater absolute distance estimation method based on a neural network and a small number of point measurements provided by an embodiment of the present invention;

[0057] Figure 4 It is a structural diagram of a distance measuring device provided by an embodiment of the present invention.

[0058] Reference numerals: laser ranging sensor 1 , camera 2 , gimbal 3 . DETAILED DESCRIPTION

[0059] The following specifically illustrates the implementation mode of the present invention in conjunction with the accompanying drawings. The embodiments are provided for illustrative purposes only and cannot be understood as limiting the present invention. The accompanying drawings are provided for reference and illustration only and do not constitute a limitation on the scope of patent protection of the present invention, because many changes can be made to the present invention without departing from the spirit and scope of the present invention.

[0060] In order to quickly and accurately obtain the absolute distance information of the underwater environment, an embodiment of the present invention provides an underwater global depth information estimation method based on a neural network and a small number of point measurements, such as Figure 1 The steps are shown in the flowchart, including the steps:

[0061] S1: Use a monocular camera to collect multiple original images of different underwater scenes to obtain an underwater original image dataset;

[0062] S2: Use the underwater original image dataset to train the self-supervised underwater depth estimation network;

[0063] S3: Use a monocular camera to collect actual measured images when measuring the underwater environment, and use the trained self-supervised underwater depth estimation network to test the measured images to obtain a depth image with depth values;

[0064] S4: Select a specific number of image coordinate points in the depth image to obtain a depth value of each image coordinate point;

[0065] S5: Use a laser distance sensor to measure the absolute distance value of the actual position corresponding to each image coordinate point;

[0066] S6: Use a neural network to fit the function between the depth value selected in step S4 and the absolute distance value measured in step S5, and calculate the absolute distance values ​​corresponding to other depth values ​​in the depth image based on the fitted function.

[0067] Step S2 specifically includes the following steps:

[0068] S21: Build a self-supervised underwater depth estimation network. The self-supervised underwater depth estimation network includes a deep network and a pre-trained optical flow network. The deep network is based on the MonoDepth2 network framework, and the optical flow network adopts the FlowNet2.0 architecture.

[0069] S22: Inputting the underwater original image data set into the optical flow network to obtain a corresponding optical flow map, and further generating a mask map of the non-occluded area in the optical flow map;

[0070] S23: performing CLAHE image enhancement and Sobel operator operations on the underwater original image data set to obtain corresponding enhancement images and first-order gradient images;

[0071] S24: Input the underwater original image data set into the deep network for training. During the training process, the deep network's own parameters are updated according to the gradient back propagation based on the first loss function. The first loss function is constructed based on the depth image generated by the deep network, the reconstructed image generated by the deep network, the optical flow image, the mask image, the enhancement image, and the first-order gradient image.

[0072] More specifically, in step S21, Figure 2 As shown in the figure, the deep network is built based on the MonoDepth2 network framework, including the Depth network and the Pose network. The Depth network is used to predict the depth of the current frame. It adopts an encoder-decoder framework. The encoder is a ResNet18 pre-trained based on ImageNet. The encoder adopts an upward expansion scheme to improve the spatial resolution.

[0073] The Pose network is also an encoder-decoder architecture. This architecture is similar to the Depth network. The difference is that the input of the Pose network is two consecutive images, and the output is the 6-DOF transformation between the input frames, represented by 6 numbers: (x, y, z) represents translation, (R x ,R y ,R z ) represents a rotation using Euler parameterization.

[0074] When using monocular data for training, the model needs to input 3 consecutive images, with ids of 0, -1, and +1 (corresponding to Figure 2 I in the original image t ,I t-1 ,I t+1 ), where frame 0 is the frame to be predicted for depth, i.e., the target frame or the current frame, and +1 and -1 are the next and previous frames of frame 0. Then, the target frame is input into the Depth Network to obtain the predicted depth. Then, the two groups of images (-1,0) and (+1,0) are input into the Pose Network to obtain the pose transformation from -1 to 0 and the pose transformation from +1 to 0. At this point, the depth of the target frame 0 and the pose transformation between the target frame and the two adjacent frames are obtained. Together with the camera's intrinsic parameter K, the reconstructed image of the target frame can be obtained (corresponding to Figure 2 Warp operation in ).

[0075] Due to the serious degradation of the underwater environment, which is characterized by low contrast, complex lighting conditions and organisms, in order to solve the above problems, the depth estimation in the atmosphere is better applied to the underwater depth estimation. In this embodiment, in step S24, the loss function (first loss function) of the self-supervised underwater depth estimation network is designed as:

[0076] L=L p +λ s L s +λ c L c +λ g L g

[0077] Among them, L p Represents the photometric loss between the enhanced image, the reconstructed image, and the mask image, L s represents the smoothness loss between the depth image and the enhancement map, L c represents the consistency loss between the optical flow information of the optical flow map, the depth information of the depth map and the mask map, L g represents the gradient loss between the first-order gradient map, the first-order gradient map of the reconstructed image and the mask image, λ s , c , g is the weight coefficient corresponding to the loss, λ s =0.0001,λ c =0.008,λ g =1.

[0078] Specifically, the luminosity loss L p Designed for:

[0079]

[0080] Among them, I′ t , They represent enhanced image and reconstructed image respectively, parameter α=0.85, SSIM(,) represents the structural similarity index, ||,||1 represents the L1 loss, and M represents the mask image.

[0081] Specifically, in order to avoid the influence of the occluded area on the self-supervised depth estimation, the consistency check of the optical flow and the image color recognition is performed to generate the mask map M of the non-occluded area. The mask map M is expressed as:

[0082] M=[w f (x)+w b (x+w f (x))] <Th∩[I1(x)-I2(x+w f (x))] <Th

[0083] Among them, I1 represents the picture of the current frame (0 frame), I2 represents the picture of the previous frame (-1 frame), x represents the position on the pixel coordinate, and w f represents the optical flow from I1 to I2, w b represents the optical flow with opposite directions, and Th represents the threshold, which is set to 1.

[0084] Specifically, the smoothing loss L s Designed for:

[0085]

[0086] Among them, D(p t ) represents the pixel point p in the depth image t The estimated distance of |·| represents the absolute value of the element. is a vector differential operator, T represents the weighted transpose of the image gradient, It also represents the enhancement map.

[0087] Specifically, the consistency loss L c is the pixel space consistency loss L ci and camera space consistency loss L cc The sum of these two components. Pixel space consistency loss L ci It is expressed as:

[0088]

[0089] Among them, p t is the two-dimensional coordinate of a pixel point in the current frame, i.e. frame 0, and are obtained through depth information and optical flow information and p t The source image corresponding to the point is the two-dimensional coordinate in the +1 frame or -1 frame, where ||.|| represents the two-dimensional distance;

[0090] Camera space consistency loss L cc It is expressed as:

[0091]

[0092] in, and Indicates that in the source camera coordinate space, the depth information and optical flow information are obtained respectively and p t The three-dimensional coordinates corresponding to the point, where ||.|| represents the three-dimensional space distance, and the source camera corresponds to the camera that collects the source image.

[0093] Specifically, the gradient loss L g It is expressed as:

[0094]

[0095] Among them, g t Represents the first-order gradient map of the current frame (frame 0), represents the first-order gradient of the reconstructed graph.

[0096] Step S4 specifically includes the following steps:

[0097] S41: normalizing the depth image pixels to obtain a corresponding normalized depth image;

[0098] S42: In the normalized depth image, at least one pixel coordinate point is selected from each of the four intervals A1, A2, A3, and A4 where the normalized pixel value is within the range A1, A2, A3, and A4. <A2<A3<A4;

[0099] S43: Determine the depth value of each selected pixel coordinate point.

[0100] In this embodiment, in step S42, the four intervals A1, A2, A3, and A4 are 0.01-0.05, 0.38-0.42, 0.68-0.72, and 0.88-0.92, respectively, and one pixel point is selected from each interval. Figure 3 shown.

[0101] For step S5, the following Figure 4 In the distance measuring device shown, step S5 specifically includes the following steps:

[0102] S51: Fix the camera and the laser distance sensor adjacent to each other, and make the direction of the laser distance sensor consistent with the optical axis direction of the camera;

[0103] S52: Calibrate the camera to obtain internal and external parameters of the camera;

[0104] S53: according to the internal and external parameters of the camera, convert the pixel coordinate point selected in step S4 from the image coordinate system to the coordinate value in the camera coordinate system and the world coordinate system;

[0105] S54: adjusting the angle of the laser ranging sensor according to the coordinate value of step S53, and measuring the absolute distance value of the current laser ranging point.

[0106] For step S52, this embodiment adopts Zhang Zhengyou's calibration method, which specifically includes the following steps:

[0107] S521: A 4×4 square chessboard plane is used as the calibration template, where the size of each square is 30 mm×30 mm, and the horizontal and vertical spacing of the chessboard is the same as the size of the square;

[0108] S522: using camera 2 to collect a plurality of calibration template images of the calibration template, wherein the posture of the calibration template in each image and the distance between the calibration template and camera 2 are different;

[0109] S523: Use the Camera Calibrator in Matlab to calibrate the camera, that is, input calib to start the camera calibration toolbox, use the corner detection algorithm to detect feature points in the image, perform correction operations, and obtain the internal and external parameters of the camera.

[0110] For step S53, the conversion formula between the image coordinate system and the camera coordinate system is:

[0111]

[0112] The conversion formula between the image coordinate system and the world coordinate system is:

[0113]

[0114] Among them, K is the internal parameter of camera 2, is the external parameter of camera 2. (u,v) is the coordinate of any point P on the normalized depth image in the image coordinate system. is the coordinate of point P in the camera coordinate system, the origin of the coordinate system is the optical center of camera 2, the X axis and Y axis are parallel to the X axis and Y axis of the image coordinate system respectively, and the Z axis is the optical axis of camera 2. w ,y w ,z w ) are the coordinates of point P in the world coordinate system.

[0115] Step S6 specifically includes the following steps:

[0116] S61: using the depth value obtained in step S4 as a data set and the absolute distance value measured in step S5 as a true value to train the polynomial function fitting neural network, and in the training process, updating the parameters of the polynomial function fitting neural network itself according to the gradient back propagation based on the second loss function;

[0117] S62: According to the polynomial function obtained by fitting the polynomial function fitting neural network after the training is completed, the absolute distance values ​​corresponding to other depth values ​​in the depth image are predicted, such as Figure 4 shown.

[0118] The polynomial function fitting neural network contains a fully connected layer, and the second loss function adopts the MSELoss loss function, which is expressed as:

[0119]

[0120] in, is the predicted value of the network, and y is the true value.

[0121] In specific applications, we only need to input the collected original image into the trained self-supervised underwater depth estimation network to obtain the corresponding depth image, and then actually measure the actual distance values ​​of a small number of points in the environment. Finally, we input the depth image and the small number of measured absolute distance values ​​into the polynomial function fitting neural network training and testing to obtain the global absolute distance information of the original underwater image.

[0122] In summary, an embodiment of the present invention provides an underwater absolute distance estimation method based on a neural network and a small number of point measurements, and obtains a fine-grained depth map by constructing, training and testing a self-supervised underwater depth estimation network. Since the depth map only contains relative distance information, namely the depth value, and there are problems such as scale inconsistency, the present invention further measures the absolute distance of a small number of points and uses the distance as the true value to train the polynomial function fitting network, so that the polynomial function between the depth value and the absolute distance can be accurately fitted, which is convenient for obtaining the global absolute distance of the original underwater image. The distance between the surrounding environment and the operating subject is one of the most important information required for the autonomous operation of underwater robots. Accurate underwater depth estimation is the cornerstone for realizing underwater autonomous exploration, and provides services for underwater robots to avoid obstacles and perform refined operations.

[0123] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A method for underwater absolute distance estimation based on neural network and a small number of point measurements, characterized in that: Includes steps: S1: Use a monocular camera to collect multiple original images of different underwater scenes to obtain an underwater original image dataset; S2: Using the underwater original image dataset to train a self-supervised underwater depth estimation network; S3: using a monocular camera to collect a measured image when actually measuring an underwater environment, and using the trained self-supervised underwater depth estimation network to test the measured image to obtain a depth image with a depth value; S4: Select a specific number of image coordinate points in the depth image to obtain a depth value of each image coordinate point; S5: using a laser distance measuring sensor to measure the absolute distance value of the actual position corresponding to each of the image coordinate points; S6: using a neural network to fit a function between the depth value selected in step S4 and the absolute distance value measured in step S5, and calculating the absolute distance values ​​corresponding to other depth values ​​in the depth image based on the fitted function; The step S2 specifically includes the following steps: S21: Building a self-supervised underwater depth estimation network, the self-supervised underwater depth estimation network includes a deep network and a pre-trained optical flow network, the deep network is based on the MonoDepth2 network framework, and the optical flow network adopts the FlowNet2.0 architecture; S22: inputting the underwater original image data set into the optical flow network to obtain a corresponding optical flow map, and further generating a mask map of the non-occluded area in the optical flow map; S23: performing CLAHE image enhancement and Sobel operator operation on the underwater original image data set to obtain corresponding enhancement images and first-order gradient images; S24: Inputting the underwater original image data set into the deep network for training. During the training process, the parameters of the deep network itself are updated according to gradient back propagation based on a first loss function. The first loss function is constructed based on the depth image generated by the deep network, the reconstructed image generated by the deep network, the optical flow image, the mask image, the enhancement image, and the first-order gradient image.

2. The underwater absolute distance estimation method based on neural network and a small number of point measurements according to claim 1 is characterized in that: In step S24, the first loss function is expressed as: L=Lp+λsLs+λcLc+λgLg Among them, Lp represents the photometric loss between the enhanced image, the reconstructed image and the mask image, Ls represents the smoothness loss between the depth image and the enhanced image, Lc represents the consistency loss between the depth information of the depth image, the optical flow information of the optical flow map and the mask image, Lg represents the gradient loss between the first-order gradient map, the first-order gradient map of the reconstructed image and the mask image, and λs, λc, and λg are the weight coefficients of the corresponding losses.

3. The underwater absolute distance estimation method based on neural network and a small number of point measurements according to claim 2 is characterized in that: The luminosity loss Lp is expressed as: Among them, It′、 They represent the enhanced image and the reconstructed image respectively, parameter α=0.85, SSIM(,) represents the structural similarity index, ‖,‖1 represents the L1 loss, and M represents the mask image; The mask image M is expressed as: M=[wf(x)+wb(x+wf(x))] <Th∩[I1(x)-I2(x+wf(x))]<Th Where I1 represents the current frame, i.e., the picture of frame 0, I2 represents the picture of the previous frame, i.e., frame -1, x represents the position on the pixel coordinates, wf represents the optical flow from I1 to I2, wb represents the optical flow with the opposite direction, and Th represents the threshold value, which is set to 1; The smoothing loss Ls is expressed as: Wherein, D(pt) represents the estimated distance of the pixel point pt in the depth image, |·| represents the absolute value of the element, is a vector differential operator, T represents the weighted transpose of the image gradient, The enhancement map is also shown.

4. The underwater absolute distance estimation method based on neural network and a small number of point measurements according to claim 3 is characterized in that: The consistency loss Lc is the sum of the pixel space consistency loss Lci and the camera space consistency loss Lcc; The pixel space consistency loss Lci is expressed as: Among them, pt is the two-dimensional coordinate of a pixel point in the current frame, i.e., frame 0. and are the two-dimensional coordinates of the source image corresponding to the point pt in the +1 frame or -1 frame obtained through the depth information and the optical flow information, respectively. Here, ‖.‖ represents the two-dimensional distance; The camera space consistency loss Lcc is expressed as: in, and It represents the three-dimensional coordinates corresponding to the point pt obtained by depth information and optical flow information in the source camera coordinate space, where ‖.‖ represents the three-dimensional space distance; The gradient loss Lg is expressed as: Among them, gt represents the first-order gradient map of the current frame, i.e., frame 0. represents the first-order gradient of the reconstructed map.

5. The underwater absolute distance estimation method based on neural network and a small number of point measurements according to any one of claims 1 to 4, characterized in that: The step S4 specifically comprises the following steps: S41: normalizing the depth image pixels to obtain a corresponding normalized depth image; S42: In the normalized depth image, at least one pixel coordinate point is selected from each of the four intervals A1, A2, A3, and A4 where the normalized pixel value is within the range A1, A2, A3, and A4. <A2<A3<A4; S43: Determine the depth value of each selected pixel coordinate point.

6. The underwater absolute distance estimation method based on neural network and a small number of point measurements according to claim 5 is characterized by: In step S42, the four intervals A1, A2, A3, and A4 are 0.01-0.05, 0.38-0.42, 0.68-0.72, and 0.88-0.92, respectively.

7. The underwater absolute distance estimation method based on neural network and a small number of point measurements according to any one of claims 1 to 4, characterized in that: The step S5 specifically comprises the following steps: S51: Fix the camera and the laser distance sensor adjacent to each other, and make the direction of the laser distance sensor consistent with the direction of the optical axis of the camera; S52: Calibrate the camera to obtain internal and external parameters of the camera; S53: according to the internal and external parameters of the camera, convert the pixel coordinate point selected in step S4 from the image coordinate system to the coordinate value in the camera coordinate system and the world coordinate system; S54: adjusting the angle of the laser ranging sensor according to the coordinate value of step S53, and measuring the absolute distance value of the current laser ranging point.

8. The underwater absolute distance estimation method based on neural network and a small number of point measurements according to any one of claims 1 to 4, characterized in that: The step S6 specifically comprises the following steps: S61: using the depth value obtained in step S4 as a data set and the absolute distance value measured in step S5 as a true value to train the polynomial function fitting neural network, and in the training process, updating the parameters of the polynomial function fitting neural network itself according to the gradient back propagation based on the second loss function; S62: predicting absolute distance values ​​corresponding to other depth values ​​in the depth image according to the polynomial function obtained by fitting the polynomial function fitting neural network after the training is completed.

9. The underwater absolute distance estimation method based on neural network and a small number of point measurements according to claim 8, characterized in that: The polynomial function fitting neural network includes a fully connected layer, and the second loss function adopts the MSELoss loss function.

Images