Underwater low-light static scene three-dimensional reconstruction and enhancement method based on neural radiance field

Abstract

The application discloses a kind of underwater low light static scene three-dimensional reconstruction and enhancement method based on neural radiation field, comprising the following steps: visible optical image data of underwater low light static scene of multiple exposure is obtained by camera;Calculate the camera internal and external parameters corresponding to the visible optical image taken;Neural radiation field model of underwater low light static scene is constructed;Visible optical image and corresponding camera internal and external parameters collected by camera are input into neural radiation field model for supervised training;Based on visual theory, construct brightness enhancement model;The trained neural radiation field is input into brightness enhancement model for supervised training, and the underwater three-dimensional scene model after removing water medium interference and brightening is output.The method of the present application can effectively reduce the influence of interference factors such as absorption and scattering of seawater on light, low light shooting noise, etc.on the three-dimensional reconstruction accuracy, significantly improve the authenticity and accuracy of underwater low light static scene three-dimensional reconstruction.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of three-dimensional vision and image processing, in particular to a method for three-dimensional reconstruction and enhancement of underwater low-light static scenes based on neural radiance fields. BACKGROUND

[0002] With the rapid development of computer vision and computational photography technology, three-dimensional reconstruction technology has been widely applied in many fields, such as virtual reality, augmented reality, cultural heritage protection and medical imaging, etc. However, underwater environment has always been a big challenge in the field of three-dimensional reconstruction due to its complex optical characteristics. The underwater lighting conditions are complex, and the scattering and absorption effects of light are significant, which leads to the poor performance of traditional optical-based three-dimensional reconstruction methods in underwater environment.

[0003] Traditional underwater three-dimensional reconstruction methods mainly rely on structured light, laser scanning and multi-view stereo vision technology. These methods can achieve certain results in shallow water environment or good lighting conditions, but their performance significantly decreases in low-light, deep water or turbid water bodies. In addition, these methods usually require complex equipment and a large amount of computing resources, making it difficult to promote in practical applications.

[0004] In recent years, the emergence of neural radiance field (NeRF) technology has brought new opportunities to the field of three-dimensional reconstruction. NeRF uses deep learning technology to represent the radiance field of a three-dimensional scene by training a neural network, which can achieve remarkable results in high-quality reconstruction and rendering. The core idea of NeRF is to implicitly encode the three-dimensional information of the scene in the weights of the neural network, and to reconstruct a high-quality three-dimensional scene from sparse image views through an optimization process.

[0005] However, the application of NeRF technology in underwater low-light static scenes still faces many challenges. The lighting conditions in underwater environment are extremely complex, and the propagation of light in water is affected by many factors, including scattering, absorption and refraction, etc. These factors make the underwater image quality low, which brings difficulties to the training and optimization of NeRF. In addition, there are a large number of suspended particles and organisms in underwater environment, which also interfere with image acquisition and processing. SUMMARY

[0006] The main purpose of the present application is to overcome the defects of the above background technology, and to provide a method for three-dimensional reconstruction and enhancement of underwater low-light static scenes based on neural radiance fields, which solves the problem of efficient and high-quality reconstruction and enhancement of underwater low-light static scene three-dimensional models in the prior art.

[0007] To achieve the above object, the application adopts the following technical solutions: a three-dimensional reconstruction and enhancement method of an underwater low-light static scene based on a neural radiation field, which comprises the following steps:

[0008] Step a: acquiring visible optical image data of the underwater low-light static scene under multi-exposure through a camera;

[0009] Step b: calculating camera internal and external parameters corresponding to the visible optical images;

[0010] Step c: physically modeling the light field and the reflection field based on the Retinex theory and the volume rendering technology, constructing a scene neural radiation field model, physically modeling the water medium based on the volume rendering technology, constructing a medium neural radiation field model, and thus constructing an overall neural radiation field model of the underwater low-light static scene;

[0011] Step d: inputting the single-exposure visible optical images collected by the camera and the corresponding camera internal and external parameters into the overall neural radiation field model for supervised training, and obtaining the reconstruction result of the underwater low-light static scene;

[0012] Step e: constructing a brightness enhancement model based on the visual theory;

[0013] Step f: based on the trained scene neural radiation field model, combining the multi-exposure visible optical images collected by the camera and the corresponding camera internal and external parameters, performing supervised training on the brightness enhancement model, and outputting an underwater three-dimensional scene model after removing the interference of the water medium and enhancing the brightness, which can render a two-dimensional visible optical image under any viewing angle.

[0014] Further, the step a specifically comprises: uniformly dividing the target scene into camera points for collection; and collecting at least two groups of visible optical images under different shutter times at each point.

[0015] Further, in the step b, the camera internal and external parameters corresponding to the visible optical images are calculated through the SFM algorithm, specifically as follows:

[0016] Step ①: using an image feature point extraction algorithm to detect key points and feature descriptors in each image, and finding matching feature point pairs in different images through the feature descriptors;

[0017] Step ②: estimating a fundamental matrix through a random sample consensus algorithm, and calculating an essential matrix by using the camera internal parameters;

[0018] Step ③: recovering multiple solutions of the camera relative rotation and translation matrix from the essential matrix through a triangulation verification method, and selecting a correct solution from the multiple solutions;

[0019] Step ④: reconstructing a three-dimensional point from the matching feature points by using a triangulation method;

[0020] Step 5: Starting from two views, gradually add new views and incrementally optimize the camera extrinsics and 3D point cloud.

[0021] Step 6: Estimate the camera intrinsic parameters through pre-calibration or self-calibration methods.

[0022] Furthermore, the step c specifically includes the following sub-steps:

[0023] Step 1: Parameterize the light: Let the starting point of the light be o(x,y,z) and the direction be

[0024] Step ②, discrete sampling: uniformly sample N points r(t i ),i=1,…,N;

[0025] Step 3, obtain the light field and reflection field: According to the Retinex theory, obtain each sampling point r(t i ) is L(r(t i ))=Φ1(r(t i ),d), the reflection field value is R(r(t i ))=Φ1(r(t i ), the object voxel density is

[0026] Step ④, obtain the color and density of the water medium: obtain each sampling point r(t i )'s medium color and medium voxel density

[0027] Step 5. Calculate the cumulative transparency: Calculate the value of each sampling point r(t i )'s cumulative transparency and medium cumulative transparency

[0028] Step 6: Differentiable volume rendering to obtain predicted images: The neural radiation field models Φ1 and Φ2 are used to predict each sampling point r(t i ) light field value L(r(t i ))、Reflection field value R(r(t i )), object voxel density Object cumulative transparency Medium color Medium voxel density and the cumulative transparency of the medium The final color value of light r is obtained by differentiable volume rendering And generate a predicted image through the final color values ​​of all lights on the image.

[0029] Further, the differentiable body rendering calculation formula is:

[0030]

[0031] wherein, is the predicted color of the object, is the predicted color of the medium, δ i is the distance between adjacent sampling points,

[0032] Further, in step d, the visible light image dataset collected by the camera with exposure parameter ∈0is supervised training, using gradient descent optimization strategy, training loss wherein, λ1, λ2, λ3 are hyperparameters; is the reconstruction radiance loss; is the light field smoothing loss; is the medium voxel density optimization loss, used to constrain the density distribution of the water medium to be as uniform as possible; is the object voxel density optimization loss, used to ensure that the scene neural radiance field model can construct the correct object geometry; after training, the neural radiance field model can be accessed from any viewing angle to obtain the corresponding two-dimensional medium-containing / non-medium reconstruction view.

[0033] Further, The reconstruction radiance loss in The formula is: Wherein sg(·) represents gradient stop, ε=10 -3 , C0, are the color values of the visible light image collected by the camera and the predicted images of the neural radiance field model Φ1 and Φ2 under exposure parameter ∈0respectively;

[0034] The light field smoothing loss The formula is: Wherein L is the two-dimensional light field image of the light field value L after differentiable body rendering, and respectively represent the gradients of the two-dimensional light field image L in the horizontal direction and the vertical direction;

[0035] The medium voxel density optimization loss The variance function is adopted, and it is assumed that the medium radiance field stops at the intersection of the light ray and the object surface during volume rendering, and the formula is: t i <Depth obj (r), wherein Var(·) is the sample variance, is the predicted object depth, t i is the distance from the sampling point to the origin of the light ray;

[0036] Object voxel density optimization loss The distribution of object voxel density on the ray r is made as dense as possible, and the non-dense distribution is punished, and the integral form is Where u and v are any two points on the ray r, The discrete form is Where t i ,t i’ are sampling points on the ray r.

[0037] Further, in step f, the brightness enhancement model is supervised training, specifically:

[0038] In the training phase, the input of the brightness enhancement model Φ3 is the sampled light field value L(r(t i )) output by the scene neural radiance field Φ1 and the brightness adjustment coefficient Where, ∈0, ∈ j ∈∈,j=1,…,M, ∈ is a set composed of exposure parameters corresponding to multi-exposure visible light image data, M=|∈|-1; The output of Φ3 is the exposure parameter ∈ j The corresponding enhanced light field value After differentiable volume rendering and combining the sampled reflectance value R(r(t i )) output by the scene neural radiance field Φ1, the exposure parameter ∈ j The corresponding predicted visible light image The training adopts gradient descent optimization strategy, and the training loss Where χ1 and χ2 are hyperparameters; is the reconstruction luminosity loss, is the color correction loss based on the gray world theory, is the contrast loss;

[0039] In the inference phase, by controlling the brightness adjustment coefficient To obtain the brightness-enhanced underwater low-light static scene.

[0040] Further, The reconstruction luminosity loss in The formula is: Where sg(·) represents gradient stop, ε=10 -3 , C j , are the color values of the visible light image collected by the camera under the exposure parameter ∈ j , and the color values of the images predicted by the neural radiance field models Φ1 and Φ2;

[0041] The color correction loss The formula is: where p, q are RGB channels, and satisfy (p, q) ∈ {(R, G), (R, B), (G, B)};

[0042] Contrast loss The formula is: where ζ is a hyperparameter.

[0043] Further, the scene neural radiance field model is a neural network composed of full connection layers, comprising: an 8-layer 256-dimensional common full connection layer, followed by three branch networks, respectively 1-layer 128-dimensional full connection layer, 1-layer 128-dimensional full connection layer, 4-layer full connection layer with 128-dimensional, 128-dimensional, 1-dimensional and 128-dimensional respectively, which are respectively used to output object voxel density value, reflection field value and light field value, non-linear activation function ReLU is used between levels, and Softplus, Sigmoid and Sigmoid activation functions are used in output layers respectively;

[0044] The medium neural radiance field model is a 4-layer full connection layer network, wherein the first 3 layers are 256-dimensional, and the last layer is 128-dimensional, which is used to output sampling point medium voxel density value and medium color value, non-linear activation function ReLU is used between levels, and Softpulus and Sigmoid activation functions are used in output layers respectively;

[0045] The brightness enhancement model is a neural network composed of 3-layer full connection layers with 256-dimensional, 128-dimensional and 128-dimensional respectively.

[0046] The application also provides an underwater low-light static scene three-dimensional reconstruction and enhancement system based on neural radiance field, comprising a microprocessor and a memory connected with each other, and the microprocessor is programmed or configured to execute the above-mentioned underwater low-light static scene three-dimensional reconstruction and enhancement method based on neural radiance field.

[0047] The application also provides a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and the computer program is used for programming or configuring a microprocessor to execute the above-mentioned underwater low-light static scene three-dimensional reconstruction and enhancement method based on neural radiance field.

[0048] The beneficial effects of the present application are: the underwater low-light static scene three-dimensional reconstruction and enhancement method based on neural radiation field proposed in the present application combines the advantages of NeRF, and conducts physical modeling according to the special optical characteristics of the underwater environment, aiming to improve the accuracy and efficiency of underwater three-dimensional reconstruction. Specifically, the present application constructs an adaptive neural network structure and an effective training strategy based on the Retinex theory and the physical properties of the underwater environment, realizing high-quality underwater three-dimensional reconstruction under low-light conditions. This method not only works effectively under low-light conditions, but also enhances the visual effect of underwater images, thereby providing higher-quality data support for subsequent applications. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0050] Figure 1 The flowchart of the underwater low-light static scene three-dimensional reconstruction and enhancement method based on neural radiation field provided by the embodiments of the present application is shown in the figure.

[0051] Figure 2 The network structure diagram of the underwater low-light static scene three-dimensional reconstruction and enhancement method based on neural radiation field provided by the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application. It should be emphasized that the following description is only exemplary and is not intended to limit the scope and application of the present application.

[0053] In combination with Figure 1 The embodiments of the present application provide an underwater low-light static scene three-dimensional reconstruction and enhancement method based on neural radiation field, and the flow includes:

[0054] Step a, acquiring visible optical image data of a multi-exposure underwater low-light static scene through a camera;

[0055] Step b, calculating the camera internal and external parameters corresponding to the visible optical image through the SFM (Structure from Motion) algorithm;

[0056] Step c, based on the Retinex theory and volume rendering technology, the physical modeling of the light field and the reflection field is carried out, the scene neural radiance field model is constructed, the physical modeling of the water medium is carried out based on the volume rendering technology, the medium neural radiance field model is constructed, and thus the overall neural radiance field model of the underwater low-light static scene is constructed;

[0057] Step d, the single-exposure visible optical image collected by the camera and the corresponding camera internal and external parameters are input into the overall neural radiance field model for supervised training, and the reconstruction result of the underwater low-light static scene is obtained;

[0058] Step e, a brightness enhancement model is constructed based on the visual theory;

[0059] Step f, based on the trained scene neural radiance field model, the multi-exposure visible optical image collected by the camera and the corresponding camera internal and external parameters are combined to supervise the training of the brightness enhancement model, and the underwater three-dimensional scene model after removing the water medium interference and brightness enhancement is output, which can render two-dimensional visible optical images under any viewing angle.

[0060] In an embodiment, the visible optical image data of the multi-exposure underwater low-light static scene is obtained by a camera, and the obtaining method is specifically as follows: 1) uniformly dividing the target scene into camera points for collection; 2) collecting at least two groups of visible optical images under different shutter times (the parameters such as photosensitivity and aperture remain the same) at each point.

[0061] In an embodiment, the camera internal and external parameters corresponding to the visible optical image are calculated by the SFM algorithm, and the specific method is as follows:

[0062] Step ①, feature detection and feature matching: using SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features) or ORB (Oriented FAST and Rotated BRIEF) image feature point extraction algorithm to detect key points and feature descriptors in each image, and finding matching feature point pairs in different images through feature descriptors;

[0063] Step ②, estimation of fundamental matrix and essential matrix: estimating the fundamental matrix by the random sample consensus algorithm (RANSAC, Random Sample Consensus), and calculating the essential matrix by using the calibration camera internal parameters;

[0064] Step ③, camera external parameter estimation: recovering multiple solutions of camera relative rotation and translation matrix from the essential matrix by the triangulation verification method, and selecting the correct solution from them;

[0065] Step ④, triangulation reconstruction of three-dimensional points: using the triangulation method to reconstruct three-dimensional points from matching feature points;

[0066] Step 5: Incremental reconstruction: Starting from two views, gradually add new views and incrementally optimize the camera extrinsics and 3D point cloud.

[0067] Step 6: Obtain camera intrinsic parameters: Estimate camera intrinsic parameters through pre-calibration or self-calibration methods.

[0068] In one embodiment, the light field and reflection field are physically modeled based on Retinex theory and volume rendering technology, and the water medium is physically modeled based on volume rendering technology, thereby constructing a neural radiation field model for underwater low-light static scenes. Specifically, the following steps are included:

[0069] Step 1: Parameterize the light: Let the starting point of the light be o(x,y,z) and the direction be Where x, y, z are the three-dimensional coordinates of the starting point of the light, θ, are the pitch angles of the light starting point;

[0070] Step ②, discrete sampling: uniformly sample N points r(t i ),i=1,…,N,r(t)=o+td, t is the sampling interval;

[0071] Step 3, obtain light field and reflection field: According to Retinex theory, light field is the lighting information provided by the light source in the scene, which affects the intensity of light received by the surface of the object. Reflection field is the characteristic of light reflected from the surface of the object, which is determined by the color and material of the object. Therefore, the light field value of the sampling point is related to the direction of the light, while the reflection field value is independent of the direction of the light. The scene neural radiation field model Φ1 is used to obtain each sampling point r(t i ) is L(r(t i ))=Φ1(r(t i ),d), the reflection field value is R(r(t i ))=

[0072] Φ1(r(t i ), the object voxel density is

[0073] Step ④, obtain the color and density of the water medium: obtain each sampling point r(t i )'s medium color and medium voxel density

[0074] Step 5. Calculate the cumulative transparency: Calculate the value of each sampling point r(t i )'s cumulative transparency and medium cumulative transparency Specifically, the cumulative transparency calculation formula is:

[0075]

[0076]

[0077] where δ k is the distance between adjacent sampling points.

[0078] Step Ⅵ, obtain the predicted image by differentiable volume rendering: predict the light field value L(r(t i ), the reflection field value R(r(t i )), the object voxel density the object cumulative transparency the medium color the medium voxel density and the medium cumulative transparency of each sampling point r(t i ) by the neural radiance field model Φ1 and Φ2, and calculate the final color value of the light ray r by differentiable volume rendering , and generate the predicted image by the final color value of all light rays on the image.

[0079] Specifically, the calculation formula of differentiable volume rendering is:

[0080]

[0081] where δ i is the distance between adjacent sampling points, is the object predicted color, is the medium predicted color, is the final cumulative transparency.

[0082] In an embodiment, the visible optical image collected by the camera and the corresponding camera internal and external parameters are input into the neural radiance field model for supervised training to obtain the reconstruction result of the underwater low-light static scene, specifically:

[0083] The visible optical image dataset collected by the camera with the exposure parameter ∈0 is used for supervised training, and a gradient descent optimization strategy is adopted, and the training loss is where λ1, λ2, and λ3 are hyperparameters.

[0084] is the reconstruction luminosity loss, where sg(·) represents gradient stop, ε=10 -3 , C0, are the color values of the visible optical image collected by the camera and the predicted image of the neural radiance field model Φ1 and Φ2 under the exposure parameter ∈0, respectively.

[0085] is the light field smoothing loss, where L is the 2D light field image of the light field value L after the differentiable body rendering, and denote the gradients of the 2D light field image L in the horizontal and vertical directions, respectively;

[0086] is the medium voxel density optimization loss, which is mainly used to constrain the density distribution of the water medium to be as uniform as possible. Here, a variance function is adopted, and it is assumed that the medium radiance field stops at the intersection of the light and the object surface during the body rendering process, thus obtaining where Var(·) is the sample variance, is the predicted object depth, t i is the distance from the sampling point to the origin of the light;

[0087] is the object voxel density optimization loss. In order to ensure that the scene neural radiance field can construct the correct object geometry, here is added to make the distribution of the object voxel density on the light r as dense as possible, and the non-dense distribution is punished. The integral form is where u and v are any two points on the light r, its discrete form is where t i ,t i’ are the sampling points on the light r.

[0088] After the training is completed, the neural radiance field model can be accessed from any viewing angle to obtain the corresponding 2D medium-containing / non-medium reconstructed view.

[0089] In an embodiment, based on the trained scene neural radiance field model, combined with the multi-exposure visible light images collected by the camera and the corresponding camera internal and external parameters, a brightness enhancement model is supervised trained, specifically:

[0090] In the training stage, the input of the brightness enhancement model Φ3 is the sampling point light field value L(r(t i )) output by the scene neural radiance field Φ1 (the exposure parameter is ∈0) and the brightness adjustment coefficient where ∈0, ∈ j ∈∈, j = 1, …, M, ∈ is a set composed of exposure parameters corresponding to multi-exposure visible light image data, M = |∈|-1. The output of Φ3 is the exposure parameter ∈ j corresponding to the enhanced light field value After differentiable body rendering and combined with the sampling point reflection field value R(r(t i )) output by the scene neural radiance field Φ1, the exposure parameter ∈ jthe corresponding predicted visible optical image

[0091] The training adopts a gradient descent optimization strategy, and the training loss where χ1, χ2 are hyperparameters;

[0092] is a reconstruction luminosity loss, where sg(·) represents gradient stop, and ε = 10 -3 , C j , are exposure parameters ∈ j The camera collects color values of the visible optical image and the predicted image of the neural radiance field model Φ1 and Φ2.

[0093] is a color correction loss based on the "gray world" theory, where p, q are RGB channels, and satisfy (p, q) ∈ {(R, G), (R, B), (G, B)};

[0094] is a contrast loss, where ζ is a hyperparameter.

[0095] In the inference stage, by controlling the brightness adjustment coefficient to obtain the brightness-enhanced underwater low-light static scene.

[0096] In an embodiment, the network structure is as shown in Figure 2 Specifically, it is:

[0097] The above scene neural radiance field model Φ1 is a neural network composed of fully connected layers, including: an 8-layer 256-dimensional shared fully connected layer, followed by three branch networks, respectively 1-layer 128-dimensional fully connected layer, 1-layer 128-dimensional fully connected layer, 4-layer fully connected layer (128-dimensional, 128-dimensional, 1-dimensional, 128-dimensional respectively) for outputting object voxel density value, reflection field value and light field value, non-linear activation function ReLU is used between levels, and Softplus, Sigmoid, Sigmoid activation functions are used in the output layer.

[0098] The above medium neural radiance field model Φ2 is a 4-layer fully connected layer network, of which the first 3 layers are 256-dimensional, and the last layer is 128-dimensional, for outputting sampling point medium voxel density value and medium color value, non-linear activation function ReLU is used between levels, and Softpulus, Sigmoid activation functions are used in the output layer.

[0099] The brightness enhancement model Φ3 is a neural network composed of three fully connected layers (256 dimensions, 128 dimensions, and 128 dimensions, respectively).

[0100] In addition, the application further provides an embodiment of a neural radiance field-based underwater low-light static scene three-dimensional reconstruction and enhancement system, which comprises a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the neural radiance field-based underwater low-light static scene three-dimensional reconstruction and enhancement method.

[0101] In addition, the application further provides an embodiment of a computer readable storage medium, which stores a computer program for being programmed or configured by a microprocessor to execute the neural radiance field-based underwater low-light static scene three-dimensional reconstruction and enhancement method.

[0102] The above is only the preferred embodiment of the application, and the protection scope of the application is not limited to the above-mentioned embodiments. Any technical solution falling within the concept of the application shall fall within the protection scope of the application. It should be noted that, for ordinary skilled persons in the art, some improvements and refinements without departing from the principles of the application shall also be considered as falling within the protection scope of the application.

Claims

1. A method for 3D reconstruction and enhancement of underwater low-light static scenes based on neural radiation fields, characterized in that: The steps include: Step a: acquiring multi-exposure visible optical image data of an underwater low-light static scene through a camera; Step b, calculating the camera internal and external parameters corresponding to the captured visible optical image; Step c: Physically modeling the light field and reflection field based on Retinex theory and volume rendering technology to construct a scene neural radiation field model; physically modeling the water medium based on volume rendering technology to construct a medium neural radiation field model, thereby constructing an overall neural radiation field model of the underwater low-light static scene; Step c specifically includes the following sub-steps: Step 1: Parameterize the light: Let the starting point of the light be o(x,y,z) and the direction be Step ②, discrete sampling: uniformly sample N points r(t i ),i=1,…,N; Step 3, obtain the light field and reflection field: According to the Retinex theory, obtain each sampling point r(t i ) is L(r(t i ))=Φ1(r(t i ),d), the reflection field value is R(r(t i ))=Φ1(r(t i ), the object voxel density is Step ④, obtain the color and density of the water medium: obtain each sampling point r(t i )'s medium color and medium voxel density Step 5. Calculate the cumulative transparency: Calculate the value of each sampling point r(t i )'s cumulative transparency and medium cumulative transparency Step 6: Differentiable volume rendering to obtain predicted images: The neural radiation field models Φ1 and Φ2 are used to predict each sampling point r(t i ) light field value L(r(t i ))、Reflection field value R(r(t i )), object voxel density Object cumulative transparency Medium color Medium voxel density and the cumulative transparency of the medium The final color value of light r is obtained by differentiable volume rendering And generate a predicted image through the final color values ​​of all lights on the image; Step d: Input the single-exposure visible optical image captured by the camera and the corresponding camera internal and external parameters into the overall neural radiation field model for supervised training to obtain the reconstruction result of the underwater low-light static scene; Step e: constructing a brightness enhancement model based on visual theory; Step f: Based on the trained scene neural radiation field model, combined with the multi-exposure visible optical images collected by the camera and the corresponding internal and external parameters of the camera, supervised training of the brightness enhancement model is performed to output a brightened underwater three-dimensional scene model after removing water medium interference, which can render a two-dimensional visible optical image at any viewing angle.

2. The method for 3D reconstruction and enhancement of underwater low-light static scenes based on neural radiation fields according to claim 1 is characterized in that: The step a specifically comprises: evenly dividing the target scene into acquisition camera points; and acquiring at least two groups of visible optical images with different shutter times at each point.

3. The method for 3D reconstruction and enhancement of underwater low-light static scenes based on neural radiation fields according to claim 1 is characterized in that: In step b, the camera internal and external parameters corresponding to the visible optical image are calculated using the SFM algorithm, specifically: Step 1: Use the image feature point extraction algorithm to detect key points and feature descriptors in each image, and find matching feature point pairs in different images through feature descriptors; Step 2: Estimate the basic matrix through random sampling consistency algorithm and calculate the essential matrix using the calibrated camera intrinsic parameters; Step 3: Recover multiple sets of solutions of the camera's relative rotation and translation matrices from the essential matrix through triangulation verification method, and select the correct solution from them; Step 4: Use triangulation method to reconstruct 3D points from matching feature points; Step 5: Starting from two views, gradually add new views and incrementally optimize the camera extrinsics and 3D point cloud. Step 6: Estimate the camera intrinsic parameters through pre-calibration or self-calibration methods.

4. The method for 3D reconstruction and enhancement of underwater low-light static scenes based on neural radiation fields according to claim 1 is characterized in that: The calculation formula for differentiable volume rendering is: in, Predicting color for objects, Predict the color for the medium, δ i is the distance between adjacent sampling points, 5. The method for 3D reconstruction and enhancement of underwater low-light static scenes based on neural radiation fields according to claim 1 is characterized in that: In step d, supervised training is performed using a visible optical image dataset with exposure parameters ∈ 0 collected by the camera, using a gradient descent optimization strategy, and the training loss Among them, λ1, λ2, λ3 are hyperparameters; To reconstruct the luminosity loss; is the light field smoothing loss; Optimize the loss for the medium voxel density, which is used to constrain the density distribution of the water medium to make it as uniform as possible; The loss is optimized for the object voxel density to ensure that the scene neural radiation field model can construct the correct object geometry; after training, the neural radiation field model can be accessed from any perspective to obtain the corresponding two-dimensional media / media-free reconstruction view.

6. The method for 3D reconstruction and enhancement of underwater low-light static scenes based on neural radiation fields according to claim 5, characterized in that: Reconstructed photometric loss in The formula is: where sg(·) indicates gradient stop, ε=10 -3 ,C0, The color values ​​of the visible optical image collected by the camera under exposure parameter ∈ 0 and the image predicted by the neural radiation field model Φ1 and Φ2 respectively; Light field smoothing loss The formula is: Where L is the two-dimensional light field image after the light field value L is rendered by the differentiable body, and Represent the gradients of the two-dimensional light field image L in the horizontal and vertical directions respectively; Dielectric voxel density optimization loss Using the variance function and assuming that the medium radiation field stops at the intersection of the light and the object surface during volume rendering, the formula is: Where Var(·) is the sample variance, To predict the depth of the object, t i is the distance from the sampling point to the origin of the light, δ i is the distance between adjacent sampling points; Object voxel density optimization loss Make the distribution of object voxel density on ray r as dense as possible and penalize non-dense distribution. Its integral form is Where u and v are any two points on the ray r, Its discrete form is where t i ,t i ' is the sampling point on ray r.

7. The method for 3D reconstruction and enhancement of underwater low-light static scenes based on neural radiation fields according to claim 5, characterized in that: In step f, supervised training is performed on the brightness enhancement model, specifically: During the training phase, the input of the brightness enhancement model Φ3 is the light field value L(r(t i )) and brightness adjustment coefficient Among them,∈0,∈ j ∈∈, j=1,…,M, ∈ is a set of exposure parameters corresponding to multi-exposure visible optical image data, M=|∈|-1; Φ3 output is the exposure parameter ∈ j Corresponding enhanced light field value After differentiable volume rendering and combining with the output of the scene neural radiation field Φ1, the reflection field value R(r(t i )), get the exposure parameter ∈ j The corresponding predicted visible optical image The training adopts the gradient descent optimization strategy, and the training loss Among them, χ1 and χ2 are hyperparameters; To reconstruct the photometric loss, is the color correction loss based on the gray world theory, is the contrast loss; In the inference stage, by controlling the brightness adjustment coefficient To obtain underwater low-light static scenes with enhanced brightness.

8. The method for 3D reconstruction and enhancement of underwater low-light static scenes based on neural radiation fields according to claim 7, characterized in that: Reconstructed photometric loss in The formula is: where sg(·) indicates gradient stop, ε=10 -3 , C j , They are exposure parameters ∈ j The lower camera collects visible optical images and the color values ​​of the images predicted by the neural radiation field models Φ1 and Φ2; Color correction loss The formula is: Where p,q are RGB channels, and satisfy (p,q)∈{(R,G),(R,B),(G,B)}; Contrast loss The formula is: Where ζ is a hyperparameter.

9. The method for 3D reconstruction and enhancement of underwater low-light static scenes based on neural radiation fields according to any one of claims 1 to 8, characterized in that: The scene neural radiation field model is a neural network composed of fully connected layers, including: an 8-layer 256-dimensional shared fully connected layer, followed by three branch networks, namely 1 layer 128-dimensional fully connected layer, 1 layer 128-dimensional fully connected layer, and 4 layers of 128-dimensional, 128-dimensional, 1-dimensional, and 128-dimensional fully connected layers, respectively, for outputting object voxel density values, reflection field values, and light field values. The nonlinear activation function ReLU is used between layers, and the output layer uses Softplus, Sigmoid, and Sigmoid activation functions respectively; The medium neural radiation field model is a 4-layer fully connected network, where the first 3 layers are 256-dimensional and the last layer is 128-dimensional, which is used to output the medium voxel density value and medium color value of the sampling point. The nonlinear activation function ReLU is used between the layers, and the output layer uses Softpulus and Sigmoid activation functions respectively. The brightness enhancement model is a neural network consisting of three fully connected layers with dimensions of 256, 128, and 128, respectively.

Images