An underwater image restoration method and system based on physical information differentiable scattering rendering

Abstract

The application discloses an underwater image restoration method and system based on physical information differentiable scattering rendering, and aims at solving the problems of missing underwater point light source two-way attenuation modeling, multi-solution ambiguity of degradation inverse problem and inability to output turbidity in white box, wherein the application inverses three-dimensional coordinates of a point light source by using double-mirror spherical high-light geometry constraints; a single scattering analytical rendering module containing two-way attenuation is constructed, a 1000*1000 two-dimensional lookup table is realized by bilinear interpolation, a 20-channel input is driven by 5-grade light intensity coupling, a physical constraint U-Net is used to jointly predict a restored image and a scattering coefficient, and a de-turbidity-turbidity self-supervised closed-loop reconstruction loss constraint is used for training; physical turbidity is output based on linear mapping within a calibration interval, and the interval is truncated according to the nearest endpoint or is expanded after secondary calibration outside the interval. The PSNR of the application is improved by 13.992 dB, and the turbidity error is only 0.098 NTU.

Description

Technical Field

[0001] This invention belongs to the field of underwater computer vision and underwater imaging physical modeling technology, specifically relating to an underwater image restoration method and system based on physically information-based differential scattering rendering. Background Technology

[0002] Underwater imaging is the physical process by which light propagates through a water medium and is received by a camera. Light in water undergoes both absorption and scattering attenuation, often explained by the Beer-Lambert attenuation law. I = I 0·exp(− β · d Describes exponential decay. β The volumetric extinction coefficient or scattering coefficient; the Jaffe-McGlamery classical underwater imaging equation further decomposes the radiation intensity received by the camera into three parts: direct component, forward scattering, and backscattering, which is the fundamental paradigm for underwater imaging physical modeling to date. Based on the above physical foundation, four technical approaches have developed in the field of underwater image restoration and enhancement: (i) traditional restoration methods based on physical models (such as DCP / UDCP, Sea-thru); (ii) pure data-driven methods based on deep learning (such as FUnIE-GAN, Shallow-UWnet, U-shape Transformer, WF-Diff); (iii) physical-data hybrid methods (such as AOD-Net, Ucolor, Polarimetric PINN); and (iv) turbidity sensing and Sim-to-Real cross-domain methods. None of the above methods have solved the problem of high-quality restoration and quantifiable turbidity inversion of underwater scenes with artificial point light sources within a unified physical framework.

[0003] A search revealed that the closest existing technology to this invention includes the following two Chinese invention patents: First, there is the Chinese invention patent application CN114037625A, entitled "An Image Restoration Method Based on a Physical Scattering Model" (applicant: Shenyang Institute of Automation, Chinese Academy of Sciences, application date: 2021). This patent application introduces real physical parameters such as the zenith angle, azimuth angle, extinction coefficient, and anisotropy constant of the light source to establish a scattering model under non-uniform illumination conditions. Its technical approach is to use the Heyney-Greenstein phase function and an improved dark-channel-like prior to sample the pure scattering region and then invert the physical quantities through nonlinear multi-parameter curve fitting. Finally, the backscattering component and attenuation component are subtracted from the degraded image to obtain the restored image. The solution process used in this patent application relies on nonlinear curve fitting, and the obtained light source parameters are scalar forms of the zenith angle and azimuth angle.

[0004] Second, the Chinese invention patent with authorization announcement number CN114463211B, entitled "An Underwater Image Enhancement Method Based on Turbidity Grading" (patentee: Huazhong University of Science and Technology, application date: 2022, authorization announcement: 2024), first divides the underwater scene into three turbidity levels—light, medium, and heavy—based on image histogram statistics. Then, images of different turbidity levels are routed to corresponding traditional image enhancement algorithm branches (including white balance, contrast-limited adaptive histogram equalization, gamma correction, etc.) for processing. Finally, the enhanced image is obtained through multi-scale fusion.

[0005] After investigation, the above-mentioned existing technologies have the following technical disadvantages: First, existing physical model-based methods generally assume that ambient light is uniformly distributed and only describe the unidirectional attenuation from the object surface to the camera. They neither describe the directional illumination characteristics of artificial LED point light sources carried by underwater robots / diving equipment, nor model the two-way attenuation mechanism along the complete optical path from the light source to the surface to the camera. Even though CN114037625A introduces parameters such as the zenith angle and azimuth angle of the light source, its solution still relies on nonlinear curve fitting. The light source is estimated in the form of scalar parameters, which cannot accurately recover the complete three-dimensional coordinates of the light source in the camera coordinate system.

[0006] Second, pure data-driven deep learning methods are black-box mappings, lacking physical interpretability and heavily reliant on the distribution of training data. In unknown turbidity scenarios outside the distribution, cross-domain generalization failures, structural distortions, and color illusions often occur. Some methods even have SSIM values ​​lower than the degenerate input ("negative optimization") and cannot output quantifiable physical parameters.

[0007] Third, existing Sim-to-Real cross-domain methods mostly rely on style transfer or adversarial domain adaptation, which only aligns the synthetic and real data at the feature level, and cannot unify the geometric and radiometric parameters of the virtual engine and the real camera at the physical level.

[0008] Fourth, both existing turbidity sensing methods and physical-data hybrid methods have significant drawbacks: the former only uses turbidity level or scalar turbidity factor to participate in algorithm branch selection or parameter modulation, without establishing a quantifiable quantitative mapping with scattering physical quantities, and without outputting NTU physical quantities; the latter, even if it uses simplified physical equations as regularization terms or embeds them into neural networks as feedforward layers, usually only characterizes unidirectional decay and is not differentiable, does not support end-to-end gradient backpropagation, and cannot achieve true coupling between physical constraints and data-driven approaches. Summary of the Invention

[0009] This invention aims to address the shortcomings of existing technologies and provides the following solutions: A method for underwater image restoration based on physically differentiable scattering rendering includes the following steps: The camera is calibrated to obtain the camera intrinsic parameter matrix. K And build an underwater imaging acquisition system that includes artificial point light sources, cameras, liquid turbidity meters and underwater illuminance meters; The three-dimensional coordinates of the artificial point light source in the camera coordinate system are inverted using specular reflective constraints from at least two specular reflectors of known radii. PL The distance from the light source to the camera is obtained. DSV ; Construct a differentiable, resolvable single scattering rendering unit that includes two-way attenuation from the light source to the surface to the camera, and perform analytical rendering of the underwater imaging process; The virtual camera and the real camera are jointly aligned geometrically and photometrically, and a digital twin synthetic dataset is constructed. Within the preset turbidity gradient range The virtual scattering coefficient is obtained based on color block matching and least squares regression fitting. β Compared with true physical turbidity T Linear mapping between NTUs And the effective working range of the linear mapping is limited to the calibrated range. Outside the interval, the output is truncated at the nearest endpoint or the interval is recalibrated to expand the interval; The same scene under multiple levels of physical light intensity I The degraded image at 0 and the corresponding normalized light intensity channel are concatenated along the channel dimension to form a multi-channel input tensor, which is then input into the physically constrained neural network. Sim-to-Real cross-domain fine-tuning of the physical constraint neural network is performed using real underwater images. The underwater image to be restored is then input into the fine-tuned physical constraint neural network, which outputs the restored image. With scattering coefficient The linear mapping is inversely solved within the specified working range to obtain the physical turbidity. ;when When the value falls outside the working range, the system will either truncate the data at the nearest endpoint or trigger a preset secondary calibration / failure alarm mechanism.

[0010] Preferably, in the camera calibration, Zhang's calibration method is used to perform stereo correction on the binocular camera to obtain the camera intrinsic parameter matrix K, wherein the focal length component in the camera intrinsic parameter matrix K... , With optical center coordinates cx , cy These are fixed values ​​that have already been calibrated.

[0011] Preferably, the three-dimensional coordinates are obtained. PL The methods include: Place at least two known radii in the underwater scene. R The mirror reflector is used as the target for high-light detection. The mirror reflector includes, but is not limited to, stainless steel mirror spheres, chrome-plated mirror spheres, nickel-plated mirror spheres, or cylindrical / ellipsoidal geometric bodies with mirror reflection characteristics. For each of the specular reflectors, the pixel coordinates of the highlight point and the radius of the sphere's outline circle are detected in the degraded image. r According to the pinhole imaging relationship Inversely solve the depth of the sphere's center in the camera coordinate system Zc And based on this, the three-dimensional coordinates of the sphere's center are obtained. C ; According to the direction of the line of sight D From the camera's optical center to the highlight point, and with C For the center of the ball, R Solving the equations of the sphere with radius r, we can obtain the specular reflection point. S and the aforementioned high-light reflection point S unit normal vector N ; According to the law of specular reflection L = V −2( V · N ) N Calculate the direction of incident light L ,in V For the reason S The unit view vector pointing to the camera's optical center; For each of the two mirror spheres, a line is obtained with a highlight reflection point. Yes Starting point, direction - Li The incident light ray, constructing an overdetermined linear equation system and the least squares solution Find the midpoint of the common perpendicular segment of the two incident rays, and use it as the three-dimensional coordinate of the point light source in the camera coordinate system. PL The final estimated value.

[0012] Preferably, the differentiable single-scatter rendering unit uses a two-dimensional lookup table with a preset resolution. By combining bilinear interpolation to achieve a differentiable lookup table, the light source-surface incident attenuation term is calculated forward. Surface-camera emission attenuation term and single backscattering term The ; The two-dimensional lookup table is Its coverage area is , The resolution is 1000×1000, and it is resident in the GPU memory; The differentiable lookup table, through... Bilinear interpolation is performed on the two-dimensional lookup table to allow the gradient to be backpropagated to the restored image via the single-scattering analytical rendering module. With scattering coefficient ; The forward calculation formula of the single scattering analytical rendering module is: ,in, To restore the radiometric values ​​of the image in the linear RGB space, The distance from the light source to the object's surface. The distance from the camera to the object's surface. For a single backscattering term; when Below the preset threshold e At that time, the module calculates in the equivalent form of a small-angle Taylor expansion. The To avoid numerical oddities; in, , I 0 represents the physical light intensity of a point light source. The water volume scattering coefficient, The scattering phase function; ; The angle parameter corresponding to the starting point of the effective scattering zone entered by the camera's line of sight. The distance from the camera to the starting point of the effective scattering zone along the line of sight; The corresponding camera's line of sight on the object's surface P The angle parameter at the termination position; For light source PL Camera optical center O Points on the surface of the object P In the triangle formed by the three elements, the camera's line of sight... D View and Light Source - Camera Baseline PL - O The angle between them.

[0013] Preferably, the method for performing the geometric and photometric joint alignment includes: Fix the focal length of the virtual camera in the virtual engine to [value]. F ue, and press The sensor width of the virtual camera is solved in reverse to ensure that the horizontal field of view error between the virtual camera and the real camera does not exceed a preset threshold. based on The Lux-Lumens spherical divergence mapping represents the illuminance of a real underwater illuminometer at a distance d. E Converted to the total luminous flux of a point light source in a virtual engine FThis aligns the radiometric measurements of the virtual camera with those of the real camera; among which, Let be the area of ​​the sphere with the light source as its center and d as its radius; A proportionally scaled-to-scale DGK standard matte color chart is constructed. The material parameters of the DGK standard matte color chart are set as follows: metallicity of 0, roughness of 1.0, and specular reflection intensity of 0.5. By traversing and searching for exposure compensation coefficients, the maximum error ΔL between the virtual camera rendered image and the real camera captured image on all color blocks in the CIELAB space L channel is not greater than a preset threshold.

[0014] Preferably, the method for obtaining the linear mapping includes: Within a preset turbidity gradient range, the liquid turbidity meter simultaneously calibrates each level of the actual physical turbidity. T NTU, and collect no less than 30 sets of corresponding real color chart images; The images of the acquired real color chart and the DGK standard matte color chart were respectively reverse-engineered. The calibration was performed, and the linear spatial mean values ​​were calculated for the R, G, and B channels on the preset color blocks of the DGK standard matte color chart. For each level of true turbidity T NTU, with L With the goal of minimizing the distance, the optimal scattering coefficient is matched from the synthetic color chart image. ,get Sample set; Regarding the The linear mapping is obtained by performing least-squares univariate linear regression on the sample set. .

[0015] Preferably, the physical constraint neural network includes: a main branch that jointly predicts the restored image and an auxiliary branch that predicts the scattering coefficients; The physical constraint neural network includes a 4-level encoder-decoder structure, skip connections, and a bottleneck layer, wherein the bottleneck layer has 512 channels. The main branch is output from the decoding end of the physical constraint neural network and mapped to a 3-channel restored image via Sigmoid activation. The auxiliary branch is predicted by the normalized scattering coefficient output from the bottleneck layer through adaptive average pooling and three fully connected layers with Dropout, and the scattering coefficient is obtained according to a preset linear inverse normalization relationship. ; Reconstructing loss using a deturbid-turbidity physical self-supervised closed loop L recon trains the physical constraint neural network; The multi-channel input tensor is generated from the same scene at 5 levels of physical light intensity. IThe degraded image at 0 is obtained by stitching together the five corresponding normalized light intensity channels along the channel dimension, wherein the five levels of physical light intensity are... I 0 corresponds to different underwater illuminance meter readings, and the total number of channels of the multi-channel input tensor is 20.

[0016] Preferably, the total loss function of the physical constraint neural network is: Among them, the deturbidity-turbidity physical self-supervised closed-loop reconstruction loss L recon is obtained by subtracting the original degraded image from the restored image output by the main branch and the scattering coefficients output by the auxiliary branch after they are re-turbidified by a differentiable single scattering rendering unit. L char is for the restored image Charbonnier loss between the image and the corresponding sharp reference image; Lβ For scattering coefficient prediction The supervised loss; the total loss function also includes a perceptual loss based on VGG16 pre-trained features and an SSIM loss based on structural similarity; l recon、 l char and lb These are the weighting coefficients for the corresponding loss terms.

[0017] The present invention also provides an underwater image restoration system based on physically based differentiable scattering rendering. The system applies the above-mentioned method and includes: a camera calibration and image acquisition module, a monocular point light source three-dimensional positioning module, a two-way attenuation single scattering differentiable analytical rendering module, a joint alignment module, a linear mapping construction module, an image stitching module, and an image restoration module. The camera calibration and image acquisition module is used to calibrate the camera and obtain the camera intrinsic parameter matrix. K And build an underwater imaging acquisition system that includes artificial point light sources, cameras, liquid turbidity meters and underwater illuminance meters; The monocular point light source 3D positioning module uses the specular geometric reflection constraints of at least two specular reflectors with known radii to invert the 3D coordinates of the artificial point light source in the camera coordinate system. PL The distance from the light source to the camera is obtained. DSV ; The dual-path attenuation single scattering differentiable analytical rendering module is used to construct a differentiable analytical single scattering rendering unit containing dual-path attenuation of light source-surface-camera, and to perform analytical rendering of the underwater imaging process. The joint alignment module is used to perform geometric and photometric joint alignment between virtual and real cameras and to construct a digital twin synthetic dataset. The linear mapping construction module is in the preset turbidity gradient range The virtual scattering coefficient is obtained based on color block matching and least squares regression fitting. β Compared with true physical turbidity T Linear mapping between NTUs And the effective working range of the linear mapping is limited to the calibrated range. Outside the interval, the output is truncated at the nearest endpoint or the interval is recalibrated to expand the interval; The image stitching module will stitch the same scene under multiple levels of physical light intensity. I The degraded image at 0 and the corresponding normalized light intensity channel are concatenated along the channel dimension to form a multi-channel input tensor, which is then input into the physically constrained neural network. The image restoration module uses real underwater images to perform Sim-to-Real cross-domain fine-tuning on the physical constraint neural network, inputs the underwater image to be restored into the fine-tuned physical constraint neural network, and outputs the restored image. With scattering coefficient The linear mapping is inversely solved within the specified working range to obtain the physical turbidity. ;when When the value falls outside the working range, the system will either truncate the data at the nearest endpoint or trigger a preset secondary calibration / failure alarm mechanism.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention is the first in the field of underwater image restoration to achieve end-to-end embedding of a differentiable scattering rendering layer. By using a 1000×1000 two-dimensional lookup table and bilinear interpolation, the gradient of the physical rendering process can penetrate into the restored image. With scattering coefficient Physical constraints and data-driven approaches are truly coupled, fundamentally avoiding the bottlenecks of traditional physical models being non-differentiable and unable to be trained end-to-end.

[0019] (2) In the DGK standard color chart scenario, the restored image of the present invention achieves a PSNR of 27.552 dB, which is 13.992 dB higher than the degraded input; the SSIM reaches 0.8912, which is 0.1548 higher; in the Armadillo scenario, the PSNR reaches 28.347 dB (13.097 dB higher) and the SSIM reaches 0.8973 (0.1715 higher). In the DGK and Armadillo test scenarios used in this invention, this invention was compared with seven state-of-the-art (SOTA) methods, including FUnIE-GAN, Shallow-UWnet, HUWIE-Net, SMDR-IS, U-shape Transformer, DPF-Net, and WF-Diff, on the same test set and with the same evaluation criteria. This invention achieved a stable positive improvement in both PSNR and SSIM relative to the degenerate input. Some purely data-driven methods exhibited a "negative optimization" phenomenon in the test scenarios, with a decrease in SSIM relative to the degenerate input. This invention effectively avoided the above problem by relying on physical constraints.

[0020] (3) This invention has the capability of white-box physical turbidity measurement. In the DGK color chart scenario, the network predicted turbidity to be 5.542 NTU, while the actual measured value by the turbidity meter was 5.640 NTU, with an error of only 0.098 NTU. This realizes a paradigm shift from black-box image translation to white-box physical turbidity measurement. This is a capability that other comparison methods do not possess structurally.

[0021] (4) The ablation experiment further verified the necessity of the various physical constraints of the present invention: compared with no L The recon variant of this invention improves PSNR by 0.722 dB and predicts scattering coefficients. The mean absolute error (Beta MAE) was reduced to nearly one-third of the former; compared to no Lβ The variant showed a 1.992 dB improvement in PSNR and a reduction in Beta MAE to the order of 1 / 8. Compared to the single-illumination input variant, the PSNR improved by 3.432 dB and the SSIM improved by 0.1482, demonstrating that the 5-level multi-illumination input played a key role in eliminating multi-solution blur.

[0022] (5) The training and inference process constructed in this invention can be trained and deployed end-to-end on a single NVIDIA RTX 3090 (24GB) GPU. It has strong engineering feasibility and can be applied to underwater visual perception, underwater robot operation, seabed topography mapping, underwater archaeology, aquaculture and water quality environment monitoring, etc., with broad application prospects. Attached Figure Description

[0023] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] Example 1 In this embodiment, as Figure 1 As shown, an underwater image restoration method based on physically differentiable scattering rendering includes the following steps: S1. Calibrate the camera and obtain the camera intrinsic parameter matrix. K An underwater imaging acquisition system was built, which includes artificial point light sources, cameras, liquid turbidity meters, and underwater illuminometers.

[0028] During camera calibration, Zhang's calibration method is used to perform stereo correction on the binocular camera to obtain the camera intrinsic parameter matrix K, and the focal length component in the camera intrinsic parameter matrix K. , With optical center coordinates cx , cy These are fixed values ​​that have already been calibrated.

[0029] In this embodiment, the camera calibration uses Zhang's Method. A 7×10 checkerboard calibration board, with each checkerboard square having a side length of 25mm, is used in a dry environment. The board is placed in 12 different poses within the camera's field of view (including frontal view, 30° left and right, 20° up and down, and three distances: near and far) to acquire original binocular calibration images. Sub-pixel coordinates of corner points are extracted from each calibration image to construct the camera projection matrix. The intrinsic parameter matrix K and distortion coefficients of the binocular camera are solved by minimizing the reprojection error. In this embodiment, the camera resolution is 2560×1440, and the calibration result is... , cx =1136.872, cy =719.744. This step also performs stereo correction based on the binaural calibration results, aligning the pixel rows of the left and right cameras to facilitate depth prior estimation.

[0030] The experimental platform for the light-shielding water tank was constructed as follows: The water tank uses a 1.2m × 0.8m × 0.8m transparent acrylic main structure, with a black matte light-absorbing film on the outside and a light-shielding cover installed. The inner wall of the tank is coated with a matte black light-absorbing coating to minimize ambient light and multiple scattering interference. The following equipment is integrated into the tank: a liquid turbidity meter (used to read the actual physical turbidity). T The system includes: NTU true value (range 0.1–100 NTU, resolution 0.01 NTU), an RS485 underwater illuminance meter (for reading illuminance E at distance d, range 0–200,000 lux), a 6500K daylight LED point light source (for artificial lighting, color temperature aligned with D65 white point), a binocular camera, and a Qt synchronous acquisition interface (for synchronously controlling camera exposure and synchronous triggering of the turbidimeter and illuminance meter, with a timestamp deviation of less than 1 millisecond). After this step, the system has the ability to stably output RGB degraded images and corresponding physical turbidimetric and illuminance true values.

[0031] S2. Invert the 3D coordinates of the artificial point light source in the camera coordinate system using specular reflective constraints from at least two specular reflectors of known radii. PL The distance from the light source to the camera is obtained. DSV .

[0032] Obtain three-dimensional coordinates PL The methods include: placing at least two known radii in the underwater scene. R Specular reflective objects are used as specular highlight detection targets. These objects include, but are not limited to, stainless steel mirror spheres, chrome-plated mirror spheres, nickel-plated mirror spheres, or cylindrical / ellipsoidal geometries with specular reflective properties. For each specular reflective object, the pixel coordinates of the highlight point and the radius of the sphere's outline are detected in the degraded image. r According to the pinhole imaging relationship Inversely solve the depth of the sphere's center in the camera coordinate system Zc And based on this, the three-dimensional coordinates of the sphere's center are obtained. C According to the direction of sight D From the camera's optical center to the highlight point, and with C For the center of the ball, R Solving the equations of the sphere with radius r, we can obtain the specular reflection point. S and the unit normal vector at that point N According to the law of specular reflection L = V −2( V · N )N Calculate the direction of incident light L ,in V For the reason S The unit view vector pointing to the camera's optical center; for each of the two specular spheres, a line is obtained with the point of specular reflection. Yes Starting point, direction - Li The incident light ray, constructing an overdetermined linear equation system and the least squares solution Find the midpoint of the common perpendicular segment of the two incident rays, and use it as the three-dimensional coordinate of the point light source in the camera coordinate system. PL The final estimated value.

[0033] In this embodiment, both surfaces are mirror-polished and have a radius of [missing information]. Standard stainless steel mirror spheres are placed at two different locations within the water tank scene, with a distance of approximately 0.30m between their centers. In this configuration, LED point light sources illuminating the surfaces of the two mirror spheres will create two clearly distinguishable highlight spots. This step is further subdivided into the following sub-steps: Spectrum detection and inverse kinematics of the sphere's center coordinates: The acquired degraded image is converted from RGB to grayscale, and the brightest pixel within the circular mask region of each steel sphere is taken as the specular point. ph The radius of the sphere profile circle was obtained using Hough circle detection. r Based on the geometry of pinhole imaging Inverse solution: depth of the ball's center Zc And combined with the intrinsic parameter matrix K Depend on , Obtain the three-dimensional coordinates of the sphere's center C In this embodiment, the three-dimensional coordinates of the centers of the two steel balls are approximately... C 1 = (−0.157, 0.045, 0.642) m and C 2 = (0.123, 0.066, 0.659) m .

[0034] Line of sight-spherical inversion to determine the direction of incident light: with the camera's optical center O =(0,0,0) starts along the line of sight. (Normalized) and the equation of the sphere Solving the equations simultaneously yields the specular reflection point. ( t (the smaller positive root of the quadratic equation) and the unit normal vector According to the law of specular reflection Calculate the direction of the incident light, where Each steel ball receives a strip of... Yes Starting point, direction - Li incident light Ri .

[0035] Least squares common perpendicular solution: parameterize the two rays as and Construct about ( t 1, t 2) Overdetermined linear equations ( A By − L 1. L 2. Construction of column vectors and their dot products. b Depend on S 2− S 1. Construction), according to Find the optimal solution, and then .

[0036] In this embodiment, the solution result is PL =(0.212, −0.025, 0.086) m After repeatedly and independently acquiring 50 frames, the standard deviation of the light source's three-dimensional coordinates was (1.8mm, 1.5mm, 2.1mm), verifying the robustness of the inversion. The distance from the light source to the camera was then determined by... In this embodiment, it is obtained that... D sv≈0.229m, which serves as the key geometric parameter for the subsequent differentiable analytical scattering rendering module.

[0037] Boundary conditions and failure degradation: When any mirror sphere is occluded by an obstacle, the highlight pixel is saturated (brightness=255), or the Hough circle detection confidence is lower than a preset threshold, this step determines that the single-frame inversion has failed and initiates one of the following degradation paths: (i) using the most recently successfully inverted image. PL (ii) Use the redundancy inversion of three or more mirror spheres and remove outliers with RANSAC; (iii) Degenerate to the ambient light uniform distribution approximation. β ·( D sp+ D vp) is simplified to unidirectional attenuation β · D (vp), at this point the invention degenerates into a unidirectional attenuation case and can still output a restored image, but the turbidity inversion accuracy decreases.

[0038] S3. Construct a differentiable, resolvable single scattering rendering unit that includes two-way attenuation from the light source to the surface to the camera, and perform analytical rendering of the underwater imaging process.

[0039] Differentiable single-scatter rendering units use a two-dimensional lookup table with a preset resolution. By combining bilinear interpolation to achieve a differentiable lookup table, the light source-surface incident attenuation term is calculated forward. Surface-camera emission attenuation term and single backscattering term The A two-dimensional lookup table is... Its coverage area is , The resolution is 1000×1000, and it is resident in the GPU memory. Differentiable lookup tables are used to... Bilinear interpolation is performed on a two-dimensional lookup table, allowing the gradient to be backpropagated to the restored image via a single-scattering analytical rendering module. With scattering coefficient The forward calculation formula for the single-scattering analytical rendering module is as follows: ,in, To restore the radiometric values ​​of the image in the linear RGB space, The distance from the light source to the object's surface. The distance from the camera to the object's surface. For a single backscattering term; when Below the preset threshold e At that time, the module calculates in the equivalent form of the small-angle Taylor expansion. The To avoid numerical singularities; among which, , I 0 represents the physical light intensity of a point light source. The water volume scattering coefficient, The scattering phase function; ; The angle parameter corresponding to the starting point of the effective scattering zone entered by the camera's line of sight. The distance from the camera to the starting point of the effective scattering zone along the line of sight; The corresponding camera's line of sight on the object's surface P The angle parameter at the termination position; For light source PL Camera optical center O Points on the surface of the object P In the triangle formed by the three elements, the camera's line of sight... D View and Light Source - Camera Baseline PL - O The angle between them.

[0040] In this embodiment, for any pixel in the scene p The corresponding three-dimensional coordinates of the object surface are: Xp (This can be given by scene depth priors or synthetic data ground truth). Define the following three key distances: This is the distance from the light source to the surface of the object; This is the distance from the camera to the object's surface; Let be the distance from the light source to the camera. In the triangle formed by the three elements, the scattering angle is defined. It is the angle between the direction of the camera's line of sight and the light source-camera baseline within the plane containing the light source, camera, and object surface.

[0041] The physical process of underwater imaging can be divided into two parts: (1) the direct signal term, which consists of the irradiance of the point source on the surface of the object (including...). The radiation is determined by inverse square attenuation, surface albedo, and two-way attenuation. For the sake of conciseness, this embodiment uses the equivalent radiation definition from computer graphics, which defines "the radiation value of a sharp image in linear RGB space". Explicitly defined That is, it absorbs the point light source. Inverse square attenuation factor and surface incident cosine factor ( i sp is the equivalent linear surface radiation after the angle between the surface normal and the incident direction of the light source; under this definition, the direct signal term is written as , respectively corresponding to incident attenuation (From light source to object surface) and emission attenuation The remaining direct component (from the object surface to the camera). The restored image predicted by the main branch of the network in this invention. That is, directly align according to the above equivalent radiation definition. Therefore, the restored image output by the inference process visually retains the distance attenuation information inherent in point light source illumination, without the need for further processing. Explicitly divide again (2) Single backscattering term The , representing the contribution of a single scattering of incident light by a water element along the camera's line of sight to a pixel.

[0042] To obtain The The analytical form of the scattering contribution of the water element integrated along the camera's line of sight is: ,in, s For the line-of-sight arc length parameter, The water volume scattering coefficient; β The volume extinction coefficient and the two satisfy the relationship ,in The water body single-scattering albedo (a known or pre-calibrated scalar constant for a given water body type, as described in this embodiment) oh Given prior information from Petzold seawater scattering data, and absorbed into A The constant factor of 0 is learned as a whole by the network's auxiliary branch, thus achieving... and β Explicit decoupling in physical quantities, rather than simple equivalence; The irradiance of an isotropic point light source at the scattering point is... The scattering phase function (in this embodiment, the Henyey-Greenstein phase function is used, and the anisotropy parameter is...) g =0.924), The distance from the water element to the light source. The distance from the water element to the camera.

[0043] To transform the above path integral into a sum This invention introduces angle variables in a lookup table format related to scalar parameters. Perform variable substitution: Let satisfy That is, line-of-sight parameters s The angle is mapped to a reference angle with the light source-camera baseline after a geometric transformation. Under this substitution, d in( s )and d out( s ) and β Weighted satisfaction ,and The quadratic factors in the path geometry are effectively absorbed.

[0044] After the above variable substitution and extraction, the integral variables are... After irrelevant constants, The Simplified to: ,in: (i) Constant factor Focus on depicting the physical light intensity of the light source I 0. Volume scattering coefficient Scattering phase function With the camera-light source baseline length in scattering geometry D sv and scattering angle sine The coupling effect; (ii) Look up the first parameter in the table Focus on characterizing scattering intensity ( β ) and scattering geometry ( The product of ) (iii) Lower limit of integration This corresponds to the starting angle of the line-of-sight segment entering the effective scattering zone. Since the starting point of the camera's line-of-sight is close to the camera's optical center in this embodiment, much smaller D sv, therefore In engineering implementation, it can be directly taken ; (iv) Maximum points The corresponding line of sight is on the surface of the object. P The termination angle at which, D vp is the distance from the camera to the surface of the object; (v) Scattering angle For light source PL Camera optical center O Points on the surface of the object P In the triangle formed by the three elements, the camera's line of sight... D View and Light Source - Camera Baseline PL - O The included angle between them can be determined by It can be obtained directly.

[0045] thus, The The evaluation of is strictly reduced to evaluating a two-dimensional lookup table. The two queries are subtracted to obtain all geometric parameters. Both can be determined by scene depth and light source position. PL The camera intrinsic parameter K is directly derived analytically, without relying on any online numerical integration. Those skilled in the art can derive it from the above variable definitions. The The original path integral (and the single scattering component of the Jaffe-McGlamery classical underwater imaging equation) are independently derived to obtain a lookup table form. Furthermore, the derivation process does not rely on any proprietary priors.

[0046] Special Integral Functions There is no closed-form solution, but its value depends only on two scalar parameters. u and v To make this module differentiable and efficient, this embodiment pre-computes the function as a two-dimensional lookup table, covering a region of... , The resolution is 1000×1000, stored as a PyTorch float32 tensor and resident in GPU memory (occupying approximately 4MB). The pre-computation process is as follows: uniformly sample 1000×1000 samples within the coverage area. Discrete points, for each The numerical integration of the composite Simpson's formula over 1024 sub-intervals was obtained. Truth value; stored as a two-dimensional tensor LUT.

[0047] During forward rendering, given the corresponding geometric parameters, the query parameters can be analytically obtained, and then the corresponding lookup table is consulted for each pixel to retrieve the value. For any query point... Let it fall into an adjacent grid point. Within the defined mesh cells, the normalized offset is defined as: The result of the bilinear interpolation lookup table is: in, and They are respectively and Normalized offset between adjacent grid points. The bilinear interpolation operator is differentiable within each grid cell and piecewise differentiable overall, allowing the gradient to be backpropagated through the physical rendering layer to the restored image and scattering coefficients.

[0048] The final forward rendering calculation formula is: ,in To restore the equivalent radiometric value of the image in linear RGB space (as defined above, the point light source has been absorbed) The inverse square decay factor is output by the main branch of the network. Reverse The result was obtained after correction. All operations in this computation are performed using native PyTorch tensor operators, without any dependence on external non-differentiable numerical libraries.

[0049] In particular, for scattering geometry (i.e., the direction of the camera's line of sight) D View and Light Source - Camera Baseline PL - O Degeneration case where the lines are almost collinear. The form tends to diverge; this embodiment adopts the following boundary handling strategy to ensure the stability of the rendering values: (i) when (This embodiment takes) e When (=1×10-3), according to the small angle limit , A first-order Taylor expansion of the path integral yields the closed-form equation. Avoid (ii) Equivalently, will appear in the denominator; Treat as a whole Directly involved in table lookup and reverse propagation, ensuring that the denominator only participates in the calculation. D sv and no longer contain (iii) During network training, for all Gradient masks are applied synchronously to pixels to prevent singularities from contaminating the backpropagated gradient. The specific choice of this boundary processing strategy is made by those skilled in the art based on the required computing power and accuracy, and does not affect the overall scope of protection of the claims of this invention.

[0050] S4. Perform geometric and photometric joint alignment between the virtual camera and the real camera, and construct a digital twin synthetic dataset.

[0051] Methods for joint geometric and photometric alignment include fixing the focal length of the virtual camera in the virtual engine to [value missing]. F ue, and press The sensor width of the virtual camera is inversely determined to ensure that the horizontal field of view error between the virtual and real cameras does not exceed a preset threshold; based on The Lux-Lumens spherical divergence mapping represents the illuminance of a real underwater illuminometer at a distance d. E Converted to the total luminous flux of a point light source in a virtual engine F This aligns the radiometric measurements of the virtual camera with those of the real camera; among which, Let d be the spherical area with the light source as the center and d as the radius; construct a proportionally twinned DGK standard matte color chart. The material parameters of the DGK standard matte color chart are set as follows: metallicity of 0, roughness of 1.0, and specular reflection intensity of 0.5. By traversing and searching for exposure compensation coefficients, the maximum error ΔL between the virtual camera rendered image and the real camera captured image on all color blocks in the CIELAB space L channel is not greater than a preset threshold.

[0052] In this embodiment, the virtual focal length is fixed inversely to the sensor size: the virtual camera focal length in the fixed virtual engine is fixed. (Based on a focal length conversion relationship consistent with that of a real camera lens), according to Inversely decode the sensor width of the virtual camera, where This refers to the image width (in pixels) of the virtual camera. In this embodiment, , ,thus Similarly, sensor height Substituting into the formula for horizontal field of view... It can calculate virtual cameras. The actual camera HFOV is calculated directly from the intrinsic parameters to be 85.543°, with an error of ≤0.001°, which is much smaller than the 0.5° threshold required by general physical simulation.

[0053] Lux-Lumens spherical divergence mapping: Illuminance read by a real underwater illuminometer at a distance d for an isotropic point light source. E (Unit: lux = lm / m²) and the total luminous flux of point light sources in the virtual engine. F (Unit: lm) satisfies the spherical divergence relation (in (The area of ​​the sphere with the light source as the center and radius d). In this step, the reference distance d is the actual physical installation distance from the point light source inside the water tank to the illuminance meter (measured directly with a steel tape measure; in this embodiment, d ≈ 35.43 cm, a known quantity), and is not derived from a formula; for each range of actual illuminance meter readings... E According to the above formula, d and E The luminous flux of the corresponding level can be directly converted. F The true value is then loaded into the virtual engine's camera lighting parameters. In this embodiment, five levels of physical light intensity are sequentially adjusted.I 0∈{0.146, 0.446, 0.746, 1.046, 1.346} (based on training reference light intensity) I The relative lux ratio obtained by ref normalization is dimensionless; corresponding to the 5 measured lux values, respectively, according to... The corresponding 5 gears are calculated. F Truth value.

[0054] DGK color chart L Channel alignment: Construct a scaled-down twin of the DGK standard matte color chart (containing 18 standard color patches), measuring 20×12.8cm; in the virtual engine, set its surface material parameters to metallicity=0, roughness=1.0, and specular reflection intensity=0.5 (approximately corresponding to pure diffuse reflection). Inverse alignment was performed on both the physically captured DGK color chart image and the virtually rendered DGK color chart image. After correction, the image is converted to the CIELAB color space. The exposure compensation coefficient of the virtual engine camera is adjusted using a traversal search method to ensure that the maximum error ΔL of all 18 color patches in the L channel does not exceed a preset threshold of 5. In this embodiment, the maximum ΔL corresponding to the finally matched exposure compensation coefficient is approximately 3.7, which meets the requirement. After this step, the synthetic image rendered by the virtual camera is consistent with that of the real camera in terms of geometric field of view, radiometrics, and exposure, providing a physical guarantee for the subsequent construction of the synthetic dataset.

[0055] Based on the aligned virtual camera and virtual point light source, a digital twin composite dataset is rendered in a virtual engine. This embodiment uses Unreal Engine 5 (UE5) as the virtual engine; five levels of physical light intensity are used. I Scattering coefficients of 0 and 10 β ( ,according to Covering the true turbidity range of 4.45 to 10.73 NTU, the Cartesian product yields 5 × 10 = 50 ( I 0, β ) combination; for each ( I 0, β The degraded images of several typical underwater scenes (including DGK standard color chart, Armadillo complex surface sculpture, rocks, biological models, etc.) are paired with corresponding clear ground truth images to create sample images.

[0056] Each of these embodiments ( I 0, β 4000 paired samples were rendered under the combined method. The entire dataset contains 50 × 4000 = 200,000 paired samples of degraded and clear images, which are divided into training, validation, and test sets in a ratio of 75:15:10. All images have a resolution of 2560 × 1440 and are stored as lossless PNGs.

[0057] S5. Within the preset turbidity gradient range The virtual scattering coefficient is obtained based on color block matching and least squares regression fitting. β Compared with true physical turbidity T Linear mapping between NTUs And the effective working range of the linear mapping is limited to the calibrated range. Outside the interval, the output is truncated at the nearest endpoint or a second calibration is performed to expand the interval.

[0058] Methods for obtaining a linear mapping include: within a preset turbidity gradient range, simultaneously calibrating each level of the actual physical turbidity using a liquid turbidimeter. T NTU, and acquire no fewer than 30 sets of corresponding real color chart images; perform inverse processing on the acquired real color chart images and DGK standard matte color chart images respectively. Calibration was performed, and the linear spatial mean values ​​were calculated for the R, G, and B channels on the preset color patches of the DGK standard matte color chart; for each level of true turbidity... T NTU, with L With the goal of minimizing the distance, the optimal scattering coefficient is matched from the synthetic color chart image. ,get Sample set; for The sample set is subjected to least squares univariate linear regression to obtain a linear mapping. .

[0059] In this embodiment, the DGK standard color chart was placed in a water tank, approximately 35 cm from the camera; 31 levels of true physical turbidity were prepared by successively adding standard turbidity suspension, covering the range of 4.45–10.73 NTU, and simultaneously calibrated by a liquid turbidity meter. T NTU true values ​​were obtained and color chart degradation images were acquired, resulting in 31 sets. Simultaneously, 1200 overlay images were extracted from the synthetic dataset. A composite color chart image with a dense mesh. Inverse processing is performed on both the real and composite images. Correction (standard sRGB EOTF) to linear RGB space, within a preset rectangular area of ​​18 color blocks. R / G / B The channel mean is used to obtain the 18×3 color block mean vector. M ( I ).

[0060] L 1. Color block matching and least squares regression: for each level T NTU, from 1200 composite color chart images, found that... smallest βAs the best in this category 31 groups were obtained. Sample; perform least squares univariate linear regression on this sample set to obtain In this embodiment k =0.1199、 b =−0.5289, R²>0.99, that is Because the regression line is in When given β <0 (physically meaningless), in this embodiment, the effective working range of the mapping is explicitly limited to the calibrated turbidity gradient range. Within this interval β Strictly non-negative, and uniformly applied during actual deployment. The non-negative truncation is used to prevent negative values ​​caused by numerical noise. During the inference phase, the inverse mapping is implemented as follows: For scenarios involving extremely clear or extremely turbid water exceeding this range, this embodiment explicitly requires a secondary calibration of the target turbidity range following the same procedure to obtain a calibration suitable for that range. The coefficients are then used for deployment, thus avoiding forced extrapolation of uncalibrated intervals.

[0061] S6. Apply multiple physical light intensities to the same scene. I The degraded image at 0 and the corresponding normalized light intensity channel are concatenated along the channel dimension to form a multi-channel input tensor, which is then input into the physically constrained neural network.

[0062] The physically constrained neural network (PCN) consists of a main branch that jointly predicts the reconstructed image and an auxiliary branch that predicts the scattering coefficients. The PCN comprises a 4-level encoder-decoder structure, skip connections, and a bottleneck layer with 512 channels. The main branch is output from the decoder of the PCN and mapped to a 3-channel reconstructed image via sigmoid activation. The auxiliary branch predicts the normalized scattering coefficients from the bottleneck layer through adaptive average pooling and three fully connected layers with Dropout, and then obtains the scattering coefficients according to a preset linear inverse normalization relationship. Loss reconstruction using a deturbidification-turbidification physical self-supervised closed loop. L Recon trains a physically constrained neural network. The multi-channel input tensor is generated from the same scene at 5 levels of physical light intensity. I The degraded image at 0 is obtained by stitching together the five corresponding normalized intensity channels along the channel dimension, resulting in five levels of physical light intensity. I 0 corresponds to different underwater illuminance meter readings, and the total number of channels for the multi-channel input tensor is 20.

[0063] The total loss function of the physically constrained neural network is: Among them, the deturbidity-turbidity physical self-supervised closed-loop reconstruction loss L recon is obtained by subtracting the original degraded image from the restored image output by the main branch and the scattering coefficients output by the auxiliary branch after they are re-turbidified by a differentiable single scattering rendering unit. L char is for the restored image Charbonnier loss between the image and the corresponding sharp reference image; Lβ For scattering coefficient prediction The supervised loss; the total loss function also includes perceptual loss based on VGG16 pre-trained features and SSIM loss based on structural similarity; l recon、 l char and lb These are the weighting coefficients for the corresponding loss terms.

[0064] In this embodiment, the pre-training of the physical constraint neural network includes: 20-channel input construction: For the same virtual scene in the training set, render it at 5 levels of physical light intensity. Degraded image below to (Each image is a 3-channel RGB image), representing multiple exposure configurations of a scene under light intensities weaker or stronger than the reference light intensity; the corresponding 5 relative intensity channels are 5 images. A single-channel image filled with constants is generated. The 5×3=15 channel degraded image above is concatenated with the 5-channel relative intensity image along the channel dimension to obtain a high-dimensional input tensor of 5×3+5=20 channels. This "multi-illumination decoupling and coupling" input design enables the network to break the multi-solution ambiguity by utilizing the physical consistency constraints between degraded images under different light intensities.

[0065] Network Structure: The backbone of the physically constrained neural network is a 4-level encoder-decoder structure. The encoder subunit 2051 consists of four downsampling blocks, each containing two 3×3 convolutions + BN + ReLU and one 2×2 max pooling with a stride of 2, with channel numbers of 64, 128, 256, and 512 respectively; the bottleneck layer is a 512-channel double convolutional block. The decoder subunit 2052 consists of four upsampling blocks, each achieving 2× upsampling through transposed convolution (or bilinear upsampling + 1×1 convolution), and concatenated with the corresponding encoding layer features via skip connections. The main branch dual-output head subunit 2053 compresses the last upsampling block (outputting a 32-channel feature map) to 3 channels through a 1×1 convolution, then normalizes it to [0, 1]3 via Sigmoid activation before outputting the restored image. In mini-batch training scenarios (batch size ≤ 4), the BatchNorm in each of the above encoder-decoder blocks can be replaced with GroupNorm (groups=8 recommended) or LayerNorm to improve mini-batch stability. This variant does not change the rest of the network structure and loss function definition. The auxiliary branch dual-output head subunit 2054 consists of a bottleneck layer followed by adaptive average pooling (output size 1×1) + 3 fully connected layers (the number of neurons is 256, 64, and 1 respectively, each followed by Dropout=0.2 and ReLU, and the last layer uses Sigmoid to output normalized scattering coefficients). Then, according to the preset linear inverse normalization relationship... (in β min=0.005 β max=0.76, corresponding to through The final scattering coefficient prediction is obtained by reverse calculation. .

[0066] Loss function construction: Total loss function Among them: (i) Deturbidization-Turbidization Physical Self-Supervised Closed-Loop Reconstruction Loss ( K =5), output the main branch With auxiliary branch output After being turbidified by the S103 rendering module, it is compared with the original degraded image. L 1. Distance; because the single scattering rendering unit is differentiable throughout its entirety, L The recon gradient can penetrate the physical rendering layer and propagate back to... and (ii) Charbonnier loss ( e =1×10⁻³), for With clear reference image (iii) Pixel-level error; scattering coefficient supervision loss Furthermore, it can be combined with VGG16 to sense loss. ( (Taking four feature layers: ReLU12, ReLU22, ReLU33, and ReLU43) and SSIM loss. The loss weight in this embodiment is... l recon=1.0、 l char=1.0、 lb =10.0, with perceptual loss and SSIM loss weights of 0.1 and 0.5, respectively.

[0067] Optimizer and training settings: AdamW optimizer (initial learning rate 1×10⁻⁴, weight decay 1×10⁻³), batch size=4, ReduceLROnPlateau scheduler (patience=10, factor=0.5), mixed precision training for 200 epochs; PSNR / SSIM validated and optimal weights saved every 5 epochs.

[0068] S7. Perform Sim-to-Real cross-domain fine-tuning on the physical constraint neural network using real underwater images. Input the underwater image to be restored into the fine-tuned physical constraint neural network and output the restored image. With scattering coefficient The physical turbidity is obtained by inverse solving the linear mapping within the specified working range. ;when When the value falls outside the working range, the system will either truncate the data at the nearest endpoint or trigger a preset secondary calibration / failure alarm mechanism.

[0069] In this embodiment, Sim-to-Real cross-domain semi-supervised fine-tuning includes: The 31 sets of degraded images from real color charts and 200 additional degraded images from real complex scenes (including 5 levels of light intensity) were used as a fine-tuning set. This fine-tuning set was divided into two subsets: (i) Color card fine-tuning subset (31 groups): Each group of samples was simultaneously calibrated with real physical turbidity. T The true NTU value (read from the liquid turbidimeter in the same frame) can be obtained from the fitted linear mapping. We can obtain the weak true value of the scattering coefficient by reverse calculation. As an auxiliary branch Weak supervision; (ii) Fine-tuning subset of complex scenes (200 images, including typical underwater targets such as rocks, biological models, and artificial structures): Each sample was also read in the same frame by a liquid turbidimeter. T The true value of NTU is obtained by reverse calculation using the same linear mapping. This subset does not have a clear reference diagram. .

[0070] Because there are no pixel-level clear reference images during the actual fine-tuning stage. This step involves the following targeted trimming of the total loss function: L char (Charbonnier pixel loss) overall disabled (weight) l char←0), avoid in In the absence of a term, the Charbonnier loss degenerates into a term that is always zero with respect to its own output, leading to network instability. L perc (VGG16 perceptual loss) and L SSIM (structural similarity loss) is simultaneously and globally disabled (weights reset to zero), both using... As an anchor point, it cannot be in the absence of Unilateral calculation at each stage; L The recon (de-turbidification-turbidation physical self-supervised closed-loop reconstruction loss) is retained and used as the dominant supervisor. l (recon=1.0), because it only relies on the network's own output. , After being re-turbidified by differentiable resolving scattering rendering, the image is compared with the actual acquired degraded image. Between L 1 distance, and Irrelevant; Lβ (Weak supervision loss of scattering coefficient, Enable () lb =10.0), The source is as described above.

[0071] Therefore, the effective total loss function during the fine-tuning phase is: , and the pre-training phase (including L char, L perc、 L (ssim) forms a contrast, constituting a two-stage training paradigm of "synthetic data fully supervised pre-training → real data physical self-supervision + weak supervision fine-tuning".

[0072] To prevent Systematic biases arise due to the noise in liquid turbidimeter measurements (typical resolution 0.01 NTU). This embodiment addresses this bias in each training batch. T The NTU true value is augmented with random jitter data of ±0.05 NTU, making It has a certain degree of flexibility within the physical confidence interval, which further improves the robustness of the model to the error of the real turbidimeter reading.

[0073] Use differential learning rate during fine-tuning: U - The backbone learning rate is 3×10⁻⁵ (preserving the geometric and texture priors learned during pre-training to avoid being overwhelmed by small sample sizes of real data), and the auxiliary branch regression learning rate is 1×10⁻³ (allowing...). (The head quickly adapts to the real scattering statistical distribution); the CosineAnnealingLR scheduler was used for fine-tuning over 200 epochs. During fine-tuning, the PSNR was verified every 5 epochs on 6 retained real color chart samples. T The NTU error is calculated, and the weight with the best overall performance is saved as the final deployment weight.

[0074] To avoid "generating simultaneously with the same linear mapping" and inverse solution T To address the potential self-verification loop introduced by "NTU", this embodiment adopts the following independent verification strategy for the final evaluation of turbidity inversion accuracy: (i) The six sets of real color card samples retained are strictly excluded in the least squares regression stage and do not participate. The coefficient fitting also does not participate in the fine-tuning stage. Supervision was conducted, and the six samples were read on-site only by a liquid turbidimeter. T (ii) In the blind test phase, the model is directly derived from the NTU true value and used as a blind test after fine-tuning convergence; Inverse solution obtained through linear mapping T NTU is obtained by subtracting the true value from the turbidimeter value. T NTU error index; (iii) In addition, the inventors simultaneously collected independent real degradation maps and turbidity ground truth values ​​on the complex Armadillo scene as a cross-scene validation set to ensure that the turbidity inversion accuracy is not affected by the actual performance of the NTU error. The data loop between the inverse kinematics formula and the inverse kinematics formula is manually supported. The 0.098 NTU turbidity error reported in this embodiment is obtained on the aforementioned independent retention set.

[0075] In this embodiment, the inference and white-box turbidity inversion output includes: (1) Geometric parameter acquisition: Using a stereo-corrected binocular camera, images of the current scene are simultaneously acquired from both the left and right eyes. A dense disparity map is obtained using a classic stereo matching algorithm (such as SGBM or a pre-trained monocular-binocular fusion deep network). ( B (Based on the binocular baseline distance) Inverse solution of the 3D coordinates of the object surface corresponding to each pixel Xp ; and thus obtain , , With scattering angle In the absence of binocular hardware, any well-known monocular underwater depth estimation model (such as the self-supervised MonoDepth series) can be used to output depth priors, and geometric parameters can be constructed according to the same formula. This invention does not limit the depth acquisition method, but only requires that a dense depth consistent with the rendering layer dimension can be provided during inference.

[0076] (2) Multi-lighting input construction: During inference, the same multi-lighting acquisition method as the training phase is preferred, that is, the system acquires the same scene at 5 levels of physical light intensity. IFive degraded images are simultaneously acquired at 0°C and stitched together into a 20-channel input tensor to maximize the preservation of the "multi-illumination coupled prior" learned during training. When the light intensity cannot be switched at the deployment site, as a degradation scheme, it is permissible to fill the five degraded image channels and the five relative light intensity channels with the same light intensity; in this case, the invention can still output... and However, the accuracy of turbidity inversion will decrease with more lighting input. Those skilled in the art can weigh the options according to their engineering needs.

[0077] (3) Network forward pass and turbidity inversion: The 20-channel input tensor is fed into the fine-tuned physical constraint neural network, and the main branch outputs a 3-channel restored image. Predicted by the output scattering coefficient of the auxiliary branch Finally, based on the fitted linear mapping The physical turbidity was obtained by reverse engineering within the specified working range [4.45, 10.73] NTU. It also synchronously outputs the "out-of-bounds flag" for samples that fall at the endpoints of the working interval.

[0078] Thus, the restored image output in this embodiment is complete. and physical turbidity T NTU together constitute the end-to-end inference result of this invention. It should be specifically noted that the linear mapping fitted in this embodiment... Only within the specified turbidity gradient range The inner case is strictly true; the result obtained during reasoning is the inverse solution. T NTU uniformly follows The output is truncated, and the system simultaneously outputs a "boundary violation flag" for samples falling at the endpoints of the interval to prompt the engineering user to perform a secondary calibration. In the DGK color chart test scenario of this embodiment, the restored image... The PSNR was 27.552 dB and SSIM was 0.8912, representing improvements of 13.992 dB and 0.1548 respectively compared to the degraded input; the network predicted turbidity. The difference from the true value of 5.640 NTU by the turbidimeter is only 0.098 NTU; in the Armadillo test scenario, PSNR=28.347 dB, SSIM=0.8973.

[0079] Example 2 This embodiment introduces several variant schemes that can be specifically adjusted based on Embodiment 1. The core steps remain unchanged in the following variants, only the sub-modules / hyperparameters are replaced or adjusted.

[0080] (a) Two-dimensional lookup table resolution: 500×500, 1000×1000, and 2000×2000 resolutions can all be used, each forming a different trade-off between memory usage, lookup table accuracy, and training speed. Those skilled in the art can flexibly choose according to the computing power deployed. The preferred resolution in this invention is 1000×1000.

[0081] (b) Number of multiple illumination input levels: 3, 5, and 7 levels of physical light intensity can all be used as multiple illumination inputs in this invention, with corresponding tensor channel numbers of 12, 20, and 28, respectively. The more levels there are, the stronger the constraint on eliminating multiple solution ambiguity. This invention prefers a 5-level 20-channel scheme as a trade-off between accuracy and video memory.

[0082] (c) Replaceable backbone network: Physical constraint loss of the present invention L recon is a general constraint independent of the backbone structure. The main branch backbone can be replaced by a level 4 U-Net or any dense prediction backbone such as U-Net++, Restormer, or SwinIR. The auxiliary branches can also be replaced by a fully connected regression head or a PointNet sparse regression head or a Vision Transformer global regression head.

[0083] (d) Multi-wavelength expansion: Expand the RGB three channels into N spectral channels (e.g., N=8 or N=16), and correspondingly expand the two-dimensional lookup table. Expanded to a 1000×1000×N three-dimensional lookup table, scattering coefficient β Modeled as a spectral dependence function This variant can further improve restoration accuracy when paired with multispectral camera hardware.

[0084] (e) Online LUT Update: In scenarios where the water scattering coefficient distribution dynamically shifts, the LUT can be updated every N training epochs according to the current... Distributed dynamic adjustment of LUT coverage area This aligns the LUT sampling density with the scattering coefficient distribution.

[0085] (f) β -NTU mapping format: This invention uses a univariate linear mapping by default. It can also be replaced by other forms of mapping such as piecewise linear or quadratic polynomial to adapt to different water body types and turbidity ranges.

[0086] Example 3 In this embodiment, an underwater image restoration system based on physically based differential scattering rendering includes: a camera calibration and image acquisition module, a monocular point light source 3D positioning module, a two-way attenuation single scattering differential analytical rendering module, a joint alignment module, a linear mapping construction module, an image stitching module, and an image restoration module.

[0087] The camera calibration and image acquisition module is used to calibrate the camera and obtain the camera intrinsic parameter matrix. K An underwater imaging acquisition system was built, which includes artificial point light sources, cameras, liquid turbidity meters, and underwater illuminometers.

[0088] The monocular point light source 3D localization module uses the specular geometric reflection constraints of at least two specular reflectors with known radii to invert the 3D coordinates of the artificial point light source in the camera coordinate system. PL The distance from the light source to the camera is obtained. DSV .

[0089] The two-way attenuation single scattering differentiable analytical rendering module is used to construct a differentiable analytical single scattering rendering unit that includes two-way attenuation from the light source to the surface to the camera, and to perform analytical rendering of the underwater imaging process.

[0090] The joint alignment module is used to perform geometric and photometric joint alignment between virtual and real cameras and to construct a digital twin synthetic dataset.

[0091] The linear mapping construction module is in the preset turbidity gradient range The virtual scattering coefficient is obtained based on color block matching and least squares regression fitting. β Compared with true physical turbidity T Linear mapping between NTUs And the effective working range of the linear mapping is limited to the calibrated range. Outside the interval, the output is truncated at the nearest endpoint or a second calibration is performed to expand the interval.

[0092] The image stitching module stitches the same scene under multiple levels of physical light intensity. I The degraded image at 0 and the corresponding normalized light intensity channel are concatenated along the channel dimension to form a multi-channel input tensor, which is then input into the physically constrained neural network.

[0093] The image restoration module uses real underwater images to perform Sim-to-Real cross-domain fine-tuning on the physical constraint neural network. The underwater image to be restored is input into the fine-tuned physical constraint neural network, and the restored image is output. With scattering coefficient The physical turbidity is obtained by inverse solving the linear mapping within the specified working range. ;when When the value falls outside the working range, the system will either truncate the data at the nearest endpoint or trigger a preset secondary calibration / failure alarm mechanism.

[0094] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for underwater image restoration based on physically differentiable scattering rendering, characterized in that, Includes the following steps: The camera is calibrated to obtain the camera intrinsic parameter matrix. K And build an underwater imaging acquisition system that includes artificial point light sources, cameras, liquid turbidity meters and underwater illuminance meters; The three-dimensional coordinates of the artificial point light source in the camera coordinate system are inverted using specular reflective constraints from at least two specular reflectors of known radii. PL The distance from the light source to the camera is obtained. Dsv ; Construct a differentiable, resolvable single scattering rendering unit that includes two-way attenuation from the light source to the surface to the camera, and perform analytical rendering of the underwater imaging process; The virtual camera and the real camera are jointly aligned geometrically and photometrically, and a digital twin synthetic dataset is constructed. Within the preset turbidity gradient range The virtual scattering coefficient is obtained based on color block matching and least squares regression fitting. β Compared with true physical turbidity T Linear mapping between NTUs And the effective working range of the linear mapping is limited to the calibrated range. Outside the interval, the output is truncated at the nearest endpoint or the interval is recalibrated to expand the interval; The same scene under multiple levels of physical light intensity I The degraded image at 0 and the corresponding normalized light intensity channel are concatenated along the channel dimension to form a multi-channel input tensor, which is then input into the physically constrained neural network. Sim-to-Real cross-domain fine-tuning of the physical constraint neural network is performed using real underwater images. The underwater image to be restored is then input into the fine-tuned physical constraint neural network, which outputs the restored image. With scattering coefficient The linear mapping is inversely solved within the specified working range to obtain the physical turbidity. ;when When the value falls outside the working range, the system will either truncate the data at the nearest endpoint or trigger a preset secondary calibration / failure alarm mechanism.

2. The underwater image restoration method based on physically differentiable scattering rendering according to claim 1, characterized in that, In the camera calibration process, Zhang's calibration method is used to perform stereo correction on the binocular camera to obtain the camera intrinsic parameter matrix K, where the focal length component is located. , With optical center coordinates cx , cy These are fixed values ​​that have already been calibrated.

3. The underwater image restoration method based on physically differentiable scattering rendering according to claim 1, characterized in that, Obtain the three-dimensional coordinates PL The methods include: Place at least two known radii in the underwater scene. R The mirror reflector is used as the target for high-light detection. The mirror reflector includes, but is not limited to, stainless steel mirror spheres, chrome-plated mirror spheres, nickel-plated mirror spheres, or cylindrical / ellipsoidal geometric bodies with mirror reflection characteristics. For each of the specular reflectors, the pixel coordinates of the highlight point and the radius of the sphere's outline circle are detected in the degraded image. r According to the pinhole imaging relationship Inversely solve the depth of the sphere's center in the camera coordinate system Zc And based on this, the three-dimensional coordinates of the sphere's center are obtained. C ; According to the direction of the line of sight D From the camera's optical center to the highlight point, and with C For the center of the ball, R Solving the equations of the sphere with radius r, we can obtain the specular reflection point. S and the aforementioned high-light reflection point S unit normal vector N ; According to the law of specular reflection L = V −2( V · N ) N Calculate the direction of incident light L ,in V For the reason S The unit view vector pointing to the camera's optical center; For each of the two mirror spheres, a line is obtained with a highlight reflection point. Si Starting point, direction - Li The incident light ray, constructing an overdetermined linear equation system and the least squares solution Find the midpoint of the common perpendicular segment of the two incident rays, and use it as the three-dimensional coordinate of the point light source in the camera coordinate system. PL The final estimated value.

4. The underwater image restoration method based on physically differentiable scattering rendering according to claim 1, characterized in that, The differentiable single-scatter rendering unit uses a two-dimensional lookup table with a preset resolution. By combining bilinear interpolation to achieve a differentiable lookup table, the light source-surface incident attenuation term is calculated forward. Surface-camera emission attenuation term and single backscattering term La ; The two-dimensional lookup table is Its coverage area is , The resolution is 1000×1000, and it is resident in the GPU memory; The differentiable lookup table, through... Bilinear interpolation is performed on the two-dimensional lookup table to allow the gradient to be backpropagated to the restored image via the single-scattering analytical rendering module. With scattering coefficient ; The forward calculation formula of the single scattering analytical rendering module is: ,in, To restore the radiometric values ​​of the image in the linear RGB space, The distance from the light source to the object's surface. The distance from the camera to the object's surface. For a single backscattering term; when Below the preset threshold ε At that time, the module calculates in the equivalent form of a small-angle Taylor expansion. La To avoid numerical oddities; in, , I 0 represents the physical light intensity of a point light source. The water volume scattering coefficient, The scattering phase function; ； The angle parameter corresponding to the starting point of the effective scattering zone entered by the camera's line of sight. The distance from the camera to the starting point of the effective scattering zone along the line of sight; The corresponding camera's line of sight on the object's surface P The angle parameter at the termination position; For light source PL Camera optical center O Points on the surface of the object P In the triangle formed by the three elements, the camera's line of sight... D View and Light Source - Camera Baseline PL - O The angle between them.

5. The underwater image restoration method based on physically differentiable scattering rendering according to claim 1, characterized in that, The method for performing the joint geometric and photometric alignment includes: Fix the focal length of the virtual camera in the virtual engine to [value]. F ue, and press The sensor width of the virtual camera is solved in reverse to ensure that the horizontal field of view error between the virtual camera and the real camera does not exceed a preset threshold. based on The Lux-Lumens spherical divergence mapping represents the illuminance of a real underwater illuminometer at a distance d. E Converted to the total luminous flux of a point light source in a virtual engine Φ This aligns the radiometric measurements of the virtual camera with those of the real camera; among which, Let be the area of ​​the sphere with the light source as its center and d as its radius; A proportionally scaled-to-scale DGK standard matte color chart is constructed. The material parameters of the DGK standard matte color chart are set as follows: metallicity of 0, roughness of 1.0, and specular reflection intensity of 0.

5. By traversing and searching for exposure compensation coefficients, the maximum error ΔL between the virtual camera rendered image and the real camera captured image on all color blocks in the CIELAB space L channel is not greater than a preset threshold.

6. The underwater image restoration method based on physically differentiable scattering rendering according to claim 5, characterized in that, The methods for obtaining the linear mapping include: Within a preset turbidity gradient range, the liquid turbidity meter simultaneously calibrates each level of the actual physical turbidity. T NTU, and collect no less than 30 sets of corresponding real color chart images; The images of the acquired real color chart and the DGK standard matte color chart were respectively reverse-engineered. The calibration was performed, and the linear spatial mean values ​​were calculated for the R, G, and B channels on the preset color blocks of the DGK standard matte color chart. For each level of true turbidity T NTU, with L With the goal of minimizing the distance, the optimal scattering coefficient is matched from the synthetic color chart image. ,get Sample set; Regarding the The linear mapping is obtained by performing least-squares univariate linear regression on the sample set. .

7. The underwater image restoration method based on physically differentiable scattering rendering according to claim 1, characterized in that, The physical constraint neural network includes: a main branch that jointly predicts the restored image and an auxiliary branch that predicts the scattering coefficient; The physical constraint neural network includes a 4-level encoder-decoder structure, skip connections, and a bottleneck layer, wherein the bottleneck layer has 512 channels. The main branch is output from the decoding end of the physical constraint neural network and mapped to a 3-channel restored image via Sigmoid activation. The auxiliary branch is predicted by the normalized scattering coefficient output from the bottleneck layer through adaptive average pooling and three fully connected layers with Dropout, and the scattering coefficient is obtained according to a preset linear inverse normalization relationship. ; Reconstructing loss using a deturbid-turbidity physical self-supervised closed loop L recon trains the physical constraint neural network; The multi-channel input tensor is generated from the same scene at 5 levels of physical light intensity. I The degraded image at 0 is obtained by stitching together the five corresponding normalized light intensity channels along the channel dimension, wherein the five levels of physical light intensity are... I 0 corresponds to different underwater illuminance meter readings, and the total number of channels of the multi-channel input tensor is 20.

8. The underwater image restoration method based on physically differentiable scattering rendering according to claim 1, characterized in that, The total loss function of the physical constraint neural network is: Among them, the deturbidity-turbidity physical self-supervised closed-loop reconstruction loss L recon is obtained by subtracting the original degraded image from the restored image output by the main branch and the scattering coefficients output by the auxiliary branch after they are re-turbidified by a differentiable single scattering rendering unit. L char is for the restored image Charbonnier loss between the image and the corresponding sharp reference image; Lβ For scattering coefficient prediction The supervised loss; the total loss function also includes a perceptual loss based on VGG16 pre-trained features and an SSIM loss based on structural similarity; λ recon、 λ char and λβ These are the weighting coefficients for the corresponding loss terms.

9. An underwater image restoration system based on physically differentiable scattering rendering, wherein the system applies the method described in any one of claims 1-8, characterized in that, include: The system includes a camera calibration and image acquisition module, a monocular point light source 3D positioning module, a two-way attenuation single scattering differentiable resolution rendering module, a joint alignment module, a linear mapping construction module, an image stitching module, and an image restoration module. The camera calibration and image acquisition module is used to calibrate the camera and obtain the camera intrinsic parameter matrix. K And build an underwater imaging acquisition system that includes artificial point light sources, cameras, liquid turbidity meters and underwater illuminance meters; The monocular point light source 3D positioning module uses the specular geometric reflection constraints of at least two specular reflectors with known radii to invert the 3D coordinates of the artificial point light source in the camera coordinate system. PL The distance from the light source to the camera is obtained. Dsv ; The dual-path attenuation single scattering differentiable analytical rendering module is used to construct a differentiable analytical single scattering rendering unit containing dual-path attenuation of light source-surface-camera, and to perform analytical rendering of the underwater imaging process. The joint alignment module is used to perform geometric and photometric joint alignment between virtual and real cameras and to construct a digital twin synthetic dataset. The linear mapping construction module is in the preset turbidity gradient range The virtual scattering coefficient is obtained based on color block matching and least squares regression fitting. β Compared with true physical turbidity T Linear mapping between NTUs And the effective working range of the linear mapping is limited to the calibrated range. Outside the interval, the output is truncated at the nearest endpoint or the interval is recalibrated to expand the interval; The image stitching module will stitch the same scene under multiple levels of physical light intensity. I The degraded image at 0 and the corresponding normalized light intensity channel are concatenated along the channel dimension to form a multi-channel input tensor, which is then input into the physically constrained neural network. The image restoration module uses real underwater images to perform Sim-to-Real cross-domain fine-tuning on the physical constraint neural network, inputs the underwater image to be restored into the fine-tuned physical constraint neural network, and outputs the restored image. With scattering coefficient The linear mapping is inversely solved within the specified working range to obtain the physical turbidity. ;when When the value falls outside the working range, the system will either truncate the data at the nearest endpoint or trigger a preset secondary calibration / failure alarm mechanism.

Images