A method for 3D scene reconstruction of low-light blurred images based on Gaussian sputtering

Abstract

This invention discloses a method for 3D scene reconstruction of low-light blurred images based on Gaussian sputtering, comprising: S1. Establishing a progressive iterative enhancement framework, setting intermediate brightness anchor points between low-light observation and target brightness; S2. Generating multiple enhanced images based on the intermediate brightness anchor points and combining histogram equalization and gamma correction techniques; S3. Performing fast deblurring processing on the enhanced images; S4. Constructing a scene representation model based on 3D Gaussian sputtering, combining a noise perception module to explicitly estimate and suppress noise; S5. Using the reconstructed rendered image as a deblurring prior for the next brightness level enhanced image to perform deblurring processing, iteratively optimizing until the target brightness is reached; S6. Generating a high-quality new perspective image based on the finally reconstructed 3D scene. This invention significantly improves rendering speed while ensuring reconstruction quality, achieving real-time 3D reconstruction, and effectively solving the problem of noise amplification in low-light environments.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, specifically to a method for reconstructing 3D scenes from low-light blurred images based on Gaussian sputtering. Background Technology

[0002] In applications such as virtual reality, augmented reality, nighttime autonomous driving, and robot navigation, high-quality 3D scene reconstruction technology is playing an increasingly important role, especially in low-light environments, such as nighttime driving and indoor dimly lit environments. It can reconstruct clear 3D scenes from low-light images, which is of great significance for improving the system's adaptability and intelligence level.

[0003] However, images in low-light environments often have the following problems: (1) insufficient brightness leads to loss of detail information; (2) sensor noise is significantly amplified under low-light conditions; (3) motion blur caused by camera shake; (4) traditional methods cannot handle multiple degradation factors at the same time. In summary, most existing 3D reconstruction methods perform well in well-lit environments, but face huge challenges in low-light environments.

[0004] Traditional solutions to the above problems mainly include: (A) Image preprocessing-based methods: first enhance and deblur the low-light image, and then perform 3D reconstruction. However, this method ignores the interdependence between tasks and is prone to introducing cumulative errors; (B) Joint optimization-based methods (LuSh-NeRF). Although this method can handle multiple degradations at the same time, the computational overhead is huge, the training time is long, and it is difficult to achieve real-time applications.

[0005] In recent years, 3D Gaussian sputtering technology, as an emerging neural rendering method, has shown great potential in the field of 3D scene reconstruction. 3DGS uses a set of 3D Gaussian ellipsoids to represent the scene. Each Gaussian contains position, covariance matrix, opacity, and spherical harmonic encoded color information. Compared with the traditional Neural Radiation Field (NeRF) method, 3DGS has the following advantages: (a) fast rendering speed, supporting real-time rendering; (b) high memory efficiency, suitable for large-scale scenes; (c) differentiable rendering, which facilitates optimization training; and (d) supports dynamic scene modeling. However, existing 3DGS methods are mainly for scenes under normal lighting conditions, and still face challenges such as noise amplification and loss of detail in low-light blur environments.

[0006] To address these technical problems, this application proposes a method for reconstructing 3D scenes from low-light blurred images based on Gaussian sputtering. Summary of the Invention

[0007] The main objective of this invention is to provide a method for reconstructing 3D scenes of low-light blurred images based on Gaussian sputtering, specifically designed for low-light environments. By fully leveraging the advantages of 3DGS technology in rendering speed and memory efficiency, this method addresses the technical problems, including noise amplification and camera motion blur, that have been raised in the background art under low-light conditions.

[0008] The present invention solves the above-mentioned technical problems by adopting the following technical solutions:

[0009] A method for reconstructing 3D scenes from low-light blurred images based on Gaussian sputtering, comprising the following steps performed by a computer device:

[0010] S1. Establish a progressive iterative enhancement framework, setting a balance between low-light observation and target brightness. An intermediate brightness anchor point is used to generate intermediate enhancement parameters through logarithmic interpolation;

[0011] S2. Based on intermediate brightness anchor points, progressive brightness enhancement is performed on low-light images, and enhanced images with multiple intermediate brightness levels are generated by combining histogram equalization and gamma correction techniques.

[0012] S3. A 2D deblurring network is used to quickly deblur the initially enhanced image, so as to decouple the deblurring and denoising tasks.

[0013] S4. Construct a scene representation model based on 3D Gaussian sputtering. The model uses a 3D Gaussian ellipsoid to represent the scene geometry and appearance, and combines a noise perception module to explicitly estimate and suppress noise.

[0014] S5. Use the reconstructed rendered image as the deblurring prior for the next brightness level enhancement image, re-execute the deblurring operation in step S3 for the next stage enhancement image, iteratively optimize and execute step S4 and step S5 until the target brightness is achieved.

[0015] S6. Generate high-quality new perspective images based on the final reconstructed 3D scene.

[0016] Preferably, the process of setting the intermediate brightness anchor point in step S1 includes:

[0017] S11. Set initialization enhancement parameters ,in The initial parameters are processed for histogram equalization. Initial parameters for gamma correction;

[0018] S12. Define target parameters ,in For the target histogram equalization parameters, The target gamma correction parameters;

[0019] S13. Generated by logarithmic interpolation intermediate parameters ,in .

[0020] Preferably, the specific operation flow for enhancing image generation in step S2 includes: according to the preset parameter interpolation sequence in S1, in each iteration stage... The corresponding parameter combination is used in the middle. Histogram equalization and gamma correction are performed sequentially on the low-light image. The subsequent iterations apply different enhancement parameters to achieve a gradual increase in image brightness, ultimately generating... A collection of enhanced images at different brightness levels.

[0021] Preferably, the specific operation process of performing histogram equalization on the low-light image includes: applying histogram equalization to the low-light image to enhance image contrast; performing global histogram analysis on the input low-light image to calculate the statistical characteristics of pixel intensity distribution; and then applying a standard histogram equalization algorithm to remap pixel values ​​through a cumulative distribution function, thereby effectively improving the overall contrast of the image and enhancing the visibility of dark areas in the image.

[0022] Preferably, the specific operation process of performing the gamma correction processing on the histogram equalized image includes: applying gamma correction to the processed image to adjust the image brightness distribution, applying gamma correction transformation to the histogram equalized image to adjust the overall brightness distribution characteristics of the image, and adjusting the brightness curve of the image through nonlinear transformation to improve the visual effect of the image.

[0023] Preferably, the specific operation process of the rapid deblurring process in step S3 includes:

[0024] S31. Use a pre-trained deblurring NAFNet network to enhance the first image. Perform initial deblurring to generate a deblurred image. This image serves as the input for the first round of 3D reconstruction, establishing a starting point for subsequent progressive processing.

[0025] S32. For subsequent enhanced images It employs a guided deblurring process based on rendered images.

[0026] Preferably, the specific operation flow of the guided image deblurring process in step S32 includes:

[0027] S321. Obtain the rendered image generated during the previous 3D reconstruction process. The image has been deblurred and denoised to obtain a high-quality image, and then parameters are obtained through differential optimization search. ;

[0028] S322. Rendering the image Apply brightness enhancement processing to make its brightness level match the current enhanced image. To maintain consistency, the final guide image is generated. This is used to provide high-quality prior knowledge for the low-light enhancement image deblurring process in the current round;

[0029] S323. Guide image Compared with the current enhanced image The image is concatenated along the channel dimension and fed into an improved NAFNet network guided by a radiation field rendering image for deblurring. This network is optimized for radiation field rendering image guidance, fully utilizing sharp features in the guiding image to direct the deblurring of the current blurred image, ultimately outputting a high-quality deblurred image. .

[0030] Preferably, the specific construction process of the scene representation model based on 3D Gaussian sputtering in step S4 includes:

[0031] S41. Point cloud initialization based on Colmap algorithm in 3D scene space Each 3D Gaussian ellipsoid contains position, covariance matrix, opacity, and spherical harmonic encoded color.

[0032] S42. By setting specified camera parameters, differentiable rasterization technology is used to project each 3D Gaussian ellipsoid onto a 2D image plane, and the final pixel color is calculated using the alpha mixing formula;

[0033] S43. Construct a noise estimation module and generate a noise map. A noise estimation module is constructed using a multilayer perceptron network. This module takes 3D spatial location coordinates and viewpoint direction as input to construct a continuous 3D noise field. For each pixel, the corresponding noise value is calculated from the noise field through ray sampling and volume rendering techniques, and finally a noise map with the same size as the input image is generated.

[0034] At this point, the deblurred image is also used to construct a 3D Gaussian sputtering model, which is used to reconstruct the scene through multi-view consistency constraints and achieve a denoising effect.

[0035] S44. The clear image obtained by 3D Gaussian sputtering rasterization rendering in step S42 is added pixel by pixel to the noise map generated in step S43 to generate the final predicted image, which is then compared with the real GT image. The reconstruction loss function between the two is optimized to improve the model parameters. The 3D Gaussian parameters and noise estimation network parameters are optimized simultaneously through end-to-end training, so that the reconstructed 3D Gaussian sputtering model can accurately separate scene content and noise components, thereby improving the reconstruction quality and robustness under low illumination conditions.

[0036] At this point, step S5 can use the image rendered by S4 as a priori to guide the deblurring process of the image in the next brightness level enhancement, and output a new deblurring result (brightness enhancement) to feed back to S4 to continue iterating until the last iteration is reached.

[0037] Preferably, the specific operation flow of the iterative optimization in step S5 includes:

[0038] S51. Search for optimal enhancement parameters in the rendered image and the next-stage low-light enhancement image using differential optimization. The parameter search range is: , , For histogram equalization parameters, Gamma correction parameters;

[0039] S52. Apply the optimal enhancement parameters obtained from the search to the current rendered image, and gradually enhance the image to a higher brightness through histogram equalization and gamma correction operations, which will serve as the prior input for the low-light enhancement image deblurring in the next stage.

[0040] S53: Repeat the above process until all are completed. Processing each brightness level.

[0041] Preferably, the new perspective image generation process in step S6 includes: based on the high-quality 3D Gaussian sputtering scene representation obtained after the last iteration of step S4, setting an arbitrary new viewing angle and camera position, using differentiable rasterization technology and alpha mixing technology, and finally outputting a high-quality new perspective image sequence.

[0042] As can be seen from the above technical solution, the present invention provides a method for 3D scene reconstruction of low-light blurred images based on Gaussian sputtering. Compared with the prior art, the present invention has the following advantages:

[0043] 1. This invention effectively solves the noise amplification problem in low-light environments by constructing a progressive iterative enhancement framework. By setting multiple intermediate brightness anchor points, it avoids excessive noise amplification caused by direct enhancement and ensures reconstruction quality.

[0044] 2. This invention decouples the deblurring and denoising tasks by using a blurred NAFNet network and a rendering-guided deblurring method. It rapidly processes images through a 2D deblurring network and explicitly estimates and suppresses noise using noise-aware 3DGS technology, thereby improving processing efficiency.

[0045] 3. By using 3D Gaussian sputtering rasterization rendering, this invention can greatly improve rendering speed while ensuring reconstruction quality, realizing real-time 3D reconstruction and providing an efficient and reliable solution for applications such as nighttime autonomous driving and robot navigation.

[0046] It should be understood that the descriptions in this section are not intended to identify key or essential features of embodiments of the invention, nor are they intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Of course, implementing any product of the invention does not necessarily require achieving all of the advantages described above simultaneously. Attached Figure Description

[0047] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0048] Figure 1 This is a flowchart illustrating the overall operation of the method of the present invention;

[0049] Figure 2 This is a schematic diagram of the process system architecture of the method of the present invention;

[0050] Figure 3 This is a schematic diagram of the noise estimator network structure of the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] For details in the embodiments, please refer to Figures 1 to 3 .

[0053] The method for 3D scene reconstruction of low-light blurred images based on Gaussian sputtering proposed in this invention embodiment, such as... Figure 2As shown, the overall architecture of this invention includes three main stages: a preprocessing stage, a progressive iterative enhancement stage, and a rendering stage. In the preprocessing stage, the low-light image is progressively enhanced through varying histogram equalization and gamma parameters to achieve progressive brightness adjustment and initial deblurring. In the progressive iterative enhancement stage, the framework progressively performs low-light enhancement, deblurring, and noise reduction. Finally, the framework completes training and renders a new perspective image using clear, normal-lit 3DGS.

[0054] In the specific implementation process, such as Figure 1 As shown, it includes the following steps:

[0055] S1: Establish a progressive iterative enhancement framework, setting a balance between low-light observation and target brightness. An intermediate brightness anchor point is used to generate intermediate enhancement parameters through logarithmic interpolation.

[0056] Specifically, S1 includes the following sub-steps:

[0057] S11: Initialize enhancement parameters ,in The initial parameters are processed for histogram equalization. Initial parameters for gamma correction;

[0058] S12: Define target parameters ,in For the target histogram equalization parameters, The target gamma correction parameters;

[0059] S13: Generated by logarithmic interpolation intermediate parameters ,in The logarithmic interpolation formula is shown below:

[0060]

[0061]

[0062] in, For the current stage index, This represents the total number of stages.

[0063] S2: Perform progressive brightness enhancement on low-light images, using histogram equalization and gamma correction techniques to generate enhanced images with multiple intermediate brightness levels.

[0064] In a specific implementation of the present invention, S2 includes the following sub-steps:

[0065] S21: For low-light images Histogram equalization is applied to enhance image contrast, as shown in the following formula:

[0066]

[0067] in, Indicates the use of parameters Histogram equalization operation, The enhanced pixel value, These are the pixel values ​​of the original low-light image;

[0068] S22: Enhanced image Gamma correction is applied to adjust the image brightness distribution. The brightness curve of the image is adjusted through nonlinear transformation to enhance the visual effect, as shown in the following formula:

[0069]

[0070] in, For the first The pixel values ​​of the enhanced image at each brightness level. To correct the nonlinear parameters of the gamma correction, For the first Sub-gamma transform operation;

[0071] S23: Repeat steps S21-S22, using the parameter interpolation sequence obtained in S13 to enhance the low-light image and generate... A collection of enhanced images with gradually increasing brightness. .

[0072] S3: Utilize a 2D deblurring network to perform fast deblurring on enhanced images, thereby decoupling the deblurring and denoising tasks.

[0073] Furthermore, in a specific embodiment, S3 includes the following sub-steps:

[0074] S31: As shown in the following formula, use the pre-trained deblurring NAFNet network to enhance the first image. Perform initial deblurring to generate a deblurred image. This image serves as the input for the first round of 3D reconstruction, establishing a starting point for subsequent progressive processing.

[0075]

[0076] in Indicates the NAFNet network. This indicates the use of the NAFNet network for image enhancement. Perform deblurring;

[0077] S32: For subsequent image enhancement Guided deblurring is employed, specifically including:

[0078] S321: Obtain the rendered image generated during the previous 3D reconstruction process. The image has been deblurred and denoised to obtain a high-quality image, and then the parameters are obtained through S5 difference optimization search. For the rendered image Apply brightness enhancement processing to make its brightness level match the current enhanced image. To maintain consistency, generate a guide image. This provides high-quality prior knowledge for the current round of low-light enhancement image deblurring processing;

[0079] S322: As shown in the following formula:

[0080]

[0081] in The improved NAFNet network is used to render images of radiation fields; ";" indicates a stitching operation. Indicates that the image is rendered using a radiation field. Guided improvement of NAFNet network for image enhancement Perform deblurring;

[0082] Guide image Compared with the current enhanced image The image is concatenated along the channel dimension and fed into a pre-trained radiation field rendering image-guided improved NAFNet network for deblurring. This network is optimized for radiation field rendering image guidance, fully utilizing sharp features in the guide image to instruct the deblurring of the current blurred image, ultimately outputting a high-quality deblurred image. .

[0083] In summary, this application decouples the deblurring and denoising tasks by using a blurred NAFNet network and a rendering-guided deblurring method. It rapidly processes images through a 2D deblurring network and explicitly estimates and suppresses noise using noise-aware 3DGS technology, thereby improving processing efficiency.

[0084] S4: Construct a scene representation model based on 3D Gaussian sputtering, using a 3D Gaussian ellipsoid to represent the scene geometry and appearance, and combining a noise perception module to explicitly estimate and suppress noise.

[0085] Furthermore, in a specific embodiment, S4 includes the following sub-steps:

[0086] S41: Point cloud initialization based on Colmap algorithm in 3D scene space Each 3D Gaussian ellipsoid contains a position. Covariance matrix Opacity Colors encoded with spherical harmonics ;

[0087] S42: For given camera parameters, differentiable rasterization is used to project each 3D Gaussian ellipsoid onto a 2D image plane. Finally, the final pixel color is calculated using the alpha blending formula, which is shown below:

[0088]

[0089] in, To influence the Gaussian count of this pixel, For the first A Gaussian color, For the first An opacity of one Gaussian unit. This is the image after alpha blending;

[0090] S43: Based on camera pose The calculation of the noise map includes the following steps:

[0091] S431: Based on camera pose Generate rays corresponding to the pixels. ;

[0092] S432: Sample along the light ray to obtain a set of sampling points. Each sampling point contains 3D coordinates x and viewing direction. ;

[0093] S433: Input the sampling points into the noise estimator network, such as... Figure 3 As shown, an MLP network is used to construct a noise estimation module. The network structure includes: an input layer that receives 3D spatial location data. and 3D light direction The features are encoded into 60-dimensional and 24-dimensional features using a positional encoding function formula, respectively. Feature extraction is then performed through three 128-dimensional hidden layers, outputting 128 features and volume density. The 128 features are concatenated with the position-encoded orientation features, and then passed through a 64-dimensional hidden layer to finally output RGB noise color.

[0094] Position encoding The function formula is:

[0095]

[0096] S434: Finally, the noise map is obtained through volume rendering calculation. As shown below:

[0097]

[0098] in, Transmittance, For the first Volume density at each sampling point The sampling interval is... For the first The noise color at each sampling point The number of sampling points is set to 64;

[0099] S44: Add the clear image obtained by 3D Gaussian sputtering rendering in step S42 to the noise image generated in step S43 pixel by pixel to generate the final predicted image, as shown in the following formula:

[0100]

[0101] in, To predict the image, Render a color map for 3DGS; ⊕ indicates element-wise addition.

[0102] At this point, the final predicted image is also used to construct a 3D Gaussian sputtering model;

[0103] S45: Then calculate the predicted image. With real images The model reconstruction loss is optimized through end-to-end training, simultaneously improving the 3D Gaussian parameters and noise estimation network parameters. This allows the reconstructed 3D Gaussian sputtering model to accurately separate scene content and noise components, improving reconstruction quality and robustness under low-light conditions. The loss function... As shown in the following formula:

[0104]

[0105] in, For real images, For weight parameters, For L1 loss, This is the structural similarity loss.

[0106] In summary, in the specific embodiments, this application, by using 3D Gaussian sputtering rasterization rendering operation, can greatly improve the rendering speed while ensuring reconstruction quality, realize real-time 3D reconstruction, and provide an efficient and reliable solution for applications such as nighttime autonomous driving and robot navigation.

[0107] At this point, S4 uses the deblurred image (the first one comes from the initial deblurring in S3, and the subsequent ones come from the deblurring results guided by the rendering in S5, with brightness increasing step by step) to construct a 3D Gaussian sputtering (3DGS) model, and reconstructs the scene through multi-view consistency constraints, while achieving a denoising effect.

[0108] Furthermore, step S5 can use the image rendered by S4 as a priori to guide the deblurring process of the image in the next brightness level enhancement, and output a new deblurring result (brightness enhancement) to feed back to S4 for continued iteration until the last iteration is reached.

[0109] S5: Use the reconstructed rendered image as the deblurring prior for the next brightness level enhancement image, re-execute the deblurring operation in step S3 on the next stage enhancement image, iteratively optimize the execution of step S4 and step S5 until the target brightness is achieved.

[0110] Furthermore, the S5 step, in its specific implementation, includes the following sub-steps:

[0111] S51: Optimize the rendered image through differential processing. and the next stage of low-light enhancement images Search for optimal enhancement parameters The parameter search range is , , For histogram equalization parameters, Gamma correction parameters;

[0112] S52: Obtain the optimal enhancement parameters from the search. Applied to the current rendered image The image is gradually upscaled to a higher brightness level through histogram equalization and gamma correction operations. This serves as the prior input for the next stage of low-light enhancement image deblurring;

[0113] S53: Repeat the above process until all are completed. Processing each brightness level.

[0114] Here, 3DGS reconstruction is performed using the initial deblurring result of the first-stage low-light image enhancement. The iterative optimization steps of S4-S5 are adopted so that each iteration uses the clean result of the previous level of brightness (after 3DGS denoising) to guide the deblurring of the next level of brightness after brightness enhancement, thereby improving brightness and quality step by step.

[0115] S6: Generates high-quality new perspective images based on the final reconstructed 3D scene.

[0116] Specifically, based on the high-quality 3D Gaussian sputtering scene representation obtained after the last iteration of step S4, an arbitrary new viewing angle and camera position are set, and differentiable rasterization technology and alpha mixing technology are used to finally output a high-quality new perspective image sequence.

[0117] In summary, this application effectively solves the noise amplification problem in low-light environments through a progressive iterative enhancement framework. By setting multiple intermediate brightness anchor points, it avoids excessive noise amplification caused by direct enhancement, thus ensuring reconstruction quality.

[0118] In a further specific embodiment, the overall process of this method is as follows:

[0119] L1. Set brightness anchor point:

[0120] Input a low-light image (blurred + noisy), set N sets of brightness anchor points, such as 0.1, 0.3, 0.5, and the target brightness is 0.5.

[0121] L2. Multi-level brightness enhancement:

[0122] Histogram equalization and gamma correction were applied to the original low-light image to generate three sets of enhanced images with different brightness levels: Level 1 brightness 0.1 (blur + noise), Level 2 brightness 0.3 (blur + noise), and Level 3 brightness 0.5 (blur + noise).

[0123] L3. Initial Deblurring:

[0124] The first-level enhanced image (brightness 0.1) is deblurred to obtain an image set (deblurred + noise) with a brightness of 0.1.

[0125] L4. Construct a noise-aware 3D Gaussian sputtering model:

[0126] A 3D Gaussian sputtering (3DGS) model is constructed using the deblurred image (first from L3 initial deblurring, then from S5 rendering-guided deblurring results, with brightness progressively increased). The scene is reconstructed and rendered using multi-view consistency constraints, while simultaneously achieving noise reduction.

[0127] At this point in the step:

[0128] First iteration: 3DGS reconstruction and rendering of the L3 image set (brightness 0.1, deblurred, with noise) to obtain an image set with consistent viewpoint (brightness 0.1, deblurred, denoised).

[0129] Subsequent iterations: The deblurred image set from subsequent L5 feedback is reconstructed and rendered using 3DGS.

[0130] L5. Iterative optimization (brightness 0.1 → 0.3):

[0131] L5.1: Enhance the image set rendered by L4 to a brightness of 0.3 through histogram equalization and gamma correction as a prior for deblurring (brightness 0.3, deblurring, small amount of noise, consistent viewpoint).

[0132] L5.2: The prior image set is stitched together with the second-level enhanced image set (brightness 0.3, blurred, noisy), and processed by 3DGS-Guided NAFNet to output a deblurred image set with a brightness of 0.3.

[0133] L5.3: Feeds the deblurring results back to L4 for 3D reconstruction and denoising.

[0134] Repeat the L4-L5 process, gradually increasing the brightness (0.1→0.3→0.5) until the target brightness of 0.5 is reached.

[0135] L6. Final Rendering:

[0136] Output the rendering result of the last L4 iteration (brightness 0.5) to obtain the final image set (brightness 0.5, deblurred, denoised, consistent viewpoint).

[0137] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the low-light blurred image 3D scene reconstruction methods based on Gaussian sputtering in the above embodiments.

[0138] It is understood that the system provided in the embodiments of the present invention corresponds to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant content can be referred to the corresponding parts of the above method.

[0139] This application also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, communication interface, and memory communicate with each other via the communication bus.

[0140] Memory, used to store computer programs;

[0141] When the processor executes the program stored in the memory, it implements the above-mentioned method for reconstructing 3D scenes of low-light blurred images based on Gaussian sputtering.

[0142] The communication bus mentioned in the aforementioned electronic devices can be a peripheral component interconnection standard bus or an extended industry standard structure bus for low-light blurred image 3D scene reconstruction based on Gaussian sputtering, etc. This communication bus can be divided into address bus, data bus, control bus, etc.

[0143] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0144] The memory may include a random access memory-based method for reconstructing 3D scenes of low-light blurred images using Gaussian sputtering, or a non-volatile memory-based method for reconstructing 3D scenes of low-light blurred images using Gaussian sputtering, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0145] The aforementioned processor can be a general-purpose processor, including a central processing unit (CPU) based on Gaussian sputtering for 3D scene reconstruction of low-light blurred images, a network processor based on Gaussian sputtering for 3D scene reconstruction of low-light blurred images, etc.; it can also be a digital signal processor (DSP) based on Gaussian sputtering for 3D scene reconstruction of low-light blurred images, an application-specific integrated circuit (ASIC) based on Gaussian sputtering for 3D scene reconstruction of low-light blurred images, a field-programmable gate array (FPGA) based on Gaussian sputtering for 3D scene reconstruction of low-light blurred images, or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0146] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, an optical medium, or a semiconductor medium, etc.

[0147] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0148] Furthermore, it should be noted that if any directional indication (such as up, down, left, right, front, back, etc.) is involved in the embodiments of the present invention, the directional indication is only used to explain the relative positional relationship and movement of each component in a specific posture. If the specific posture changes, the directional indication will also change accordingly.

[0149] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the meaning of "and / or" throughout the text includes three parallel solutions; for example, "A and / or B" includes solution A, solution B, or a solution where both A and B are satisfied simultaneously. Furthermore, in the embodiments of this invention, "multiple" refers to two or more. Moreover, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

Claims

1. A method for 3D scene reconstruction from low-light blurred images based on Gaussian sputtering, characterized in that, Comprise: S1. Establish a progressive iterative enhancement framework, set an intermediate brightness anchor point between low light observation and target brightness ; S2. Based on the intermediate brightness anchor point, the low-light image is progressively enhanced in brightness, and a plurality of intermediate brightness level enhanced images are generated by combining histogram equalization and gamma correction techniques; S3. Utilize the 2D deblurring network to perform fast deblurring processing on the enhanced image; S4. Construct a scene representation model based on 3D Gaussian sputtering, combine noise perception module to estimate and suppress noise; S5. The reconstructed rendering image is used as the deblurring prior of the next brightness level enhanced image, and the deblurring processing operation in step S3 is performed on the next stage enhanced image, and the iteration optimization of step S4 and step S5 is performed until the target brightness is reached; S6. Based on the final reconstructed 3D scene, generate high-quality new view images; The specific construction process of the scene representation model based on 3D Gaussian sputtering in step S4 comprises: S41. Initialize the point cloud based on the Colmap algorithm in the 3D scene space a 3D Gaussian ellipsoid, each Gaussian containing a position, a covariance matrix, an opacity, and a color encoded in spherical harmonics; S42. By setting the specified camera parameters, use the differentiable rasterization technology to project each 3D Gaussian ellipsoid to the 2D image plane, and calculate the final pixel color by the alpha blending formula; S43. Use a multi-layer perception network to construct a continuous 3D noise field with 3D spatial position coordinates and view direction as input, calculate the corresponding noise value from the noise field for each pixel by ray sampling and volume rendering technology, and finally generate a noise map with the same size as the input image; At this time, the deblurred image is also used to construct the 3D Gaussian sputtering model, which is used for scene reconstruction by multi-view consistency constraint; S44. Perform pixel-by-pixel addition operation on the clear image obtained by 3D Gaussian sputtering grating rendering in step S42 and the noise map generated in step S43 to generate a final prediction image, and then optimize the model parameters between the final prediction image and the real GT image reconstruction loss function, and optimize the 3D Gaussian parameters and the noise estimation network parameters simultaneously through end-to-end training.

2. The low-light foggy image 3D scene reconstruction method based on Gaussian sputtering of claim 1, wherein, The setting process of the intermediate brightness anchor point in step S1 comprises: S11. Set initialization enhancement parameters wherein is a histogram equalization processing initial parameter, is a gamma correction initial parameter; S12. Defining target parameters wherein is a target histogram equalization parameter, is a target gamma correction parameter; S13. Generate an intermediate parameter by logarithmic interpolation wherein wherein .

3. The low-light foggy image 3D scene reconstruction method based on Gaussian sputtering of claim 2, wherein, The specific operation process of enhancing image generation in the S2 step includes: according to the parameter interpolation sequence preset in S1, performing histogram equalization processing and gamma correction processing on the low-light image in each iteration stage with the corresponding parameter combination , gradually applying different enhancement parameters through times of iteration processing to realize progressive incremental improvement of image brightness, and finally generating a set of enhanced images with different brightness levels.

4. The low-light foggy image 3D scene reconstruction method based on Gaussian sputtering of claim 3, wherein, The specific operation process of the histogram equalization processing of the low-light image comprises: performing global histogram analysis on the input low-light image, calculating the pixel intensity distribution statistical characteristics, then applying the standard histogram equalization algorithm, remapping the pixel value through the cumulative distribution function, and enhancing the overall contrast of the image to enhance the visibility of the dark area in the image.

5. The low-light foggy image 3D scene reconstruction method based on Gaussian sputtering of claim 4, wherein, The specific operation process of the gamma correction processing of the histogram equalized image comprises: applying gamma correction transformation to the histogram equalized image, adjusting the overall brightness distribution characteristics of the image, and adjusting the brightness curve of the image through nonlinear transformation to improve the visual effect of the image.

6. The low-light foggy image 3D scene reconstruction method based on Gaussian sputtering of claim 2, wherein, The specific operation process of the fast deblurring processing in step S3 comprises: S31. Perform initial deblurring processing on the first enhanced image using a pre-trained deblurring NAFNet network to generate a deblurred image that serves as an input basis for the first round of 3D reconstruction . S32. For the subsequent enhanced image , , a render picture guided deblurring process is employed.

7. The low-light foggy image 3D scene reconstruction method based on Gaussian sputtering of claim 6, wherein, The specific operation process of the rendering picture guided deblurring processing in step S32 comprises: S321. Obtain the rendered image generated in the previous round of 3D reconstruction process and then obtain the parameters according to the differential optimization search ; S322. To the rendered image application of a brightness enhancement process, which brings its brightness level in line with the current enhanced image ; and ; S323. The guided image with the current enhanced image Concatenated in the channel dimension, input to the radiation field rendering image guided improved NAFNet network for deblurring processing, finally output high-quality deblurred image .

8. The low-light foggy image 3D scene reconstruction method based on Gaussian sputtering of claim 2, wherein, The specific operation process of the iteration optimization in step S5 comprises: S51. Search for optimal enhancement parameters in the rendered image and the next stage low-light enhancement image by differential optimization, the parameter search range is , , is a histogram equalization parameter, is a gamma correction parameter; S52. Apply the optimal enhancement parameter searched to the current rendering image, gradually enhance the image to a higher brightness image through histogram equalization and gamma correction operations, and input it as the deblurring prior of the next stage low-light enhanced image; S53: repeat the above process until the processing of all brightness levels is completed.

9. The low-light foggy image 3D scene reconstruction method based on Gaussian sputtering of claim 1, wherein, The new view image generation process in step S6 comprises: based on the high-quality 3D Gaussian sputtering scene representation obtained after the last iteration of step S4, setting an arbitrary new observation view and camera position, using differentiable rasterization technology and alpha blending technology, and finally outputting a high-quality new view image sequence.

Images