Low-light image enhancement method and device, storage medium, and electronic device

Abstract

The present invention provides a method and device for enhancing low-light images, a storage medium, and an electronic device, wherein the method comprises: obtaining a low-light image of a low-light scene, wherein the low-light image is an image with brightness below a threshold; obtaining a camera pose and a scene sparse point cloud of the low-light image; performing three-dimensional scene reconstruction on the low-light image using the camera pose and the scene sparse point cloud to obtain a Gaussian sphere distribution of the low-light image; constructing an absorptivity attribute of the Gaussian sphere distribution, wherein the absorptivity attribute is used to describe an object's ability to absorb radiant energy projected onto its surface; and performing illumination enhancement on the Gaussian sphere distribution based on the absorptivity attribute to obtain a scene reconstructed view of the low-light image. The embodiments of the present invention solve the technical problem of missing image details in low-light scenes in the related art, and can effectively preserve the scene details of low-light images.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for enhancing low-light images, a storage medium, and an electronic device. Background Art

[0002] Among related technologies, 3D scene reconstruction and novel view synthesis are long-standing and complex tasks in computer vision. Their goal is to recover the 3D structure of a scene from a set of 2D images. This technology plays a crucial role in a variety of applications, including virtual reality (VR) and autonomous driving. In recent years, implicit representation methods, particularly Neural Radiance Fields (NeRF) and its derivatives, have made significant progress in generating realistic images from arbitrary viewpoints. Despite the excellent performance of NeRF in creating realistic images, there is a growing demand for faster and more efficient rendering methods, especially for applications requiring real-time performance. 3DGS (3D Gaussian Splatting) is a recently proposed novel 3D scene representation and view synthesis method. 3DGS explicitly represents the scene using millions of 3D Gaussian functions, which are projected onto the image plane during rendering. Compared to implicit representation methods, 3DGS leverages the advantages of a differentiable rendering pipeline and point-based rendering techniques, providing an efficient computation and rendering process while avoiding the high computational cost of traditional NeRF methods.

[0003] While 3DGS technology has achieved significant progress in rendering efficiency and scene dynamics control, its performance remains limited in low-light conditions. In low-light conditions, such as at night, on cloudy days, or in underground spaces, light reflection from object surfaces is weak, resulting in images affected by backlighting, uneven lighting, low contrast, and dense noise. This makes it difficult to capture ideal effects and key information. Due to the low number of photons, camera noise is also relatively amplified, further burying scene details and distorting object colors. These characteristics of low-light images make it difficult for existing 3DGS technology to generate high-quality new perspective images.

[0004] In the field of view synthesis, enhancing low-light images and restoring their details are crucial to improving the quality of 3D scene reconstruction. Currently, two-dimensional low-light image enhancement technology is a direct and key step to improve the usability of new view synthesis. Although directly enhancing the brightness of the input image seems simple, existing methods have some limitations in maintaining lighting consistency between multiple views. Existing low-light image enhancement techniques usually focus on improving the brightness of a single image, but often ignore the importance of lighting consistency between different images. In addition, deep learning-based enhancement methods often learn specific brightness mappings from training data, which limits the generalization ability in unknown or wild scenes.

[0005] When processing images captured under low-light conditions, existing methods have difficulty in achieving high-fidelity color information capture and effective scene detail recovery, resulting in low quality of 3D scene reconstruction and new view synthesis.

[0006] For the above-mentioned problems existing in related technologies, no efficient and accurate solutions have been found yet. Summary of the Invention

[0007] The present invention provides a low-light image enhancement method and device, a storage medium, and an electronic device to solve technical problems in related technologies.

[0008] According to one embodiment of the present invention, a method for enhancing low-light images is provided, comprising: acquiring a low-light image of a low-light scene, wherein the low-light image is an image with brightness below a threshold; acquiring a camera pose and a scene sparse point cloud of the low-light image; performing three-dimensional scene reconstruction on the low-light image using the camera pose and the scene sparse point cloud to obtain a Gaussian sphere distribution of the low-light image; constructing an absorptivity attribute of the Gaussian sphere distribution, wherein the absorptivity attribute is used to describe an object's ability to absorb radiant energy projected onto its surface; and performing illumination enhancement on the Gaussian sphere distribution according to the absorptivity attribute to obtain a scene reconstructed view of the low-light image.

[0009] Optionally, obtaining the camera pose and scene sparse point cloud of the low-light image includes: performing image normalization processing on the low-light image to obtain a standard low-light image; performing camera pose estimation on the standard low-light image to obtain a camera pose, and performing motion recovery on objects in the standard low-light image to obtain a scene sparse point cloud.

[0010] Optionally, the camera pose and the scene sparse point cloud are used to perform three-dimensional scene reconstruction on the low-light image to obtain a Gaussian sphere distribution of the low-light image, including: initializing the scene sparse point cloud into a set of parameterized three-dimensional Gaussian basis elements representing the three-dimensional scene; using the camera pose to project the parameterized three-dimensional Gaussian basis elements to a two-dimensional imaging plane to obtain a two-dimensional Gaussian distribution; and performing three-dimensional scene reconstruction on the two-dimensional Gaussian distribution to obtain a Gaussian sphere distribution of the low-light image.

[0011] Optionally, performing a three-dimensional scene reconstruction on the two-dimensional Gaussian distribution to obtain a Gaussian sphere distribution of the low-light image includes: determining the depth information of the two-dimensional Gaussian distribution in three-dimensional space; and sorting the two-dimensional Gaussian distribution on the imaging plane according to the depth information to obtain the Gaussian sphere distribution of the low-light image.

[0012] Optionally, constructing the absorption rate attribute of the Gaussian sphere distribution includes: obtaining the Gaussian center coordinates of the Gaussian sphere distribution; converting the Gaussian center coordinates into a high-dimensional vector to obtain the spatial characteristics of the Gaussian sphere distribution; and using the Gaussian center coordinates and the Gaussian sphere distribution spatial characteristics to construct the absorption rate attribute of the Gaussian sphere distribution.

[0013] Optionally, using the Gaussian center coordinates and the Gaussian sphere distribution spatial features to construct the absorption rate attribute of the Gaussian sphere distribution includes: inputting the Gaussian center coordinates and the Gaussian sphere distribution spatial features into the first fully connected layer of the multi-layer perceptron MLP network, and outputting the camera pose feature vector and the Gaussian sphere distribution feature vector; splicing the camera pose feature vector and the Gaussian sphere distribution feature vector and inputting them into the second fully connected layer of the MLP network, and outputting the scene features; and using the activation function in the MLP network to activate the scene features to obtain the absorption rate attribute.

[0014] Optionally, the Gaussian sphere distribution is subjected to illumination enhancement according to the absorptivity attribute to obtain a scene reconstruction view of the low-light image, including: projecting the Gaussian sphere distribution onto a two-dimensional plane, and performing low-light mixed rendering according to the absorptivity attribute to obtain a low-light scene map; projecting the Gaussian sphere distribution onto a two-dimensional plane, and performing enhanced illumination mixed rendering on the Gaussian sphere distribution to obtain an enhanced illumination scene map; projecting the Gaussian sphere distribution onto a two-dimensional plane, and performing illumination intensity mixed rendering according to the absorptivity attribute to obtain a scene illumination intensity map; integrating the low-light scene map, the enhanced illumination scene map, and the scene illumination intensity map to obtain a scene reconstruction view of the low-light image.

[0015] Optionally, projecting the Gaussian sphere distribution onto a two-dimensional plane and performing low-light mixed rendering according to the absorption rate attribute to obtain a low-light scene graph includes: using the following mixed rendering formula to calculate the low-light scene graph : ;in, is the color of each pixel in the scene image under low light conditions, N is the number of elements in the set of Gaussian sphere distributions, i is the index representing the current accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorptivity property, It is the color of the three-dimensional Gaussian ball distribution i projected onto the two-dimensional imaging plane. It is a three-dimensional vector representing the values of the three RGB color channels. It is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene image under low light conditions.

[0016] Optionally, the Gaussian sphere distribution is projected onto a two-dimensional plane, and enhanced illumination mixed rendering is performed on the Gaussian sphere distribution to obtain an enhanced illumination scene graph, including: using the following mixed rendering formula to calculate the enhanced illumination scene graph : ;in, To enhance the color of each pixel in the lighting scene image, N is the number of elements in the set of Gaussian sphere distributions, i is the index, representing the current accumulated three-dimensional Gaussian sphere distribution, It is the color of the three-dimensional Gaussian ball distribution i projected onto the two-dimensional imaging plane. It is a three-dimensional vector representing the values of the three RGB color channels. It is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the enhanced lighting scene image.

[0017] Optionally, the Gaussian sphere distribution is projected onto a two-dimensional plane, and illumination intensity mixed rendering is performed according to the absorption rate attribute to obtain a scene illumination intensity map, including: using the following mixed rendering formula to calculate the scene illumination intensity map : ;in, is the color of each pixel in the scene light intensity image, N is the number of elements in the set of Gaussian sphere distribution, i is the index, representing the current accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorptivity property, is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene light intensity map.

[0018] According to another embodiment of the present invention, a low-light image enhancement device is provided, including: a first acquisition module, used to acquire a low-light image of a low-light scene, wherein the low-light image is an image with brightness lower than a threshold; a second acquisition module, used to acquire the camera pose and the scene sparse point cloud of the low-light image; a reconstruction module, used to perform three-dimensional scene reconstruction on the low-light image using the camera pose and the scene sparse point cloud to obtain a Gaussian sphere distribution of the low-light image; a construction module, used to construct an absorptivity attribute of the Gaussian sphere distribution, wherein the absorptivity attribute is used to describe the ability of an object to absorb radiant energy projected onto its surface; and an enhancement module, used to perform illumination enhancement on the Gaussian sphere distribution according to the absorptivity attribute to obtain a scene reconstructed view of the low-light image.

[0019] Optionally, the second acquisition module includes: a first processing unit, used to perform image normalization processing on the low-light image to obtain a standard low-light image; a second processing unit, used to perform camera pose estimation on the standard low-light image to obtain the camera pose, and perform motion recovery on objects in the standard low-light image to obtain a sparse point cloud of the scene.

[0020] Optionally, the reconstruction module includes: an initialization unit for initializing the scene sparse point cloud into a set of parameterized three-dimensional Gaussian primitives representing the three-dimensional scene; a projection unit for projecting the parameterized three-dimensional Gaussian primitives onto a two-dimensional imaging plane using the camera pose to obtain a two-dimensional Gaussian distribution; and a reconstruction unit for performing three-dimensional scene reconstruction on the two-dimensional Gaussian distribution to obtain a Gaussian sphere distribution of the low-light image.

[0021] Optionally, the reconstruction unit includes: a determination subunit, used to determine the depth information of the two-dimensional Gaussian distribution in three-dimensional space; and a sorting subunit, used to sort the two-dimensional Gaussian distribution on the imaging plane according to the depth information to obtain the Gaussian sphere distribution of the low-light image.

[0022] Optionally, the construction module includes: an acquisition unit for acquiring the Gaussian center coordinates of the Gaussian sphere distribution; a conversion unit for converting the Gaussian center coordinates into a high-dimensional vector to obtain the spatial characteristics of the Gaussian sphere distribution; and a construction unit for using the Gaussian center coordinates and the Gaussian sphere distribution spatial characteristics to construct the absorption rate properties of the Gaussian sphere distribution.

[0023] Optionally, the construction unit includes: a first processing unit, used to input the Gaussian center coordinates and the Gaussian sphere distribution spatial features into the first fully connected layer of the multi-layer perceptron MLP network, and output a camera pose feature vector and a Gaussian sphere distribution feature vector; a second processing unit, used to splice the camera pose feature vector and the Gaussian sphere distribution feature vector and input them into the second fully connected layer of the MLP network, and output scene features; an activation unit, used to use the activation function in the MLP network to activate the scene features to obtain the absorption rate attribute.

[0024] Optionally, the enhancement module includes: a first rendering unit, used to project the Gaussian sphere distribution onto a two-dimensional plane, and perform low-light mixed rendering according to the absorption rate attribute to obtain a low-light scene map; a second rendering unit, used to project the Gaussian sphere distribution onto a two-dimensional plane, and perform enhanced illumination mixed rendering on the Gaussian sphere distribution to obtain an enhanced illumination scene map; a third rendering unit, used to project the Gaussian sphere distribution onto a two-dimensional plane, and perform illumination intensity mixed rendering according to the absorption rate attribute to obtain a scene illumination intensity map; an integration unit, used to integrate the low-light scene map, the enhanced illumination scene map and the scene illumination intensity map to obtain a scene reconstruction view of the low-light image.

[0025] Optionally, the first rendering unit includes: a calculation subunit, configured to calculate a low-light scene image using the following hybrid rendering formula: : ;in, is the color of each pixel in the scene image under low light conditions, N is the number of elements in the set of Gaussian sphere distributions, i is the index representing the current accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorptivity property, It is the color of the three-dimensional Gaussian ball distribution i projected onto the two-dimensional imaging plane. It is a three-dimensional vector representing the values of the three RGB color channels. It is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene image under low light conditions.

[0026] Optionally, the second rendering unit includes: a calculation subunit, configured to calculate an enhanced lighting scene graph using the following hybrid rendering formula: : ;in, To enhance the color of each pixel in the lighting scene image, N is the number of elements in the set of Gaussian sphere distributions, i is the index, representing the current accumulated three-dimensional Gaussian sphere distribution, It is the color of the three-dimensional Gaussian ball distribution i projected onto the two-dimensional imaging plane. It is a three-dimensional vector representing the values of the three RGB color channels. It is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the enhanced lighting scene image.

[0027] Optionally, the third rendering unit includes: a calculation subunit, configured to calculate a scene illumination intensity map using the following hybrid rendering formula: : ;in, is the color of each pixel in the scene light intensity image, N is the number of elements in the set of Gaussian sphere distribution, i is the index, representing the current accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorptivity property, is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene light intensity map.

[0028] According to another aspect of an embodiment of the present application, a storage medium is further provided, which includes a stored program, and the above steps are executed when the program is run.

[0029] According to another aspect of an embodiment of the present application, an electronic device is also provided, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; wherein: the memory is used to store computer programs; the processor is used to execute the steps in the above method by running the program stored in the memory.

[0030] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the steps in the above method.

[0031] Beneficial effects of the present invention:

[0032] 1. Improve the accuracy of 3D reconstruction and illumination enhancement of low-light scene images, and be able to reconstruct high-quality scene reconstruction views under poor lighting conditions, effectively preserving scene details in low-light images;

[0033] 2. By obtaining the camera pose and sparse point cloud of the low-light image, the camera's position and orientation information at the time of shooting can be determined, and the view of the low-light image under different camera poses can be reconstructed. Combined with the sparse point cloud of the scene, the geometric information of the 3D scene reconstruction can be obtained, thereby enhancing the accuracy of 3D scene reconstruction in low light. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0035] Figure 1 This is a hardware structure block diagram of a car according to an embodiment of the present invention;

[0036] Figure 2 is a flow chart of a method for enhancing a low-light image according to an embodiment of the present invention;

[0037] Figure 3 is a schematic diagram of the principle of 3D reconstruction and illumination enhancement of low-light scene images in an embodiment of the present invention;

[0038] Figure 4 is a schematic diagram of an MLP network in an embodiment of the present invention;

[0039] Figure 5 1 is a schematic diagram comparing an image before and after being enhanced by illumination in an embodiment of the present invention;

[0040] Figure 6 4 is a structural block diagram of a low-light image enhancement device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of this application. It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0042] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0043] Example 1

[0044] The method embodiment provided in the first embodiment of the present application can be executed in a car, a server, a computer, a processor, an automatic driving / assisted driving / intelligent driving controller or a similar processing device. Taking running on a car as an example, Figure 1 This is a hardware structure diagram of a car according to an embodiment of the present invention. Figure 1 As shown, a car may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data. Optionally, the above-mentioned car may also include a transmission device 106 for communication functions and an input and output device 108. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned automobile. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0045] The memory 104 can be used to store vehicle programs, for example, software programs and modules of application software, such as the vehicle program corresponding to a method for enhancing low-light images of a vehicle in an embodiment of the present invention. The processor 102 executes the vehicle program stored in the memory 104 to execute various functional applications and data processing, thereby implementing the above-mentioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories may be connected to the vehicle via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0046] Transmission device 106 is used to receive or transmit data via a network. A specific example of such a network may include a wireless network provided by the vehicle's communications provider. In one embodiment, transmission device 106 includes a network interface controller (NIC), which can connect to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0047] In this embodiment, a method for enhancing low-light images is provided. Figure 2 FIG. 1 is a flow chart of a method for enhancing a low-light image according to an embodiment of the present invention. Figure 2As shown, the process includes the following steps:

[0048] Step S202, obtaining a low-light image of a low-light scene, wherein the low-light image is an image with brightness lower than a threshold;

[0049] Optionally, a low-light image refers to one or more images captured by an image sensor in a low-light scene due to the low-light environment. The low-light image contains image details hidden by the low light and color distortion caused by the low light.

[0050] Step S204, obtaining the camera pose and scene sparse point cloud of the low-light image;

[0051] The camera pose in this embodiment refers to the camera's position and orientation in three-dimensional space. A sparse scene point cloud refers to point cloud data that is unevenly distributed in three-dimensional space. Due to low light conditions, some areas have dense point clouds, while others have sparse point clouds, resulting in a sparse distribution of point cloud data.

[0052] Step S206, performing three-dimensional scene reconstruction on the low-light image using the camera pose and the scene sparse point cloud to obtain a Gaussian sphere distribution of the low-light image;

[0053] Step S208: constructing an absorptivity attribute of the Gaussian sphere distribution, wherein the absorptivity attribute is used to describe the ability of an object to absorb radiation energy projected onto its surface;

[0054] Step S210 : performing illumination enhancement on the Gaussian sphere distribution according to the absorptivity attribute to obtain a scene reconstruction view of the low-light image.

[0055] The scene reconstructed view in this embodiment is a two-dimensional image obtained by enhancing the illumination of the low-light image.

[0056] Through the above steps, a low-light image of a low-light scene is obtained, wherein the low-light image is an image with brightness lower than a threshold; the camera pose and the scene sparse point cloud of the low-light image are obtained; the camera pose and the scene sparse point cloud are used to perform three-dimensional scene reconstruction on the low-light image to obtain a Gaussian sphere distribution of the low-light image; an absorptivity attribute of the Gaussian sphere distribution is constructed, wherein the absorptivity attribute is used to describe the ability of an object to absorb radiant energy projected onto its surface; the Gaussian sphere distribution is illuminated according to the absorptivity attribute to obtain a scene reconstructed view of the low-light image, which solves the technical problem of missing image details in low-light scenes in related technologies and can effectively retain the scene details of the low-light image.

[0057] Optionally, obtaining the camera pose and scene sparse point cloud of the low-light image includes: performing image normalization processing on the low-light image to obtain a standard low-light image; performing camera pose estimation on the standard low-light image to obtain a camera pose, and performing motion recovery on objects in the standard low-light image to obtain a scene sparse point cloud.

[0058] The standard low-light image is subjected to pose estimation using a preset camera pose estimation software to obtain the camera pose, and the scene sparse point cloud corresponding to the standard low-light image is estimated using a structure-from-motion algorithm.

[0059] In embodiments of the present invention, camera pose estimation software refers to software that estimates the camera pose of an image using techniques such as feature detection and matching, incremental SfM (Structure from Motion), and multi-view stereo matching. For example, the preset camera pose estimation software may employ COLMAP, an open-source 3D reconstruction tool for recovering the structure of a 3D scene from a 2D image.

[0060] In this embodiment, the camera pose and the sparse point cloud of the scene are used to perform three-dimensional scene reconstruction on the low-light image. The process of obtaining the Gaussian sphere distribution and its absorptivity attribute of the low-light image can be implemented by a preset three-dimensional scene algorithm, and the three-dimensional scene algorithm can be encapsulated as a pre-trained model. The three-dimensional scene algorithm of this embodiment is an improved 3DGS (3D Gaussian Splatting) algorithm based on the 3DGS algorithm. The 3DGS algorithm is a technology for three-dimensional scene reconstruction and rendering. It uses a set of Gaussian ellipsoids to model the scene and uses rasterization technology to render these Gaussian ellipsoids into images, thereby achieving an efficient rendering tool. On the basis of the original 3DGS, a new attribute, namely the absorptivity attribute, is introduced for each Gaussian ellipsoid. The improved 3DGS algorithm includes a multilayer perceptron (MLP) network, and the MLP network is used to construct the absorptivity attribute. Figure 3 This is a schematic diagram of the principle of 3D reconstruction and illumination enhancement of low-light scene images in an embodiment of the present invention. The original low-light image is processed to obtain a sparse point cloud, which is then processed again to obtain the Gaussian sphere distribution feature vector (Gaussian ellipsoid mean, spherical harmonic function coefficients). Combined with the camera pose feature vector, the absorption rate attribute is obtained, and then hybrid rendering is used to obtain a scene reconstruction view with illumination enhancement.

[0061] Before using the preset 3D scene algorithm to reconstruct the 3D scene, the preset 3D scene algorithm is also trained and optimized. This includes: using a tone mapping loss function to adjust the image consistency of the 3D scene algorithm, using an improved structural similarity loss function to adjust the image contrast and structure consistency of the 3D scene algorithm, using an exposure control loss function to control the exposure level of the 3D scene algorithm, and using a scale loss function to control the size of the Gaussian sphere distribution of the scene.

[0062] In one example, the tone mapping loss function (Tone-mapped L1 Loss, L it-MAE ): This loss function introduces a nonlinear tone mapping function Φ(x) , the original L1 loss is modified to improve the consistency of image tones under different lighting conditions. The tone mapping function is a piecewise cubic polynomial that is used to adjust the weights of dark and bright areas, so that the network pays more attention to the restoration of details during training. The tone mapping function is defined as follows:

[0063]

[0064] in, x represents the pixel value in the image, a is the first adjustable coefficient, b is the second adjustable coefficient. In the embodiment of the present invention, .

[0065] The specific definition of the tone mapping loss function is given by the following formula:

[0066]

[0067] in, is the tone mapping loss, This is a low-light image rendered by the algorithm. A real low-light image.

[0068] In one example, the improved structural similarity loss function (csSSIM Loss, L csSSIM ): The traditional SSIM metric takes into account the brightness, contrast, and structural information of the image. In this invention, the SSIM metric is improved by removing the brightness comparison part and retaining only the contrast and structural information to enhance the network's ability to learn scene details and reduce the impact of image brightness. This loss function promotes the network to generate images with high detail consistency by comparing the similarity between the enhanced image and the real image in contrast and structure. The specific definition of the improved csSSIM metric is given by the following formula:

[0069]

[0070] in, x Represents the data in the first image window, y Represents the data in the second image window, and In turn, x and y The variance of express x and y The covariance between is a small constant used to maintain numerical stability. Further, the csSSIM loss function is defined as:

[0071]

[0072] Where W is the total number of image windows, i Indicates the current image window number, For the i The enhanced image rendered under the image window sequence number is For the i Real low-light images under image window numbers.

[0073] In one example, the exposure control loss function (Exposure Control Loss, L exp ): This loss function is used to control the exposure level of the image, preventing overexposure or underexposure, and ensuring the image's visual naturalness and consistency. By calculating the average brightness of a local area in the image and comparing it with a preset target brightness level E, this loss function forces the network to output an image that matches the brightness of the real scene. The specific definition of this loss function is given by the following formula:

[0074]

[0075] in, M Represents the number of non-overlapping local regions into which the image is divided, each region is 16×16 pixels in size, Y k Indicates the k The average intensity value of the local area in the enhanced image, E is the average brightness value. Optionally, the average brightness value is set to 0.6.

[0076] In one example, the scale loss function (Scale Loss, L scal): This loss function is used to control the size of the 3D Gaussian ellipsoid in the scene to improve the expressiveness and richness of details of the scene. By setting a desired average Gaussian size e , and calculates the difference between the size of all Gaussian ellipsoids in the scene and the expected value. This loss function makes the size distribution of the Gaussian ellipsoids generated by the network more reasonable. The specific definition of this loss function is given by the following formula:

[0077]

[0078] in, P represents the total number of Gaussian ellipsoids, S k Indicates the k The average size of a Gaussian ellipsoid, where the average size is the average of the lengths of the three axes of the Gaussian ellipsoid, e is an artificially defined average size, which is a hyperparameter used to control the level of Gaussian sphere size. e is set to 0.1.

[0079] Total Loss, L ) is the weighted sum of the above four loss functions, expressed as:

[0080]

[0081] Among them, λ1, λ2 and λ3 are non-negative parameters used to balance the weights of various loss functions. Optional, λ 1 =0.5,λ 2 =0.1 and λ 3 =0.075 .

[0082] In one implementation of this embodiment, the camera pose and the scene sparse point cloud are used to perform three-dimensional scene reconstruction on the low-light image to obtain a Gaussian sphere distribution of the low-light image, including: initializing the scene sparse point cloud into a set of parameterized three-dimensional Gaussian basis elements representing the three-dimensional scene; using the camera pose to project the parameterized three-dimensional Gaussian basis elements onto a two-dimensional imaging plane to obtain a two-dimensional Gaussian distribution; and performing three-dimensional scene reconstruction on the two-dimensional Gaussian distribution to obtain a Gaussian sphere distribution of the low-light image.

[0083] Optionally, after obtaining the Gaussian sphere distribution of the low-light image, the Gaussian sphere distribution can be subjected to transparency blending to obtain color parameters for each pixel on the imaging plane, and the Gaussian sphere distribution can be rendered as a target image based on the color parameters. By selecting and switching the viewing angle of the target image, it is possible to reconstruct light-enhanced views from other viewing angles of the scene containing the low-light image.

[0084] In one example, performing a three-dimensional scene reconstruction on the two-dimensional Gaussian distribution to obtain the Gaussian sphere distribution of the low-light image includes: determining the depth information of the two-dimensional Gaussian distribution in the three-dimensional space; and sorting the two-dimensional Gaussian distribution on the imaging plane according to the depth information to obtain the Gaussian sphere distribution of the low-light image.

[0085] The depth in this three-dimensional space refers to the vertical distance (height) of the two-dimensional Gaussian distribution from the imaging plane.

[0086] In one implementation of this embodiment, constructing the absorption rate attribute of the Gaussian sphere distribution includes: obtaining the Gaussian center coordinates of the Gaussian sphere distribution; converting the Gaussian center coordinates into a high-dimensional vector to obtain the spatial characteristics of the Gaussian sphere distribution; and using the Gaussian center coordinates and the Gaussian sphere distribution spatial characteristics to construct the absorption rate attribute of the Gaussian sphere distribution.

[0087] In one example, using the Gaussian center coordinates and the Gaussian sphere distribution spatial features to construct the absorption rate attribute of the Gaussian sphere distribution includes: inputting the Gaussian center coordinates and the Gaussian sphere distribution spatial features into the first fully connected layer of the multi-layer perceptron MLP network, and outputting the camera pose feature vector and the Gaussian sphere distribution feature vector; splicing the camera pose feature vector and the Gaussian sphere distribution feature vector and inputting them into the second fully connected layer of the MLP network, and outputting the scene features; using the activation function in the MLP network to activate the scene features to obtain the absorption rate attribute.

[0088] Figure 4 : This is a schematic diagram of the MLP network in an embodiment of the present invention. The Gaussian ellipsoid mean and spherical harmonic function coefficients in the Gaussian sphere distribution feature vector are spliced with the camera pose feature vector and input into the MLP network to output the absorption rate attribute θ.

[0089] In this embodiment, the Gaussian sphere distribution is subjected to illumination enhancement according to the absorptivity attribute to obtain a scene reconstruction view of the low-light image, including: projecting the Gaussian sphere distribution onto a two-dimensional plane, and performing low-light mixed rendering according to the absorptivity attribute to obtain a low-light scene map; projecting the Gaussian sphere distribution onto a two-dimensional plane, and performing enhanced illumination mixed rendering on the Gaussian sphere distribution to obtain an enhanced illumination scene map; projecting the Gaussian sphere distribution onto a two-dimensional plane, and performing illumination intensity mixed rendering according to the absorptivity attribute to obtain a scene illumination intensity map; integrating the low-light scene map, the enhanced illumination scene map, and the scene illumination intensity map to obtain a scene reconstruction view of the low-light image.

[0090] Optionally, projecting the Gaussian sphere distribution onto a two-dimensional plane and performing low-light mixed rendering according to the absorption rate attribute to obtain a low-light scene graph includes: using the following mixed rendering formula to calculate the low-light scene graph :

[0091] ;

[0092] in, is the color of each pixel in the scene image under low light conditions, N is the number of elements in the set of Gaussian sphere distributions, i is the index representing the current accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorptivity property, It is the color of the three-dimensional Gaussian ball distribution i projected onto the two-dimensional imaging plane. It is a three-dimensional vector representing the values of the three RGB color channels. It is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene image under low light conditions.

[0093] Optionally, the Gaussian sphere distribution is projected onto a two-dimensional plane, and enhanced illumination mixed rendering is performed on the Gaussian sphere distribution to obtain an enhanced illumination scene graph, including: using the following mixed rendering formula to calculate the enhanced illumination scene graph :

[0094] ;

[0095] in, To enhance the color of each pixel in the lighting scene image, N is the number of elements in the set of Gaussian sphere distributions, i is the index, representing the current accumulated three-dimensional Gaussian sphere distribution, It is the color of the three-dimensional Gaussian ball distribution i projected onto the two-dimensional imaging plane. It is a three-dimensional vector representing the values of the three RGB color channels. It is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the enhanced lighting scene image.

[0096] Optionally, the Gaussian sphere distribution is projected onto a two-dimensional plane, and illumination intensity mixed rendering is performed according to the absorption rate attribute to obtain a scene illumination intensity map, including: using the following mixed rendering formula to calculate the scene illumination intensity map :

[0097] ;

[0098] in, is the color of each pixel in the scene light intensity image, N is the number of elements in the set of Gaussian sphere distribution, i is the index, representing the current accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorptivity property, is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene light intensity map.

[0099] Figure 5 3 is a schematic diagram comparing images before and after illumination enhancement in an embodiment of the present invention, illustrating two scenes, indoor and outdoor. After illumination enhancement, the image details are more obvious and richer.

[0100] By adopting the solution of this embodiment, by calculating the camera pose of the low-light image, the position information and direction information of the camera during shooting can be determined, and low-light images under different camera poses can be obtained. In addition, combined with the calculated sparse point cloud information of the scene, geometric information during three-dimensional scene reconstruction can be obtained, thereby enhancing the accuracy of three-dimensional scene reconstruction under low light. In addition, by performing illumination enhancement on the three-dimensional reconstructed scene image, the reconstruction effect of the low-light image under low light can be further improved, so that image processing and three-dimensional reconstruction in a low-light environment can still obtain high-quality three-dimensional scene images even when the lighting conditions are not ideal, and can effectively retain the scene details of the low-light image.

[0101] Through the description of the above embodiments, those skilled in the art will clearly understand that the methods according to the above embodiments can be implemented using software plus the necessary general-purpose hardware platform. Of course, hardware can also be used, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, or optical disk) and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0102] Example 2

[0103] This embodiment also provides a low-light image enhancement device for implementing the above-mentioned embodiments and preferred implementations. Details already described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

[0104] Figure 6 FIG. 1 is a structural block diagram of a low-light image enhancement device according to an embodiment of the present invention. Figure 6 As shown, the device includes:

[0105] A first acquisition module 60 is configured to acquire a low-light image of a low-light scene, wherein the low-light image is an image with a brightness lower than a threshold;

[0106] A second acquisition module 62 is configured to acquire a camera pose and a scene sparse point cloud of the low-light image;

[0107] a reconstruction module 64, configured to perform three-dimensional scene reconstruction on the low-light image using the camera pose and the scene sparse point cloud to obtain a Gaussian sphere distribution of the low-light image;

[0108] A construction module 66 is configured to construct an absorptivity attribute of the Gaussian sphere distribution, wherein the absorptivity attribute is used to describe the ability of an object to absorb radiation energy projected onto its surface;

[0109] The enhancement module 68 is configured to perform illumination enhancement on the Gaussian sphere distribution according to the absorption rate attribute to obtain a scene reconstruction view of the low-light image.

[0110] Optionally, the second acquisition module includes: a first processing unit, used to perform image normalization processing on the low-light image to obtain a standard low-light image; a second processing unit, used to perform camera pose estimation on the standard low-light image to obtain the camera pose, and perform motion recovery on objects in the standard low-light image to obtain a sparse point cloud of the scene.

[0111] Optionally, the reconstruction module includes: an initialization unit for initializing the scene sparse point cloud into a set of parameterized three-dimensional Gaussian primitives representing the three-dimensional scene; a projection unit for projecting the parameterized three-dimensional Gaussian primitives onto a two-dimensional imaging plane using the camera pose to obtain a two-dimensional Gaussian distribution; and a reconstruction unit for performing three-dimensional scene reconstruction on the two-dimensional Gaussian distribution to obtain a Gaussian sphere distribution of the low-light image.

[0112] Optionally, the reconstruction unit includes: a determination subunit, used to determine the depth information of the two-dimensional Gaussian distribution in three-dimensional space; and a sorting subunit, used to sort the two-dimensional Gaussian distribution on the imaging plane according to the depth information to obtain the Gaussian sphere distribution of the low-light image.

[0113] Optionally, the construction module includes: an acquisition unit for acquiring the Gaussian center coordinates of the Gaussian sphere distribution; a conversion unit for converting the Gaussian center coordinates into a high-dimensional vector to obtain the spatial characteristics of the Gaussian sphere distribution; and a construction unit for using the Gaussian center coordinates and the Gaussian sphere distribution spatial characteristics to construct the absorption rate properties of the Gaussian sphere distribution.

[0114] Optionally, the construction unit includes: a first processing unit, used to input the Gaussian center coordinates and the Gaussian sphere distribution spatial features into the first fully connected layer of the multi-layer perceptron MLP network, and output a camera pose feature vector and a Gaussian sphere distribution feature vector; a second processing unit, used to splice the camera pose feature vector and the Gaussian sphere distribution feature vector and input them into the second fully connected layer of the MLP network, and output scene features; an activation unit, used to use the activation function in the MLP network to activate the scene features to obtain the absorption rate attribute.

[0115] Optionally, the enhancement module includes: a first rendering unit, used to project the Gaussian sphere distribution onto a two-dimensional plane, and perform low-light mixed rendering according to the absorption rate attribute to obtain a low-light scene map; a second rendering unit, used to project the Gaussian sphere distribution onto a two-dimensional plane, and perform enhanced illumination mixed rendering on the Gaussian sphere distribution to obtain an enhanced illumination scene map; a third rendering unit, used to project the Gaussian sphere distribution onto a two-dimensional plane, and perform illumination intensity mixed rendering according to the absorption rate attribute to obtain a scene illumination intensity map; an integration unit, used to integrate the low-light scene map, the enhanced illumination scene map and the scene illumination intensity map to obtain a scene reconstruction view of the low-light image.

[0116] Optionally, the first rendering unit includes: a calculation subunit, configured to calculate a low-light scene image using the following hybrid rendering formula: : ;in, is the color of each pixel in the scene image under low light conditions, N is the number of elements in the set of Gaussian sphere distributions, i is the index representing the current accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorptivity property, It is the color of the three-dimensional Gaussian ball distribution i projected onto the two-dimensional imaging plane. It is a three-dimensional vector representing the values of the three RGB color channels. It is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene image under low light conditions.

[0117] Optionally, the second rendering unit includes: a calculation subunit, configured to calculate an enhanced lighting scene graph using the following hybrid rendering formula: : ;in, To enhance the color of each pixel in the lighting scene image, N is the number of elements in the set of Gaussian sphere distributions, i is the index, representing the current accumulated three-dimensional Gaussian sphere distribution, It is the color of the three-dimensional Gaussian ball distribution i projected onto the two-dimensional imaging plane. It is a three-dimensional vector representing the values of the three RGB color channels. It is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the enhanced lighting scene image.

[0118] Optionally, the third rendering unit includes: a calculation subunit, configured to calculate a scene illumination intensity map using the following hybrid rendering formula: : ;in, is the color of each pixel in the scene light intensity image, N is the number of elements in the set of Gaussian sphere distribution, i is the index, representing the current accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorptivity property, is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene light intensity map.

[0119] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0120] Example 3

[0121] An embodiment of the present invention further provides a storage medium storing a computer program, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0122] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0123] S1, acquiring a low-light image of a low-light scene, wherein the low-light image is an image with brightness lower than a threshold;

[0124] S2, obtaining the camera pose and scene sparse point cloud of the low-light image;

[0125] S3, performing three-dimensional scene reconstruction on the low-light image using the camera pose and the scene sparse point cloud to obtain a Gaussian sphere distribution of the low-light image;

[0126] S4, constructing an absorptivity attribute of the Gaussian sphere distribution, wherein the absorptivity attribute is used to describe the absorption capability of an object to radiation energy projected onto its surface;

[0127] S5. Perform illumination enhancement on the Gaussian sphere distribution according to the absorption rate attribute to obtain a scene reconstruction view of the low-light image.

[0128] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0129] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0130] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0131] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0132] S1, acquiring a low-light image of a low-light scene, wherein the low-light image is an image with brightness lower than a threshold;

[0133] S2, obtaining the camera pose and scene sparse point cloud of the low-light image;

[0134] S3, performing three-dimensional scene reconstruction on the low-light image using the camera pose and the scene sparse point cloud to obtain a Gaussian sphere distribution of the low-light image;

[0135] S4, constructing an absorptivity attribute of the Gaussian sphere distribution, wherein the absorptivity attribute is used to describe the absorption capability of an object to radiation energy projected onto its surface;

[0136] S5. Perform illumination enhancement on the Gaussian sphere distribution according to the absorption rate attribute to obtain a scene reconstruction view of the low-light image.

[0137] Optionally, specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0138] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0139] Through the description of the above embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a general hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the relevant technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0140] It should be understood that the terms used herein are for the purpose of describing specific example embodiments only and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "comprise", "include", "contain" and "have" are inclusive and therefore specify the presence of stated features, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring them to be performed in the specific order described or illustrated, unless the order of execution is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0141] The foregoing description is intended only to provide specific embodiments of the present invention, which will enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein, but is intended to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for enhancing low-light images, characterized in that: include: Acquire a low-light image of a low-light scene, wherein the low-light image is an image with a brightness lower than a threshold; Obtaining a camera pose and a scene sparse point cloud of the low-light image; Reconstructing a three-dimensional scene of the low-light image using the camera pose and the scene sparse point cloud to obtain a Gaussian sphere distribution of the low-light image; Constructing an absorptivity attribute of the Gaussian sphere distribution, wherein the absorptivity attribute is used to describe the absorption capability of an object to radiation energy projected onto its surface; performing illumination enhancement on the Gaussian sphere distribution according to the absorptivity attribute to obtain a scene reconstruction view of the low-light image; wherein, performing illumination enhancement on the Gaussian sphere distribution according to the absorptivity attribute to obtain a scene reconstruction view of the low-light image includes: projecting the Gaussian sphere distribution onto a two-dimensional plane, and performing low-light mixed rendering according to the absorptivity attribute to obtain a low-light scene graph; projecting the Gaussian sphere distribution onto a two-dimensional plane, and performing enhanced illumination mixed rendering on the Gaussian sphere distribution to obtain an enhanced illumination scene graph; projecting the Gaussian sphere distribution onto a two-dimensional plane, and performing illumination intensity mixed rendering according to the absorptivity attribute to obtain a scene illumination intensity graph; and integrating the low-light scene graph, the enhanced illumination scene graph, and the scene illumination intensity graph to obtain a scene reconstruction view of the low-light image; The Gaussian sphere distribution is projected onto a two-dimensional plane, and low-light mixed rendering is performed according to the absorption rate attribute to obtain a low-light scene image, which includes: using the following mixed rendering formula to calculate the color of each pixel in the low-light scene image : ;in, is the color of each pixel in the scene image under low light conditions, N is the number of elements in the set of Gaussian sphere distributions, i is the index representing the current accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorptivity property, It is the color of the three-dimensional Gaussian ball distribution i projected onto the two-dimensional imaging plane. It is a three-dimensional vector representing the values of the three RGB color channels. is a value between 0 and 1, which is used to represent the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene image under low light conditions; The Gaussian sphere distribution is projected onto a two-dimensional plane, and the Gaussian sphere distribution is subjected to enhanced illumination mixed rendering to obtain an enhanced illumination scene graph, including: using the following mixed rendering formula to calculate the color of each pixel in the enhanced illumination scene graph : ;in, To enhance the color of each pixel in the lighting scene image, N is the number of elements in the set of Gaussian sphere distributions, i is the index, representing the current accumulated three-dimensional Gaussian sphere distribution, It is the color of the three-dimensional Gaussian ball distribution i projected onto the two-dimensional imaging plane. It is a three-dimensional vector representing the values of the three RGB color channels. is a value between 0 and 1, which is used to represent the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the enhanced lighting scene image; The Gaussian sphere distribution is projected onto a two-dimensional plane, and light intensity mixed rendering is performed according to the absorption rate attribute to obtain a scene light intensity map, including: using the following mixed rendering formula to calculate the color of each pixel in the scene light intensity map : ;in, is the color of each pixel in the scene light intensity image, N is the number of elements in the set of Gaussian sphere distribution, i is the index, representing the current accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorptivity property, is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene light intensity map.

2. The method according to claim 1, characterized in that Obtaining the camera pose and scene sparse point cloud of the low-light image includes: performing image standardization processing on the low-light image to obtain a standard low-light image; Performing camera pose estimation on the standard low-light image to obtain the camera pose, and performing motion recovery on objects in the standard low-light image to obtain a scene sparse point cloud.

3. The method according to claim 1, characterized in that Reconstructing the low-light image in three dimensions using the camera pose and the scene sparse point cloud to obtain a Gaussian sphere distribution of the low-light image includes: Initializing the scene sparse point cloud into a set of parameterized three-dimensional Gaussian primitives representing the three-dimensional scene; Projecting the parameterized three-dimensional Gaussian basis element onto a two-dimensional imaging plane using the camera pose to obtain a two-dimensional Gaussian distribution; A three-dimensional scene reconstruction is performed on the two-dimensional Gaussian distribution to obtain a Gaussian sphere distribution of the low-light image.

4. The method according to claim 3, characterized in that Performing a three-dimensional scene reconstruction on the two-dimensional Gaussian distribution to obtain a Gaussian sphere distribution of the low-light image includes: Determining depth information of the two-dimensional Gaussian distribution in three-dimensional space; The two-dimensional Gaussian distribution is sorted on an imaging plane according to the depth information to obtain a Gaussian sphere distribution of the low-light image.

5. The method according to claim 1, wherein The absorptivity properties for constructing the Gaussian sphere distribution include: Obtaining the Gaussian center coordinates of the Gaussian sphere distribution; Converting the Gaussian center coordinates into a high-dimensional vector to obtain Gaussian sphere distribution spatial characteristics; The Gaussian center coordinates and the Gaussian sphere distribution spatial characteristics are used to construct the absorption rate attribute of the Gaussian sphere distribution.

6. The method according to claim 5, characterized in that Constructing the absorption rate attribute of the Gaussian sphere distribution using the Gaussian center coordinates and the Gaussian sphere distribution spatial characteristics includes: Input the Gaussian center coordinates and the Gaussian sphere distribution spatial features into the first fully connected layer of the MLP network, and output the camera pose feature vector and the Gaussian sphere distribution feature vector; The camera pose feature vector and the Gaussian sphere distribution feature vector are concatenated and input into the second fully connected layer of the MLP network to output scene features; The scene feature is activated using an activation function in the MLP network to obtain the absorption rate attribute.

7. A low-light image enhancement device, characterized in that: include: A first acquisition module is configured to acquire a low-light image of a low-light scene, wherein the low-light image is an image with a brightness lower than a threshold; A second acquisition module is used to obtain the camera pose and scene sparse point cloud of the low-light image; a reconstruction module, configured to perform three-dimensional scene reconstruction on the low-light image using the camera pose and the scene sparse point cloud to obtain a Gaussian sphere distribution of the low-light image; A construction module, configured to construct an absorptivity attribute of the Gaussian sphere distribution, wherein the absorptivity attribute is used to describe an object's ability to absorb radiation energy projected onto its surface; an enhancement module, configured to perform illumination enhancement on the Gaussian sphere distribution according to the absorptivity attribute to obtain a scene reconstruction view of the low-light image; The enhancement module includes: a first rendering unit, configured to project the Gaussian sphere distribution onto a two-dimensional plane, and perform low-light mixed rendering according to the absorptivity attribute to obtain a low-light scene graph; a second rendering unit, configured to project the Gaussian sphere distribution onto a two-dimensional plane, and perform enhanced light mixed rendering on the Gaussian sphere distribution to obtain an enhanced light scene graph; a third rendering unit, configured to project the Gaussian sphere distribution onto a two-dimensional plane, and perform light intensity mixed rendering according to the absorptivity attribute to obtain a scene light intensity graph; an integration unit, configured to integrate the low-light scene graph, the enhanced light scene graph, and the scene light intensity graph to obtain a scene reconstructed view of the low-light image; The first rendering unit includes a calculation subunit for calculating the low-light scene image using the following hybrid rendering formula: : ;in, is the color of each pixel in the scene image under low light conditions, N is the number of elements in the set of Gaussian sphere distributions, i is the index representing the current accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorptivity property, It is the color of the three-dimensional Gaussian ball distribution i projected onto the two-dimensional imaging plane. It is a three-dimensional vector representing the values of the three RGB color channels. is a value between 0 and 1, which is used to represent the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene image under low light conditions; The second rendering unit includes: a calculation subunit for calculating the enhanced lighting scene graph using the following hybrid rendering formula: : ;in, To enhance the color of each pixel in the lighting scene image, N is the number of elements in the set of Gaussian sphere distributions, i is the index, representing the current accumulated three-dimensional Gaussian sphere distribution, It is the color of the three-dimensional Gaussian ball distribution i projected onto the two-dimensional imaging plane. It is a three-dimensional vector representing the values of the three RGB color channels. is a value between 0 and 1, which is used to represent the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the enhanced lighting scene image; The third rendering unit includes: a calculation subunit, which is used to calculate the scene illumination intensity map using the following hybrid rendering formula: : ;in, is the color of each pixel in the scene light intensity image, N is the number of elements in the set of Gaussian sphere distribution, i is the index, representing the current accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorptivity property, is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene light intensity map.

8. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 6 when executed.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 6.

Images