Low-illumination image enhancement method and device, storage medium and electronic device

Abstract

The invention provides a low-illumination image enhancement method and device, a storage medium and an electronic device.The method comprises the steps that a low-illumination image of a low-illumination scene is acquired, and the brightness of the low-illumination image is lower than a threshold value; obtaining a camera pose and a scene sparse point cloud of the low-illumination image; performing three-dimensional scene reconstruction on the low-illumination image by adopting the camera pose and the scene sparse point cloud to obtain Gaussian ball distribution of the low-illumination image; the absorptivity attribute of the Gaussian ball distribution is constructed, and the absorptivity attribute is used for describing the absorptivity of an object to radiation energy projected to the surface of the object; and performing illumination enhancement on the Gaussian ball distribution according to the absorptivity attribute to obtain a scene reconstruction view of the low-illumination image. Through the embodiment of the invention, the technical problem of image detail missing in a low-illumination scene in related technologies is solved, and scene details of the low-illumination image can be effectively reserved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a method and device for enhancing low-light images, a storage medium, and an electronic device. Background Art

[0002] In the related art, 3D scene reconstruction and novel view synthesis are long-standing complex tasks in the field of computer vision, and the goal is to recover the 3D structure of a scene from a set of 2D images. This technology plays a crucial role in many application fields such as virtual reality (VR) and autonomous driving. In recent years, implicit representation methods, especially Neural Radiance Fields (hereinafter referred to as NeRF) and its derivative models, have made remarkable progress in generating realistic images from arbitrary viewpoints. Although NeRF performs well in creating realistic images, the market's demand for faster and more efficient rendering methods is increasing, especially for applications that require real-time performance. 3DGS (3D Gaussian Splatting) technology is a new 3D scene representation and view synthesis method proposed in recent years. The 3DGS technology explicitly represents the scene by using millions of 3D Gaussian functions, and these Gaussian functions are projected onto the image plane during the rendering process. Compared with implicit representation methods, the 3DGS technology takes advantage of differentiable rendering pipelines and point-based rendering techniques, providing an efficient calculation and rendering process and avoiding the high computational cost problem of traditional NeRF methods.

[0003] Although the 3DGS technology has made remarkable progress in rendering efficiency and scene dynamic control, its performance is still limited under low-light conditions. Under low-light conditions, such as at night, on cloudy days, or in underground spaces, the light reflection on the surface of objects is weak, resulting in images being affected by backlighting, uneven illumination, low contrast, and dense noise, which makes it difficult to achieve ideal capture effects and key information. Due to the small number of photons, the noise of the camera is also relatively amplified, further burying scene details and distorting object colors. These characteristics of low-light images make it difficult for existing 3DGS technologies to generate high-quality novel view images.

[0004] In the field of view synthesis, enhancing low-light images and restoring their details are crucial for improving the quality of 3D scene reconstruction. Currently, 2D low-light image enhancement technology is a direct and key step in improving the usability of new view synthesis. Although directly enhancing the brightness of the input image seems simple, existing methods have some limitations in maintaining lighting consistency among multiple views. Existing low-light image enhancement techniques usually focus on increasing the brightness of a single image but often neglect the importance of lighting consistency between different images. In addition, deep learning-based enhancement methods tend to learn specific brightness mappings from training data, which limits their generalization ability in unknown or wild scenarios.

[0005] When processing images captured under low-light conditions, existing methods are difficult to achieve high-fidelity color information capture and effective scene detail restoration, resulting in low quality of 3D scene reconstruction and new view synthesis.

[0006] Regarding the above problems existing in the related technology, no efficient and accurate solution has been found yet. Summary of the Invention

[0007] The present invention provides a method and device for enhancing low-light images, a storage medium, and an electronic device to solve the technical problems in the related technology.

[0008] According to an embodiment of the present invention, a method for enhancing a low-light image is provided, including: obtaining a low-light image of a low-light scene, where the low-light image is an image with a brightness lower than a threshold; obtaining the camera pose and scene sparse point cloud of the low-light image; performing 3D scene reconstruction on the low-light image using the camera pose and the scene sparse point cloud to obtain the Gaussian sphere distribution of the low-light image; constructing the absorption rate attribute of the Gaussian sphere distribution, where the absorption rate attribute is used to describe the absorption ability of an object for the radiation energy projected onto its surface; and performing light enhancement on the Gaussian sphere distribution according to the absorption rate attribute to obtain the scene reconstruction view of the low-light image.

[0009] Optionally, obtaining the camera pose and scene sparse point cloud of the low-light image includes: performing image normalization processing on the low-light image to obtain a standard low-light image; performing camera pose estimation on the standard low-light image to obtain the camera pose, and performing motion recovery on the objects in the standard low-light image to obtain the scene sparse point cloud.

[0010] Optionally, performing three-dimensional scene reconstruction on the low-light image using the camera pose and the scene sparse point cloud to obtain the Gaussian sphere distribution of the low-light image, including: initializing the scene sparse point cloud as a set of parameterized three-dimensional Gaussian basis elements representing the three-dimensional scene; projecting the parameterized three-dimensional Gaussian basis elements onto a two-dimensional imaging plane using the camera pose to obtain a two-dimensional Gaussian distribution; and performing three-dimensional scene reconstruction on the two-dimensional Gaussian distribution to obtain the Gaussian sphere distribution of the low-light image.

[0011] Optionally, performing three-dimensional scene reconstruction on the two-dimensional Gaussian distribution to obtain the Gaussian sphere distribution of the low-light image includes: determining the depth information of the two-dimensional Gaussian distribution in three-dimensional space; sorting the two-dimensional Gaussian distribution on the imaging plane according to the depth information to obtain the Gaussian sphere distribution of the low-light image.

[0012] Optionally, constructing the absorption rate attribute of the Gaussian sphere distribution includes: obtaining the Gaussian center coordinates of the Gaussian sphere distribution; converting the Gaussian center coordinates into a high-dimensional vector to obtain the spatial characteristics of the Gaussian sphere distribution; and constructing the absorption rate attribute of the Gaussian sphere distribution using the Gaussian center coordinates and the spatial characteristics of the Gaussian sphere distribution.

[0013] Optionally, constructing the absorption rate attribute of the Gaussian sphere distribution using the Gaussian center coordinates and the spatial characteristics of the Gaussian sphere distribution includes: inputting the Gaussian center coordinates and the spatial characteristics of the Gaussian sphere distribution into the first fully connected layer of the multi-layer perceptron (MLP) network to output a camera pose feature vector and a Gaussian sphere distribution feature vector; concatenating the camera pose feature vector and the Gaussian sphere distribution feature vector and inputting the result into the second fully connected layer of the MLP network to output a scene feature; and activating the scene feature using the activation function in the MLP network to obtain the absorption rate attribute.

[0014] Optionally, enhancing the illumination of the Gaussian sphere distribution according to the absorption rate attribute to obtain the scene reconstruction view of the low-light image, including: projecting the Gaussian sphere distribution onto a two-dimensional plane and performing low-light hybrid rendering according to the absorption rate attribute to obtain a scene map under low light; projecting the Gaussian sphere distribution onto a two-dimensional plane and performing enhanced illumination hybrid rendering on the Gaussian sphere distribution to obtain an enhanced illumination scene map; projecting the Gaussian sphere distribution onto a two-dimensional plane and performing illumination intensity hybrid rendering according to the absorption rate attribute to obtain a scene illumination intensity map; and integrating the scene map under low light, the enhanced illumination scene map, and the scene illumination intensity map to obtain the scene reconstruction view of the low-light image.

[0015] Optionally, project the Gaussian spherical distribution onto a two-dimensional plane and perform low-light hybrid rendering according to the absorption rate attribute to obtain a scene graph under low light, including: calculating the scene graph under low light using the following hybrid rendering formula : ; where is the color of each pixel in the scene graph under low light, N is the number of elements in the set of the Gaussian spherical distribution, i is the index, representing the currently accumulated three-dimensional Gaussian spherical distribution, is the absorption coefficient of the absorption rate attribute, is the color of the three-dimensional Gaussian spherical distribution i projected onto the two-dimensional imaging plane, which is a three-dimensional vector representing the values of the RGB three color channels, is a value between 0 and 1, used to represent the contribution degree of the three-dimensional Gaussian spherical distribution i to the pixel color in the scene graph under low light.

[0016] Optionally, project the Gaussian spherical distribution onto a two-dimensional plane and perform enhanced-light hybrid rendering on the Gaussian spherical distribution to obtain an enhanced-light scene graph, including: calculating the enhanced-light scene graph using the following hybrid rendering formula : ; where is the color of each pixel in the enhanced-light scene graph, N is the number of elements in the set of the Gaussian spherical distribution, i is the index, representing the currently accumulated three-dimensional Gaussian spherical distribution, is the color of the three-dimensional Gaussian spherical distribution i projected onto the two-dimensional imaging plane, which is a three-dimensional vector representing the values of the RGB three color channels, is a value between 0 and 1, used to represent the contribution degree of the three-dimensional Gaussian spherical distribution i to the pixel color in the enhanced-light scene graph.

[0017] Optionally, project the Gaussian spherical distribution onto a two-dimensional plane and perform light intensity hybrid rendering according to the absorption rate attribute to obtain a scene light intensity graph, including: calculating the scene light intensity graph using the following hybrid rendering formula : ; where is the color of each pixel in the scene light intensity image, N is the number of elements in the set of the Gaussian spherical distribution, i is the index, representing the currently accumulated three-dimensional Gaussian spherical distribution, is the absorption coefficient of the absorption rate attribute, is a value between 0 and 1, used to represent the contribution degree of the three-dimensional Gaussian spherical distribution i to the pixel color in the scene light intensity graph.

[0018] According to another embodiment of the present invention, an enhancement device for low-light images is provided, including: a first acquisition module, configured to acquire a low-light image of a low-light scene, where the low-light image is an image with a brightness lower than a threshold; a second acquisition module, configured to acquire the camera pose and the scene sparse point cloud of the low-light image; a reconstruction module, configured to perform three-dimensional scene reconstruction on the low-light image by using the camera pose and the scene sparse point cloud to obtain the Gaussian sphere distribution of the low-light image; a construction module, configured to construct the absorption rate attribute of the Gaussian sphere distribution, where the absorption rate attribute is used to describe the absorption ability of an object to the radiation energy projected onto its surface; an enhancement module, configured to perform light enhancement on the Gaussian sphere distribution according to the absorption rate attribute to obtain the scene reconstruction view of the low-light image.

[0019] Optionally, the second acquisition module includes: a first processing unit, configured to perform image normalization processing on the low-light image to obtain a standard low-light image; a second processing unit, configured to perform camera pose estimation on the standard low-light image to obtain the camera pose, and perform motion recovery on the objects in the standard low-light image to obtain the scene sparse point cloud.

[0020] Optionally, the reconstruction module includes: an initialization unit, configured to initialize the scene sparse point cloud into a set of parameterized three-dimensional Gaussian basis elements representing a three-dimensional scene; a projection unit, configured to project the parameterized three-dimensional Gaussian basis elements onto a two-dimensional imaging plane by using the camera pose to obtain a two-dimensional Gaussian distribution; a reconstruction unit, configured to perform three-dimensional scene reconstruction on the two-dimensional Gaussian distribution to obtain the Gaussian sphere distribution of the low-light image.

[0021] Optionally, the reconstruction unit includes: a determination subunit, configured to determine the depth information of the two-dimensional Gaussian distribution in three-dimensional space; a sorting subunit, configured to sort the two-dimensional Gaussian distribution on the imaging plane according to the depth information to obtain the Gaussian sphere distribution of the low-light image.

[0022] Optionally, the construction module includes: an acquisition unit, configured to acquire the Gaussian center coordinates of the Gaussian sphere distribution; a conversion unit, configured to convert the Gaussian center coordinates into a high-dimensional vector to obtain the spatial feature of the Gaussian sphere distribution; a construction unit, configured to construct the absorption rate attribute of the Gaussian sphere distribution by using the Gaussian center coordinates and the spatial feature of the Gaussian sphere distribution.

[0023] Optionally, the building unit includes: a first processing unit configured to input the Gaussian center coordinates and the Gaussian sphere distribution spatial features into the first fully-connected layer of the first part of a multi-layer perceptron (MLP) network, and output a camera pose feature vector and a Gaussian sphere distribution feature vector; a second processing unit configured to splice the camera pose feature vector and the Gaussian sphere distribution feature vector and then input the result into the second fully-connected layer of the MLP network to output a scene feature; and an activation unit configured to perform activation processing on the scene feature by using an activation function in the MLP network to obtain the absorption rate attribute.

[0024] Optionally, the enhancement module includes: a first rendering unit configured to project the Gaussian sphere distribution onto a two-dimensional plane and perform low-light mixing rendering according to the absorption rate attribute to obtain a scene graph under low light; a second rendering unit configured to project the Gaussian sphere distribution onto a two-dimensional plane and perform enhanced-lighting mixing rendering on the Gaussian sphere distribution to obtain an enhanced-lighting scene graph; a third rendering unit configured to project the Gaussian sphere distribution onto a two-dimensional plane and perform lighting intensity mixing rendering according to the absorption rate attribute to obtain a scene lighting intensity graph; and an integration unit configured to integrate the scene graph under low light, the enhanced-lighting scene graph, and the scene lighting intensity graph to obtain a scene reconstruction view of the low-light image.

[0025] Optionally, the first rendering unit includes: a calculation subunit configured to calculate the scene graph under low light by using the following mixing rendering formula : ; where is the color of each pixel in the scene graph under low light, N is the number of elements in the set of the Gaussian sphere distributions, i is an index representing the currently accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorption rate attribute, is the color of the three-dimensional Gaussian sphere distribution i projected onto the two-dimensional imaging plane, which is a three-dimensional vector representing the values of the RGB three color channels, is a value between 0 and 1, used to represent the contribution degree of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene graph under low light.

[0026] Optionally, the second rendering unit includes: a calculation subunit configured to calculate the enhanced-lighting scene graph by using the following mixing rendering formula : ; where is the color of each pixel in the enhanced-lighting scene graph, N is the number of elements in the set of the Gaussian sphere distributions, i is an index representing the currently accumulated three-dimensional Gaussian sphere distribution, is the color of the three-dimensional Gaussian sphere distribution i projected onto the two-dimensional imaging plane, which is a three-dimensional vector representing the values of the RGB three color channels, is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the enhanced lighting scene graph.

[0027] Optionally, the third rendering unit includes: a calculation subunit, configured to calculate a scene illumination intensity map using the following mixed rendering formula: : ;in, is the color of each pixel in the scene illumination intensity image, N is the number of elements in the set of Gaussian sphere distribution, i is the index, representing the current accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorptivity property, is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene light intensity map.

[0028] According to another aspect of an embodiment of the present application, a storage medium is further provided, which includes a stored program, and the above steps are executed when the program is run.

[0029] According to another aspect of an embodiment of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein: the memory is used to store computer programs; the processor is used to execute the steps in the above method by running the program stored in the memory.

[0030] The embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the steps in the above method.

[0031] Beneficial effects of the present invention: 1. Improve the accuracy of 3D reconstruction and illumination enhancement of low-light scene images, and be able to reconstruct high-quality scene reconstruction views under unfavorable lighting conditions, effectively preserving scene details of low-light images; 2. By obtaining the camera pose and sparse point cloud of the low-light image, the position and direction information of the camera during shooting can be determined, and the view of the low-light image under different camera poses can be reconstructed. Combined with the sparse point cloud of the scene, the geometric information during 3D scene reconstruction can be obtained, thereby enhancing the accuracy of 3D scene reconstruction under low light conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1It is a hardware structure block diagram of an automobile according to an embodiment of the present invention; Figure 2 It is a flowchart of a method for enhancing low-light images according to an embodiment of the present invention; Figure 3 It is a schematic diagram of three-dimensional reconstruction and light enhancement of low-light scene images in an embodiment of the present invention; Figure 4 It is a schematic diagram of the principle of the MLP network in an embodiment of the present invention; Figure 5 It is a comparison schematic diagram of an image before and after light enhancement in an embodiment of the present invention; Figure 6 It is a structure block diagram of an apparatus for enhancing low-light images according to an embodiment of the present invention. Detailed implementation manners

[0033] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, product or device including a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0035] Embodiment 1 The method embodiment provided in the first embodiment of the present application can be executed in an automobile, a server, a computer, a processor, an autonomous driving / assisted driving / smart driving controller or a similar processing device. Taking running on an automobile as an example, Figure 1 It is a hardware structure block diagram of an automobile according to an embodiment of the present invention. As Figure 1 shown, the automobile may include one or more ( Figure 1Only one processor 102 is shown (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a field programmable gate array FPGA), and a memory 104 for storing data. Optionally, the above-mentioned vehicle may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 The structure shown is only schematic and does not limit the structure of the above-mentioned vehicle. For example, the vehicle may further include more or fewer components than Figure 1 shown in, or have a different configuration from Figure 1 that shown.

[0036] The memory 104 can be used to store vehicle programs. For example, software programs and modules of application software, such as the vehicle program corresponding to the method for enhancing low-light images of a vehicle in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the vehicle program stored in the memory 104, that is, the above-mentioned method is implemented. The memory 104 may include a high-speed random access memory, and may further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the vehicle through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0037] The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of the vehicle. In one instance, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0038] In this embodiment, a method for enhancing low-light images is provided. Figure 2 is a flowchart of a method for enhancing low-light images according to an embodiment of the present invention, as Figure 2 shown, and the process includes the following steps: Step S202, obtaining a low-light image of a low-light scene, where the low-light image is an image with a brightness lower than a threshold; Optionally, a low-light image refers to one or more images captured by an image sensor in a low-light scene due to the limitation of the low-light environment. There are image details obscured by the low light and color distortion caused by the low light in the low-light image.

[0039] Step S204: Obtain the camera pose and the scene sparse point cloud of the low-light image; In this embodiment, the camera pose refers to the position and orientation of the camera in three-dimensional space. The scene sparse point cloud refers to point cloud data that is unevenly distributed in three-dimensional space. Due to the influence of low light, the point cloud is dense in some areas and sparse in some places, resulting in the sparsity of the distribution of the point cloud data.

[0040] Step S206: Perform three-dimensional scene reconstruction on the low-light image using the camera pose and the scene sparse point cloud to obtain the Gaussian sphere distribution of the low-light image; Step S208: Construct the absorption rate attribute of the Gaussian sphere distribution, where the absorption rate attribute is used to describe the absorption ability of an object to the radiant energy projected onto its surface; Step S210: Enhance the illumination of the Gaussian sphere distribution according to the absorption rate attribute to obtain the scene reconstruction view of the low-light image.

[0041] The scene reconstruction view in this embodiment is a two-dimensional image after the illumination of the low-light image is enhanced.

[0042] Through the above steps, a low-light image of a low-light scene is obtained, where the low-light image is an image with a brightness lower than a threshold; the camera pose and the scene sparse point cloud of the low-light image are obtained; three-dimensional scene reconstruction is performed on the low-light image using the camera pose and the scene sparse point cloud to obtain the Gaussian sphere distribution of the low-light image; the absorption rate attribute of the Gaussian sphere distribution is constructed, where the absorption rate attribute is used to describe the absorption ability of an object to the radiant energy projected onto its surface; the illumination of the Gaussian sphere distribution is enhanced according to the absorption rate attribute to obtain the scene reconstruction view of the low-light image, solving the technical problem of missing image details in the low-light scene in the related art and effectively retaining the scene details of the low-light image.

[0043] Optionally, obtaining the camera pose and the scene sparse point cloud of the low-light image includes: performing image normalization processing on the low-light image to obtain a standard low-light image; performing camera pose estimation on the standard low-light image to obtain the camera pose, and performing motion recovery on the objects in the standard low-light image to obtain the scene sparse point cloud.

[0044] Use the preset camera pose estimation software to estimate the pose of the standard low-light image, and obtain the camera pose. Use the structure from motion algorithm to estimate the sparse point cloud of the scene corresponding to the standard low-light image.

[0045] In the embodiments of the present invention, the camera pose estimation software refers to software that estimates the camera pose of an image by using technologies such as feature detection and matching, incremental SfM (Structure from Motion), and multi-view stereo matching included in the software. For example, the preset camera pose estimation software can adopt the COLMAP software. COLMAP is an open-source 3D reconstruction tool used to recover the structure of a 3D scene from 2D images.

[0046] In this embodiment, the process of performing 3D scene reconstruction on the low-light image by using the camera pose and the sparse point cloud of the scene to obtain the Gaussian sphere distribution and its absorption rate attribute of the low-light image can be implemented by a preset 3D scene algorithm, and the 3D scene algorithm can be encapsulated as a pre-trained model. The 3D scene algorithm in this embodiment is an improved 3DGS algorithm based on the 3DGS (3D Gaussian Splatting) algorithm. The 3DGS algorithm is a technology for 3D scene reconstruction and rendering. It models the scene by using a set of Gaussian ellipsoids and uses rasterization technology to render these Gaussian ellipsoids into images, thereby realizing an effective rendering tool. On the basis of the original 3DGS, a new attribute, that is, the absorption rate attribute, is introduced for each Gaussian ellipsoid. The improved 3DGS algorithm includes a Multilayer Perceptron (MLP) network, and the MLP network is used to construct the absorption rate attribute. Figure 3 It is the schematic diagram of 3D reconstruction and light enhancement of the low-light scene image in the embodiments of the present invention. The original low-light image is processed to obtain a sparse point cloud, and then processed to obtain the Gaussian sphere distribution feature vector (Gaussian ellipsoid mean, spherical harmonic function coefficient). Combining with the camera pose feature vector, the absorption rate attribute is obtained, and then through hybrid rendering, the light-enhanced scene reconstruction view is obtained.

[0047] Before performing 3D scene reconstruction by using the preset 3D scene algorithm, it also includes training and optimizing the preset 3D scene algorithm. It includes: adjusting the image consistency of the 3D scene algorithm by using the tone mapping loss function, adjusting the image contrast and structure consistency of the 3D scene algorithm by using the improved structural similarity loss function, adjusting the exposure level control of the 3D scene algorithm by using the exposure control loss function, and adjusting the size control of the scene Gaussian sphere distribution by using the scale loss function.

[0048] In one example, the tone-mapped L1 loss function ( L it-MAE ): By introducing a non-linear tone mapping function, this loss function modifies the original L1 loss to improve the image tone consistency under different lighting conditions. The tone mapping function is a piecewise cubic polynomial used to adjust the weights of dark and bright regions, enabling the network to pay more attention to the restoration of details during training. The tone mapping function is defined as follows: Φ(x) where

[0049] represents the pixel value in the image, x is the first adjustable coefficient, a is the second adjustable coefficient, and in the embodiments of the present invention, b is taken.

[0050] The specific definition of the tone-mapped loss function is given by the following formula:

[0051] where is the tone-mapped loss, is the low-light image rendered by the algorithm, is the real low-light image.

[0052] In one example, the improved structural similarity loss function (csSSIM Loss, L csSSIM ): The traditional SSIM metric takes into account the brightness, contrast, and structural information of the image. In the present invention, the SSIM metric is improved by removing the brightness comparison part and only retaining the contrast and structural information to enhance the network's learning ability for scene details and reduce the influence of image brightness. This loss function promotes the network to generate images with highly detailed consistency by comparing the similarity in contrast and structure between the enhanced image and the real image. The specific definition of the improved csSSIM metric is given by the following formula:

[0053] where x represents the data in the window of the first image, y represents the data in the window of the second image, and represent in sequence x and y the variances of represents x and y the covariance between ​is a tiny constant used to maintain numerical stability. Further, define the csSSIM loss function as:

[0054] where W is the total number of image windows, i represents the current image window serial number, is the enhanced image rendered under the i -th image window serial number, is the real low-light image under the i -th image window serial number.

[0055] In one example, the Exposure Control Loss function (Exposure Control Loss, L exp ): This loss function is used to control the exposure level of the image, prevent overexposure or underexposure, and ensure the visual naturalness and consistency of the image. By calculating the average brightness of local regions in the image and comparing it with a preset target brightness level E, this loss function prompts the image output by the network to match the real scene in terms of brightness. The specific definition of this loss function is given by the following formula:

[0056] where, M represents the number of non-overlapping local regions into which the image is divided, and the size of each region is 16×16 pixels, Y k represents the average intensity value of the k -th local region in the enhanced image, E is the average brightness value. Optionally, the average brightness value is set to 0.6.

[0057] In one example, the Scale Loss function (Scale Loss, L scal ): This loss function is used to control the size of the 3D Gaussian ellipsoid in the scene to improve the expressiveness of the scene and the richness of details. By setting a desired average Gaussian size e , and calculating the difference between the sizes of all Gaussian ellipsoids in the scene and this expected value, this loss function prompts the network to generate a more reasonable Gaussian ellipsoid size distribution. The specific definition of this loss function is given by the following formula:

[0058] where, P represents the total number of Gaussian ellipsoids, S k represents the kThe average size of a Gaussian ellipsoid, where the average size is the average of the lengths of the three axes of the Gaussian ellipsoid, e is an artificially defined average size, which is a hyperparameter used to control the level of the Gaussian sphere size. In the experiment, e can it is set to 0.1.

[0059] The total loss function (Total Loss, L ) is the weighted sum of the above four loss functions, expressed as:

[0060] where λ 1 , λ 2 and λ 3 are non - negative parameters used to balance the weights of each loss function. Optionally, λ 1 = 0.5, λ 2 =0.1 and λ 3 =0.075 .

[0061] In an implementation manner of this embodiment, the low - light image is reconstructed in a three - dimensional scene using the camera pose and the scene sparse point cloud to obtain the Gaussian sphere distribution of the low - light image, including: initializing the scene sparse point cloud as a set of parameterized three - dimensional Gaussian basis elements representing the three - dimensional scene; projecting the parameterized three - dimensional Gaussian basis elements onto a two - dimensional imaging plane using the camera pose to obtain a two - dimensional Gaussian distribution; and performing three - dimensional scene reconstruction on the two - dimensional Gaussian distribution to obtain the Gaussian sphere distribution of the low - light image.

[0062] Optionally, after obtaining the Gaussian sphere distribution of the low - light image, the Gaussian sphere distribution can be subjected to transparency blending to obtain the color parameters of each pixel point on the imaging plane, and the Gaussian sphere distribution can be rendered as a target image according to the color parameters. By selecting and switching the viewing angle of the target image, views with enhanced lighting in other viewing angles of the scene where the low - light image is located can be reconstructed.

[0063] In an example, performing three - dimensional scene reconstruction on the two - dimensional Gaussian distribution to obtain the Gaussian sphere distribution of the low - light image includes: determining the depth information of the two - dimensional Gaussian distribution in three - dimensional space; sorting the two - dimensional Gaussian distribution on the imaging plane according to the depth information to obtain the Gaussian sphere distribution of the low - light image.

[0064] The depth in this three - dimensional space refers to the vertical distance (height) from the two - dimensional Gaussian distribution to the imaging plane.

[0065] In one implementation manner of this embodiment, constructing the absorption rate attribute of the Gaussian sphere distribution includes: obtaining the Gaussian center coordinates of the Gaussian sphere distribution; converting the Gaussian center coordinates into a high-dimensional vector to obtain the spatial characteristics of the Gaussian sphere distribution; and constructing the absorption rate attribute of the Gaussian sphere distribution by using the Gaussian center coordinates and the spatial characteristics of the Gaussian sphere distribution.

[0066] In one example, constructing the absorption rate attribute of the Gaussian sphere distribution by using the Gaussian center coordinates and the spatial characteristics of the Gaussian sphere distribution includes: inputting the Gaussian center coordinates and the spatial characteristics of the Gaussian sphere distribution into the first fully connected layer of the multi-layer perceptron (MLP) network to output a camera pose feature vector and a Gaussian sphere distribution feature vector; concatenating the camera pose feature vector and the Gaussian sphere distribution feature vector and then inputting them into the second fully connected layer of the MLP network to output a scene feature; and using the activation function in the MLP network to perform activation processing on the scene feature to obtain the absorption rate attribute.

[0067] Figure 4 It is the schematic diagram of the MLP network in the embodiment of the present invention. The Gaussian ellipsoid mean and spherical harmonic function coefficients in the Gaussian sphere distribution feature vector are concatenated with the camera pose feature vector and then input into the MLP network to output the absorption rate attribute θ.

[0068] In this embodiment, enhancing the illumination of the Gaussian sphere distribution according to the absorption rate attribute to obtain the scene reconstruction view of the low-light image includes: projecting the Gaussian sphere distribution onto a two-dimensional plane and performing low-light hybrid rendering according to the absorption rate attribute to obtain a scene graph under low light; projecting the Gaussian sphere distribution onto a two-dimensional plane and performing enhanced illumination hybrid rendering on the Gaussian sphere distribution to obtain an enhanced illumination scene graph; projecting the Gaussian sphere distribution onto a two-dimensional plane and performing illumination intensity hybrid rendering according to the absorption rate attribute to obtain a scene illumination intensity graph; and integrating the scene graph under low light, the enhanced illumination scene graph, and the scene illumination intensity graph to obtain the scene reconstruction view of the low-light image.

[0069] Optionally, projecting the Gaussian sphere distribution onto a two-dimensional plane and performing low-light hybrid rendering according to the absorption rate attribute to obtain a scene graph under low light includes: calculating the scene graph under low light by using the following hybrid rendering formula : ; where is the color of each pixel in the scene graph under low light, N is the number of elements in the set of the Gaussian sphere distribution, i is an index representing the currently accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorption rate attribute, The color projected from the three-dimensional Gaussian sphere distribution i onto the two-dimensional imaging plane is a three-dimensional vector representing the values of the RGB three color channels. It is a value between 0 and 1, used to represent the contribution degree of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene graph under low light.

[0070] Optionally, project the Gaussian sphere distribution onto a two-dimensional plane and perform enhanced illumination mixing rendering on the Gaussian sphere distribution to obtain an enhanced illumination scene graph, including: calculating the enhanced illumination scene graph using the following mixing rendering formula : ; Wherein, is the color of each pixel in the enhanced illumination scene graph, N is the number of elements in the set of the Gaussian sphere distributions, i is the index, representing the currently accumulated three-dimensional Gaussian sphere distribution. The color projected from the three-dimensional Gaussian sphere distribution i onto the two-dimensional imaging plane is a three-dimensional vector representing the values of the RGB three color channels. It is a value between 0 and 1, used to represent the contribution degree of the three-dimensional Gaussian sphere distribution i to the pixel color in the enhanced illumination scene graph.

[0071] Optionally, project the Gaussian sphere distribution onto a two-dimensional plane and perform illumination intensity mixing rendering according to the absorption rate attribute to obtain a scene illumination intensity graph, including: calculating the scene illumination intensity graph using the following mixing rendering formula : ; Wherein, is the color of each pixel in the scene illumination intensity image, N is the number of elements in the set of the Gaussian sphere distributions, i is the index, representing the currently accumulated three-dimensional Gaussian sphere distribution. is the absorption coefficient of the absorption rate attribute. It is a value between 0 and 1, used to represent the contribution degree of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene illumination intensity graph.

[0072] Figure 5 This is a comparison schematic diagram of the image before and after being enhanced by illumination in the embodiments of the present invention, showing two scenarios of indoor and outdoor. After the illumination is enhanced, the details of the image are more obvious and rich.

[0073] By adopting the scheme of this embodiment, by calculating the camera pose of the low-light image, the position information and direction information of the camera during shooting can be determined, and the low-light image under different camera poses can be obtained. In addition, combined with the calculated sparse point cloud information of the scene, the geometric information during three-dimensional scene reconstruction can be obtained, thereby enhancing the accuracy of three-dimensional scene reconstruction under low light. In addition, by performing illumination enhancement on the three-dimensional reconstructed scene image, the reconstruction effect of the low-light image under low light can be further improved, so that image processing and three-dimensional reconstruction in a low-light environment can still obtain high-quality three-dimensional scene images even when the lighting conditions are not ideal, and can effectively retain the scene details of the low-light image.

[0074] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0075] Example 2 In this embodiment, a low-light image enhancement device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0076] Figure 6 is a structural block diagram of a low-light image enhancement device according to an embodiment of the present invention. Figure 6 As shown, the device comprises: A first acquisition module 60, configured to acquire a low-light image of a low-light scene, wherein the low-light image is an image with a brightness lower than a threshold; A second acquisition module 62, configured to acquire a camera pose and a scene sparse point cloud of the low-light image; A reconstruction module 64, configured to perform three-dimensional scene reconstruction on the low-light image using the camera pose and the scene sparse point cloud to obtain a Gaussian sphere distribution of the low-light image; A construction module 66 for constructing the absorption rate attribute of the Gaussian sphere distribution, where the absorption rate attribute is used to describe the absorption ability of an object to the radiation energy projected onto its surface; An enhancement module 68 for enhancing the illumination of the Gaussian sphere distribution according to the absorption rate attribute to obtain a scene reconstruction view of the low-light image.

[0077] Optionally, the second acquisition module includes: a first processing unit for performing image normalization processing on the low-light image to obtain a standard low-light image; a second processing unit for performing camera pose estimation on the standard low-light image to obtain a camera pose, and performing motion recovery on the objects in the standard low-light image to obtain a scene sparse point cloud.

[0078] Optionally, the reconstruction module includes: an initialization unit for initializing the scene sparse point cloud into a set of parameterized three-dimensional Gaussian basis elements representing a three-dimensional scene; a projection unit for projecting the parameterized three-dimensional Gaussian basis elements onto a two-dimensional imaging plane by using the camera pose to obtain a two-dimensional Gaussian distribution; a reconstruction unit for performing three-dimensional scene reconstruction on the two-dimensional Gaussian distribution to obtain the Gaussian sphere distribution of the low-light image.

[0079] Optionally, the reconstruction unit includes: a determination subunit for determining the depth information of the two-dimensional Gaussian distribution in three-dimensional space; a sorting subunit for sorting the two-dimensional Gaussian distribution on the imaging plane according to the depth information to obtain the Gaussian sphere distribution of the low-light image.

[0080] Optionally, the construction module includes: an acquisition unit for acquiring the Gaussian center coordinates of the Gaussian sphere distribution; a conversion unit for converting the Gaussian center coordinates into a high-dimensional vector to obtain the spatial feature of the Gaussian sphere distribution; a construction unit for constructing the absorption rate attribute of the Gaussian sphere distribution by using the Gaussian center coordinates and the spatial feature of the Gaussian sphere distribution.

[0081] Optionally, the construction unit includes: a first processing unit for inputting the Gaussian center coordinates and the spatial feature of the Gaussian sphere distribution into the first fully connected layer of the first part of the multi-layer perceptron MLP network, and outputting a camera pose feature vector and a Gaussian sphere distribution feature vector; a second processing unit for concatenating the camera pose feature vector and the Gaussian sphere distribution feature vector and inputting them into the second fully connected layer of the MLP network, and outputting a scene feature; an activation unit for activating the scene feature by using the activation function in the MLP network to obtain the absorption rate attribute.

[0082] Optionally, the enhancement module includes: a first rendering unit configured to project the Gaussian sphere distribution onto a two-dimensional plane and perform low-light mixing rendering according to the absorption rate attribute to obtain a scene graph under low light; a second rendering unit configured to project the Gaussian sphere distribution onto a two-dimensional plane and perform enhanced-light mixing rendering on the Gaussian sphere distribution to obtain an enhanced-light scene graph; a third rendering unit configured to project the Gaussian sphere distribution onto a two-dimensional plane and perform light intensity mixing rendering according to the absorption rate attribute to obtain a scene light intensity graph; and an integration unit configured to integrate the scene graph under low light, the enhanced-light scene graph, and the scene light intensity graph to obtain a scene reconstruction view of the low-light image.

[0083] Optionally, the first rendering unit includes: a calculation subunit configured to calculate the scene graph under low light by using the following mixing rendering formula : ; where is the color of each pixel in the scene graph under low light, N is the number of elements in the set of the Gaussian sphere distribution, i is an index representing the currently accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorption rate attribute, is the color of the three-dimensional Gaussian sphere distribution i projected onto the two-dimensional imaging plane, which is a three-dimensional vector representing the values of the RGB three color channels, is a value between 0 and 1, which is used to represent the contribution degree of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene graph under low light.

[0084] Optionally, the second rendering unit includes: a calculation subunit configured to calculate the enhanced-light scene graph by using the following mixing rendering formula : ; where is the color of each pixel in the enhanced-light scene graph, N is the number of elements in the set of the Gaussian sphere distribution, i is an index representing the currently accumulated three-dimensional Gaussian sphere distribution, is the color of the three-dimensional Gaussian sphere distribution i projected onto the two-dimensional imaging plane, which is a three-dimensional vector representing the values of the RGB three color channels, is a value between 0 and 1, which is used to represent the contribution degree of the three-dimensional Gaussian sphere distribution i to the pixel color in the enhanced-light scene graph.

[0085] Optionally, the third rendering unit includes: a calculation subunit configured to calculate the scene light intensity graph by using the following mixing rendering formula : ; where is the color of each pixel in the scene light intensity image, N is the number of elements in the set of the Gaussian sphere distribution, i is an index representing the currently accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorption rate attribute, is a value between 0 and 1, and is used to represent the contribution degree of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene illumination intensity map.

[0086] It should be noted that the above-mentioned various modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above-mentioned modules are all located in the same processor; or, the above-mentioned various modules are respectively located in different processors in any combination form.

[0087] Embodiment 3 The embodiment of the present invention also provides a storage medium, in which a computer program is stored, and the computer program is set to execute the steps in any one of the above method embodiments when running.

[0088] Optionally, in this embodiment, the above storage medium can be set to store a computer program for executing the following steps: S1. Obtain a low-light image of a low-light scene, where the low-light image is an image with a brightness lower than a threshold; S2. Obtain the camera pose and the scene sparse point cloud of the low-light image; S3. Perform three-dimensional scene reconstruction on the low-light image by using the camera pose and the scene sparse point cloud to obtain the Gaussian sphere distribution of the low-light image; S4. Construct the absorption rate attribute of the Gaussian sphere distribution, where the absorption rate attribute is used to describe the absorption ability of an object to the radiation energy projected onto its surface; S5. Perform illumination enhancement on the Gaussian sphere distribution according to the absorption rate attribute to obtain the scene reconstruction view of the low-light image.

[0089] Optionally, in this embodiment, the above storage medium may include but is not limited to: various media such as USB flash drives, read-only memories (ROM for short), random access memories (RAM for short), mobile hard disks, magnetic disks, or optical discs that can store computer programs.

[0090] The embodiment of the present invention also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is set to run the computer program to execute the steps in any one of the above method embodiments.

[0091] Optionally, the above electronic device may further include a transmission device and an input / output device, where the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0092] Optionally, in this embodiment, the above-mentioned processor may be configured to execute the following steps through a computer program: S1. Obtain a low-light image of a low-light scene, where the low-light image is an image with a brightness lower than a threshold; S2. Obtain the camera pose and the scene sparse point cloud of the low-light image; S3. Perform three-dimensional scene reconstruction on the low-light image by using the camera pose and the scene sparse point cloud to obtain the Gaussian sphere distribution of the low-light image; S4. Construct the absorption rate attribute of the Gaussian sphere distribution, where the absorption rate attribute is used to describe the absorption ability of an object to the radiation energy projected onto its surface; S5. Enhance the illumination of the Gaussian sphere distribution according to the absorption rate attribute to obtain the scene reconstruction view of the low-light image.

[0093] Optionally, specific examples in this embodiment may refer to the examples described in the above-mentioned embodiments and optional implementation manners, and will not be elaborated herein.

[0094] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0095] Through the description of the above implementation manners, those skilled in the art can clearly understand that each implementation manner can be implemented by means of software plus a general hardware platform, and of course, can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the related technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0096] It should be understood that the terms used herein are for the purpose of describing particular example embodiments only and are not intended to be limiting. Unless the context clearly dictates otherwise, the singular forms "a", "an", and "the" as used herein may also include the plural forms. The terms "comprising", "including", "containing", and "having" are inclusive and thus specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring them to be performed in the particular order described or illustrated, unless the order of performance is expressly stated. It should also be understood that additional or alternative steps may be used.

[0097] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for enhancing a low-light image, characterized in that: include: Acquire a low-light image of a low-light scene, wherein the low-light image is an image with brightness lower than a threshold; Obtaining a camera pose and a scene sparse point cloud of the low-light image; Reconstructing the low-light image in three dimensions using the camera pose and the scene sparse point cloud to obtain a Gaussian sphere distribution of the low-light image; Constructing an absorptivity attribute of the Gaussian sphere distribution, wherein the absorptivity attribute is used to describe the absorption capability of an object to radiation energy projected onto its surface; The Gaussian sphere distribution is subjected to illumination enhancement according to the absorption rate attribute to obtain a scene reconstruction view of the low-light image.

2. The method according to claim 1, characterized in that Obtaining the camera pose and scene sparse point cloud of the low-light image includes: Performing image standardization processing on the low-light image to obtain a standard low-light image; The camera pose is estimated on the standard low-light image to obtain the camera pose, and the motion of the object in the standard low-light image is restored to obtain a scene sparse point cloud.

3. The method according to claim 1, characterized in that The three-dimensional scene reconstruction of the low-light image is performed using the camera posture and the scene sparse point cloud to obtain a Gaussian sphere distribution of the low-light image, including: Initializing the scene sparse point cloud into a set of parameterized three-dimensional Gaussian primitives representing the three-dimensional scene; Projecting the parameterized three-dimensional Gaussian basis element onto a two-dimensional imaging plane using the camera pose to obtain a two-dimensional Gaussian distribution; A three-dimensional scene reconstruction is performed on the two-dimensional Gaussian distribution to obtain a Gaussian sphere distribution of the low-light image.

4. The method according to claim 3, characterized in that Performing three-dimensional scene reconstruction on the two-dimensional Gaussian distribution to obtain the Gaussian sphere distribution of the low-light image includes: Determining depth information of the two-dimensional Gaussian distribution in three-dimensional space; The two-dimensional Gaussian distribution is sorted on an imaging plane according to the depth information to obtain a Gaussian sphere distribution of the low-light image.

5. The method according to claim 1, characterized in that The absorptivity properties for constructing the Gaussian sphere distribution include: Obtain the Gaussian center coordinates of the Gaussian sphere distribution; Convert the Gaussian center coordinates into a high-dimensional vector to obtain Gaussian sphere distribution space characteristics; The Gaussian center coordinates and the Gaussian sphere distribution spatial characteristics are used to construct the absorption rate attribute of the Gaussian sphere distribution.

6. The method according to claim 5, characterized in that The absorption rate attribute of the Gaussian sphere distribution is constructed by using the Gaussian center coordinates and the Gaussian sphere distribution spatial characteristics, including: Input the Gaussian center coordinates and the Gaussian sphere distribution spatial features into the fully connected layer of the first part of the multi-layer perceptron MLP network, and output the camera pose feature vector and the Gaussian sphere distribution feature vector; The camera pose feature vector and the Gaussian sphere distribution feature vector are concatenated and input into the second fully connected layer of the MLP network to output scene features; The scene feature is activated using an activation function in the MLP network to obtain the absorption rate attribute.

7. The method according to claim 1, characterized in that Performing illumination enhancement on the Gaussian sphere distribution according to the absorption rate attribute to obtain a scene reconstruction view of the low-light image includes: Projecting the Gaussian sphere distribution onto a two-dimensional plane, and performing low-light mixed rendering according to the absorption rate attribute to obtain a low-light scene image; Projecting the Gaussian sphere distribution onto a two-dimensional plane, and performing enhanced illumination mixed rendering on the Gaussian sphere distribution to obtain an enhanced illumination scene graph; The Gaussian sphere distribution is projected onto a two-dimensional plane, and light intensity mixed rendering is performed according to the absorption rate attribute to obtain a scene light intensity map; The low-light scene graph, the enhanced-light scene graph, and the scene light intensity graph are integrated to obtain a scene reconstruction view of the low-light image.

8. The method according to claim 7, characterized in that The Gaussian sphere distribution is projected onto a two-dimensional plane, and low-light mixed rendering is performed according to the absorption rate attribute to obtain a low-light scene graph including: The following mixed rendering formula is used to calculate the scene map under low light : ； in, is the color of each pixel in the scene image under low light, N is the number of elements in the set of Gaussian sphere distribution, i is the index, representing the current accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorptivity property, It is the color projected onto the two-dimensional imaging plane by the three-dimensional Gaussian ball distribution i. It is a three-dimensional vector representing the values ​​of the three RGB color channels. is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene image under low light conditions.

9. The method according to claim 7, characterized in that: The Gaussian sphere distribution is projected onto a two-dimensional plane, and enhanced illumination mixed rendering is performed on the Gaussian sphere distribution to obtain an enhanced illumination scene graph, including: The enhanced lighting scene graph is calculated using the following hybrid rendering formula: : ； in, To enhance the color of each pixel in the illumination scene image, N is the number of elements in the set of Gaussian sphere distribution, i is the index, representing the current accumulated three-dimensional Gaussian sphere distribution, It is the color projected onto the two-dimensional imaging plane by the three-dimensional Gaussian ball distribution i. It is a three-dimensional vector representing the values ​​of the three RGB color channels. is a value between 0 and 1 that indicates the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the enhanced lighting scene graph.

10. The method according to claim 7, characterized in that The Gaussian sphere distribution is projected onto a two-dimensional plane, and illumination intensity mixed rendering is performed according to the absorption rate attribute to obtain a scene illumination intensity map, including: The scene illumination intensity map is calculated using the following hybrid rendering formula: : ； in, is the color of each pixel in the scene illumination intensity image, N is the number of elements in the set of Gaussian sphere distribution, i is the index, representing the current accumulated three-dimensional Gaussian sphere distribution, is the absorption coefficient of the absorptivity property, is a value between 0 and 1 that represents the contribution of the three-dimensional Gaussian sphere distribution i to the pixel color in the scene light intensity map.

11. A low-light image enhancement device, characterized in that: include: A first acquisition module, configured to acquire a low-light image of a low-light scene, wherein the low-light image is an image with a brightness lower than a threshold; A second acquisition module, used to acquire the camera pose and scene sparse point cloud of the low-light image; A reconstruction module, configured to perform three-dimensional scene reconstruction on the low-light image using the camera pose and the scene sparse point cloud to obtain a Gaussian sphere distribution of the low-light image; A construction module, used to construct an absorptivity attribute of the Gaussian sphere distribution, wherein the absorptivity attribute is used to describe the absorption capacity of an object to radiation energy projected onto its surface; An enhancement module is used to perform illumination enhancement on the Gaussian sphere distribution according to the absorption rate attribute to obtain a scene reconstruction view of the low-light image.

12. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 10 when executed.

13. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 10.

Images