Weak light image enhancement method and device, equipment and storage medium

Abstract

The invention discloses a weak light image enhancement method and device, equipment and a storage medium, and relates to the technical field of image enhancement, and the method comprises the steps: obtaining a to-be-enhanced image and a target normal illumination image, and optimizing an initial image enhancement model to obtain a target image enhancement model; performing feature extraction and manifold distribution estimation on the target normal illumination image by using the initial image enhancement model to obtain first distribution manifold information; performing feature extraction and manifold distribution estimation on the to-be-enhanced image by using the target image enhancement model to obtain second distribution estimation information; fusing the first distribution manifold information and the second distribution estimation information by using a target image enhancement model, and establishing a mapping relation between target high-dimensional feature information vectors and feature information vectors by using the target image enhancement model based on a fused target high-dimensional manifold information space; and obtaining a target projection and target manifold distribution according to the mapping relationship, and decoding the target manifold distribution by using a decoder to obtain a target image enhancement result. And image adaptive enhancement is realized.

Description

Technical Field

[0001] The present invention relates to the field of image enhancement technology, and in particular to a method, device, equipment and storage medium for enhancing a weak-light image. Background Art

[0002] Current low-light image enhancement methods can be broadly categorized into those based on traditional image processing and those based on deep learning. Traditional techniques typically improve image quality by enhancing overall image brightness, adjusting contrast, or enhancing local details. However, these techniques are susceptible to noise, which can lead to loss of image detail and degradation of image quality. On the other hand, deep learning-based low-light image enhancement methods have made significant progress in recent years, leveraging large-scale datasets and complex network structures to achieve a combined enhancement of image brightness and detail.

[0003] Although artificial intelligence has achieved significant breakthroughs in various fields using algorithmic models, low-light images are often captured with high levels of noise, making it difficult to accurately restore the natural light field distribution within the image. Most enhancement methods focus solely on improving overall brightness while neglecting the fidelity of reconstructing local lighting features, resulting in a loss of realism in the enhanced image. Furthermore, lighting conditions in low-light environments are complex and varied, potentially including point sources, diffuse light, reflected light, and other scenarios. Existing methods are typically optimized for specific lighting scenarios, making it difficult to achieve adaptive processing for a variety of lighting conditions.

[0004] In summary, how to suppress noise interference in complex low-light environments, truly restore the natural light field distribution, and achieve adaptive enhancement for various lighting conditions is an urgent problem that needs to be solved. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a low-light image enhancement method, device, equipment, and storage medium that can suppress noise interference in complex low-light environments, truly restore the natural light field distribution, and achieve adaptive enhancement under various lighting conditions. The specific solution is as follows:

[0006] In a first aspect, the present application provides a method for enhancing a low-light image, comprising:

[0007] Acquire an image to be enhanced in a preset weak light scene and a target normal light image in a preset normal light scene, construct an initial image enhancement model, and optimize the initial image enhancement model to obtain a target image enhancement model;

[0008] Inputting the target normal illumination image into the initial image enhancement model, and performing preset feature extraction and manifold distribution estimation operations on the target normal illumination image using the initial image enhancement model to obtain first distribution manifold information;

[0009] Inputting the image to be enhanced into the target image enhancement model, and performing preset feature extraction and manifold distribution estimation operations on the image to be enhanced using the target image enhancement model to obtain second distribution estimation information;

[0010] Using the target image enhancement model to fuse the first distribution manifold information and the second distribution estimation information to determine a target high-dimensional manifold information space, and based on the target high-dimensional manifold information space and using the target image enhancement model, establish a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a corresponding feature information vector in a preset label space;

[0011] According to the mapping relationship, the target projection corresponding to the image to be enhanced is obtained based on preset conditions, the target manifold distribution that meets the preset feature distribution conditions is determined based on the target projection corresponding to the image to be enhanced, and the target manifold distribution is decoded using the decoder in the target image enhancement model to obtain the target image enhancement result.

[0012] Optionally, the step of acquiring the image to be enhanced in a preset weak light scene and the target normal light image in a preset normal light scene includes:

[0013] Using a preset image acquisition device to perform preset periodic imaging acquisition operations on a target to be imaged in a preset weak light scene and a preset normal light scene based on a preset frequency, so as to obtain a weak light image set in the preset weak light scene and a normal light image set in the preset normal light scene;

[0014] Based on a preset image acquisition requirement, an image to be enhanced and a target normal-light image are determined from the low-light image set and the normal-light image set.

[0015] Optionally, the image enhancement model includes a backbone network encoder module, a feature manifold estimation and establishment module, and a feature map selection module.

[0016] Optionally, performing preset feature extraction and manifold distribution estimation operations on the image to be enhanced by using the target image enhancement model to obtain second distribution estimation information includes:

[0017] Using the backbone network encoder module in the target image enhancement model, a preset overall illumination coverage map estimation operation is performed on the image to be enhanced to obtain illumination distribution characteristics of the image to be enhanced;

[0018] Inputting the illumination distribution feature into the deep coding layer of the backbone network encoder module, and performing light field distribution estimation and analysis on the backbone network encoder module to obtain hierarchical distribution information of the image to be enhanced;

[0019] Second distribution estimation information of the image to be enhanced is obtained based on the illumination distribution feature and the hierarchical distribution information.

[0020] Optionally, the using the target image enhancement model to fuse the first distribution manifold information with the second distribution estimation information to determine a target high-dimensional manifold information space includes:

[0021] Using the first loss function and the second loss function of the feature manifold estimation and establishment module in the target image enhancement model, based on a preset dimension, a preset information interactive fusion operation is performed on the first distribution manifold information and the second distribution estimation information to establish a target high-dimensional manifold information space;

[0022] The first loss function is used to determine the information features of the image to be enhanced, and the second loss function is used to fuse the label features of the target normal illumination image with the image to be enhanced.

[0023] Optionally, the establishing a mapping relationship between the target high-dimensional feature information vector in the preset sample space and the corresponding feature information vector in the preset label space by using the target image enhancement model based on the target high-dimensional manifold information space includes:

[0024] Utilizing the feature manifold estimation and establishment module of the target image enhancement model, sampling and estimating the image to be enhanced based on the illumination distribution characteristics of the image to be enhanced, so as to obtain a corresponding target high-dimensional manifold estimation result;

[0025] Training a target operator based on the target high-dimensional manifold estimation result, optimizing the parameters of the target operator in combination with preset operator constraints to obtain an optimized target operator, and constructing a mapping function according to the optimized target operator;

[0026] According to the mapping function, a mapping relationship is established between the target high-dimensional feature information vector of the preset sample space and the feature information vector of the preset label space in the target high-dimensional manifold information space.

[0027] Optionally, acquiring a target projection corresponding to the image to be enhanced based on a preset condition according to the mapping relationship, and determining a target manifold distribution that satisfies a preset feature distribution condition based on the target projection corresponding to the image to be enhanced, includes:

[0028] performing a preset data processing operation on the high-dimensional manifold information in the target high-dimensional manifold information space using the third loss function and the fourth loss function in the feature map selection module to obtain processed high-dimensional manifold information;

[0029] Projecting the processed high-dimensional manifold information onto a preset low-dimensional space according to the mapping relationship to generate a corresponding projection set;

[0030] Acquire a target projection corresponding to the image to be enhanced from the projection set according to a preset condition;

[0031] Determining a target manifold distribution that satisfies a preset image similarity condition based on the target projection corresponding to the image to be enhanced;

[0032] Among them, the third loss function is used to eliminate information that meets the preset redundancy condition in the high-dimensional manifold information of the target high-dimensional manifold information space, and the fourth loss function is used to expand the high-dimensional manifold information of the target high-dimensional manifold information space.

[0033] In a second aspect, the present application provides a low-light image enhancement device, comprising:

[0034] The model acquisition module is used to acquire the image to be enhanced in a preset weak light scene and the target normal light image in a preset normal light scene, construct an initial image enhancement model, and optimize the initial image enhancement model to obtain the target image enhancement model;

[0035] a first information acquisition module, configured to input the target normal illumination image into the initial image enhancement model, and perform preset feature extraction and manifold distribution estimation operations on the target normal illumination image using the initial image enhancement model to obtain first distribution manifold information;

[0036] a second information acquisition module, configured to input the image to be enhanced into the target image enhancement model, and perform preset feature extraction and manifold distribution estimation operations on the image to be enhanced using the target image enhancement model to obtain second distribution estimation information;

[0037] a mapping relationship establishment module, configured to use the target image enhancement model to fuse the first distribution manifold information with the second distribution estimation information to determine a target high-dimensional manifold information space, and to establish a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a corresponding feature information vector in a preset label space using the target image enhancement model based on the target high-dimensional manifold information space;

[0038] An enhancement result acquisition module is used to obtain the target projection corresponding to the image to be enhanced based on preset conditions according to the mapping relationship, determine the target manifold distribution that meets the preset feature distribution conditions based on the target projection corresponding to the image to be enhanced, and use the decoder in the target image enhancement model to decode the target manifold distribution to obtain the target image enhancement result.

[0039] In a third aspect, the present application provides an electronic device, comprising:

[0040] Memory, used to store computer programs;

[0041] The processor is used to execute the computer program to implement the aforementioned weak-light image enhancement method.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned low-light image enhancement method is implemented.

[0043] In summary, the present application obtains an image to be enhanced in a preset weak light scene and a target normal lighting image in a preset normal lighting scene, constructs an initial image enhancement model, and optimizes the initial image enhancement model to obtain a target image enhancement model; the target normal lighting image is input into the initial image enhancement model, and the initial image enhancement model is used to perform preset feature extraction and manifold distribution estimation operations on the target normal lighting image to obtain first distribution manifold information; then, the image to be enhanced is input into the target image enhancement model, and the target image enhancement model is used to perform preset feature extraction and manifold distribution estimation operations on the image to be enhanced to obtain second distribution estimation information; The target image enhancement model is used to fuse the first distribution manifold information and the second distribution estimation information to determine the target high-dimensional manifold information space. Based on the target high-dimensional manifold information space, the target image enhancement model is used to establish a mapping relationship between the target high-dimensional feature information vector in the preset sample space and the corresponding feature information vector in the preset label space; according to the mapping relationship, the target projection corresponding to the image to be enhanced is obtained based on preset conditions, and the target manifold distribution that meets the preset feature distribution conditions is determined based on the target projection corresponding to the image to be enhanced. The target manifold distribution is decoded using the decoder in the target image enhancement model to obtain the target image enhancement result. As can be seen from the above, the present application obtains a preset weak light scene image to be enhanced and a target image of a normal lighting scene, constructs and optimizes an initial image enhancement model to obtain a target image enhancement model; the target normal lighting image is input into the initial model to extract features and estimate the manifold distribution to obtain first distribution manifold information, and the image to be enhanced is input into the target image enhancement model to extract features and estimate the manifold distribution to obtain second distribution estimation information; the target image enhancement model is used to fuse the two types of distribution information to determine the target high-dimensional manifold information space, and based on this space, a mapping relationship between the preset sample space target high-dimensional feature information vector and the preset label space corresponding feature information vector is established in the target image enhancement model; according to the mapping relationship, the target projection corresponding to the image to be enhanced is obtained according to the preset conditions, and the target manifold distribution that meets the preset feature distribution conditions is determined accordingly, and the target manifold distribution is decoded by the target image enhancement model decoder to obtain the target image enhancement result. In this way, by constructing and optimizing the image enhancement model, combining the feature extraction, manifold distribution estimation and fusion of multi-scene images, and cross-space feature vector mapping and manifold distribution decoding, it is possible to optimize the quality of low-light images. By optimizing the features and adjusting the manifold distribution of low-light images, the model can effectively improve the image brightness, contrast and detail clarity to improve the visual effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0045] Figure 1 This is a flow chart of a weak light image enhancement method disclosed in this application;

[0046] Figure 2 This is a schematic diagram of a loss function guiding feature interaction fusion disclosed in this application;

[0047] Figure 3 A schematic diagram of an image to be enhanced, a target normal illumination image, and a target image enhancement result disclosed in this application;

[0048] Figure 4 A schematic diagram of a high-dimensional manifold establishment module and a feature map selection module disclosed in this application;

[0049] Figure 5 This is a schematic structural diagram of a weak light image enhancement device disclosed in this application;

[0050] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0052] At present, low-light image enhancement methods can be mainly divided into methods based on traditional image processing and methods based on deep learning. Technologies based on traditional methods usually improve image quality by enhancing overall image brightness, adjusting contrast or enhancing local details, but are easily interfered by noise, resulting in loss of image details and quality degradation. On the other hand, low-light image enhancement methods based on deep learning have made significant progress in recent years, and have achieved joint enhancement of image brightness and details by utilizing large-scale data sets and complex network structures. In order to solve the above technical problems, the present application discloses a low-light image enhancement method, device, equipment and storage medium, which can suppress noise interference in complex low-light environments, truly restore the distribution of natural light fields, and achieve adaptive enhancement of various lighting conditions.

[0053] See also Figure 1 As shown, an embodiment of the present invention discloses a method for enhancing a weak light image, comprising:

[0054] Step S11: obtaining an image to be enhanced in a preset weak light scene and a target normal light image in a preset normal light scene, constructing an initial image enhancement model, and optimizing the initial image enhancement model to obtain a target image enhancement model.

[0055] In this embodiment, in order to obtain an image to be enhanced in a preset weak light scene of a preset size and a target normal light image in a preset normal light scene, a preset image acquisition device can be used to perform preset periodic imaging acquisition operations on the target to be imaged in the preset weak light scene and the preset normal light scene based on a preset frequency to obtain a weak light image set in the preset weak light scene and a normal light image set in the preset normal light scene; based on the preset image acquisition requirements, the image to be enhanced and the target normal light image are determined from the weak light image set and the normal light image set. Specifically, a preset image acquisition device is used to perform preset periodic imaging acquisition on the imaging target at a preset frequency in a preset weak light scene and a normal light scene to obtain a weak light image set and a normal light image set. The images are screened according to the preset image acquisition requirements. For example, the signal-to-noise ratio, contrast, motion blur and other indicators are calculated for the weak light image set, and images with a signal-to-noise ratio below a threshold and a low contrast are selected as images to be enhanced; for the normal light image set, images with a high signal-to-noise ratio, high contrast, no obvious blur and exposure abnormalities, and which are spatially and temporally aligned with the images to be enhanced are selected as target normal light images.

[0056] In addition, there are two ways to obtain environmental images in indoor and outdoor scenes. If you need to obtain the image to be enhanced and the target normal lighting image in an indoor scene, you can control the indoor lighting to perform imaging acquisition and label image acquisition on the indoor image to obtain a dark light image and a good lighting image in a weak light scene consistent with the target to be enhanced, wherein the layout of the indoor scene remains consistent under various lighting conditions. If you need to obtain the image to be enhanced and the target normal lighting image in an outdoor scene, you can use a capture camera to perform imaging acquisition of the target to be enhanced according to preset parameters in the same outdoor scene at different time periods to obtain target normal lighting images in various lighting scenes consistent with the image to be enhanced.

[0057] It can be understood that after obtaining the image to be enhanced and the target normal lighting image, an initial image enhancement model is constructed, and then the parameters in the initial image enhancement model are optimized to obtain the target image enhancement model. Among them, both the initial image enhancement model and the target image enhancement model include a backbone network encoder module, a feature manifold estimation and establishment module, and a feature mapping selection module.

[0058] Step S12: input the target normal illumination image into the initial image enhancement model, and use the initial image enhancement model to perform preset feature extraction and manifold distribution estimation operations on the target normal illumination image to obtain first distribution manifold information.

[0059] In this embodiment, the target normal illumination image is input into the initial image enhancement model. The backbone network encoder module in the initial image enhancement model first extracts features of the target normal illumination image based on the hierarchical structure of the convolutional neural network through convolution kernels of different sizes and step sizes, such as basic visual features such as edges, textures, and shapes, as well as advanced features at the semantic level. At the same time, manifold distribution estimation is performed. The distribution morphology of the extracted features in the high-dimensional space can be analyzed and modeled by using a manifold learning algorithm based on deep learning, such as a variational autoencoder or a related mechanism in a generative adversarial network. By calculating statistics such as similarity and distance metrics between features, the distribution law of the feature data in the manifold space is estimated, and finally the first distribution manifold information that can characterize the feature distribution characteristics of the target normal illumination image is obtained.

[0060] Step S13: input the image to be enhanced into the target image enhancement model, and use the target image enhancement model to perform preset feature extraction and manifold distribution estimation operations on the image to be enhanced to obtain second distribution estimation information.

[0061] In this embodiment, the image to be enhanced is input into the target image enhancement model, and the backbone network encoder module in the target image enhancement model is used to perform a preset overall illumination coverage map estimation operation on the image to be enhanced to obtain illumination distribution characteristics of the image to be enhanced; the illumination distribution characteristics are input into the deep network coding layer of the backbone network encoder module, and the backbone network encoder module is subjected to light field distribution estimation and analysis to obtain hierarchical distribution information of the image to be enhanced; and second distribution estimation information of the image to be enhanced is obtained based on the illumination distribution characteristics and the hierarchical distribution information. Specifically, the shallow coding layer of the target image enhancement model is used to perform illumination coverage map estimation on the image to be enhanced to generate illumination distribution characteristics, and the illumination distribution characteristics and the characteristics of the image to be enhanced are combined and input into the deep coding layer to perform light field distribution analysis to obtain distribution information of each level, and the encoder is used to perform distribution estimation on the distribution information of each level to generate feature manifold distribution estimation information of the image to be enhanced.

[0062] Step S14: Use the target image enhancement model to fuse the first distribution manifold information and the second distribution estimation information to determine the target high-dimensional manifold information space, and use the target image enhancement model based on the target high-dimensional manifold information space to establish a mapping relationship between the target high-dimensional feature information vector in the preset sample space and the corresponding feature information vector in the preset label space.

[0063] In this embodiment, after obtaining the first distribution estimation information and the second distribution estimation information, it is necessary to use the first loss function and the second loss function of the feature manifold estimation and establishment module in the target image enhancement model to perform a preset information interactive fusion operation on the first distribution manifold information and the second distribution estimation information based on a preset dimension to establish a target high-dimensional manifold information space; wherein, the first loss function is used to determine the information features of the image to be enhanced, and the second loss function is used to fuse the label features of the target normal illumination image with the image to be enhanced. In addition, the feature manifold estimation and establishment module of the target image enhancement model is used to perform sampling estimation on the image to be enhanced based on the illumination distribution features of the image to be enhanced to obtain a corresponding target high-dimensional manifold estimation result; based on the target high-dimensional manifold estimation result, the target operator is trained, and the parameters of the target operator are optimized in combination with preset operator constraints to obtain an optimized target operator, and a mapping function is constructed based on the optimized target operator; according to the mapping function, a mapping relationship is established in the target high-dimensional manifold information space between the target high-dimensional feature information vector of the preset sample space and the feature information vector of the preset label space.

[0064] Specifically, after obtaining the first distribution estimation information and the second distribution estimation information, as shown in FIG. Figure 2 As shown, the first loss function and the second loss function of the feature manifold estimation and establishment module in the target image enhancement model are used to interactively fuse the first distribution manifold information and the second distribution estimation information based on the preset dimension to establish a target high-dimensional manifold information space. It should be pointed out that when performing information interactive fusion, the feature manifold estimation and establishment module is used to perform sampling estimation based on the illumination distribution characteristics of the image to be enhanced to obtain the target high-dimensional manifold estimation result. Then, based on the result, the target operator is trained, and the operator parameters are optimized in combination with the preset operator constraints to obtain the optimized target operator and construct a mapping function. According to the mapping function, a mapping relationship between the sample space feature vector and the label space feature vector is established in the target high-dimensional manifold information space.

[0065] The first loss function is used to redefine the information features of the image to be enhanced on a high-dimensional manifold, and the second loss function is used to fuse the label feature information of the target normal illumination image. For example, the first loss function is the LPSR (Loss of Particular Solution Reconstruction Optimization) loss function, and its specific form is:

[0066] ;

[0067] in, is a metric representation; is the model function; are model parameters; x is the prior target normal light image and its corresponding feature input; min is the minimum value of the loss function; is the output result obtained after the input image x passes through the network with parameter θ.

[0068] The second loss function is the ManiLoss (Loss of Manifold) loss function, which is the manifold information integration and interaction optimization loss. Its specific form is:

[0069] ;

[0070] in, is a metric representation; is the model function; x is the prior target normal light image and its corresponding feature input; min is the minimum value of the loss function; z is the dark light image and its corresponding feature input.

[0071] Step S15: obtaining a target projection corresponding to the image to be enhanced based on preset conditions according to the mapping relationship, determining a target manifold distribution that meets preset feature distribution conditions based on the target projection corresponding to the image to be enhanced, and decoding the target manifold distribution using a decoder in the target image enhancement model to obtain a target image enhancement result.

[0072] In this embodiment, the third loss function and the fourth loss function in the feature map selection module are used to perform preset data processing operations on the high-dimensional manifold information in the target high-dimensional manifold information space to obtain processed high-dimensional manifold information; the processed high-dimensional manifold information is projected onto a preset low-dimensional space according to the mapping relationship to generate a corresponding projection set; the target projection corresponding to the image to be enhanced is obtained from the projection set according to preset conditions; and the target manifold distribution that meets the preset image similarity condition is determined based on the target projection corresponding to the image to be enhanced; wherein the third loss function is used to eliminate information that meets the preset redundant condition in the high-dimensional manifold information in the target high-dimensional manifold information space, and the fourth loss function is used to expand the high-dimensional manifold information in the target high-dimensional manifold information space. Specifically, the feature map selection module is used to project the high-dimensional manifold in the high-dimensional manifold information space onto a low-dimensional distribution based on the third loss function and the fourth loss function, and the target projection closest to the low-light image is determined from the generated projections according to preset judgment conditions. The third loss function is used to remove information that meets a preset redundancy condition from the high-dimensional manifold information in the target high-dimensional manifold information space, and the fourth loss function is used to expand the high-dimensional manifold information in the target high-dimensional manifold information space. For example, the third loss function is an LPX (Loss of projection to X) loss function, whose specific form is:

[0073] ;

[0074] in, is a metric representation; is the model function; are model parameters; is a frozen parameter, that is, no gradient backpropagation is performed; z is a dark and low-light image and its corresponding feature input; Is the function identifier.

[0075] The fourth loss function can be the LTM (Loss of tighten manifold) loss function, and its specific form is:

[0076] ;

[0077] in, is a metric representation; is the model function; are model parameters; is a frozen parameter, that is, no gradient backpropagation is performed; z is a dark and low-light image and its corresponding feature input; Is the function identifier.

[0078] Furthermore, the feature mapping selection module is used to map and associate the high-dimensional feature vector with the low-dimensional feature vector of the target label to obtain the associated features. The decoder in the target image enhancement model performs corresponding sampling decoding on the obtained target manifold distribution to obtain the following: Figure 3 The target image enhancement result corresponding to the image to be enhanced is shown in FIG. In this way, the backbone network of the image enhancement model can enhance the image to be enhanced by feature fusion and feature mapping of the input features under the estimated distribution.

[0079] As can be seen from the above, the embodiment of the present application obtains a preset weak light scene image to be enhanced and a normal lighting scene target image, constructs and optimizes an initial image enhancement model to obtain a target image enhancement model; the target normal lighting image is input into the initial model to extract features and estimate the manifold distribution to obtain first distribution manifold information, and the image to be enhanced is input into the target image enhancement model to extract features and estimate the manifold distribution to obtain second distribution estimation information; the target image enhancement model is used to fuse the two types of distribution information to determine the target high-dimensional manifold information space, and based on this space, a mapping relationship between the preset sample space target high-dimensional feature information vector and the preset label space corresponding feature information vector is established in the target image enhancement model; according to the mapping relationship, the target projection corresponding to the image to be enhanced is obtained according to the preset conditions, and the target manifold distribution that meets the preset feature distribution conditions is determined accordingly, and the target manifold distribution is decoded by the target image enhancement model decoder to obtain the target image enhancement result. In this way, by constructing and optimizing the image enhancement model, combining the feature extraction, manifold distribution estimation and fusion of multi-scene images, and cross-space feature vector mapping and manifold distribution decoding, it is possible to optimize the quality of low-light images. By optimizing the features and adjusting the manifold distribution of low-light images, the model can effectively improve the image brightness, contrast and detail clarity to improve the visual effect.

[0080] Based on the previous embodiment, this application discloses a method for enhancing low-light images, which can suppress noise interference, truly restore the natural light field distribution in complex low-light environments, and achieve adaptive enhancement under various lighting conditions. Figure 4 The low-light image enhancement method shown is described in detail.

[0081] In the results section, we propose a method for circumventing noise to locate the image features to be enhanced, Sj, while also optimizing the mapping path from the target features Dj to Sj. To further improve the mapping from Dj to Sj, we aim to construct a high-dimensional manifold information space that integrates information from the target and input manifolds. This can be divided into four main parts: the first part is manifold modeling, the second part is the process of constructing the feature distribution on the manifold, the third part is the construction of the manifold topological properties, and the fourth part is the optimization construction of Lie groups.

[0082] The first part mainly describes the manifold modeling process of image features. Assume that the image dataset is X={x1,x2,…,x n}, where each x i ∈R d , d is the image dimension. These images exist in the high-dimensional space R d However, although d may be large due to the high resolution of the image, its features are usually distributed on a low-dimensional manifold. Assume that the dataset X actually lies on a low-dimensional manifold , M is a k-dimensional smooth differential manifold, and k ≪ d. Specifically, there exists a locally smooth embedding mapping function:

[0083] ;

[0084] In order to embed the features of image data into the differentiable manifold, a mapping function from the original image space to the manifold is constructed:

[0085] ;

[0086] Where ψ is the mapping function; R is the set of real numbers; d is the dimension; is a d-dimensional real number set; M is a manifold; i is a natural number, i=1,2,...,n; q is an element cluster on M; is the i-th element on M; is the meaning of the i-th high-dimensional feature data x.

[0087] The mapping function ψ transforms the original high-dimensional data x i Mapped to a low-dimensional representation on the manifold M . Using the intrinsic properties of neural networks, constraints and , ensuring that both are infinitely smooth functions. In addition, the mapping preserves the data structure of the original image samples: .

[0088] The second part mainly describes the distribution of features on the manifold. For an image dataset, each image I can be mapped into a feature space using neural network feature extraction methods. These features are further embedded on a smooth and differentiable manifold M. Let G be a Lie group whose group operation μ is defined on the manifold M and is a linear combination or transformation of the feature vectors. In incoherent imaging systems, since image information is based on the superposition of light intensities, these features can be viewed as a linear system, so μ must be smooth. At the same time, the backpropagation mechanism in the neural network ensures the existence of inverse elements.

[0089] However, in coherent imaging systems, due to the interference effects of coherent waves, the propagation and interaction of signals in space are no longer simply linear superpositions. The output image depends not only on the intensity distribution of the input image but also on the phase characteristics of the waves. Therefore, nonlinear operators are introduced into the construction of neural networks to accommodate the interference and phase effects in coherent imaging systems. This effectively changes the metric space in which the image is embedded, allowing it to approximately satisfy group properties under the action of nonlinear operators.

[0090] In the nonlinear operator space T, a new metric d is constructed to measure the distance between features:

[0091] ;

[0092] Where d is the metric function; T() is the nonlinear operator space; x and y are the input elements respectively; is the second normal form; ɑ is the coefficient; is the mapping function.

[0093] The third part mainly describes the construction of manifold topology properties. Using the hierarchical relationship of neural networks, we can satisfy the , where T1 and T2 belong to two nonlinear transformations of the operator set 𝒯; is a function composite operation, that is, first performing T2 and then T1; x is the input image. Based on the definition of the nonlinear operator space T, an approximate group structure is constructed. This method establishes a high-dimensional manifold information space, integrating information from the target domain and the input domain.

[0094] The fourth part mainly describes the optimization construction of Lie groups. The target space o(u,v) is defined as the target domain x, and the image space to be enhanced I(x,y) is defined as the source domain z. Since each piece of information in x and z is an image, its metric ρ can directly use the Euclidean distance of intensity information. It can be inferred that x and z satisfy positive definiteness, symmetry and triangle inequality under the metric ρ, thus forming a topological space. It is further assumed that there exists a high-dimensional smooth manifold f x and f z , fitting the data through the neural network, with input x and output f x A one-to-one correspondence is established between them, so that on the manifold f x A bijective relationship is formed between and X, f z The relationship between and Z is similar. Based on the linear properties of the incoherent imaging system, the correspondence between the target sample and the input also satisfies a one-to-one mapping. Through the fitting characteristics of the neural network, it can be assumed that f and its inverse mapping are continuous. Therefore, f x and X have homeomorphic properties, f z And Z as well. Finally, f x and f zThey are embedded into two low-dimensional differentiable manifolds M1 and M2, and the corresponding Lie groups are G1 and G2. By constructing the Cartesian product, a new Lie group is obtained , the group operations and Lie algebras are defined as: and Among them, (g1, g2) is a two-tuple element of a Lie group, with sub-elements g1 and g2; (h1, h2) is similar, representing another group element; is the union of two linear spaces or Lie algebras. The newly constructed high-dimensional Lie group G maximizes the degree of freedom and integrates all potential relations within G1×G2. On this basis, by introducing the constraint , where L is the constraint function and n is the bias. The group velocity of the high-dimensional Lie group G is optimized, and the invariant feature map is used to further achieve dimensionality reduction. Ultimately, the constructed optimal Lie group G′ retains all information in the target and input domains with minimal dimensions, providing a convenient channel for information extraction from the target domain.

[0095] See also Figure 5 As shown, an embodiment of the present invention discloses a weak light image enhancement device, comprising:

[0096] The model acquisition module 11 is used to acquire an image to be enhanced in a preset weak light scene and a target normal light image in a preset normal light scene, construct an initial image enhancement model, and optimize the initial image enhancement model to obtain a target image enhancement model;

[0097] A first information acquisition module 12 is configured to input the target normal illumination image into the initial image enhancement model, and perform preset feature extraction and manifold distribution estimation operations on the target normal illumination image using the initial image enhancement model to obtain first distribution manifold information;

[0098] a second information acquisition module 13, configured to input the image to be enhanced into the target image enhancement model, and perform preset feature extraction and manifold distribution estimation operations on the image to be enhanced using the target image enhancement model to obtain second distribution estimation information;

[0099] a mapping relationship establishing module 14, configured to use the target image enhancement model to fuse the first distribution manifold information with the second distribution estimation information to determine a target high-dimensional manifold information space, and to establish a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a corresponding feature information vector in a preset label space using the target image enhancement model based on the target high-dimensional manifold information space;

[0100] The enhancement result acquisition module 15 is used to obtain the target projection corresponding to the image to be enhanced based on the preset conditions according to the mapping relationship, determine the target manifold distribution that meets the preset feature distribution conditions based on the target projection corresponding to the image to be enhanced, and use the decoder in the target image enhancement model to decode the target manifold distribution to obtain the target image enhancement result.

[0101] As can be seen from the above, the present application obtains a preset weak light scene image to be enhanced and a target image of a normal lighting scene, constructs and optimizes an initial image enhancement model to obtain a target image enhancement model; the target normal lighting image is input into the initial model to extract features and estimate the manifold distribution to obtain first distribution manifold information, and the image to be enhanced is input into the target image enhancement model to extract features and estimate the manifold distribution to obtain second distribution estimation information; the target image enhancement model is used to fuse the two types of distribution information to determine the target high-dimensional manifold information space, and based on this space, a mapping relationship between the preset sample space target high-dimensional feature information vector and the preset label space corresponding feature information vector is established in the target image enhancement model; according to the mapping relationship, the target projection corresponding to the image to be enhanced is obtained according to the preset conditions, and the target manifold distribution that meets the preset feature distribution conditions is determined accordingly, and the target manifold distribution is decoded by the target image enhancement model decoder to obtain the target image enhancement result. In this way, by constructing and optimizing the image enhancement model, combining the feature extraction, manifold distribution estimation and fusion of multi-scene images, and cross-space feature vector mapping and manifold distribution decoding, it is possible to optimize the quality of low-light images. By optimizing the features and adjusting the manifold distribution of low-light images, the model can effectively improve the image brightness, contrast and detail clarity to improve the visual effect.

[0102] In some specific implementations, the model acquisition module 11 may specifically include:

[0103] An image set acquisition unit is configured to use a preset image acquisition device to perform a preset periodic imaging acquisition operation on a target to be imaged in a preset weak light scene and a preset normal light scene based on a preset frequency, so as to obtain a weak light image set in the preset weak light scene and a normal light image set in the preset normal light scene;

[0104] The image determination unit is configured to determine, based on a preset image acquisition requirement, an image to be enhanced and a target normal-light image from the weak-light image set and the normal-light image set.

[0105] In some specific embodiments, the image enhancement model includes a backbone network encoder module, a feature manifold estimation and establishment module, and a feature map selection module.

[0106] In some specific implementations, the second information acquisition module 13 may specifically include:

[0107] an illumination distribution feature acquisition unit, configured to perform a preset overall illumination coverage map estimation operation on the image to be enhanced using the backbone network encoder module in the target image enhancement model, so as to obtain illumination distribution features of the image to be enhanced;

[0108] a hierarchical distribution information acquisition unit, configured to input the illumination distribution feature into a deep network coding layer of the backbone network encoder module, and perform light field distribution estimation and analysis on the backbone network encoder module to obtain hierarchical distribution information of the image to be enhanced;

[0109] The second distribution estimation information acquiring unit is configured to acquire second distribution estimation information of the image to be enhanced based on the illumination distribution feature and the hierarchical distribution information.

[0110] In some specific implementations, the mapping relationship establishing module 14 may specifically include:

[0111] A space establishment unit is used to utilize the first loss function and the second loss function of the feature manifold estimation and establishment module in the target image enhancement model to perform a preset information interactive fusion operation on the first distribution manifold information and the second distribution estimation information based on a preset dimension to establish a target high-dimensional manifold information space; wherein, the first loss function is used to determine the information features of the image to be enhanced, and the second loss function is used to fuse the label features of the target normal illumination image with the image to be enhanced.

[0112] A result acquisition unit is configured to utilize the feature manifold estimation and establishment module of the target image enhancement model to perform sampling estimation on the image to be enhanced based on the illumination distribution characteristics of the image to be enhanced, so as to obtain a corresponding target high-dimensional manifold estimation result;

[0113] A mapping function construction unit is used to train a target operator based on the target high-dimensional manifold estimation result, optimize the parameters of the target operator in combination with preset operator constraints to obtain an optimized target operator, and construct a mapping function according to the optimized target operator;

[0114] A mapping relationship establishing unit is used to establish a mapping relationship between the target high-dimensional feature information vector of the preset sample space and the feature information vector of the preset label space in the target high-dimensional manifold information space according to the mapping function.

[0115] In some specific implementations, the enhancement result acquisition module 15 may specifically include:

[0116] a processed high-dimensional manifold information acquisition unit, configured to perform a preset data processing operation on the high-dimensional manifold information in the target high-dimensional manifold information space using the third loss function and the fourth loss function in the feature map selection module to obtain processed high-dimensional manifold information;

[0117] a projection set generating unit, configured to project the processed high-dimensional manifold information onto a preset low-dimensional space according to the mapping relationship to generate a corresponding projection set;

[0118] a target projection acquisition unit, configured to acquire a target projection corresponding to the image to be enhanced from the projection set according to a preset condition;

[0119] A target manifold distribution determination unit is used to determine a target manifold distribution that meets a preset image similarity condition based on the target projection corresponding to the image to be enhanced; wherein the third loss function is used to eliminate information that meets a preset redundant condition in the high-dimensional manifold information of the target high-dimensional manifold information space, and the fourth loss function is used to expand the high-dimensional manifold information of the target high-dimensional manifold information space.

[0120] Furthermore, the embodiment of the present application also discloses an electronic device, Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram should not be considered as any limitation to the scope of application of the present application.

[0121] Figure 6 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the low-light image enhancement method disclosed in any of the aforementioned embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0122] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0123] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0124] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, and can be Windows Server, NetWare, Unix, Linux, etc. In addition to including a computer program capable of implementing the low-light image enhancement method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 may further include a computer program capable of implementing other specific tasks.

[0125] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when executed by a processor, the computer program implements the aforementioned low-light image enhancement method. The specific steps of this method can be referred to the corresponding contents disclosed in the aforementioned embodiments and will not be repeated here.

[0126] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0127] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0128] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0129] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0130] The above is a detailed introduction to the technical solution provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for enhancing a weak light image, characterized in that: include: Acquire an image to be enhanced in a preset weak light scene and a target normal light image in a preset normal light scene, construct an initial image enhancement model, and optimize the initial image enhancement model to obtain a target image enhancement model; Inputting the target normal illumination image into the initial image enhancement model, and performing preset feature extraction and manifold distribution estimation operations on the target normal illumination image using the initial image enhancement model to obtain first distribution manifold information; Inputting the image to be enhanced into the target image enhancement model, and performing preset feature extraction and manifold distribution estimation operations on the image to be enhanced using the target image enhancement model to obtain second distribution estimation information; Using the target image enhancement model to fuse the first distribution manifold information and the second distribution estimation information to determine a target high-dimensional manifold information space, and based on the target high-dimensional manifold information space and using the target image enhancement model, establish a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a corresponding feature information vector in a preset label space; According to the mapping relationship, the target projection corresponding to the image to be enhanced is obtained based on preset conditions, the target manifold distribution that meets the preset feature distribution conditions is determined based on the target projection corresponding to the image to be enhanced, and the target manifold distribution is decoded using the decoder in the target image enhancement model to obtain the target image enhancement result.

2. The method for enhancing weak light images according to claim 1, wherein: The step of acquiring the image to be enhanced in a preset weak light scene and the target normal light image in a preset normal light scene includes: Using a preset image acquisition device to perform preset periodic imaging acquisition operations on a target to be imaged in a preset weak light scene and a preset normal light scene based on a preset frequency, so as to obtain a weak light image set in the preset weak light scene and a normal light image set in the preset normal light scene; Based on a preset image acquisition requirement, an image to be enhanced and a target normal-light image are determined from the low-light image set and the normal-light image set.

3. The method for enhancing a weak light image according to claim 1 or 2, wherein: The image enhancement model includes a backbone network encoder module, a feature manifold estimation and establishment module, and a feature map selection module.

4. The method for enhancing weak light images according to claim 3, wherein: The using the target image enhancement model to perform preset feature extraction and manifold distribution estimation operations on the image to be enhanced to obtain second distribution estimation information includes: Using the backbone network encoder module in the target image enhancement model, a preset overall illumination coverage map estimation operation is performed on the image to be enhanced to obtain illumination distribution characteristics of the image to be enhanced; Inputting the illumination distribution feature into the deep coding layer of the backbone network encoder module, and performing light field distribution estimation and analysis on the backbone network encoder module to obtain hierarchical distribution information of the image to be enhanced; Second distribution estimation information of the image to be enhanced is obtained based on the illumination distribution feature and the hierarchical distribution information.

5. The method for enhancing weak light images according to claim 3, wherein: The step of fusing the first distribution manifold information and the second distribution estimation information using the target image enhancement model to determine a target high-dimensional manifold information space includes: Using the first loss function and the second loss function of the feature manifold estimation and establishment module in the target image enhancement model, based on a preset dimension, a preset information interactive fusion operation is performed on the first distribution manifold information and the second distribution estimation information to establish a target high-dimensional manifold information space; The first loss function is used to determine the information features of the image to be enhanced, and the second loss function is used to fuse the label features of the target normal illumination image with the image to be enhanced.

6. The method for enhancing a weak light image according to claim 4, wherein: The method of establishing a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a corresponding feature information vector in a preset label space by using the target image enhancement model based on the target high-dimensional manifold information space includes: Utilizing the feature manifold estimation and establishment module of the target image enhancement model, sampling and estimating the image to be enhanced based on the illumination distribution characteristics of the image to be enhanced, so as to obtain a corresponding target high-dimensional manifold estimation result; Training a target operator based on the target high-dimensional manifold estimation result, optimizing the parameters of the target operator in combination with preset operator constraints to obtain an optimized target operator, and constructing a mapping function according to the optimized target operator; According to the mapping function, a mapping relationship is established between the target high-dimensional feature information vector of the preset sample space and the feature information vector of the preset label space in the target high-dimensional manifold information space.

7. The method for enhancing a weak light image according to claim 6, wherein: The step of obtaining a target projection corresponding to the image to be enhanced based on a preset condition according to the mapping relationship, and determining a target manifold distribution that satisfies a preset feature distribution condition based on the target projection corresponding to the image to be enhanced, includes: performing a preset data processing operation on the high-dimensional manifold information in the target high-dimensional manifold information space using the third loss function and the fourth loss function in the feature map selection module to obtain processed high-dimensional manifold information; Projecting the processed high-dimensional manifold information onto a preset low-dimensional space according to the mapping relationship to generate a corresponding projection set; Acquire a target projection corresponding to the image to be enhanced from the projection set according to a preset condition; Determining a target manifold distribution that satisfies a preset image similarity condition based on the target projection corresponding to the image to be enhanced; Among them, the third loss function is used to eliminate information that meets the preset redundancy condition in the high-dimensional manifold information of the target high-dimensional manifold information space, and the fourth loss function is used to expand the high-dimensional manifold information of the target high-dimensional manifold information space.

8. A weak light image enhancement device, characterized in that: include: The model acquisition module is used to acquire the image to be enhanced in a preset weak light scene and the target normal light image in a preset normal light scene, construct an initial image enhancement model, and optimize the initial image enhancement model to obtain the target image enhancement model; a first information acquisition module, configured to input the target normal illumination image into the initial image enhancement model, and perform preset feature extraction and manifold distribution estimation operations on the target normal illumination image using the initial image enhancement model to obtain first distribution manifold information; a second information acquisition module, configured to input the image to be enhanced into the target image enhancement model, and perform preset feature extraction and manifold distribution estimation operations on the image to be enhanced using the target image enhancement model to obtain second distribution estimation information; a mapping relationship establishment module, configured to use the target image enhancement model to fuse the first distribution manifold information with the second distribution estimation information to determine a target high-dimensional manifold information space, and to establish a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a corresponding feature information vector in a preset label space using the target image enhancement model based on the target high-dimensional manifold information space; An enhancement result acquisition module is used to obtain the target projection corresponding to the image to be enhanced based on preset conditions according to the mapping relationship, determine the target manifold distribution that meets the preset feature distribution conditions based on the target projection corresponding to the image to be enhanced, and use the decoder in the target image enhancement model to decode the target manifold distribution to obtain the target image enhancement result.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the method for enhancing a weak-light image according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that Used to store a computer program; wherein, when the computer program is executed by a processor, the weak-light image enhancement method according to any one of claims 1 to 7 is implemented.

Images