A weak light image enhancement method, device, equipment and storage medium

Abstract

The application discloses a weak light image enhancement method and device, equipment and storage medium, relates to the technical field of image enhancement, and comprises the following steps: acquiring a to-be-enhanced image and a target normal light image, and optimizing an initial image enhancement model to obtain a target image enhancement model; performing feature extraction and manifold distribution estimation on the target normal light image by using the initial image enhancement model to obtain first distribution manifold information; performing feature extraction and manifold distribution estimation on the to-be-enhanced image by using the target image enhancement model to obtain second distribution estimation information; fusing the first distribution manifold information and the second distribution estimation information by using the target image enhancement model, establishing a mapping relationship between a target high-dimensional feature information vector and a feature information vector based on fused target high-dimensional manifold information space, and acquiring a target projection and a target manifold distribution according to the mapping relationship; and decoding the target manifold distribution by using a decoder to obtain a target image enhancement result. Image adaptive enhancement is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image enhancement, in particular to a weak light image enhancement method, device, equipment and storage medium. BACKGROUND

[0002] The current weak light image enhancement method can be mainly divided into a method based on traditional image processing and a method based on deep learning. The technology based on the traditional method usually enhances the image quality by enhancing the overall image brightness, adjusting the contrast or local detail enhancement, but is easily disturbed by noise, thereby causing the loss of image details and the decline of quality. On the other hand, the weak light image enhancement method based on deep learning has made remarkable progress in recent years, and realizes the joint enhancement of image brightness and details by using a large-scale data set and a complex network structure.

[0003] Although artificial intelligence has made great breakthroughs in various fields by using algorithm models, the weak light image is usually accompanied by high-level noise when shooting, and the natural light field distribution in the image is difficult to be accurately restored. Most enhancement methods only focus on the improvement of overall brightness, while ignoring the fidelity reconstruction of local lighting features, resulting in the loss of reality of the enhanced image. At the same time, the lighting conditions in the low light environment are complex and changeable, which may include point light source, diffuse light, reflected light and other conditions. The existing method is usually optimized for a specific lighting scene, and it is difficult to realize adaptive processing for multiple lighting conditions.

[0004] In summary, how to suppress noise interference, restore the natural light field distribution, and realize adaptive enhancement for multiple lighting conditions in a complex low light environment is a problem to be solved at present. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a weak light image enhancement method, device, equipment and storage medium, which can suppress noise interference, restore the natural light field distribution, and realize adaptive enhancement for multiple lighting conditions in a complex low light environment. The specific scheme is as follows:

[0006] In a first aspect, the present application provides a weak light image enhancement method, comprising:

[0007] obtaining a to-be-enhanced image in a preset weak light scene and a target normal lighting image in a preset normal lighting scene, constructing an initial image enhancement model, and optimizing the initial image enhancement model to obtain a target image enhancement model;

[0008] inputting the target normal lighting image into the initial image enhancement model, and performing a preset feature extraction and manifold distribution estimation operation on the target normal lighting image by using the initial image enhancement model to obtain first distribution manifold information;

[0009] inputting the image to be enhanced into the target image enhancement model, performing preset feature extraction and manifold distribution estimation operations on the image to be enhanced by using the target image enhancement model to obtain second distribution estimation information;

[0010] fusing the first distribution manifold information and the second distribution estimation information by using the target image enhancement model to determine a target high-dimensional manifold information space, and establishing a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a corresponding feature information vector in a preset label space based on the target high-dimensional manifold information space by using the target image enhancement model;

[0011] obtaining a target projection corresponding to the image to be enhanced based on a preset condition according to the mapping relationship, determining a target manifold distribution satisfying a preset feature distribution condition based on the target projection corresponding to the image to be enhanced, and decoding the target manifold distribution by using a decoder in the target image enhancement model to obtain a target image enhancement result.

[0012] Optionally, the obtaining of the image to be enhanced under the preset weak light scene and the target normal illumination image under the preset normal illumination scene comprises:

[0013] performing a preset periodic imaging collection operation on an imaging target in the preset weak light scene and the preset normal illumination scene based on a preset frequency by using a preset image collection device to obtain a weak light image set under the preset weak light scene and a normal illumination image set under the preset normal illumination scene;

[0014] determining the image to be enhanced and the target normal illumination image from the weak light image set and the normal illumination image set based on a preset image collection requirement.

[0015] Optionally, the image enhancement model comprises a backbone network encoder module, a feature manifold estimation and establishment module, and a feature mapping selection module.

[0016] Optionally, the performing of the preset feature extraction and manifold distribution estimation operations on the image to be enhanced by using the target image enhancement model to obtain the second distribution estimation information comprises:

[0017] performing a preset overall illumination coverage map estimation operation on the image to be enhanced by using the backbone network encoder module in the target image enhancement model to obtain illumination distribution features of the image to be enhanced;

[0018] inputting the illumination distribution features into a network deep encoding layer of the backbone network encoder module, and performing light field distribution estimation and analysis on the backbone network encoder module to obtain hierarchical distribution information of the image to be enhanced;

[0019] obtain second distribution estimation information of the image to be enhanced based on the illumination distribution characteristics and the hierarchical distribution information.

[0020] Optionally, the fusing the first distribution manifold information and the second distribution estimation information by using the target image enhancement model to determine a target high-dimensional manifold information space comprises:

[0021] performing a preset information interaction fusion operation on the first distribution manifold information and the second distribution estimation information based on a preset dimension by using the first loss function and the second loss function of the feature manifold estimation and establishment module in the target image enhancement model to establish a target high-dimensional manifold information space;

[0022] The first loss function is used to determine information characteristics of the image to be enhanced, and the second loss function is used to fuse label characteristics of the target normal illumination image with the image to be enhanced.

[0023] Optionally, the establishing a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a corresponding feature information vector in a preset label space based on the target high-dimensional manifold information space by using the target image enhancement model comprises:

[0024] performing sampling estimation on the image to be enhanced based on illumination distribution characteristics of the image to be enhanced by using the feature manifold estimation and establishment module of the target image enhancement model to obtain a corresponding target high-dimensional manifold estimation result;

[0025] training a target operator based on the target high-dimensional manifold estimation result, and optimizing parameters of the target operator in combination with a preset operator constraint condition to obtain an optimized target operator, and constructing a mapping function according to the optimized target operator;

[0026] establishing a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a feature information vector in a preset label space in the target high-dimensional manifold information space according to the mapping function.

[0027] Optionally, the obtaining a target projection corresponding to the image to be enhanced based on a preset condition and determining a target manifold distribution satisfying a preset feature distribution condition based on the target projection corresponding to the image to be enhanced comprises:

[0028] performing a preset data processing operation on high-dimensional manifold information in the target high-dimensional manifold information space by using a third loss function and a fourth loss function in the feature mapping selection module to obtain processed high-dimensional manifold information;

[0029] Projecting the processed high-dimensional manifold information onto a preset low-dimensional space according to the mapping relationship to generate a corresponding projection set;

[0030] Acquire a target projection corresponding to the image to be enhanced from the projection set according to a preset condition;

[0031] Determining a target manifold distribution that satisfies a preset image similarity condition based on the target projection corresponding to the image to be enhanced;

[0032] Among them, the third loss function is used to eliminate information that meets the preset redundancy condition in the high-dimensional manifold information of the target high-dimensional manifold information space, and the fourth loss function is used to expand the high-dimensional manifold information of the target high-dimensional manifold information space.

[0033] In a second aspect, the present application provides a low-light image enhancement device, comprising:

[0034] The model acquisition module is used to acquire the image to be enhanced in a preset weak light scene and the target normal light image in a preset normal light scene, construct an initial image enhancement model, and optimize the initial image enhancement model to obtain the target image enhancement model;

[0035] a first information acquisition module, configured to input the target normal illumination image into the initial image enhancement model, and perform preset feature extraction and manifold distribution estimation operations on the target normal illumination image using the initial image enhancement model to obtain first distribution manifold information;

[0036] a second information acquisition module, configured to input the image to be enhanced into the target image enhancement model, and perform preset feature extraction and manifold distribution estimation operations on the image to be enhanced using the target image enhancement model to obtain second distribution estimation information;

[0037] a mapping relationship establishment module, configured to use the target image enhancement model to fuse the first distribution manifold information with the second distribution estimation information to determine a target high-dimensional manifold information space, and to establish a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a corresponding feature information vector in a preset label space using the target image enhancement model based on the target high-dimensional manifold information space;

[0038] An enhancement result acquisition module is used to obtain the target projection corresponding to the image to be enhanced based on preset conditions according to the mapping relationship, determine the target manifold distribution that meets the preset feature distribution conditions based on the target projection corresponding to the image to be enhanced, and use the decoder in the target image enhancement model to decode the target manifold distribution to obtain the target image enhancement result.

[0039] In a third aspect, the present application provides an electronic device, comprising:

[0040] a memory for storing a computer program;

[0041] a processor for executing the computer program to implement the weak light image enhancement method as described above.

[0042] In a fourth aspect, the present application provides a computer readable storage medium for storing a computer program; wherein the computer program is executed by a processor to implement the weak light image enhancement method as described above.

[0043] In summary, the present application obtains a to-be-enhanced image in a preset weak light scene and a target normal illumination image in a preset normal illumination scene, constructs an initial image enhancement model, and optimizes the initial image enhancement model to obtain a target image enhancement model; inputs the target normal illumination image into the initial image enhancement model, performs preset feature extraction and manifold distribution estimation operations on the target normal illumination image by using the initial image enhancement model to obtain first distribution manifold information, then inputs the to-be-enhanced image into the target image enhancement model, performs preset feature extraction and manifold distribution estimation operations on the to-be-enhanced image by using the target image enhancement model to obtain second distribution estimation information; fuses the first distribution manifold information and the second distribution estimation information by using the target image enhancement model to determine a target high-dimensional manifold information space, establishes a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a corresponding feature information vector in a preset label space based on the target high-dimensional manifold information space and by using the target image enhancement model; obtains a target projection corresponding to the to-be-enhanced image based on a preset condition according to the mapping relationship, determines a target manifold distribution satisfying a preset feature distribution condition based on the target projection corresponding to the to-be-enhanced image, and decodes the target manifold distribution by using a decoder in the target image enhancement model to obtain a target image enhancement result. As can be seen from the above, the present application obtains a to-be-enhanced image in a preset weak light scene and a target image in a normal illumination scene, constructs and optimizes an initial image enhancement model to obtain a target image enhancement model; inputs a target normal illumination image into an initial model to extract features and estimate a manifold distribution to obtain first distribution manifold information, inputs a to-be-enhanced image into a target image enhancement model to extract features and estimate a manifold distribution to obtain second distribution estimation information; determines a target high-dimensional manifold information space by fusing two types of distribution information by using a target image enhancement model, establishes a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a corresponding feature information vector in a preset label space in the target image enhancement model based on the space, obtains a target projection corresponding to the to-be-enhanced image according to the mapping relationship and a preset condition, determines a target manifold distribution satisfying a preset feature distribution condition based on this, and decodes the target manifold distribution by using a decoder in the target image enhancement model to obtain a target image enhancement result. In this way, by constructing and optimizing an image enhancement model, combining feature extraction, manifold distribution estimation and fusion of multiple scene images, and cross-space feature vector mapping and manifold distribution decoding, the effect of optimizing low-illumination image quality can be achieved, and the feature optimization and manifold distribution adjustment of the model on the weak light image can effectively improve the image brightness, contrast, and detail clarity to improve the visual effect. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only a part of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on the provided drawings.

[0045] Figure 1 A flow chart of a weak light image enhancement method disclosed by the present application;

[0046] Figure 2 A schematic diagram of a loss function guided feature interaction fusion disclosed by the present application;

[0047] Figure 3 A schematic diagram of a to-be-enhanced image, a target normal illumination image and a target image enhancement result disclosed by the present application;

[0048] Figure 4 A schematic diagram of a high-dimensional manifold establishment module and a feature mapping selection module disclosed by the present application;

[0049] Figure 5 A structural schematic diagram of a weak light image enhancement device disclosed by the present application;

[0050] Figure 6 A structural diagram of an electronic device disclosed by the present application. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0052] At present, the weak light image enhancement method can be mainly divided into a method based on traditional image processing and a method based on deep learning. The technology based on the traditional method usually enhances the image quality by enhancing the overall image brightness, adjusting the contrast or local detail enhancement, but is easily disturbed by noise, thereby causing image detail loss and quality decline. On the other hand, the weak light image enhancement method based on deep learning has made significant progress in recent years, and through the use of large-scale data sets and complex network structures, the joint enhancement of image brightness and details is realized. In order to solve the above technical problems, the present application discloses a weak light image enhancement method, device, equipment and storage medium, which can suppress noise interference in a complex low light environment, restore natural light field distribution, and realize adaptive enhancement under various illumination conditions.

[0053] Referring to Figure 1 As shown in the embodiments of the application, a weak light image enhancement method is disclosed, comprising:

[0054] In step S11, a to-be-enhanced image under a preset weak light scene and a target normal illumination image under a preset normal illumination scene are acquired, an initial image enhancement model is constructed, and the initial image enhancement model is optimized to obtain a target image enhancement model.

[0055] In this embodiment, in order to acquire the to-be-enhanced image under the preset weak light scene and the target normal illumination image under the preset normal illumination scene of a preset size, a preset periodic imaging acquisition operation can be performed on the to-be-imaged target in the preset weak light scene and the preset normal illumination scene based on a preset frequency by using a preset image acquisition device, so as to obtain a weak light image set under the preset weak light scene and a normal illumination image set under the preset normal illumination scene; and the to-be-enhanced image and the target normal illumination image are determined from the weak light image set and the normal illumination image set based on a preset image acquisition requirement. Specifically, the to-be-imaged target is periodically imaged and acquired in the preset weak light scene and the normal illumination scene based on the preset frequency by using the preset image acquisition device, so as to obtain the weak light image set and the normal illumination image set; and the images are screened according to the preset image acquisition requirement, for example, the signal-to-noise ratio, the contrast, the motion blur, and the like of the weak light image set are calculated, and the image with a signal-to-noise ratio lower than a threshold and a low contrast is selected as the to-be-enhanced image; and the normal illumination image set is selected, and the image with a high signal-to-noise ratio, a high contrast, no obvious blur, no abnormal exposure, and a time and space alignment with the to-be-enhanced image is selected as the target normal illumination image.

[0056] In addition, the environment images under indoor and outdoor scenes can be acquired in two ways. If the to-be-enhanced image and the target normal illumination image under the indoor scene are required to be acquired, the indoor image can be imaged and acquired and the label image can be acquired by controlling the indoor light, so as to obtain the dark light image and the well-illuminated image under the weak light scene consistent with the to-be-enhanced target, wherein the arrangement of the indoor scene remains consistent under multiple lights. If the to-be-enhanced image and the target normal illumination image under the outdoor scene are required to be acquired, the same scene at different time periods can be imaged and acquired by using the acquisition camera according to the preset parameters, so as to obtain the target normal illumination image under multiple illumination scenes consistent with the to-be-enhanced image.

[0057] It can be understood that after the to-be-enhanced image and the target normal illumination image are acquired, the initial image enhancement model is constructed, and then the parameters in the initial image enhancement model are optimized to obtain the target image enhancement model, wherein the initial image enhancement model and the target image enhancement model both include the backbone network encoder module, the feature manifold estimation and construction module, and the feature mapping selection module.

[0058] In step S12, the target normal illumination image is input into the initial image enhancement model, and the initial image enhancement model is used to perform preset feature extraction and manifold distribution estimation on the target normal illumination image to obtain first distribution manifold information.

[0059] In this embodiment, the target normal illumination image is input into the initial image enhancement model, and the backbone network encoder module in the initial image enhancement model first extracts features from the target normal illumination image according to the hierarchical structure of the convolutional neural network, such as basic visual features (e.g., edges, textures, shapes, etc.) and semantic-level high-level features, by using convolution kernels of different sizes and steps. At the same time, the manifold distribution estimation is performed, and the distribution pattern of the extracted features in the high-dimensional space can be analyzed and modeled by using a manifold learning algorithm based on deep learning, such as a related mechanism in a variational autoencoder or a generative adversarial network. By calculating the similarity, distance metric, and other statistics between the features, the distribution rule of the feature data in the manifold space is estimated, and finally the first distribution manifold information that can represent the feature distribution characteristics of the target normal illumination image is obtained.

[0060] In step S13, the image to be enhanced is input into the target image enhancement model, and the target image enhancement model is used to perform preset feature extraction and manifold distribution estimation on the image to be enhanced to obtain second distribution estimation information.

[0061] In this embodiment, the image to be enhanced is input into the target image enhancement model, and the backbone network encoder module in the target image enhancement model is used to perform preset overall illumination coverage map estimation on the image to be enhanced to obtain illumination distribution features of the image to be enhanced. The illumination distribution features are input into the network deep encoding layer of the backbone network encoder module, and the backbone network encoder module is subjected to light field distribution estimation and analysis to obtain hierarchical distribution information of the image to be enhanced. Based on the illumination distribution features and the hierarchical distribution information, second distribution estimation information of the image to be enhanced is obtained. Specifically, the shallow encoding layer of the target image enhancement model is used to perform illumination coverage map estimation on the image to be enhanced to generate illumination distribution features, the illumination distribution features and the features of the image to be enhanced are input into the deep encoding layer for light field distribution analysis to obtain hierarchical distribution information, and the encoder is used to perform distribution estimation on the hierarchical distribution information to generate feature manifold distribution estimation information of the image to be enhanced.

[0062] In step S14, the first distribution manifold information and the second distribution estimation information are fused by using the target image enhancement model to determine a target high-dimensional manifold information space. Based on the target high-dimensional manifold information space, the target image enhancement model is used to establish a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a corresponding feature information vector in a preset label space.

[0063] In this embodiment, after obtaining the first distribution estimation information and the second distribution estimation information, the first loss function and the second loss function of the feature manifold estimation and establishment module in the target image enhancement model are used to perform a preset information interaction fusion operation on the first distribution manifold information and the second distribution estimation information based on a preset dimension to establish a target high-dimensional manifold information space. The first loss function is used to determine the information features of the image to be enhanced, and the second loss function is used to fuse the label features of the target normal illumination image with the image to be enhanced. In addition, the feature manifold estimation and establishment module of the target image enhancement model is used to sample and estimate the image to be enhanced based on the illumination distribution features of the image to be enhanced to obtain a corresponding target high-dimensional manifold estimation result. A target operator is trained based on the target high-dimensional manifold estimation result, and the parameters of the target operator are optimized in combination with a preset operator constraint condition to obtain an optimized target operator. A mapping function is constructed according to the optimized target operator. The target high-dimensional feature information vector of the preset sample space and the feature information vector of the preset label space are mapped in the target high-dimensional manifold information space according to the mapping function.

[0064] Specifically, after obtaining the first distribution estimation information and the second distribution estimation information, as shown in Figure 2 the first loss function and the second loss function of the feature manifold estimation and establishment module in the target image enhancement model are used to perform an interaction fusion on the first distribution manifold information and the second distribution estimation information based on a preset dimension to establish a target high-dimensional manifold information space. It should be noted that when performing information interaction fusion, the feature manifold estimation and establishment module is used to perform sampling estimation based on the illumination distribution features of the image to be enhanced to obtain a target high-dimensional manifold estimation result. Then, a target operator is trained based on the result, the operator parameters are optimized in combination with a preset operator constraint condition to obtain an optimized target operator and construct a mapping function. The mapping relationship between the feature vectors of the sample space and the label space is established in the target high-dimensional manifold information space according to the mapping function.

[0065] The first loss function is used to redefine the information features of the image to be enhanced on a high-dimensional manifold, and the second loss function is used to fuse the label feature information of the target normal illumination image. For example, the first loss function is the LPSR (Loss of Particular Solution Reconstruction Optimization) loss function, and its specific form is:

[0066] ;

[0067] in, is a metric representation; is the model function; are model parameters; x is the prior target normal light image and its corresponding feature input; min is the minimum value of the loss function; is the output result obtained after the input image x passes through the network with parameter θ.

[0068] The second loss function is the ManiLoss (Loss of Manifold) loss function, which is the manifold information integration and interaction optimization loss. Its specific form is:

[0069] ;

[0070] in, is a metric representation; is the model function; x is the prior target normal light image and its corresponding feature input; min is the minimum value of the loss function; z is the dark light image and its corresponding feature input.

[0071] Step S15: obtaining a target projection corresponding to the image to be enhanced based on preset conditions according to the mapping relationship, determining a target manifold distribution that meets preset feature distribution conditions based on the target projection corresponding to the image to be enhanced, and decoding the target manifold distribution using a decoder in the target image enhancement model to obtain a target image enhancement result.

[0072] In this embodiment, the third loss function and the fourth loss function in the feature mapping selection module are used to perform a preset data processing operation on the high-dimensional manifold information in the target high-dimensional manifold information space, to obtain processed high-dimensional manifold information; the processed high-dimensional manifold information is projected to a preset low-dimensional space according to the mapping relationship, to generate a corresponding projection set; a target projection corresponding to the image to be enhanced is obtained from the projection set according to a preset condition; a target manifold distribution satisfying a preset image similarity condition is determined based on the target projection corresponding to the image to be enhanced; the third loss function is used to eliminate information in the high-dimensional manifold information of the target high-dimensional manifold information space that satisfies a preset redundancy condition, and the fourth loss function is used to expand the high-dimensional manifold information of the target high-dimensional manifold information space. Specifically, the feature mapping selection module is used to project the high-dimensional manifold information in the high-dimensional manifold information space to a low-dimensional distribution based on the third loss function and the fourth loss function, and a target projection closest to the weak light image is determined from the generated projections according to a preset judgment condition. The third loss function is used to eliminate information in the high-dimensional manifold information of the target high-dimensional manifold information space that satisfies a preset redundancy condition, and the fourth loss function is used to expand the high-dimensional manifold information of the target high-dimensional manifold information space. For example, the third loss function is an LPX (Loss of projection to X) loss function, and its specific form is:

[0073] ;

[0074] wherein, is a metric representation; is a model function; is a model parameter; is a frozen parameter, i.e., no gradient backpropagation; z is a dark weak light image and its corresponding feature input; is a function identifier.

[0075] The fourth loss function can be an LTM (Loss of tighten manifold) loss function, and its specific form is:

[0076] ;

[0077] wherein, is a metric representation; is a model function; is a model parameter; is a frozen parameter, i.e., no gradient backpropagation; z is a dark weak light image and its corresponding feature input; is a function identifier.

[0078] Further, using the feature mapping selection module, the high-dimensional feature vector is mapped and associated with the low-dimensional feature vector of the target label to obtain the associated feature. The target manifold distribution obtained is decoded by the decoder in the target image enhancement model to obtain the target image enhancement result corresponding to the image to be enhanced as shown in Figure 3 Thus, the backbone network of the image enhancement model can enhance the image to be enhanced under the estimated distribution by feature fusion and feature mapping of the input features.

[0079] As can be seen from the above, the embodiments of the present application obtain a target image to be enhanced and a normal light scene target image, construct and optimize an initial image enhancement model to obtain a target image enhancement model; input the target normal light image into the initial model to extract features and estimate the manifold distribution to obtain first distribution manifold information, input the image to be enhanced into the target image enhancement model to extract features and estimate the manifold distribution to obtain second distribution estimation information; determine a target high-dimensional manifold information space by fusing the two types of distribution information using the target image enhancement model, and establish a mapping relationship between the target high-dimensional feature information vector of the preset sample space and the corresponding feature information vector of the preset label space in the target image enhancement model based on the space; obtain the target projection corresponding to the image to be enhanced according to the mapping relationship under the preset condition, and determine the target manifold distribution that meets the preset feature distribution condition, and decode the target manifold distribution by the decoder of the target image enhancement model to obtain the target image enhancement result. In this way, by constructing and optimizing the image enhancement model, combining feature extraction, manifold distribution estimation and fusion of multiple scene images, and cross-space feature vector mapping and manifold distribution decoding, the effect of optimizing low light image quality can be achieved, and the image brightness, contrast and detail clarity are effectively improved by feature optimization and manifold distribution adjustment of the model to improve the visual effect.

[0080] Based on the above embodiment, the present application discloses a weak light image enhancement method, which can suppress noise interference, restore natural light field distribution, and realize adaptive enhancement under various light conditions. Next, the weak light image enhancement method as shown in Figure 4 will be described in detail.

[0081] In the results section, a method of bypassing noise to locate the feature Sj of the image to be enhanced is proposed, and the mapping path from the target feature Dj to Sj is optimized. To further improve the mapping from Dj to Sj, a high-dimensional manifold information space is constructed to integrate the information of the target manifold and the input manifold. It can include four main parts, the first part is the manifold modeling, the second part is the feature distribution construction process on the manifold, the third part is the construction of the manifold topological property, and the fourth part is the optimization of the Lie group.

[0082] The first part mainly describes the manifold modeling process of image features. Assume that the image dataset is X={x1,x2,…,x n}, where each x i ∈R d , d is the image dimension. These images exist in the high-dimensional space R d However, although d may be large due to the high resolution of the image, its features are usually distributed on a low-dimensional manifold. Assume that the dataset X actually lies on a low-dimensional manifold , M is a k-dimensional smooth differential manifold, and k ≪ d. Specifically, there exists a locally smooth embedding mapping function:

[0083] ;

[0084] In order to embed the features of image data into the differentiable manifold, a mapping function from the original image space to the manifold is constructed:

[0085] ;

[0086] Where ψ is the mapping function; R is the set of real numbers; d is the dimension; is a d-dimensional real number set; M is a manifold; i is a natural number, i=1,2,...,n; q is an element cluster on M; is the i-th element on M; is the meaning of the i-th high-dimensional feature data x.

[0087] The mapping function ψ transforms the original high-dimensional data x i Mapped to a low-dimensional representation on the manifold M . Using the intrinsic properties of neural networks, constraints and , ensuring that both are infinitely smooth functions. In addition, the mapping preserves the data structure of the original image samples: .

[0088] The second part mainly describes the distribution of features on the manifold. For an image dataset, each image I can be mapped into a feature space using neural network feature extraction methods. These features are further embedded on a smooth and differentiable manifold M. Let G be a Lie group whose group operation μ is defined on the manifold M and is a linear combination or transformation of the feature vectors. In incoherent imaging systems, since image information is based on the superposition of light intensities, these features can be viewed as a linear system, so μ must be smooth. At the same time, the backpropagation mechanism in the neural network ensures the existence of inverse elements.

[0089] However, in coherent imaging systems, due to the interference effects of coherent waves, the propagation and interaction of signals in space are no longer simply linear superpositions. The output image depends not only on the intensity distribution of the input image but also on the phase characteristics of the waves. Therefore, nonlinear operators are introduced into the construction of neural networks to accommodate the interference and phase effects in coherent imaging systems. This effectively changes the metric space in which the image is embedded, allowing it to approximately satisfy group properties under the action of nonlinear operators.

[0090] In the nonlinear operator space T, a new metric d is constructed to measure the distance between features:

[0091] ;

[0092] Where d is the metric function; T() is the nonlinear operator space; x and y are the input elements respectively; is the second normal form; ɑ is the coefficient; is the mapping function.

[0093] The third part mainly describes the construction of manifold topology properties. Using the hierarchical relationship of neural networks, we can satisfy the , where T1 and T2 belong to two nonlinear transformations of the operator set 𝒯; is a function composite operation, that is, first performing T2 and then T1; x is the input image. Based on the definition of the nonlinear operator space T, an approximate group structure is constructed. This method establishes a high-dimensional manifold information space, integrating information from the target domain and the input domain.

[0094] The fourth part mainly describes the optimization construction of Lie groups. The target space o(u,v) is defined as the target domain x, and the image space to be enhanced I(x,y) is defined as the source domain z. Since each piece of information in x and z is an image, its metric ρ can directly use the Euclidean distance of intensity information. It can be inferred that x and z satisfy positive definiteness, symmetry and triangle inequality under the metric ρ, thus forming a topological space. It is further assumed that there exists a high-dimensional smooth manifold f x and f z , fitting the data through the neural network, with input x and output f x A one-to-one correspondence is established between them, so that on the manifold f x A bijective relationship is formed between and X, f z The relationship between and Z is similar. Based on the linear properties of the incoherent imaging system, the correspondence between the target sample and the input also satisfies a one-to-one mapping. Through the fitting characteristics of the neural network, it can be assumed that f and its inverse mapping are continuous. Therefore, f x and X have homeomorphic properties, f z And Z as well. Finally, f x and f zThe two low-dimensional differentiable manifolds M1 and M2 are embedded into two Lie groups G1 and G2 respectively. By constructing a Cartesian product, a new Lie group G is obtained , the group operation and Lie algebra are defined as and . Wherein, (g1, g2) is a binary tuple element of a Lie group, and g1 and g2 are sub-elements; (h1, h2) is similar, representing another group element; is the union of two linear spaces or Lie algebras. The newly constructed high-dimensional Lie group G maximizes the degrees of freedom and integrates all potential relationships within G1 x G2. On this basis, by introducing a constraint condition , wherein L is a constraint function, n is a bias, the group velocity of the high-dimensional Lie group G is optimized, and the dimensionality reduction is further realized by using the invariant feature mapping. Finally, the optimal Lie group G' retains all the information of the target domain and the input domain in the minimum dimension, providing a convenient channel for information extraction of the target domain.

[0095] Referring to FIG. 1 Figure 5 , the embodiment of the present application discloses a weak light image enhancement device, comprising:

[0096] A model acquisition module 11 is configured to acquire a to-be-enhanced image in a preset weak light scene and a target normal illumination image in a preset normal illumination scene, construct an initial image enhancement model, and optimize the initial image enhancement model to obtain a target image enhancement model.

[0097] A first information acquisition module 12 is configured to input the target normal illumination image into the initial image enhancement model, perform preset feature extraction and manifold distribution estimation operations on the target normal illumination image by using the initial image enhancement model, and obtain first distribution manifold information.

[0098] A second information acquisition module 13 is configured to input the to-be-enhanced image into the target image enhancement model, perform preset feature extraction and manifold distribution estimation operations on the to-be-enhanced image by using the target image enhancement model, and obtain second distribution estimation information.

[0099] A mapping relationship establishment module 14 is configured to fuse the first distribution manifold information and the second distribution estimation information by using the target image enhancement model, determine a target high-dimensional manifold information space, and establish a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a corresponding feature information vector in a preset label space by using the target image enhancement model based on the target high-dimensional manifold information space.

[0100] The enhanced result obtaining module 15 is configured to obtain a target projection corresponding to the image to be enhanced based on the preset condition according to the mapping relationship, determine a target manifold distribution satisfying a preset feature distribution condition based on the target projection corresponding to the image to be enhanced, and decode the target manifold distribution by using a decoder in the target image enhancement model to obtain a target image enhancement result.

[0101] As can be seen from the above, the preset weak light scene image to be enhanced and the target image under normal illumination are obtained, an initial image enhancement model is constructed and optimized to obtain a target image enhancement model; the target image under normal illumination is input into the initial model to extract features and estimate a manifold distribution to obtain first distribution manifold information, the image to be enhanced is input into the target image enhancement model to extract features and estimate a manifold distribution to obtain second distribution estimation information; the target image enhancement model is used to fuse the two types of distribution information to determine a target high-dimensional manifold information space, and a mapping relationship between a preset sample space target high-dimensional feature information vector and a corresponding feature information vector in a preset label space is established in the target image enhancement model based on the space; the target projection corresponding to the image to be enhanced is obtained according to the mapping relationship and the preset condition, and the target manifold distribution satisfying the preset feature distribution condition is determined accordingly, and the target image enhancement result is obtained by decoding the target manifold distribution by using the decoder in the target image enhancement model. In this way, by constructing and optimizing the image enhancement model, combining feature extraction, manifold distribution estimation and fusion of multiple scene images, and cross-space feature vector mapping and manifold distribution decoding, the effect of optimizing low-illumination image quality can be achieved, and the image brightness, contrast and detail clarity are effectively improved by feature optimization and manifold distribution adjustment of the model to improve the visual effect.

[0102] In some specific embodiments, the model obtaining module 11 can specifically include:

[0103] The image set obtaining unit is configured to perform a preset periodic imaging and collecting operation on the target to be imaged in a preset weak light scene and a preset normal illumination scene based on a preset frequency by using a preset image collecting device, to obtain a weak light image set in the preset weak light scene and a normal illumination image set in the preset normal illumination scene.

[0104] The image determining unit is configured to determine the image to be enhanced and the target normal illumination image from the weak light image set and the normal illumination image set based on a preset image collecting requirement.

[0105] In some specific embodiments, the image enhancement model includes a backbone network encoder module, a feature manifold estimation and establishment module, and a feature mapping selection module.

[0106] In some specific embodiments, the second information obtaining module 13 can specifically include:

[0107] The light distribution feature acquisition unit is configured to perform a preset overall light coverage map estimation operation on the image to be enhanced by using the backbone network encoder module in the target image enhancement model, so as to obtain the light distribution feature of the image to be enhanced.

[0108] The hierarchical distribution information acquisition unit is configured to input the light distribution feature into a deep network encoding layer of the backbone network encoder module, and perform light field distribution estimation and analysis on the backbone network encoder module to obtain the hierarchical distribution information of the image to be enhanced.

[0109] The second distribution estimation information acquisition unit is configured to acquire second distribution estimation information of the image to be enhanced based on the light distribution feature and the hierarchical distribution information.

[0110] In some specific embodiments, the mapping relationship establishing module 14 can specifically include:

[0111] The space establishing unit is configured to perform a preset information interaction fusion operation on the first distribution manifold information and the second distribution estimation information based on a preset dimension by using a first loss function and a second loss function of the feature manifold estimation and establishment module in the target image enhancement model, to establish a target high-dimensional manifold information space; the first loss function is configured to determine the information feature of the image to be enhanced, and the second loss function is configured to fuse the label feature of the target normal light image with the image to be enhanced.

[0112] The result acquisition unit is configured to perform sampling estimation on the image to be enhanced based on the light distribution feature of the image to be enhanced by using the feature manifold estimation and establishment module of the target image enhancement model, to obtain a corresponding target high-dimensional manifold estimation result.

[0113] The mapping function construction unit is configured to train a target operator based on the target high-dimensional manifold estimation result, optimize parameters of the target operator in combination with a preset operator constraint condition to obtain an optimized target operator, and construct a mapping function according to the optimized target operator.

[0114] The mapping relationship establishing unit is configured to establish a mapping relationship between a target high-dimensional feature information vector of a preset sample space and a feature information vector of a preset label space in the target high-dimensional manifold information space according to the mapping function.

[0115] In some specific embodiments, the enhancement result acquisition module 15 can specifically include:

[0116] The processed high-dimensional manifold information acquisition unit is configured to perform a preset data processing operation on the high-dimensional manifold information in the target high-dimensional manifold information space by using the third loss function and the fourth loss function in the feature mapping selection module, to obtain processed high-dimensional manifold information.

[0117] The projection set generation unit is configured to project the processed high-dimensional manifold information to a preset low-dimensional space according to the mapping relationship, to generate a corresponding projection set.

[0118] The target projection acquisition unit is configured to acquire a target projection corresponding to the image to be enhanced from the projection set according to a preset condition.

[0119] The target manifold distribution determination unit is configured to determine a target manifold distribution satisfying a preset image similarity condition based on the target projection corresponding to the image to be enhanced.

[0120] Further, the embodiment of the present application further discloses an electronic device, Figure 6 is an electronic device 20 structure diagram according to an exemplary embodiment, the contents in the figure cannot be considered as any limitation on the use range of the present application.

[0121] Figure 6 A structure schematic diagram of an electronic device 20 provided by the embodiment of the present application. The electronic device 20 can specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26. Wherein, the memory 22 is used for storing computer programs, the computer programs are loaded and executed by the processor 21, to realize the related steps in the weak light image enhancement method disclosed by any of the preceding embodiments. In addition, the electronic device 20 in the embodiment can be an electronic computer.

[0122] In the embodiment, the power supply 23 is used for providing working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which is not limited specifically herein; the input / output interface 25 is used for acquiring external input data or outputting data to the outside world, and the specific interface type can be selected according to the specific application needs, which is not limited specifically herein.

[0123] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0124] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, and can be Windows Server, NetWare, Unix, Linux, etc. In addition to including a computer program capable of implementing the low-light image enhancement method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 may further include a computer program capable of implementing other specific tasks.

[0125] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when executed by a processor, the computer program implements the aforementioned low-light image enhancement method. The specific steps of this method can be referred to the corresponding contents disclosed in the aforementioned embodiments and will not be repeated here.

[0126] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0127] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0128] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0129] Finally, it needs to be pointed out that in this document, relational terms such as first and second and the like can only be intended to distinguish one entity or operation from another entity or operation without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the statement "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus including the stated element.

[0130] The above detailed description of the technical solutions provided by the present application has been provided, and the principles and implementation modes of the present application have been described by applying specific examples. The above description of the examples is only for the purpose of helping to understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed, and the above description of the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for enhancing a weak light image, characterized in that: include: Acquire an image to be enhanced in a preset weak light scene and a target normal light image in a preset normal light scene, construct an initial image enhancement model, and optimize the initial image enhancement model to obtain a target image enhancement model; Inputting the target normal illumination image into the initial image enhancement model, and performing preset feature extraction and manifold distribution estimation operations on the target normal illumination image using the initial image enhancement model to obtain first distribution manifold information; Inputting the image to be enhanced into the target image enhancement model, and performing preset feature extraction and manifold distribution estimation operations on the image to be enhanced using the target image enhancement model to obtain second distribution estimation information; Using the target image enhancement model to fuse the first distribution manifold information and the second distribution estimation information to determine a target high-dimensional manifold information space, and based on the target high-dimensional manifold information space and using the target image enhancement model, establish a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a corresponding feature information vector in a preset label space; According to the mapping relationship, the target projection corresponding to the image to be enhanced is obtained based on preset conditions, the target manifold distribution that meets the preset feature distribution conditions is determined based on the target projection corresponding to the image to be enhanced, and the target manifold distribution is decoded using the decoder in the target image enhancement model to obtain the target image enhancement result.

2. The method for enhancing weak light images according to claim 1, wherein: The step of acquiring the image to be enhanced in a preset weak light scene and the target normal light image in a preset normal light scene includes: Using a preset image acquisition device to perform preset periodic imaging acquisition operations on a target to be imaged in a preset weak light scene and a preset normal light scene based on a preset frequency, so as to obtain a weak light image set in the preset weak light scene and a normal light image set in the preset normal light scene; Based on a preset image acquisition requirement, an image to be enhanced and a target normal-light image are determined from the low-light image set and the normal-light image set.

3. The method for enhancing a weak light image according to claim 1 or 2, wherein: The image enhancement model includes a backbone network encoder module, a feature manifold estimation and establishment module, and a feature map selection module.

4. The method for enhancing weak light images according to claim 3, wherein: The using the target image enhancement model to perform preset feature extraction and manifold distribution estimation operations on the image to be enhanced to obtain second distribution estimation information includes: Using the backbone network encoder module in the target image enhancement model, a preset overall illumination coverage map estimation operation is performed on the image to be enhanced to obtain illumination distribution characteristics of the image to be enhanced; Inputting the illumination distribution feature into the deep coding layer of the backbone network encoder module, and performing light field distribution estimation and analysis on the backbone network encoder module to obtain hierarchical distribution information of the image to be enhanced; Second distribution estimation information of the image to be enhanced is obtained based on the illumination distribution feature and the hierarchical distribution information.

5. The method for enhancing weak light images according to claim 3, wherein: The step of fusing the first distribution manifold information and the second distribution estimation information using the target image enhancement model to determine a target high-dimensional manifold information space includes: Using the first loss function and the second loss function of the feature manifold estimation and establishment module in the target image enhancement model, based on a preset dimension, a preset information interactive fusion operation is performed on the first distribution manifold information and the second distribution estimation information to establish a target high-dimensional manifold information space; The first loss function is used to determine the information features of the image to be enhanced, and the second loss function is used to fuse the label features of the target normal illumination image with the image to be enhanced.

6. The method for enhancing a weak light image according to claim 4, wherein: The method of establishing a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a corresponding feature information vector in a preset label space by using the target image enhancement model based on the target high-dimensional manifold information space includes: Utilizing the feature manifold estimation and establishment module of the target image enhancement model, sampling and estimating the image to be enhanced based on the illumination distribution characteristics of the image to be enhanced, so as to obtain a corresponding target high-dimensional manifold estimation result; Training a target operator based on the target high-dimensional manifold estimation result, optimizing the parameters of the target operator in combination with preset operator constraints to obtain an optimized target operator, and constructing a mapping function according to the optimized target operator; According to the mapping function, a mapping relationship is established between the target high-dimensional feature information vector of the preset sample space and the feature information vector of the preset label space in the target high-dimensional manifold information space.

7. The method for enhancing a weak light image according to claim 6, wherein: The step of obtaining a target projection corresponding to the image to be enhanced based on a preset condition according to the mapping relationship, and determining a target manifold distribution that satisfies a preset feature distribution condition based on the target projection corresponding to the image to be enhanced, includes: performing a preset data processing operation on the high-dimensional manifold information in the target high-dimensional manifold information space using the third loss function and the fourth loss function in the feature map selection module to obtain processed high-dimensional manifold information; Projecting the processed high-dimensional manifold information onto a preset low-dimensional space according to the mapping relationship to generate a corresponding projection set; Acquire a target projection corresponding to the image to be enhanced from the projection set according to a preset condition; Determining a target manifold distribution that satisfies a preset image similarity condition based on the target projection corresponding to the image to be enhanced; Among them, the third loss function is used to eliminate information that meets the preset redundancy condition in the high-dimensional manifold information of the target high-dimensional manifold information space, and the fourth loss function is used to expand the high-dimensional manifold information of the target high-dimensional manifold information space.

8. A device for enhancing weak light images, characterized in that: include: The model acquisition module is used to acquire the image to be enhanced in a preset weak light scene and the target normal light image in a preset normal light scene, construct an initial image enhancement model, and optimize the initial image enhancement model to obtain the target image enhancement model; a first information acquisition module, configured to input the target normal illumination image into the initial image enhancement model, and perform preset feature extraction and manifold distribution estimation operations on the target normal illumination image using the initial image enhancement model to obtain first distribution manifold information; a second information acquisition module, configured to input the image to be enhanced into the target image enhancement model, and perform preset feature extraction and manifold distribution estimation operations on the image to be enhanced using the target image enhancement model to obtain second distribution estimation information; a mapping relationship establishment module, configured to use the target image enhancement model to fuse the first distribution manifold information with the second distribution estimation information to determine a target high-dimensional manifold information space, and to establish a mapping relationship between a target high-dimensional feature information vector in a preset sample space and a corresponding feature information vector in a preset label space using the target image enhancement model based on the target high-dimensional manifold information space; An enhancement result acquisition module is used to obtain the target projection corresponding to the image to be enhanced based on preset conditions according to the mapping relationship, determine the target manifold distribution that meets the preset feature distribution conditions based on the target projection corresponding to the image to be enhanced, and use the decoder in the target image enhancement model to decode the target manifold distribution to obtain the target image enhancement result.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the method for enhancing a weak-light image according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that Used to store a computer program; wherein, when the computer program is executed by a processor, the weak-light image enhancement method according to any one of claims 1 to 7 is implemented.

Images