Method for enhancing weak light image in narrow space

Abstract

The invention discloses a method for enhancing a weak light image in a narrow space, and the method comprises the steps: obtaining a to-be-processed image, inputting the to-be-processed image into a trained dual-conversion training architecture, and outputting a clear target image after the to-be-processed image is processed through the dual-conversion training architecture; wherein the double-conversion training architecture comprises a bright state sub-model, a dark state sub-model and an enhanced mapping library embedded in the bright state sub-model; the bright state sub-model is used for receiving a dark noise image and predicting and outputting a corresponding clear target image; the dark state sub-model estimates distribution parameters required for generating the dark noise image from the clear target image output by the bright state sub-model based on the statistical hypothesis that the pixel intensity of the dark noise image obeys Gaussian mixture-Poisson distribution; and generating dark noise pixels according to the estimated parameter sampling, and synthesizing a dark noise image. The method can effectively solve the problems of image color distortion detection and noise suppression in a dark environment and a narrow and small space in the prior art.

Description

Technical Field

[0001] This invention belongs to the field of aerospace manufacturing technology and relates to a method for image enhancement processing in low light conditions in confined spaces. Background Technology

[0002] The aerospace manufacturing industry has an urgent need for high-precision visual inspection, especially in confined spaces such as aircraft engine compartments, fuel tank enclosed areas, fuel tank pipelines, and the interior of the fuselage. Visual sensors, such as endoscopes and miniature cameras, are required for defect detection, assembly verification, and maintenance monitoring. However, these scenarios generally present the following imaging challenges: 1. Low-light environments: In confined spaces where sufficient lighting equipment cannot be installed, the requirements for the responsiveness and detection rate of visual sensor components are extremely high. Under insufficient light radiation, the image signal-to-noise ratio is significantly reduced, and sensor noise (such as thermal noise, shot noise, 1 / f noise) and low-light color shift problems become prominent, making it impossible for visual images to accurately reproduce the real scene.

[0003] 2. Mechanical jitter interference: Camera jitter caused by robotic arm movement, processing equipment vibration, or manual hand-held shooting can lead to motion blur and pixel displacement, further exacerbating image degradation, reducing visual inspection efficiency, and posing a risk of missed or missed detections.

[0004] 3. Complex noise coupling: High sensitivity gain amplification noise under low light conditions, combined with jitter blur and interference from metal surface reflections, forms a non-uniform, multi-modal noise distribution.

[0005] 4. For the future of intelligent manufacturing, robots are considered the best physical entities for embodied intelligence. Visual image processing not only provides embodied robots with the ability to perceive their environment, but also provides a solid technological foundation for them to perform complex tasks, collaborate with humans, and adapt to dynamic environments. Further optimizing image quality and improving detection efficiency and environmental perception capabilities are of great significance for promoting the digital transformation and upgrading of aerospace manufacturing.

[0006] The aforementioned problems severely impact the detection in confined spaces and the image quality of machine vision. There is an urgent need to develop an image enhancement technology for low-light environments to improve visual image quality and preserve image details. Summary of the Invention

[0007] The purpose of this invention is to provide a method for enhancing images in low light conditions in confined spaces, which can be used to solve the problems of color distortion and noise suppression in existing dark environments and confined spaces.

[0008] To achieve the above objectives, the present invention employs the following technical solution: A method for enhancing low-light images in confined spaces includes: The image to be processed is acquired and input into a pre-trained dual-transformation training architecture. The dual-transformation training architecture then outputs a clear target image after processing the image to be processed. Wherein: The dual-conversion training architecture includes a bright-state sub-model, a dark-state sub-model, and an enhancement mapping library embedded in the bright-state sub-model. The bright-state sub-model receives the dark noise image and predicts the corresponding sharp target image. The dark-state sub-model estimates the distribution parameters required to generate the dark noise image from the sharp target image output by the bright-state sub-model based on the statistical assumption that the pixel intensity of the dark noise image follows a mixture of Gaussian and Poisson distributions. Dark noise pixels are sampled and generated according to the estimated parameters to synthesize the dark noise image.

[0009] Furthermore, the bright-state sub-model and the dark-state sub-model form a closed-loop domain transformation relationship; the dark noise image input to the bright-state sub-model is the output of the dark-state sub-model; the clear target image output by the bright-state sub-model is the input of the dark-state sub-model. This constitutes a closed-loop dual transformation learning mechanism for collaborative training, enabling the two sub-models to generate training samples for each other.

[0010] Furthermore, the bright state sub-model includes a denoising and brightening module and an attention fusion module, which are trained collaboratively through feature interaction and parameter sharing; The denoising and brightening module consists of multiple identical sub-modules. The dark noise image is input into the bright state sub-model as the input to its first sub-module. The output of the first sub-module is used as the input to the second sub-module and the enhancement mapping library. The output of the last sub-module is used as the input to the first sub-module in the attention fusion module. The output of each of the remaining sub-modules in the denoising and brightening module is used as the input to its next-level sub-module and the corresponding sub-module in the attention fusion module. The input feature map of each submodule in the attention fusion module consists of two parts: The input to the first submodule of the attention fusion module includes the outputs of the last two submodules of the denoising and brightening module; the input to the second submodule of the attention fusion module is the output of the previous submodule and the output of the third-to-last submodule of the denoising and brightening module, and so on; the input to the last submodule of the attention fusion module is the output of the previous submodule and the output of the second submodule of the denoising and brightening module, and its output serves as the input to the enhancement mapping library.

[0011] Furthermore, each submodule of the noise reduction and brightening module adopts a cascaded structure, sequentially containing a first 1×1 convolutional layer for compressing the number of channels in the input feature map; after the first 1×1 convolutional layer, it splits into two branches, each branch consisting of two 3×3 convolutional layers connected in sequence for image feature extraction; the two branches are combined and connected to a second 1×1 convolutional layer to restore the number of channels in the extracted feature map to the target output channel number; the output of the second 1×1 convolutional layer is subtracted element-wise from the original input of the submodule and then split into two branches, one branch using a third 1×1 convolutional layer and the other branch using two 3×3 convolutional layers connected in sequence; the two branches are combined and added element-wise to obtain the final output of the submodule.

[0012] Furthermore, the attention fusion module comprises multiple identical sub-modules, each including: Region Interest Pooling Layer: Used to locate and extract key feature regions from the input feature map, and transform the regions into feature maps of fixed size, providing spatially consistent feature representations for subsequent processing; Two sequentially connected 3×3 convolutional layers: Used to perform convolution operations on the feature maps output by the Region Interest Pooling Layer to extract local spatial features; Upsampling Layer: Used to restore the spatial resolution of the feature map after 3×3 convolution processing to the resolution of the input feature map; Sigmoid Function: Used to apply the Sigmoid activation function pixel by pixel to the upsampled feature map to generate an attention weight map in the range [0, 1]; Element-wise Multiplication Unit: In this unit, the attention weight map is multiplied with the input feature map of this submodule and then divided into two branches. One branch uses a 1×1 convolutional layer, and the other branch uses two sequentially connected 3×3 convolutional layers. The two branches are then summed element-wise as the final output of this submodule.

[0013] Furthermore, the enhancement mapping library employs a neural network structure to generate clear target images with different brightness and peak signal-to-noise ratios for the input feature map. The input feature map of the enhancement mapping library includes the output of the last sub-module of the attention fusion module and the output of the first sub-module of the denoising and brightening module.

[0014] Furthermore, the enhanced mapping library utilizes diverse loss functions. Optimize the parameter update process:

[0015] in, It is a point-to-point regression model. To perceive loss; The output of the neural network is the first A clear target image, Represents a real image; Indicates the weights of the loss function. The number of sharp target images output; The definition is as follows:

[0016] in, It is a structural similarity index. λ 1 and λ 2 represents the image correlation coefficient, with values ​​of 1.0 and 0.1 respectively. It is an L1 norm; The definition is as follows:

[0017] in, This indicates that the perceptual loss is calculated using a pre-trained VGG-19 network; It is an L2 norm.

[0018] Furthermore, the dark-state sub-model is implemented by a feature extraction network, which embeds a residual network module, a noise generator, a neural network module, and a global tone adjustment module; it includes two modes in total: One mode consists of a residual network module and a noise generator. The residual network module uses skip connections to ensure stable transfer of feature information between layers. The noise generator is used to estimate noise parameters from the clear target image output by the bright sub-model to extract noise patterns. Another mode includes a neural network module and a global tone adjustment module. The neural network module is used to extract feature information from the clear target image, and the global tone adjustment module is used to adjust the overall brightness, contrast, and color balance of the deep feature map output by the neural network module to simulate the visual degradation effect in low-light environments and output a pseudo-dark image. The extracted noise patterns are fused with the output pseudo-dark image to obtain a dark noise image as the output of the dark sub-model.

[0019] Furthermore, the dual-conversion training architecture adopts a phased training mode: In the first stage, the dark state sub-model and the bright state sub-model are pre-trained. The dark state sub-model is trained using benchmark images from the benchmark dataset, while the bright state sub-model is trained using low-quality images from the low-quality dataset. In the second stage, the dark state sub-model and the bright state sub-model are jointly trained. The outputs of the bright state sub-model and the dark state sub-model are used as each other's inputs, thus forming a closed-loop dual-transformation learning mechanism for collaborative training, enabling the two sub-models to generate new training samples for each other.

[0020] A terminal device includes a processor, a memory, and a computer program stored in the memory; when the processor executes the computer program, it implements a method for enhancing low-light images in confined spaces.

[0021] A computer-readable storage medium storing a computer program; when executed by a processor, the computer program implements a method for enhancing a low-light image in a confined space.

[0022] Compared with the prior art, the present invention has the following technical features: 1. Improve image quality and enhance the visual experience.

[0023] In low-light environments, traditional methods struggle to simultaneously enhance brightness and preserve detail, leading to distortion or noise issues. This invention utilizes a neural network dual-transformation learning architecture to achieve intelligent optimization processing of low-light images. The bright-state sub-model extracts and enhances key information, while the dark-state sub-model suppresses noise and restores details. This design improves the overall brightness and contrast of the image, preserves texture and color, resulting in a clearer, more natural enhanced image and improved visual experience. A noise generator enhances adaptability to complex noise, ensuring high-quality image output even in low-light conditions.

[0024] 2. It has strong versatility and adaptability.

[0025] The dual-conversion learning architecture proposed in this invention is suitable for image enhancement needs in various low-light environments, including confined space detection and machine vision. The bright-state sub-model and the dark-state sub-model work together to maintain stable performance under complex lighting conditions, avoiding the distortion or over-enhancement problems caused by traditional methods with a single processing mode. This technology allows for flexible adjustment of model parameters and training strategies, quickly adapting to the application requirements of different scenarios.

[0026] 3. Highly efficient training, low resource consumption, and economical.

[0027] This invention employs a dual-transformation learning architecture to reduce model training time and computational resource consumption. By training bright-state and dark-state sub-models in parallel, it avoids the performance bottleneck of traditional single-path models in complex scenarios. The residual network design improves learning efficiency, reduces the gradient vanishing problem, and makes the training process more stable and efficient. In practical applications, this technology reduces hardware deployment costs and shortens the training and optimization cycle, demonstrating economic practicality.

[0028] Based on the above characteristics, this invention provides an efficient, intelligent and highly adaptable ideal solution for image enhancement in low-light environments. It is applicable to fields such as confined space detection, industrial machine vision, and security monitoring, significantly improving image quality and practical application effects. Attached Figure Description

[0029] Figure 1 This is a structural diagram of the dual-conversion training architecture in this invention; Figure 2 This is a structural diagram of the noise reduction and brightness enhancement module; Figure 3 This is a structural diagram of the attention fusion module; Figure 4 This is a comparison image before and after image enhancement in one embodiment of the present invention. Detailed Implementation

[0030] This invention provides a method for image enhancement in confined spaces and low light conditions, addressing challenges such as low signal-to-noise ratio and poor image quality in visual inspection images of confined spaces like aircraft engine compartments, fuel tank enclosed areas, fuel tank pipelines, and the aircraft interior, which suffer from unwanted objects, defects, and poor assembly quality. This invention innovatively proposes an end-to-end dual-conversion neural network training architecture with collaborative training capabilities. This architecture employs a modular design, integrating two dedicated models: a bright state sub-model and a dark state sub-model. This innovative architecture significantly improves the quality assessment metrics of images in low-light environments, including Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM), while effectively reducing the computational resource consumption required for model training and inference. Furthermore, this architecture introduces the design concept of a bright state enhancement mapping library, creatively solving the inherent complex many-to-one (or one-to-many) mapping problem in image brightening. By constructing a diverse set of brightness features, the system can adaptively learn and generate enhancement results under different lighting conditions, meeting the visual preferences of different scenarios while maintaining low storage space consumption and computational complexity. The method of this invention can improve image quality in confined spaces or dark environments, thereby increasing the detection efficiency and accuracy of visual inspection; at the same time, it can enhance the perception ability and dynamic environment adaptability of intelligent robots in complex environments.

[0031] See appendix Figures 1 to 3 The present invention provides a method for enhancing low-light images in confined spaces, comprising the following steps: Step 1: Image acquisition and dataset construction.

[0032] Construct a low-quality dataset, which contains low-light and high-noise images; at the same time, collect the corresponding benchmark images to form a benchmark dataset; the low-quality dataset and the benchmark dataset together constitute the training set.

[0033] Specifically, the low-light and high-noise images refer to images acquired using conventional image acquisition equipment (such as ordinary cameras) in a closed, low-brightness experimental environment simulating a real spatial scene; the reference image refers to an image acquired using professional equipment (such as a professional high-resolution camera) in an open, high-brightness environment that corresponds to the low-quality images.

[0034] Step 2: Construct a dual-conversion training architecture.

[0035] Construct a dual-transformation training architecture, including a bright-state sub-model, a dark-state sub-model, and an enhancement mapping library embedded within the bright-state sub-model, such as... Figure 1 As shown; where the bright state sub-model is used to receive the dark noise image and predict the corresponding clear target image; the dark state sub-model is based on the statistical assumption that the pixel intensity of the dark noise image follows a mixture Gaussian-Poisson distribution, and its function is to estimate the distribution parameters required to generate the dark noise image from the clear target image output by the bright state sub-model; and to sample and generate dark noise pixels according to the estimated parameters to synthesize the dark noise image.

[0036] The bright-state sub-model and the dark-state sub-model form a closed-loop domain transformation relationship: The dark, noisy image input to the bright-state sub-model becomes the output of the dark-state sub-model; conversely, the sharp target image output by the bright-state sub-model becomes the input of the dark-state sub-model. This forms a closed-loop dual-transformation learning mechanism for collaborative training, enabling the two sub-models to generate training samples for each other. Specifically, the dark-state sub-model generates dark, noisy images as training samples for the bright-state sub-model, and the bright-state sub-model generates sharp target images as training samples for the dark-state sub-model. This dual-transformation training architecture provides self-supervised signals, utilizing the input / output sample pairs generated by each other to provide supervision for the other's image transformation / reconstruction task in the absence of paired real data.

[0037] The purpose of using a one-to-one mapping between the bright and dark sub-models is to increase and decrease the size of the image feature space, thereby increasing the receptive field of the neural network, expanding the image capture area, and increasing the amount of image information that the neural network can process.

[0038] The specific structure of the dual-conversion training architecture is as follows: (1) Bright state sub-model.

[0039] The bright state sub-model includes a denoising and brightening module and an attention fusion module; the denoising and brightening module and the attention fusion module are trained collaboratively through feature interaction and parameter sharing to jointly optimize the image enhancement effect.

[0040] (1-1) Noise reduction and brightness enhancement module.

[0041] The noise reduction and brightening module includes multiple identical sub-modules; such as Figure 1As shown, the dark noise image is input into the bright state sub-model as the input to its first sub-module. The output of the first sub-module is used as the input to the second sub-module and the enhancement mapping library. The output of the last sub-module is used as the input to the first sub-module in the attention fusion module. The output of each of the remaining sub-modules in the denoising and brightening module is used as the input to its next-level sub-module and the corresponding sub-module in the attention fusion module.

[0042] like Figure 2 As shown, each sub-module is implemented using a 1×1 convolutional network to increase image input features, reduce computation, and perform denoising and brightness estimation functions; based on the 1×1 convolution operation, cross-channel feature fusion is achieved while keeping the feature map spatial dimension unchanged.

[0043] Specifically, each submodule adopts a cascaded structure, sequentially containing a first 1×1 convolutional layer to compress the number of channels in the input feature map, thereby significantly reducing the number of model parameters. After the first 1×1 convolutional layer, it splits into two branches, each consisting of two 3×3 convolutional layers connected in sequence, for performing the core image feature extraction. The two branches are then combined and connected to a second 1×1 convolutional layer to restore the number of channels in the feature map to the target output channel number. The output of the second 1×1 convolutional layer is subtracted element-wise from the original input of the submodule and then split into two branches. One branch uses a third 1×1 convolutional layer, and the other branch uses two 3×3 convolutional layers connected in sequence. The two branches are then combined element-wise and added together to obtain the final output of the submodule.

[0044] (1-2) Attention fusion module.

[0045] The attention fusion module is used to generate region-channel maps and effectively fuse image features at different levels; the attention fusion module also includes several identical sub-modules, such as... Figure 3 As shown, each submodule includes: Region of Interest Pooling (RoI Pooling): Locates and extracts key feature regions from the input feature map, transforming these regions into fixed-size feature maps to provide spatially consistent feature representations for subsequent processing. Two sequentially connected 3×3 convolutional layers: Perform convolution operations on the feature maps output by the RoI Pooling layers to extract local spatial features. Upsampling layer: Restores the spatial resolution of the feature map after 3×3 convolution to the resolution of the input feature map. Sigmoid function: Applies the Sigmoid activation function pixel-by-pixel to the upsampled feature map, generating an attention weight map within the range [0, 1]. High-weight regions focus on key image features, while low-weight regions correspond to background or secondary information. Element-wise multiplication unit: In this unit, the attention weight map after applying the Sigmoid function is multiplied by the input feature map of this submodule, resulting in two branches. One branch uses a 1×1 convolutional layer, and the other branch uses two sequentially connected 3×3 convolutional layers. The two branches are then summed element-wise as the final output of this submodule.

[0046] (1-3) Feature cross-linking relationship between the noise reduction and brightening module and the attention fusion module.

[0047] The attention fusion module has two fewer sub-modules than the noise reduction and brightening module, such as... Figure 1 In the example, the noise reduction and brightening module contains 5 sub-modules, and the attention fusion module contains 3 sub-modules.

[0048] The input feature map of each submodule in the attention fusion module consists of two parts: The input to the first submodule of the attention fusion module includes the outputs of the last two submodules of the denoising and brightening module; the input to the second submodule of the attention fusion module is the output of the previous submodule and the output of the third-to-last submodule of the denoising and brightening module, and so on; the input to the last submodule of the attention fusion module is the output of the previous submodule and the output of the second submodule of the denoising and brightening module, and its output serves as the input to the enhancement mapping library.

[0049] (1-4) Enhance the mapping library.

[0050] This enhancement mapping library generates sharp target images with varying brightness and peak signal-to-noise ratios from input feature maps. It employs a neural network architecture, such as the U-Net network. The input feature map includes the output of the last submodule of the attention fusion module and the output of the first submodule of the denoising and brightening module. The library utilizes diverse loss functions. The parameter update process is optimized to ensure that the generated multiple sharp target images are reliable; that is, by utilizing the constraints of a diversified loss function, the generated sharp target images cover a variety of possible and reasonable images. Diversified loss function Specifically as follows:

[0051] in, It is a point-to-point regression model. Perceptual loss is used to ensure that the output of each feature branch of the neural network is as close as possible to the original. Figure 1 To; The output of the neural network is the first A clear target image, This represents real images, i.e., low-quality images in the training set; Indicates the weights of the loss function. The number of clear target images output.

[0052] The definition is as follows:

[0053] in, It is a structural similarity index. λ 1 and λ 2 represents the image correlation coefficient, with values ​​of 1.0 and 0.1 respectively. It is an L1 norm.

[0054] The definition is as follows:

[0055] in, This indicates that the perceptual loss is calculated using a pre-trained VGG-19 network; It is an L2 norm.

[0056] (2) Dark state sub-model.

[0057] The dark-state sub-model is based on the statistical assumption that the pixel intensity of the dark noise image follows a mixture of Gaussian and Poisson distributions. It estimates the distribution parameters required to generate the dark noise image from the clear target image output by the bright-state sub-model. Dark noise pixels are then sampled and generated according to the estimated parameters to synthesize the dark noise image.

[0058] The dark-state sub-model is implemented by a feature extraction network, which embeds a residual network module, a noise generator, a neural network module, and a global tone adjustment module; it includes two modes: One mode consists of a residual network module and a noise generator. The residual network module utilizes skip connections to effectively mitigate the gradient vanishing problem during feature extraction network training, ensuring stable transfer of feature information between layers. The noise generator is used to estimate noise parameters from the clear target image output by the bright-state sub-model to extract noise patterns. Another mode includes a neural network module and a global tone adjustment module. The neural network module is used to extract feature information from the clear target image, and the global tone adjustment module is used to adjust the overall brightness, contrast, and color balance of the deep feature map (or decoded image) output by the neural network module to simulate the visual degradation effect in low-light environments and output a pseudo-dark-state image. The extracted noise patterns are fused with the output pseudo-dark-state image to inject noise characteristics matching real low-light images into the pseudo-dark-state image, improving the realism of the simulation, and obtaining a noisy dark image with low-light conditions as the output of the dark-state sub-model.

[0059] Step 3: Use the training set to complete the phased training of the dual-transformation training architecture.

[0060] In the first stage, the dark state sub-model and the bright state sub-model are pre-trained, with each stage lasting 5 epochs.

[0061] The dark state sub-model is trained using benchmark images from the benchmark dataset, while the bright state sub-model is trained using low-quality images from the low-quality dataset.

[0062] During pre-training of the dark and bright sub-models, the resolution of the sample images (baseline image and low-quality image) in the corresponding dataset is reduced to 1 / 8 of the original image, and bilateral filtering is applied to the downsampled sample images for noise reduction. The sub-models are then trained. Based on this, the training focuses on the mean estimation branch to minimize the impact of noise on the mean estimation. This stage uses a loss function for optimization training to ensure that the sub-models can more accurately capture the structural information of the images.

[0063] In the second stage, a dual-transformation training architecture is jointly trained at full resolution.

[0064] The dark-state and bright-state sub-models are jointly trained for a total of n epochs (the exact number depends on the dataset). The sharp target image output by the bright-state sub-model serves as the input to the dark-state sub-model, thus forming a closed-loop dual-transformation learning mechanism for co-training, allowing the two sub-models to generate new training samples for each other. During training, the batch size is set to 1. A two-stage strategy is used to set the learning rate: the initial learning rate is set to 10. -4 And after completing epoch / 2, reduce the learning rate to 10. -5 .

[0065] To improve the model's generalization ability, multi-scale preprocessing was performed on the input sample images for the dark and bright sub-models during the joint training phase: first, the images were scaled to between 0.8 and 1.0 times their original size, and then a 512×512 image patch was randomly cropped as a training sample. Data augmentation was then performed using random flipping and rotation. Simultaneously, a reflection-filling strategy was employed during training to reduce the impact of boundary effects. For the optimizer, the Adam optimizer was used for model updates, with a momentum parameter of 0.9 and a weight decay coefficient of 0.0001. The entire experiment was implemented using the PyTorch framework. This setup ensures the model has strong adaptability and robustness under different lighting conditions.

[0066] Keeping the pre-trained mean estimation component unchanged, only the parameters of the full model are updated. Specifically, a loss function is used to optimize the overall output, ensuring that the dual-transformation training architecture generates results highly consistent with the true values. This staged training strategy helps to further improve the overall performance and detail recovery capability of the model while ensuring the accuracy of structural information.

[0067] In practical applications, the image to be processed is acquired and input into the dark state sub-model in the trained dual-conversion training architecture. The output of the dark state sub-model is used as the input of the bright state sub-model. Finally, the bright state sub-model outputs the clear target image after processing the image to be processed.

[0068] Model performance using the dual-transformation training architecture: Evaluations showed that this dual-conversion training architecture significantly improved the visual quality of low-light images, producing results with high clarity and complete detail retention, such as... Figure 4 As shown, the peak signal-to-noise ratio (PSNR) of the output image reached 22.73 dB, which is 2 times higher than that of the original image, the structural similarity index (SSIM) reached 0.86, and the training time of the model was only 63.2 ms.

[0069] This embodiment, through the innovative neural network dual-transformation model design and the synergistic training of the bright and dark sub-models, significantly improves the peak noise ratio and image quality of images in low-light environments, further enhancing the confidence of visual detection results. At the same time, the model has good versatility and is suitable for image quality improvement needs under various working conditions.

[0070] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for narrow space low light image enhancement, characterized in that, The method comprises the following steps: obtaining a to-be-processed image, inputting the to-be-processed image into a trained dual conversion training architecture, and outputting a clear target image processed by the to-be-processed image by using the dual conversion training architecture; wherein: the dual conversion training architecture comprises a bright state submodel, a dark state submodel, and an enhanced mapping library embedded in the bright state submodel; the bright state submodel is used for receiving a dark noise image and predicting and outputting a corresponding clear target image; the dark state submodel estimates and generates distribution parameters required for generating the dark noise image from the clear target image output by the bright state submodel based on the statistical assumption that the pixel intensity of the dark noise image obeys a mixed Gaussian-Poisson distribution; and the dark noise image is synthesized according to the estimated parameters.

2. The method of tight-space low-light image enhancement of claim 1, wherein, The bright state submodel and the dark state submodel form a closed-loop domain conversion relationship; the dark noise image input into the bright state submodel is the output of the dark state submodel; the clear target image output by the bright state submodel is the input of the dark state submodel, thereby forming a closed-loop dual conversion learning mechanism for collaborative training, so that the two submodels can generate training samples for each other.

3. The method of tight-space low-light image enhancement of claim 1, wherein, The bright state submodel comprises a denoising and brightening module and an attention fusion module, and the denoising and brightening module and the attention fusion module are collaboratively trained through feature interaction and parameter sharing; The denoising and brightening module comprises a plurality of identical submodules; the dark noise image is input into the bright state submodel as the input of the first submodule, the output of the first submodule is input into the second submodule and the enhanced mapping library, and the output of the last submodule is input into the first submodule of the attention fusion module; and the output of each submodule in the denoising and brightening module is input into the next-level submodule and the corresponding submodule of the attention fusion module. The input feature map of each submodule in the attention fusion module comprises two parts: The input of the first submodule of the attention fusion module comprises the outputs of the last two submodules of the denoising and brightening module; The input of the second submodule of the attention fusion module comprises the output of the previous submodule and the output of the third-from-last submodule of the denoising and brightening module, and so on; the input of the last submodule of the attention fusion module comprises the output of the previous submodule and the output of the second submodule of the denoising and brightening module, and the output thereof is input into the enhanced mapping library.

4. The method of tight-space low-light image enhancement of claim 1, wherein, Each submodule of the denoising and brightening module adopts a cascaded structure, sequentially comprising a first 1×1 convolutional layer for compressing the channel number of the input feature map; after the first 1×1 convolutional layer, two branches are formed, each branch comprising two 3×3 convolutional layers connected in sequence for image feature extraction; the two branches are combined and connected to a second 1×1 convolutional layer for restoring the channel number of the extracted feature map to a target output channel number; The output of the second 1×1 convolutional layer and the original input of the submodule are subtracted element by element and then divided into two branches, one branch adopts a third 1×1 convolutional layer, and the other branch adopts two 3×3 convolutional layers connected in sequence, the two branches are combined and added element by element, and the result is taken as the final output of the submodule.

5. The method of tight-space low-light image enhancement of claim 1, wherein, The attention fusion module comprises a plurality of identical submodules, and each submodule comprises: The region of interest pooling layer is used for locating and extracting key feature regions from the input feature map and transforming the regions into a fixed-size feature map to provide a spatially consistent feature representation for subsequent processing; the two 3*3 convolution layers connected in sequence are used for performing convolution operations on the feature map output by the region of interest pooling layer to extract local spatial features; the up-sampling layer is used for restoring the spatial resolution of the feature map processed by the 3*3 convolution to the resolution of the input feature map; the sigmoid function is used for applying a sigmoid activation function to the up-sampled feature map pixel by pixel to generate an attention weight map in the range of [0, 1]; and the element-wise multiplication unit is used for multiplying the attention weight map with the input feature map of the sub-module, and then dividing the result into two branches, one of which adopts a 1*1 convolution layer, and the other of which adopts two 3*3 convolution layers connected in sequence, and the two branches are added element by element after being combined to serve as the final output of the sub-module.

6. The method of tight-space low-light image enhancement of claim 1, wherein, The enhanced mapping library adopts a neural network structure and is used for generating clear target images with different brightness and peak signal-to-noise ratios for the input feature map, and the input feature map of the enhanced mapping library includes the output of the last sub-module of the attention fusion module and the output of the first sub-module of the de-noising and brightening module.

7. The method of tight-space low-light image enhancement of claim 1, wherein, Enhanced mapping repository utilizes diversified loss function Optimized parameter update process: wherein, is a point-to-point regression model, is a perceptual loss; is the first clear target image output by the neural network, represents the real image; represents the loss function weight, is the number of clear target images output. The following definitions apply: wherein, is a structural similarity index, The dark state sub-model is implemented by a feature extraction network, and the feature extraction network is embedded with a residual network module, a noise generator, a neural network module and a global tone adjustment module; and includes two modes in total: 1 and One mode is composed of the residual network module and the noise generator, the residual network module uses a skip connection to ensure stable transmission of feature information between levels; and the noise generator is used for estimating noise parameters from the clear target image output by the bright state sub-model to extract a noise pattern; the other mode includes the neural network module and the global tone adjustment module, the neural network module is used for extracting feature information of the clear target image, and the global tone adjustment module is used for adjusting the overall brightness, contrast and color balance of the deep feature map output by the neural network module to simulate the visual degradation effect in a dark light environment and output a pseudo-dark state picture; and the extracted noise pattern and the output pseudo-dark state picture are fused to obtain a dark noise image as the output of the dark state sub-model. 2 are image correlation coefficients, respectively, having values of 1.0 and 0.1, respectively, is an L1 norm; The following definitions apply: wherein, represents computing perceptual loss using a pre-trained VGG-19 network; is the L2 norm.

8. The method of tight-space low-light image enhancement of claim 1, wherein, The dual conversion training architecture adopts a phased training mode: In the first phase, the dark state sub-model and the bright state sub-model are pre-trained; wherein the dark state sub-model is trained by using the reference images in the reference data set; and the bright state sub-model is trained by using the low-quality images in the low-quality data set; 9. The method of tight-space low-light image enhancement of claim 1, wherein, In the second phase, the dark state sub-model and the bright state sub-model are jointly trained; the outputs of the bright state sub-model and the dark state sub-model are used as the inputs of each other, thereby forming a closed-loop dual conversion learning mechanism for collaborative training, so that the two sub-models can generate new training samples for each other. The processor executes the computer program to implement the method for enhancing a weak light image in a narrow space according to any one of claims 1-9. ​ 10.A terminal device, comprising a processor, a memory, and a computer program stored in the memory; characterized in that, ​

Images