Low-illumination space target image brightness enhancement method and system
By constructing a lightweight convolutional neural network based on unsupervised deep learning and the Retinex algorithm, the brightness of low-light spatial target images is decomposed and enhanced, solving the problem of insufficient image brightness under low-light conditions. This achieves efficient image enhancement and detail preservation, adapting to the changing lighting conditions in the space environment.
Patent Information
- Application Number
- CN202411007554.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-07-25
AI Technical Summary
Existing technologies struggle to effectively enhance the brightness of space target images under low-light conditions while preserving important target details, and often introduce noise and detail distortion, making them unsuitable for the unstable lighting conditions in space.
A lightweight convolutional neural network based on unsupervised deep learning and the Retinex algorithm is used to construct a brightness enhancement network. The network is trained using a no-reference loss function and combines exposure loss, spatial loss, and illumination smoothing loss to decompose the image into illumination and reflection layers for image brightness enhancement. The image is then fused using a weighted average.
It significantly improves image brightness and visual quality under low-light conditions, preserves important space target information, adapts to different lighting conditions, and improves the efficiency and accuracy of space exploration and monitoring.
Smart Images

Figure CN119006346A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of on-orbit image processing, and in particular relates to a method and system for enhancing the brightness of a low-light space target image. Background Art
[0002] Images taken under low-light conditions usually suffer from many problems, such as increased noise, reduced contrast, and loss of details. These problems are particularly prominent in space environments, because the lighting conditions in space environments are not only low but also changeable. In addition, the particularity of space targets, such as satellites and space stations, makes the improvement of image quality particularly important and challenging. At present, common image brightness enhancement techniques include histogram equalization, local enhancement algorithms, and deep learning-based methods. Although these methods can effectively enhance image brightness and contrast under general conditions, they often introduce over-enhancement problems, such as noise amplification and detail distortion, when processing extreme low-light space images. In addition, these technologies are often not optimized for the special properties of space targets (such as material, shape, and reflective properties).
[0003] For example, the prior art with document number CN114862722A discloses a method for implementing image brightness enhancement and a processing terminal, the method comprising: exposing the original image to obtain N exposure images; converting each exposure image from RGB to YUV to obtain a YUV exposure image; extracting a color scale distribution map from the YUV exposure image, in which the color scale distribution map is divided into dark, mid-tone and highlight areas in order from small to large brightness, assigning a weight value to each pixel of the YUV exposure image, and assigning successively increasing weight values to the pixels in the dark, mid-tone and highlight areas in turn, taking the pixel values obtained by weighted averaging the pixels at the same position according to their respective weights as the target pixel value, reconstructing the target image according to the target pixel value by position, and the target image is used as the image after brightness enhancement. This prior art can not affect the original image area that was originally exposed normally, the brightness of the dark part is improved, and the original color is restored. The prior art with document number CN115861101A discloses a low-light image enhancement method based on deep separable convolution. It includes: step S1, collecting monitoring videos of tunnel construction during the day and night and dividing them into frame images to enhance low-light images; step S2, building a self-calibrated low-light image enhancement model based on deep separable convolution, determining model parameters, neural network loss function, and optimizing model performance to the best; step S3, using the trained low-light image enhancement network for dim day and night frame images to enhance image brightness and obtain images after light enhancement. The illumination estimated by this method maintains good smoothness, and obtains very superior performance in both image quality and reasoning speed, and can quickly improve image quality in complex tunnel environments.
[0004] It can be seen that, in the existing technology, for space targets, such as satellites or space debris, which usually have complex geometric shapes and material properties, whose performance in low-light conditions is completely different from that of objects on the ground, how to enhance the brightness of low-light space target images has not been proposed and provided by anyone.
[0005] Therefore, when enhancing such images, it is necessary to consider their unique reflective characteristics and the influence of background illumination, as well as how to retain important spatial target information while enhancing image brightness.
[0006] Given the shortcomings of existing technologies, it is particularly important to develop a brightness enhancement method specifically for low-light space target images. This method should be able to effectively handle image noise and improve the dynamic range of the image while retaining important target details. In addition, considering the real-time requirements of space missions, the method should also have high computational efficiency and adaptability so that it can be quickly adjusted and applied under different observation conditions and mission requirements. Summary of the invention
[0007] The technical problems to be solved by the present invention are:
[0008] The purpose of the present invention is to provide a method and system for enhancing the brightness of low-light space target images, so as to solve the problem that in the low-light environment that is common in space and in the unique space environment with extremely unstable lighting conditions, the acquired image brightness is often insufficient due to the obstruction of sunlight or the lack of cosmic background light, thereby affecting the image quality and subsequent image analysis and application.
[0009] The technical solution adopted by the present invention to solve the above technical problems is:
[0010] A method for enhancing the brightness of a low-light space target image, the method comprising:
[0011] Step 1: Get low-light images
[0012] The definition of low-light images is: low-light images refer to images acquired under extremely weak light conditions, with the characteristics of average brightness less than 30 luminance units, signal-to-noise ratio less than 20 dB, contrast less than 0.1, color fidelity CIEDE2000 color difference greater than 5, and dynamic range less than 40 dB; at the same time, the images must also meet the requirements of space on-orbit, including complex lighting conditions, special target characteristics and environmental adaptability; this definition can provide clear standards and guidance for the application of image brightness enhancement methods;
[0013] Step 2: Build a brightness enhancement network
[0014] A lightweight convolutional neural network based on unsupervised deep learning and the Retinex algorithm is constructed as a brightness enhancement network, and a lightweight convolutional neural network (CNN) is used to estimate the lighting components of low-light images.
[0015] The network receives a low-light image as input, decomposes the input image into an illumination layer and a reflection layer, and then outputs the reflection layer as the result of image brightness enhancement;
[0016] Step 3: Brightness enhancement network model training
[0017] During the network training process, the network is trained through no-reference loss functions such as exposure loss, spatial loss and lighting smoothness loss to avoid dependence on paired training data; the no-reference loss function can effectively train the network to achieve image enhancement without real labels; the exposure loss is used to brighten the underexposed areas in the image, the spatial loss ensures the spatial consistency of the image after enhancement, and the lighting smoothness loss encourages smooth changes in the lighting layer to conform to the natural changes in lighting in the real world;
[0018] Step 4: Fusion processing
[0019] After obtaining the image with enhanced lighting conditions through the neural network, it is fused with the original image; the pixel values corresponding to the two images are weighted averaged, and the fusion effect is determined by calculating the peak signal-to-noise ratio; if it does not meet the requirements, return to step three for network training;
[0020] Step 5: Verify the data enhancement effect after brightness enhancement network training:
[0021] After obtaining the fused image, the network model is tested and optimized. According to the training effect and performance of the network model, the parameters are adjusted and continuously optimized until the enhanced low-light space target image quality assessment requirements are met.
[0022] Furthermore, in step one, the low-light image refers to a low-light image taken by a satellite or space detection equipment in a space environment, or produced using a simulation engine as an input.
[0023] Further, in step 2, the lightweight convolutional neural network (CNN) is based on the U-Net architecture and is used for illumination estimation of low-light images. The network comprises seven layers, each layer uses a 3×3 convolution kernel, the number of output channels of the first six layers is 8, and the number of output channels of the last layer is 1. The network structure includes skip connections to maintain image details and reduce information loss during training;
[0024] The brightness enhancement network estimates the illumination and reflection components respectively. The illumination estimation uses the deep features extracted by the encoder to estimate the illumination component of the image. The reflection estimation divides the original input image by the estimated illumination component to obtain the reflection component of the image. The established Retinex model is as follows:
[0025] I=R×L
[0026] In the formula, the image I can be modeled as the element-by-element product of the illumination component L and the reflection component R, where L represents the illumination of the object by the light source, which usually changes slowly and is evenly distributed; R represents the color and properties of the object itself, which is related to the inherent properties of the object and is not affected by the light intensity.
[0027] Furthermore, the specific steps in step three are:
[0028] (1) Initializing the brightness enhancement network model: Based on the pre-defined lightweight convolutional neural network architecture, all weights of the brightness enhancement network model are initialized using the Xavier initialization method. Xavier initialization can ensure that the network weights have a reasonable distribution in the early stage of training, which helps to improve the stability and convergence speed of model training.
[0029] (2) Define the loss function: Construct a composite loss function including exposure loss, spatial loss, lighting loss, and smoothness loss to optimize the training process of the low-light image brightness enhancement network; the content loss part of the composite loss function is used to ensure that the enhanced image is semantically consistent with the original low-light image and maintain the original information and details of the image; the spatial loss is intended to adjust the overall style of the enhanced image to meet specific visual standards; the regularization term added to the composite loss function helps prevent network overfitting and promotes the generalization ability of the model under different low-light conditions; (In this way, the composite loss function provides a strong training basis for the effective enhancement of low-light images, thereby achieving high-quality image output)
[0030] (3) Training and monitoring: After training is completed, the trained model is used to perform brightness enhancement on actual low-light images. By comparing the enhanced images with the original images, not only the visual effects of the images are evaluated, but also the quality of the generated data, the convergence speed of the model, and the accuracy of style transfer are comprehensively examined to comprehensively evaluate the performance and practicality of the model.
[0031] (4) Verification of brightness enhancement effect and performance: Use the trained brightness enhancement network model to perform brightness enhancement processing on the in-orbit target image in space; compare the brightness enhancement image with the original image to determine the brightness enhancement effect, consider the image improvement quality, model convergence speed, and network noise resistance, and thus comprehensively evaluate the performance of the network model.
[0032] Furthermore, the brightness network model for high enhancement of low-light images is implemented through unsupervised learning training. The core lies in the design of a reference-free loss function. The loss function includes four components: exposure loss, spatial loss, and lighting and smoothness loss:
[0033] Exposure loss is defined as follows:
[0034]
[0035] Where M is the number of non-overlapping local regions where the image is segmented, and R k is the average intensity of the kth region in the reflectance image, and E is the preset desired exposure level, which is set to 0.6 here; this loss function improves the overall brightness by enhancing the underexposed areas in the image to achieve the ideal exposure level, which helps to obtain consistent image quality under different brightness conditions;
[0036] The spatial loss function is defined as follows:
[0037]
[0038] Where K is the number of local regions considered in the image, Ω(i) represents the four neighboring regions (upper, lower, left, and right) surrounding region i, and R and I represent the average local brightness of the enhanced and original images, respectively. This loss maintains the similarity in spatial consistency between the enhanced image and the original low-light image, thereby maintaining the integrity of the image structure.
[0039] The lighting loss function is defined as follows:
[0040]
[0041] In the formula, here and They are the gradient operations in the horizontal and vertical directions, representing the smoothness of the illumination component L in these two directions;
[0042] Image smoothing loss function definition:
[0043]
[0044] Where: I represents the image, I[i,j] represents the pixel value of the image at position (i,j). This formula calculates the gradient values between all adjacent pixels of the image, sums them up and takes the square root, thereby quantifying the overall smoothness of the image; the loss function represents the sum of the intensity differences between adjacent pixels in the image, which represents a measure of the local smoothness of the image; |I[i,j+1]-I[i,j] and |I[i+1,j]-I[i,j] are the differences between adjacent pixels in the horizontal and vertical directions of the image, and these differences represent the gradients of the image in these directions; I[i,j+1]-I[i,j] and |I[i+1,j]-I[i,j] are the differences between adjacent pixels in the horizontal and vertical directions of the image, and the differences represent the gradients of the image in these directions;
[0045] The total loss function is set as:
[0046] L total =λ1L exp +λ2L spa +λ3L smo +λ4L TV
[0047] Where λ1λ2λ3λ4 represents the weight coefficients for balancing the importance of each loss term, and these weight coefficients can be adjusted according to specific task requirements.
[0048] Furthermore, in step three, the network training is based on the Pytorch deep learning framework on the Ubuntu 20.04 system, and the training uses the NVIDIA 3060 GPU; the initial learning rate is set to 0.0001; the batch size is set to 8; the training cycle is 40 cycles; and L2 regularization is used.
[0049] Furthermore, the specific steps of the fusion process in step 4 are:
[0050] After obtaining the image with enhanced lighting conditions through the neural network, it is fused with the original image; the pixel values corresponding to the two images are weighted averaged, and the formula is:
[0051]
[0052] Among them I fused (x, y) is the pixel value of the fused image at (x, y), I i (x, y) is the pixel value of the original image and the image after illumination enhancement at (x, y), w i is the corresponding weighting coefficient, which is dynamically adjusted according to the enhancement effect. The enhancement effect is determined by calculating the peak signal-to-noise ratio. The calculation formula for the peak signal-to-noise ratio is:
[0053]
[0054] In the formula, MAXI is the maximum possible pixel value of the image (such as 255 for an 8-bit image); where MSE is the mean square error, calculated as follows:
[0055]
[0056] Where I is the original image, K is the fused image, m and n are the number of rows and columns of the image respectively; after obtaining the peak signal ratio, the weighting coefficient is dynamically adjusted and optimized.
[0057] Furthermore, the specific process in step five includes the following steps:
[0058] 1) Data preprocessing and preparation
[0059] A series of preprocessing operations are performed on the collected low-light image dataset: including normalizing the pixel values in the image to ensure that they are in the range of 0 to 1, so as to ensure the consistency of the data during processing; in addition, the image is resized according to the requirements of the brightness enhancement model, such as resizing the image to 256×256 pixels; some data enhancement techniques that do not change the semantic content of the image are also applied to enhance the diversity of the dataset, including random rotation and flipping;
[0060] 2) Quality assessment before enhancement
[0061] Establish quality assessment indicators for brightness enhancement: image brightness level, detail visibility, and noise level; then conduct a preliminary assessment of the images in the dataset based on these indicators and record their initial quality status to provide a reference standard for subsequent enhancement effect comparison;
[0062] 3) Enhanced quality assessment
[0063] Use non-reference image quality evaluation indicators (such as NIQE and BRISQUE) to evaluate or measure the enhanced quality of the enhanced image; compare the quality scores before and after enhancement to measure the degree of image brightness improvement, and adjust and optimize the network parameters based on the evaluation results;
[0064] The quality assessment indicators of non-reference images include NIQE and / or BRISQUE.
[0065] NIQE is a scoring system based on the statistical properties of natural scenes and does not rely on specific image content or any type of reference image;
[0066] BRISQUE is also a reference-free image quality assessment algorithm that predicts image quality by analyzing the spatial domain features of the image; BRISQUE focuses on the natural properties of the image (especially the local spatial features of the image, such as the texture and edges of the image);
[0067] 4) Domain adaptability evaluation and generalization ability test
[0068] By quantitatively analyzing performance indicators such as accuracy and recall, the domain adaptability and generalization ability of the model are evaluated; based on the test results, the network model and data processing flow are optimized.
[0069] A low-light space target image brightness enhancement system, the system has a program module corresponding to the steps of the low-light space target image brightness enhancement method, and executes the steps in the low-light space target image brightness enhancement method when running.
[0070] A computer-readable storage medium stores a computer program, wherein the computer program is configured to implement the steps of the method for enhancing the brightness of a low-light space target image when called by a processor.
[0071] A low-light space target image processing device, the low-light space target image processing device includes at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the low-light space target image brightness enhancement method.
[0072] The present invention has the following beneficial technical effects:
[0073] The present invention is a method for enhancing the image brightness of effective space targets under low-light conditions based on an image brightness enhancement network, and is specifically proposed for enhancing the brightness of low-light space target images. The space environment is unique and the lighting conditions are extremely unstable. The brightness of the acquired images is often insufficient due to the obstruction of sunlight or the lack of cosmic background light, which affects the image quality and subsequent image analysis and application. Through the present invention, the brightness and visibility of images captured in low-light environments in space can be significantly improved, thereby improving the efficiency and accuracy of space detection, monitoring and research.
[0074] When performing brightness enhancement for low-light space target images, the present invention specifically considers their unique reflective properties and the influence of background lighting, as well as how to retain important space target information while enhancing image brightness. The method of the present invention can effectively process image noise and improve the dynamic range of the image while retaining important target details. In addition, the present invention takes into account the real-time needs of space missions and has high computational efficiency and adaptability so that it can be quickly adjusted and applied under different observation conditions and mission requirements. The efficient space target image brightness enhancement method and system provided by the present invention solves the low-light problem commonly encountered in space environments and significantly improves the quality of images captured under low-light conditions. The brightness enhancement effect is as follows: Figure 6 , Figure 7 and Figure 8 As shown, the following are the specific effects of the present invention in enhancing image brightness:
[0075] (1) Image brightness improvement
[0076] The present invention effectively improves the brightness of space target images captured under extremely low light conditions by combining deep learning technology with Retinex theory. This makes low-quality images originally captured under conditions of sunlight obstruction or insufficient cosmic background light become brighter and clearer.
[0077] (2) Improved image quality
[0078] In addition to the brightness increase, the system also optimizes the contrast and detail of the image, which significantly improves the visual quality of the image. This quality improvement makes the image more suitable for detailed analysis and processing
[0079] (3) Adaptive lighting changes
[0080] The system is designed to adapt to different lighting conditions and can automatically adjust processing parameters according to different ambient lighting conditions to ensure optimal image enhancement effects under various lighting conditions.
[0081] (4) Improved space detection and monitoring efficiency:
[0082] By providing clearer images, the present invention significantly improves the efficiency and accuracy of space exploration, monitoring and research, which is particularly important for applications such as satellite remote sensing, celestial observation and space station monitoring. BRIEF DESCRIPTION OF THE DRAWINGS
[0083] Figure 1 This is a flow chart of the space target image data enhancement method based on deep learning. Figure 2 is the first original image of a low-light space target. Figure 3 is the second original image of a low-light space target. Figure 4is the third low-light space target original image, Figure 5 This is the network structure diagram for image brightness enhancement;
[0084] Figure 6 This is the first target image corresponding to the brightness enhancement. Figure 7 This is the second target image corresponding to the brightness enhancement. Figure 8 This is the third target image corresponding to brightness enhancement. DETAILED DESCRIPTION
[0085] The following is based on the attached Figure 1-8 The present invention is described in detail.
[0086] Specific implementation method 1: This implementation method provides a method for enhancing the brightness of a low-light space target image, and the method steps include:
[0087] Step 1: Get low-light images
[0088] The images of space targets on orbit are under complex and changeable lighting conditions. These images are usually taken by satellites or other space detection equipment in a space environment. Due to the limitation of the angle of sunlight or the obstruction of other celestial bodies, the images are under low-light conditions. In the present invention, in order to simulate the low-brightness environment of real space on orbit, a simulation engine is used to produce a low-light image as input.
[0089] Step 2: Build a brightness enhancement network
[0090] The present invention proposes a neural network based on the combination of unsupervised deep learning and Retinex theory, and uses a lightweight convolutional neural network (CNN) to estimate the illumination component of low-light images. The network receives a low-light image as input, estimates the illumination layer of the image, and then decomposes the input image into an illumination layer and a reflection layer according to the Retinex theory. Finally, the reflection layer is the result of image brightness enhancement. The network in the present invention is based on the U-Net architecture, which is a lightweight convolutional neural network designed specifically for illumination estimation of low-light images. The network contains seven layers, each layer uses a 3×3 convolution kernel, the number of output channels of the first six layers is 8, and the number of output channels of the last layer is 1, which is specifically used for illumination estimation. In addition, the network structure contains skip connections to maintain image details and reduce information loss during training.
[0091] Step 3: Brightness enhancement network model training
[0092] An unsupervised learning strategy is adopted in this network. The network is trained through no-reference loss functions such as exposure loss, spatial loss and lighting smoothness loss, thus avoiding the dependence on paired training data. This no-reference loss function can effectively train the network to achieve image enhancement without real labels. Among them, the exposure loss is used to brighten the underexposed areas in the image, the spatial loss ensures the spatial consistency of the image after enhancement, and the lighting smoothness loss encourages smooth changes in the lighting layer to match the natural changes in lighting in the real world.
[0093] Specific steps:
[0094] (1) Initialize the brightness enhancement network model: According to the pre-defined lightweight convolutional neural network architecture, use the Xavier initialization method to initialize all weights of the brightness enhancement network model. Xavier initialization can ensure that the network weights have a reasonable distribution in the early stage of training, which helps to improve the stability and convergence speed of model training.
[0095] (2) Define the loss function: construct a training network with exposure loss, spatial loss, and illumination flatness and smoothness losses. In the present invention, a composite loss function is designed to optimize the training process of the low-light image brightness enhancement network. This loss function combines content loss to ensure that the enhanced image is semantically consistent with the original low-light image, thereby maintaining the original information and details of the image. The spatial loss aims to adjust the overall style of the enhanced image to meet specific visual standards. The added regularization term helps prevent network overfitting and promotes the generalization ability of the model under different low-light conditions. In this way, the composite loss function provides a strong training basis for the effective enhancement of low-light images, thereby achieving high-quality image output.
[0096] (3) Training and monitoring: After training, the trained model is used to perform brightness enhancement on actual low-light images. By comparing the enhanced images with the original images, not only the visual effects of the images are evaluated, but also the quality of the generated data, the convergence speed of the model, and the accuracy of style transfer are comprehensively examined to comprehensively evaluate the performance and practicality of the model.
[0097] (4) Brightness enhancement effect and performance verification: Use the trained brightness enhancement network model to perform brightness enhancement processing on the on-orbit target image. By comparing the brightness enhanced image with the original image, not only the brightness enhancement effect is judged, but also the image improvement quality, model convergence speed, and network noise resistance are comprehensively considered to comprehensively evaluate the performance of the network model.
[0098] Step 4: Fusion processing
[0099] After obtaining the image with enhanced lighting conditions through the neural network, it is fused with the original image. The pixel values corresponding to the two images are weighted averaged, and the enhancement effect is determined by calculating the peak signal-to-noise ratio.
[0100] Step 5: Verify the data enhancement effect after brightness enhancement network training
[0101] Get the fused image, then test and optimize the network model, and adjust the parameters according to the training effect and performance of the network model. The specific process includes the following steps:
[0102] 1. Data preprocessing and preparation
[0103] A series of preprocessing operations are performed on the collected low-light image dataset. This includes normalizing the pixel values in the image to ensure that they are in the range of 0 to 1, thereby ensuring the consistency of the data during processing. In addition, the images are resized according to the requirements of the brightness enhancement model, such as resizing the images to 256×256 pixels. In order to enhance the diversity of the dataset, some data augmentation techniques that do not change the semantic content of the image are also applied, such as random rotation and flipping.
[0104] 2. Quality Assessment Before Enhancement
[0105] In this step, the present invention will establish a series of quality assessment indicators for brightness enhancement, such as the brightness level of the image, the visibility of details and the noise level, etc. Then, the images in the data set are preliminarily evaluated according to these indicators, and their initial quality status is recorded to provide a reference standard for the subsequent comparison of enhancement effects.
[0106] 3. Enhanced Quality Assessment
[0107] The quality of the enhanced image will be evaluated. In order to objectively evaluate the enhancement effect, the present invention will use non-reference image quality evaluation indicators, such as NIQE and BRISQUE, to measure the enhanced quality of the image. The present invention will compare the quality scores before and after enhancement to measure the degree of improvement in image brightness, and adjust and optimize the network parameters based on the evaluation results. NIQE is a scoring system based on the statistical characteristics of natural scenes. It does not depend on the specific image content or any type of reference image. BRISQUE is also a non-reference image quality assessment algorithm that predicts image quality by analyzing the spatial domain characteristics of the image. BRISQUE focuses on the natural properties of the image, especially the local spatial characteristics of the image, such as the texture and edges of the image.
[0108] 4. Domain adaptability evaluation and generalization ability test
[0109] The present invention proposes to analyze performance indicators through quantitative indicators, such as accuracy, recall, etc., to evaluate the domain adaptability and generalization ability of the model. The present invention further optimizes the network model or data processing flow based on the test results.
[0110] Specific implementation method 2: The present invention can obtain a low-light image by enhancing the image brightness through the system to obtain an enhanced image. The low-light space target image brightness enhancement method and system are as follows: Figure 1 shown.
[0111] Step 1: Get low-light images
[0112] First, an advanced graphics simulation platform is used to create a detailed three-dimensional space scene and build an accurately simulated space target model. By carefully simulating the lighting effects of sunlight and other celestial bodies, including sunrise, sunset, and scenarios where the sun is blocked, complex and changeable low-light conditions are generated. In these scenes, multiple virtual cameras are set at different positions and angles. These cameras are controlled by writing automated scripts to follow preset trajectories or dynamically adjust the shooting path according to the light intensity to capture a wide range of low-light images. The collected images are then annotated in detail, including lighting conditions and camera settings, to form a comprehensive image dataset, providing high-quality basic data for the training and verification of image brightness enhancement models. This method not only avoids costly experiments in actual space environments, but also provides a controllable, efficient and repeatable method for data acquisition, greatly improving the research and development efficiency and accuracy of space target image brightness enhancement systems. Among them Figure 2 Figure 3 Figure 4 Some images of the dataset.
[0113] Step 2: Build a brightness enhancement network
[0114] The brightness enhancement network uses a lightweight convolutional neural network (CNN). Figure 5As shown in the figure, the network contains multiple convolutional layers, each of which uses a 3x3 convolution kernel and skip connections to retain image details and reduce information loss during training. The 3x3 convolution kernel helps capture image detail information while reducing the computational complexity of the model. The number of output channels of the first six layers of the network is set to 8, and the number of output channels of the last layer is set to 1, which is specifically used for lighting estimation. The skip connection used by the network is to directly connect the encoder part of the network to the corresponding decoder part, so as to retain more detail information during the image enhancement process and prevent information loss in the convolution layer. The network architecture maintains the same network architecture as U-net, with a symmetrical encoder and decoder structure. Then, using the Retinex theory, the brightness enhancement network restores the true reflection properties of the image by estimating and adjusting the illumination component.
[0115] The brightness enhancement network estimates the illumination and reflection components respectively, where the illumination estimation uses the deep features extracted by the encoder to estimate the illumination component of the image, and the reflection estimation divides the original input image by the estimated illumination component to obtain the reflection component of the image. This operation is based on the Retinex theory. The Retinex model established in the present invention is as follows:
[0116] I=R×L
[0117] In the formula, the image I can be modeled as the element-by-element product of the illumination component L and the reflection component R, where L represents the illumination of the object by the light source, which usually changes slowly and is evenly distributed; R represents the color and properties of the object itself, which is related to the inherent properties of the object and is not affected by the light intensity.
[0118] Step 3: Brightness network model training
[0119] The low-light image enhancement method proposed in the present invention is implemented through an unsupervised learning training, the core of which lies in its unique reference-free loss function design. This loss function includes four components: exposure loss, spatial loss, illumination loss, and smoothness loss.
[0120] Exposure loss is defined as follows:
[0121]
[0122] Where M is the number of non-overlapping local regions where the image is segmented, and R k is the average intensity of the kth region in the reflectance image, and E is the preset desired exposure level, which is set to 0.6 here. This loss function improves the overall brightness by enhancing the underexposed areas in the image to achieve the ideal exposure level. This helps to achieve consistent image quality under different brightness conditions.
[0123] The spatial loss function is defined as follows:
[0124]
[0125] Where K is the number of local regions considered in the image, Ω(i) represents the four neighboring regions (upper, lower, left, and right) surrounding region i, and R and I represent the average local brightness of the enhanced and original images, respectively. This loss maintains the similarity between the enhanced image and the original low-light image in terms of spatial consistency, thereby maintaining the integrity of the image structure.
[0126] The lighting loss function is defined as follows:
[0127]
[0128] In the formula, here and They are the gradient operations in the horizontal and vertical directions, respectively, representing the smoothness of the lighting component L in these two directions.
[0129] Image smoothing loss function definition:
[0130]
[0131] Where: I represents the image, I[i,j] represents the pixel value of the image at position (i,j). This formula calculates the gradient values between all adjacent pixels of the image, sums them up and takes the square root, thereby quantifying the overall smoothness of the image. This loss function represents the sum of the intensity differences between adjacent pixels in the image, and represents a measure of the local smoothness of the image. are the differences between adjacent pixels in the horizontal and vertical directions of the image, respectively, and these differences represent the gradients of the image in these directions. I[i,j+1]-I[i,j] and |I[i+1,j]-I[i,j] are the differences between adjacent pixels in the horizontal and vertical directions of the image, respectively, and these differences represent the gradients of the image in these directions.
[0132] The total loss function is set as:
[0133] L total =λ1L exp +λ2L spa +λ3L smo +λ4L TV
[0134] Where λ1λ2λ3λ4 represents the weight coefficients for balancing the importance of each loss term, and these weight coefficients can be adjusted according to specific task requirements.
[0135] Combined with the comprehensive expression of the above loss function, the network in the present invention not only enhances the brightness of the image, but also maintains the spatial structure and smooth lighting transition of the image, which is crucial to achieving natural and realistic image enhancement effects.
[0136] The specific steps are:
[0137] (1) Training set preparation
[0138] The brightness enhancement network uses images taken in a variety of low-light environments as a dataset to ensure that the dataset contains a variety of lighting conditions and scenes to enhance the generalization ability of the model. The present invention preprocesses the image data before inputting the network, including standardizing the image size (such as 256×256 pixels) and normalizing the pixel values.
[0139] (2) Optimizer settings
[0140] When training the brightness enhancement network model, the optimizer selected is the Adam optimizer. For the image enhancement task, the learning rate of each parameter is adaptively adjusted by calculating the first-order moment estimate and the second-order moment estimate of the gradient to obtain the optimal image brightness enhancement result.
[0141] (3) Hyperparameter setting
[0142] The specific hyperparameters in the network model are set as follows: Initial learning rate: set to 0.0001. This is a common starting value that is suitable for most cases and is small enough to avoid excessive weight updates in the early stages of training, thereby avoiding potential instability; Batch size: set to 8. It can run effectively on most GPUs while maintaining memory efficiency and sufficient data diversity during training. Smaller batches can provide more frequent model updates and help converge faster; Training cycle (Epochs): set to 40 cycles. This value is determined based on experimental observations and is sufficient to enable the network to converge on a given data set without overfitting. Regularization technology: L2 regularization is used to prevent model overfitting and improve the generalization ability of the model.
[0143] Step 4: Fusion processing
[0144] After obtaining the image with enhanced lighting conditions through the neural network, it is fused with the original image. The pixel values corresponding to the two images are weighted averaged, and the formula is:
[0145]
[0146] Among them I fused (x, y) is the pixel value of the fused image at (x, y), I i (x, y) is the pixel value of the original image and the image after illumination enhancement at (x, y), wi is the corresponding weighting coefficient, which is dynamically adjusted according to the enhancement effect. The enhancement effect is determined by calculating the peak signal-to-noise ratio. The calculation formula for the peak signal-to-noise ratio is:
[0147]
[0148] In the formula, MAX I is the maximum possible pixel value of the image (such as 255 for an 8-bit image). Where MSE is the mean square error, calculated as follows:
[0149]
[0150] Here I is the original image, K is the fused image, m and n are the number of rows and columns of the image respectively. After obtaining the peak signal ratio, the weighting coefficient is dynamically adjusted and optimized.
[0151] Step 5: Verify the data enhancement effect after brightness enhancement network training
[0152] Get the fused image, then test and optimize the network model, and adjust the parameters according to the training effect and performance of the network model. The specific process includes the following steps:
[0153] 1. Data preprocessing and preparation
[0154] A series of preprocessing operations are performed on the collected low-light image dataset. This includes normalizing the pixel values in the image to ensure that they are in the range of 0 to 1, thereby ensuring the consistency of the data during processing. In addition, the images are resized according to the requirements of the brightness enhancement model, such as resizing the images to 256×256 pixels. In order to enhance the diversity of the dataset, some data augmentation techniques that do not change the semantic content of the image are also applied, such as random rotation and flipping.
[0155] 2. Quality Assessment Before Enhancement
[0156] In this step, the present invention will establish a series of quality assessment indicators for brightness enhancement, such as the brightness level of the image, the visibility of details and the noise level, etc. Then, the images in the data set are preliminarily evaluated according to these indicators, and their initial quality status is recorded to provide a reference standard for the subsequent comparison of enhancement effects.
[0157] 3. Enhanced Quality Assessment
[0158] The quality of the enhanced image will be evaluated. In order to objectively evaluate the enhancement effect, the present invention will use non-reference image quality evaluation indicators, such as NIQE and BRISQUE, to measure the enhanced quality of the image. The present invention will compare the quality scores before and after enhancement to measure the degree of image brightness improvement, and adjust and optimize the network parameters according to the evaluation results.
[0159] NIQE is a scoring system based on the statistical properties of natural scenes. It does not rely on specific image content or any type of reference image. NIQE works by first learning a statistical model from a set of undistorted images that captures the normal visual quality characteristics of the image. The model is then used to evaluate the quality of the test image, generating a quality score by comparing the statistical properties of the test image with the learned model. The lower the NIQE score, the higher the quality of the image, which is closer to the natural look and feel.
[0160] BRISQUE is also a reference-free image quality assessment algorithm that predicts image quality by analyzing the spatial domain features of the image. BRISQUE focuses on the natural properties of the image, especially the local spatial features of the image, such as the texture and edges of the image. It uses machine learning methods to predict the visual quality of the image, and does not require prior knowledge of the distortion type during training. It models some common characteristics of natural images (for example, the edge distribution of the image should conform to specific statistical laws), and then uses the model to evaluate the quality of the test image. The lower the BRISQUE score, the better, indicating that the image quality is closer to the natural state observed by the human eye.
[0161] 4. Domain adaptability evaluation and generalization ability test
[0162] The present invention proposes to analyze performance indicators such as accuracy, recall, etc. with reference to Table 2 to evaluate the domain adaptability and generalization ability of the model. The present invention further optimizes the network model or data processing flow based on the test results.
[0163] Table 2 Brightness enhancement network evaluation performance indicators
[0164] Performance Indicators describe importance Accuracy The proportion of correctly classified samples to the total samples Reflects the overall performance of the model Recall The proportion of correctly identified positive classes Measures the model's ability to identify positive samples Precision The proportion of positive classes that are identified as positive Measures the model's ability to exclude negative samples F1 Score The harmonic mean of precision and recall An indicator that comprehensively considers precision and recall, suitable for unbalanced data sets AUC value (Area Under Curve) The area under the ROC curve measures the overall performance of the classifier Comprehensively evaluate model performance without being affected by decision thresholds
[0165] In summary, the method proposed in the present invention solves the practical technical problem of the present invention. The method of the present invention has been verified through simulation experiments and practical applications, and the technical effects and practicality claimed by the present invention have been verified.
[0166] The algorithm (method) proposed in the present invention is the underlying technical core of the present invention, and various products can be derived based on the algorithm.
[0167] Based on the algorithm (method) proposed in the present invention, a low-light space target image brightness enhancement system is developed using a programming language. The system has program modules corresponding to the steps of the above-mentioned technical solution, and executes the steps of the above-mentioned low-light space target image brightness enhancement method during operation.
[0168] The computer program of the developed system (software) is stored on a computer-readable storage medium, and the computer program is configured to implement the steps of the above-mentioned low-light space target image brightness enhancement method when called by a processor, that is, the present invention is materialized on a carrier to become a computer program product.
[0169] The present invention also provides a low-light space target image processing device, the low-light space target image processing device includes at least one processor and a memory connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned low-light space target image brightness enhancement method. The low-light space target image processing device is used as a terminal intelligent product of the present invention and is applied to oil and gas pipeline network fault diagnosis.
[0170] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0171] The computer programs (also referred to as programs, software, software applications, or codes) of the present invention include machine instructions for programmable processors, and these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or device (e.g., disk, optical disk, memory, programmable logic device PLD) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.
[0172] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, they are all within the scope of protection of the present invention.
Claims
1. A method for enhancing the brightness of a low-light space target image, characterized in that: The method comprises: Step 1: Get low-light images The definition of low-light images is: low-light images refer to images acquired under extremely weak light conditions, with the characteristics of average brightness less than 30 luminance units, signal-to-noise ratio less than 20 dB, contrast less than 0.1, color fidelity CIEDE2000 color difference greater than 5, and dynamic range less than 40 dB; at the same time, the images must also meet the requirements of space on-orbit, including complex lighting conditions, special target characteristics and environmental adaptability; Step 2: Build a brightness enhancement network A lightweight convolutional neural network based on unsupervised deep learning and the Retinex algorithm is constructed as a brightness enhancement network, and a lightweight convolutional neural network (CNN) is used to estimate the lighting components of low-light images. The network receives a low-light image as input, decomposes the input image into an illumination layer and a reflection layer, and then outputs the reflection layer as the result of image brightness enhancement; Step 3: Brightness enhancement network model training During the network training process, the network is trained through no-reference loss functions such as exposure loss, spatial loss and lighting smoothness loss to avoid dependence on paired training data; the no-reference loss function can effectively train the network to achieve image enhancement without real labels; the exposure loss is used to brighten the underexposed areas in the image, the spatial loss ensures the spatial consistency of the image after enhancement, and the lighting smoothness loss encourages smooth changes in the lighting layer to conform to the natural changes in lighting in the real world; Step 4: Fusion processing After obtaining the image with enhanced lighting conditions through the neural network, it is fused with the original image; the pixel values corresponding to the two images are weighted averaged, and the fusion effect is determined by calculating the peak signal-to-noise ratio; if it does not meet the requirements, return to step three for network training; Step 5: Verify the data enhancement effect after brightness enhancement network training: After obtaining the fused image, the network model is tested and optimized. According to the training effect and performance of the network model, the parameters are adjusted and continuously optimized until the enhanced low-light space target image quality assessment requirements are met.
2. The method for enhancing the brightness of a low-light space target image according to claim 1, characterized in that: In step 2, The lightweight convolutional neural network (CNN) is based on the U-Net architecture and is used for illumination estimation of low-light images. The network consists of seven layers, each layer uses a 3×3 convolution kernel, the number of output channels of the first six layers is 8, and the number of output channels of the last layer is 1. The network structure contains skip connections to preserve image details and reduce information loss during training; The brightness enhancement network estimates the illumination and reflection components respectively. The illumination estimation uses the deep features extracted by the encoder to estimate the illumination component of the image. The reflection estimation divides the original input image by the estimated illumination component to obtain the reflection component of the image. The established Retinex model is as follows: I=R×L In the formula, the image I can be modeled as the element-by-element product of the illumination component L and the reflection component R, where L represents the illumination of the object by the light source, which usually changes slowly and is evenly distributed; R represents the color and properties of the object itself, which is related to the inherent properties of the object and is not affected by the light intensity.
3. The method for enhancing brightness of a low-light space target image according to claim 2, characterized in that: The specific steps in step three are: (1) Initializing the brightness enhancement network model: Based on the pre-defined lightweight convolutional neural network architecture, all weights of the brightness enhancement network model are initialized using the Xavier initialization method. Xavier initialization can ensure that the network weights have a reasonable distribution in the early stage of training, which helps to improve the stability and convergence speed of model training. (2) Define the loss function: Construct a composite loss function including exposure loss, spatial loss, lighting loss, and smoothness loss to optimize the training process of the low-light image brightness enhancement network; the content loss part of the composite loss function is used to ensure that the enhanced image is semantically consistent with the original low-light image and maintain the original information and details of the image; the spatial loss is intended to adjust the overall style of the enhanced image to meet specific visual standards; the regularization term added to the composite loss function helps prevent network overfitting and promotes the generalization ability of the model under different low-light conditions; (In this way, the composite loss function provides a strong training basis for the effective enhancement of low-light images, thereby achieving high-quality image output) (3) Training and monitoring: After training is completed, the trained model is used to perform brightness enhancement on actual low-light images; By comparing the enhanced images with the original images, we not only evaluate the visual effects of the images, but also comprehensively examine the quality of the generated data, the convergence speed of the model, and the accuracy of style transfer, and comprehensively evaluate the performance and practicality of the model. (4) Brightness enhancement effect and performance verification: Use the trained brightness enhancement network model to perform brightness enhancement processing on the on-orbit target image; By comparing the brightness enhanced image with the original image, the brightness improvement effect is judged, and the image improvement quality, the speed of model convergence and the noise resistance of the network are considered to comprehensively evaluate the performance of the network model.
4. The method for enhancing brightness of a low-light space target image according to claim 3, characterized in that: The brightness network model for high enhancement of low-light images is implemented through unsupervised learning training. The core lies in the design of a reference-free loss function. The loss function includes four components: exposure loss, spatial loss, and lighting and smoothness loss: Exposure loss is defined as follows: Where M is the number of non-overlapping local regions where the image is segmented, and R k is the average intensity of the kth region in the reflectance image, and E is the preset desired exposure level, which is set to 0.6 here; this loss function improves the overall brightness by enhancing the underexposed areas in the image to achieve the ideal exposure level, which helps to obtain consistent image quality under different brightness conditions; The spatial loss function is defined as follows: Where K is the number of local regions considered in the image, Ω(i) represents the four neighboring regions (upper, lower, left, and right) surrounding region i, and R and I represent the average local brightness of the enhanced and original images, respectively. This loss maintains the similarity in spatial consistency between the enhanced image and the original low-light image, thereby maintaining the integrity of the image structure. The lighting loss function is defined as follows: In the formula, here and They are the gradient operations in the horizontal and vertical directions, representing the smoothness of the illumination component L in these two directions; Image smoothing loss function definition: Where: I represents the image, and I[i,j] represents the pixel value of the image at position (i,j). This formula calculates the gradient values between all adjacent pixels of the image, sums them up and takes the square root, thereby quantifying the overall smoothness of the image; the loss function represents the sum of the intensity differences between adjacent pixels in the image, which represents a measure of the local smoothness of the image; are the differences between adjacent pixels of the image in the horizontal and vertical directions, respectively, and these differences represent the gradients of the image in these directions; I[i,j+1]-I[i,j] and |I[i+1,j]-I[i,j] are the differences between adjacent pixels of the image in the horizontal and vertical directions, respectively, and the differences represent the gradients of the image in these directions; The total loss function is set as: L total =λ1L exp +λ2L spa +λ3L smo +λ4L TV Where λ1λ2λ3λ4 represents the weight coefficients for balancing the importance of each loss term, and these weight coefficients can be adjusted according to specific task requirements.
5. The method for enhancing brightness of a low-light space target image according to claim 4, characterized in that: In step three, the network training is based on the Pytorch deep learning framework on the Ubuntu 20.04 system, and the training uses the NVIDIA3060GPU; the initial learning rate is set to 0.0001; the batch size is set to 8; the training cycle is 40 cycles; and L2 regularization is used.
6. A method for enhancing brightness of a low-light space target image according to claim 3, 4 or 5, characterized in that: The specific steps of the fusion processing in step 4 are: After obtaining the image with enhanced lighting conditions through the neural network, it is fused with the original image; the pixel values corresponding to the two images are weighted averaged, and the formula is: Among them I fused (x, y) is the pixel value of the fused image at (x, y), I i (x, y) is the pixel value of the original image and the image after illumination enhancement at (x, y), w i is the corresponding weighting coefficient, which is dynamically adjusted according to the enhancement effect. The enhancement effect is determined by calculating the peak signal-to-noise ratio. The calculation formula for the peak signal-to-noise ratio is: In the formula, MAX I is the maximum possible pixel value of the image (such as 255 for an 8-bit image); Where MSE is the mean square error, calculated as follows: Where I is the original image, K is the fused image, m and n are the number of rows and columns of the image respectively; after obtaining the peak signal ratio, the weighting coefficient is dynamically adjusted and optimized.
7. A method for enhancing brightness of a low-light space target image according to claims 3, 4 and 5, characterized in that: The specific process in step five includes the following steps: 1) Data preprocessing and preparation A series of preprocessing operations are performed on the collected low-light image dataset: including normalizing the pixel values in the image to ensure that they are in the range of 0 to 1, so as to ensure the consistency of the data during processing; in addition, the image is resized according to the requirements of the brightness enhancement model, such as resizing the image to 256×256 pixels; some data enhancement techniques that do not change the semantic content of the image are also applied to enhance the diversity of the dataset, including random rotation and flipping; 2) Quality assessment before enhancement Establish quality assessment indicators for brightness enhancement: image brightness level, detail visibility, and noise level; then conduct a preliminary assessment of the images in the dataset based on these indicators and record their initial quality status to provide a reference standard for subsequent enhancement effect comparison; 3) Enhanced quality assessment Use non-reference image quality assessment metrics (such as NIQE and BRISQUE) to evaluate or measure the enhancement quality of the enhanced image; The quality scores before and after enhancement will be compared to measure the degree of improvement in image brightness, and the network parameters will be adjusted and optimized based on the evaluation results; The quality assessment indicators of non-reference images include NIQE and / or BRISQUE. NIQE is a scoring system based on the statistical properties of natural scenes and does not rely on specific image content or any type of reference image; BRISQUE is also a reference-free image quality assessment algorithm that predicts image quality by analyzing the spatial domain features of the image; BRISQUE focuses on the natural properties of the image (especially the local spatial features of the image, such as the texture and edges of the image); 4) Domain adaptability evaluation and generalization ability test By quantitatively analyzing performance indicators: accuracy and recall, the domain adaptability and generalization ability of the model are evaluated; Based on the test results, optimize the network model and data processing flow.
8. A low-light space target image brightness enhancement system, characterized by: The system has a program module corresponding to the steps of any one of claims 1 to 7, and executes the steps in the low-light space target image brightness enhancement method when running.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of the low-light space target image brightness enhancement method described in any one of claims 1 to 7 when called by a processor.
10. A low-light space target image processing device, characterized in that: The low-light space target image processing device includes at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the low-light space target image brightness enhancement method described in any one of claims 1-7.
Citation Information
Patent Citations
Low-illumination color image enhancement method based on Retinex and convolutional neural network
CN110232661A
Port container number identification method based on image enhancement
CN115830585A
Double-branch low-illumination image enhancement method based on Retinex theory
CN117994155A
Low-light image enhancement method based on deep Retinex
JP7493867B1
Cited By
Pseudo-supervised learning traffic image quality enhancement method based on edge information guidance
CN119559075A
An edge information guidance-based pseudo-supervised learning traffic image quality enhancement method
CN119559075B
ROV underwater adaptive illumination algorithm, storage medium and equipment
CN119676574A
A method, storage medium, and device for adaptive underwater lighting in ROVs
CN119676574B
A training method of a low-light image enhancement model based on zero reference learning
CN122780122B
