A polar unmanned vehicle image enhancement method and system based on multi-scale learning

Abstract

A polar unmanned vehicle image enhancement method and system based on multi-scale learning relates to the field of image exposure correction. This method addresses the problem that existing exposure correction and contrast enhancement methods can generally only correct underexposure errors but cannot improve image quality. The method includes: acquiring image data; constructing an image exposure correction architecture using an improved Laplacian pyramid, the image exposure correction architecture comprising: four subnets, the first subnet being a four-layer encoder-decoder network, the second and third subnets each being a three-layer encoder-decoder network; and the fourth subnet being a three-layer encoder-decoder network; and performing color correction and detail enhancement on the image data using the image exposure correction architecture. The present invention is applicable to the field of surface unmanned vehicle target recognition.

Description

Technical Field

[0001] The present invention relates to the field of image exposure correction, and in particular to a polar unmanned vehicle image enhancement method based on multi-scale learning. Background Art

[0002] Unmanned surface vehicles (USVs) are crucial in the field of marine equipment. They possess autonomous planning and navigation capabilities, enabling them to independently perform tasks such as environmental monitoring, target detection, and security patrols. They hold significant strategic significance in both military and civilian applications. With the continuous advancement of technology, their autonomy, intelligence, modularity, and integration are continuously improving, enabling them to replace humans in complex maritime environments to complete various dangerous and challenging tasks, thus possessing extremely high application value.

[0003] On the other hand, global warming and increased human activity are posing unprecedented challenges to the polar ecosystem. Influenced by the Atlantic warm current, the Arctic climate is relatively hospitable to wildlife. However, since the 1950s, the rapid melting of polar ice sheets due to global warming has had a significant impact on wildlife habitats. If the rate of climate change outstrips the ability of wildlife to adapt to new climatic conditions or migrate, species will be at risk of extinction. Rapid ice loss not only threatens wildlife but also impacts the global climate, posing a serious challenge to humanity. As the global population continues to grow, the impact of human activities on the polar regions will only intensify, making polar scientific research all the more crucial.

[0004] Currently, in polar environments, perception equipment usually relies on optoelectronic cameras. However, due to the harsh polar environment, even the extremely harsh conditions of polar night and polar day, the images obtained by optoelectronic equipment usually suffer from underexposure and overexposure, which has a great impact on target recognition of surface unmanned boats in polar scientific expeditions.

[0005] Traditional exposure correction and contrast enhancement methods typically rebalance image intensity values ​​through the image histogram. Tone curve adjustment is also used to correct image exposure errors. This process can be achieved by directly using input image information or relying on a trained deep learning model. Most previous work is based on Retinex theory, which assumes that an incorrectly exposed image can be regarded as the result of pixel-by-pixel multiplication of an image with correct exposure settings and an illumination map. Therefore, these methods aim to predict the illumination map to restore the target image to normal exposure. However, these methods can usually only correct underexposure errors and cannot improve image quality. Summary of the Invention

[0006] Aiming at the problem that existing exposure correction and contrast enhancement methods can only correct underexposure errors but cannot improve image quality, the present invention proposes a polar unmanned vehicle image enhancement method based on multi-scale learning, the method comprising: collecting image data;

[0007] An image exposure correction architecture is constructed by improving the Laplacian pyramid. The image exposure correction architecture includes four subnetworks, the first subnetwork is a four-layer encoder-decoder network, the second and third subnetworks are both three-layer encoder-decoder networks, and the fourth subnetwork is a three-layer encoder-decoder network.

[0008] Perform color correction and detail enhancement on image data using an image exposure correction framework.

[0009] Furthermore, a preferred method is proposed, wherein the first subnet is used to process the low-frequency layer X(4), and the low-frequency layer X(4) is upsampled by a 2×2×3 transposed convolution with a stride of 2. The generated image is added to the intermediate frequency layer X(3) output by the first intermediate frequency band of the Laplacian pyramid and sent to the second subnet.

[0010] Furthermore, a preferred method is proposed, wherein the second subnet is used to process the upsampled input from the first subnet, generate a residual layer, and upsample through a 2×2×3 transposed convolution with a stride of 2; the generated image is added to the intermediate frequency layer X(2) output of the second intermediate frequency band of the Laplacian pyramid and passed to the third subnet.

[0011] Furthermore, a preferred embodiment is proposed in which the first conversion layer in the fourth subnet encoder outputs 16 channels.

[0012] Furthermore, a preferred embodiment is proposed, wherein the method further includes connecting a preprocessed image as a guidance image to the first sub-network.

[0013] Furthermore, a preferred embodiment is proposed, wherein the method for obtaining the pre-processed image is to pre-correct the image illumination based on the retinex theory and the LIME method.

[0014] Furthermore, a preferred method is proposed, wherein the method based on retinex theory and LIME performs pre-correction on image illumination, including:

[0015] The Retinex-based degradation model is:

[0016] L=R×T

[0017] Among them, L is the dark image, R is the restored image, and T is the illumination enhancement image;

[0018] The maximum RGB value of each pixel is used as the illumination enhancement map T of the corresponding pixel in the image;

[0019] Calculate the average illumination T of the image mean ;

[0020] According to the average light T mean Calculate the restored image R:

[0021] R=L×0.5 / T mean .

[0022] Based on the same inventive concept, the present invention also proposes a polar unmanned vehicle image enhancement system based on multi-scale learning, the system comprising:

[0023] A data acquisition unit, configured to acquire image data;

[0024] An image exposure correction architecture construction unit is configured to construct an image exposure correction architecture by using an improved Laplacian pyramid. The image exposure correction architecture includes four subnetworks, wherein the first subnetwork is a four-layer encoder-decoder network, the second subnetwork and the third subnetwork are both three-layer encoder-decoder networks, and the fourth subnetwork is a three-layer encoder-decoder network.

[0025] The image enhancement unit is used to perform color correction and detail enhancement on the image data through an image exposure correction architecture.

[0026] Based on the same inventive concept, the present invention also proposes a computer device, including a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a polar unmanned boat image enhancement method based on multi-scale learning according to any one of the above.

[0027] Based on the same inventive concept, the present invention also proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of a polar unmanned boat image enhancement method based on multi-scale learning as described in any of the above items are executed.

[0028] The present invention is beneficial in that:

[0029] Most existing exposure correction methods focus solely on the single issue of underexposure, typically improving image brightness and making details in dark areas more visible, but often neglecting image color restoration and contrast enhancement. In practical applications, especially for images captured by unmanned boats in polar environments, underexposure is often accompanied by color distortion, loss of detail, and low image contrast. The method proposed in this invention constructs an image exposure correction architecture by improving the Laplacian pyramid and employs a multi-scale learning approach to process images at multiple levels. This not only addresses the underexposure issue but also effectively improves the image's color, detail, and contrast, significantly enhancing image quality.

[0030] The role of the Laplacian pyramid in image processing is to extract different frequency components of an image in layers, helping to better adjust image information at different scales. The proposed method utilizes a modified Laplacian pyramid structure to design an exposure correction architecture consisting of four subnetworks, including multiple encoder-decoder structures (four-layer and three-layer encoder-decoder networks, respectively). This multi-scale structure processes images at different levels: the four-layer encoder-decoder network primarily captures global image information and adjusts the overall brightness, color, and detail of the image; the three-layer encoder-decoder network (the second and third subnetworks) focuses on enhancing mid-frequency and high-frequency details, respectively, improving local detail and texture, and enhancing contrast and layering; the fourth subnetwork (also a three-layer encoder-decoder network) further refines the detail level, fine-tuning exposure and contrast to avoid image distortion caused by overprocessing. This multi-scale processing approach allows the system to gradually refine the image at different spatial scales, ensuring that details in different frequency bands are appropriately enhanced, avoiding local overexposure or underexposure, and improving the visual quality of the entire image.

[0031] Images captured in polar environments often exhibit severe color deviation and loss of detail, especially in low-light or backlit conditions. The complex variations in polar illumination make it difficult for traditional image processing methods to achieve ideal results. Traditional methods often have certain deficiencies in color and detail enhancement, and can only address exposure issues but are unable to restore details and restore natural colors. The image exposure correction architecture of the present invention enables color correction and detail enhancement to be performed simultaneously with exposure correction. In particular, the use of a multi-scale structure enables the optimization of image color information at different levels, restoring true colors, enhancing local details of the image, and improving the visual effect of the entire image.

[0032] The method proposed in this paper not only effectively corrects underexposure but also enhances image contrast, making images more natural and clear in terms of brightness, detail, and color reproduction, significantly improving image quality. Through a multi-scale Laplacian pyramid architecture, image processing at different scales can be more precisely controlled, avoiding the potential loss of detail or overprocessing associated with simple global processing methods. Through adaptive adjustment of the deep network model, it can effectively handle polar imagery captured in diverse environments, particularly those in complex and low-light environments, significantly improving image usability and quality.

[0033] The present invention is applied to the field of surface unmanned boat target recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a processing flow chart of a polar unmanned vehicle image enhancement method based on multi-scale learning according to embodiment 1;

[0035] Figure 2 Schematic diagram of the image exposure correction architecture according to the eleventh embodiment;

[0036] Figure 3 This is a diagram of the MSPEC network structure according to the eleventh embodiment;

[0037] Figure 4 This is a comparison chart of the exposure correction effect under haze conditions described in the eleventh embodiment;

[0038] Figure 5 This is a comparison diagram of the snow exposure correction effect described in the eleventh embodiment. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in combination with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0040] Implementation method 1, see Figure 1 This embodiment describes a polar unmanned vehicle image enhancement method based on multi-scale learning, the method comprising:

[0041] Collect image data;

[0042] An image exposure correction architecture is constructed by improving the Laplacian pyramid. The image exposure correction architecture includes four subnetworks, the first subnetwork is a four-layer encoder-decoder network, the second and third subnetworks are both three-layer encoder-decoder networks, and the fourth subnetwork is a three-layer encoder-decoder network.

[0043] Perform color correction and detail enhancement on image data using an image exposure correction framework.

[0044] Most existing exposure correction methods focus solely on the single issue of underexposure, typically only improving the brightness of the image and making details in dark areas more visible, but often neglecting image color restoration and contrast enhancement. In practical applications, especially for images taken by unmanned boats in polar environments, underexposure is often accompanied by color distortion, loss of detail, and low image contrast. The present invention constructs an image exposure correction architecture by improving the Laplacian pyramid and adopts a multi-scale learning approach to process images at multiple levels. This not only solves the underexposure problem, but also effectively improves the image's color, detail, and contrast, thereby significantly improving image quality.

[0045] The role of the Laplacian pyramid in image processing is to extract different frequency components of an image in layers, helping to better adjust image information at different scales. This paper utilizes a modified Laplacian pyramid structure to design an exposure correction architecture consisting of four subnetworks, including multiple encoder-decoder structures (four-layer and three-layer encoder-decoder networks, respectively). This multi-scale structure processes images at different levels: the four-layer encoder-decoder network primarily captures global image information and adjusts the overall brightness, color, and detail of the image; the three-layer encoder-decoder network (the second and third subnetworks) focuses on enhancing mid-frequency and high-frequency details, respectively, improving local detail and texture, and enhancing contrast and layering; the fourth subnetwork (also a three-layer encoder-decoder network) further refines the detail level, fine-tuning exposure and contrast to avoid image distortion caused by overprocessing. This multi-scale processing approach allows the system to refine the image layer by layer at different spatial scales, ensuring that details in different frequency bands are appropriately enhanced, avoiding local overexposure or underexposure, and improving the visual quality of the entire image.

[0046] Images captured in polar environments often exhibit severe color deviation and loss of detail, especially in low-light or backlit conditions. The complex variations in polar illumination make it difficult for traditional image processing methods to achieve ideal results. Traditional methods often have certain deficiencies in color and detail enhancement, and can only address exposure issues but are unable to restore details and restore natural colors. The image exposure correction architecture of the present invention enables color correction and detail enhancement to be performed simultaneously with exposure correction. In particular, the use of a multi-scale structure enables the optimization of image color information at different levels, restoring true colors, enhancing local details of the image, and improving the visual effect of the entire image.

[0047] This invention not only effectively corrects underexposure but also enhances image contrast, making images more natural and clear in terms of brightness, detail, and color reproduction, significantly improving image quality. Through a multi-scale Laplacian pyramid architecture, image processing at different scales can be more precisely controlled, avoiding the loss of detail or overprocessing that can result from simple global processing methods. Through adaptive adjustment of the deep network model, it can effectively handle polar imagery captured in different environments, especially those in complex and low-light environments, greatly improving image usability and quality.

[0048] Implementation method 2. This implementation method further limits the polar unmanned boat image enhancement method based on multi-scale learning described in implementation method 1. The first subnet is used to process the low-frequency layer X(4). The low-frequency layer X(4) is upsampled by 2×2×3 transposed convolution with a stride of 2. The generated image is added to the intermediate frequency layer X(3) output by the first intermediate frequency band of the Laplacian pyramid and sent to the second subnet.

[0049] This implementation uses a Laplacian pyramid to hierarchically extract information from different frequency bands, such as low and mid-frequency bands, ensuring that image features at all scales are fully processed. Upsampling the low-frequency layers while integrating details from the mid-frequency layers enhances the image's layering, making the final output image visually clearer and more natural.

[0050] After upsampling the low-frequency layer, it is added to the mid-frequency layer to effectively compensate for the lack of image detail. In the low-frequency region, the overall image outline and color information are relatively smooth, while the mid-frequency region typically contains image texture and detail information. By fusing these two components, a more realistic and richer image output can be obtained.

[0051] Transposed convolution upsampling can restore low-resolution images to higher-resolution images, thereby refining local features. This is particularly important for image processing for polar unmanned vehicles, as polar environments often feature high-contrast backgrounds, complex textures, and lighting variations. Enhancing local details can effectively aid image understanding and subsequent processing.

[0052] Multi-scale learning methods can effectively enhance the robustness of models, especially in complex environments such as low light, snow, and ice in polar regions. Fusion of features at different scales helps the model handle a variety of environmental factors, improving the stability and effectiveness of image enhancement.

[0053] Implementation method three. This implementation method is a further limitation of the polar unmanned boat image enhancement method based on multi-scale learning described in implementation method one. The second subnet is used to process the upsampled input from the first subnet, generate a residual layer, and upsample through a 2×2×3 transposed convolution with a stride of 2; the generated image is added to the intermediate frequency layer X(2) output of the second intermediate frequency band of the Laplacian pyramid and passed to the third subnet.

[0054] The residual layer generated by the second subnet is added to the output of the second intermediate frequency band of the Laplacian pyramid (intermediate frequency layer X(2)). This process can effectively combine low-frequency and high-frequency information, thereby enhancing the details and edges of the image. The second subnet performs upsampling through a 2×2×3 transposed convolution with a stride of 2. Transposed convolution (also called deconvolution) can effectively restore low-resolution images to higher resolutions while maintaining the coherence of image structure and edges. A stride of 2 means that the upsampling scale is 2 times, which can effectively restore the spatial dimension of the image. By adding it to the intermediate frequency band of the Laplacian pyramid, the image details can be further refined and restored, reducing possible blur and noise. By introducing residual learning, the residual layer generated by the second subnet helps reduce the complexity of model training and accelerate convergence. Residual learning can optimize image quality by learning errors while retaining the original image structure. The residual layer provides a more refined correction, thereby improving the enhancement effect of image details.

[0055] Implementation method 4: This implementation method further limits the polar unmanned boat image enhancement method based on multi-scale learning described in implementation method 1. The first conversion layer output in the fourth subnet encoder has 16 channels.

[0056] The output of the first conversion layer has 16 channels, which means it can extract more feature information. Generally speaking, increasing the number of channels allows the network to capture more features in the spatial and channel dimensions. This is particularly important for image enhancement tasks, especially in images of polar unmanned vessels, which may suffer from low contrast, insufficient lighting, and noise interference. More channels help the network learn richer image features, thereby improving the quality of image enhancement.

[0057] In a multi-scale learning framework, image information at different scales is typically processed through multiple subnetworks and ultimately fused. By specifying a 16-channel output for the first conversion layer in the fourth subnetwork encoder, the network's ability to transfer features between scales is enhanced, ensuring better fusion of low-level and high-level features during multi-scale learning. This is crucial for recovering image detail and enhancing global information, especially when processing images from complex environments (such as those in polar regions with low contrast or under unusual weather conditions).

[0058] Implementation method 5: This implementation method further limits the polar unmanned boat image enhancement method based on multi-scale learning described in implementation method 1, and the method also includes connecting a preprocessed image as a guidance image to the first sub-network.

[0059] Implementation method six: This implementation method further limits the polar unmanned boat image enhancement method based on multi-scale learning described in implementation method five. The method for obtaining the preprocessed image is: pre-correcting the image illumination based on retinex theory and LIME method.

[0060] Combined with the fifth embodiment, the advantages of this embodiment are explained. In polar environments, image quality may be greatly affected by factors such as strong lighting changes, shadows and reflections. Using Retinex theory for lighting pre-correction can effectively eliminate the effects caused by uneven lighting, making the details and colors of the image more realistic and balanced, thereby improving the visual quality of the image. The LIME (Local Interpretable Model-Independent Explanation) method makes the enhancement process more transparent and easy to debug by providing interpretability for the image processing model. LIME can help understand and optimize the contribution of different factors in the image enhancement process, for example, which parts of the lighting correction are most important for the enhancement effect. This can better adjust the image processing strategy for unmanned boats in polar environments, especially in low-light and complex environments, thereby improving the robustness and effectiveness of the algorithm.

[0061] Multi-scale learning is crucial in image processing. In polar environments, images may exhibit varying scale characteristics, such as distant terrain and near-field details. Multi-scale learning allows the model to simultaneously focus on both detailed and global image information, enabling more comprehensive image enhancement at all levels. This method, combining multi-scale learning with illumination pre-correction, ensures robust enhancement results across different scales, restoring detail while improving overall image quality.

[0062] By connecting the preprocessed image as a guidance image to the first sub-network, corrections and optimizations can be made beforehand during the enhancement process. This approach reduces the complexity of subsequent network processing, thereby improving the efficiency and accuracy of the entire image enhancement process. The guidance image acts like "prior knowledge," helping the neural network learn and optimize enhancement strategies more quickly, particularly in complex or harsh environments (such as polar regions), enabling it to better cope with low-quality original images.

[0063] By combining illumination pre-correction with multi-scale learning, this method effectively reduces the impact of noise on image quality. Enhancing the stability of this method is particularly important in polar environments, where weather and illumination vary dramatically. By applying Retinex theory to correct illumination, the negative impact of uneven illumination on images can be significantly reduced, improving the robustness of image processing.

[0064] Implementation 7: This implementation further defines the polar unmanned vehicle image enhancement method based on multi-scale learning described in Implementation 6. The method based on retinex theory and LIME pre-corrects image illumination, including:

[0065] The Retinex-based degradation model is:

[0066] L=R×T

[0067] Among them, L is the dark image, R is the restored image, and T is the illumination enhancement image;

[0068] The maximum RGB value of each pixel is used as the illumination enhancement map T of the corresponding pixel in the image;

[0069] Calculate the average illumination T of the image mean ;

[0070] According to the average light T mean Calculate the restored image R:

[0071] R=L×0.5 / T mean .

[0072] Embodiment 8: This embodiment describes a polar unmanned vehicle image enhancement system based on multi-scale learning, the system comprising:

[0073] A data acquisition unit, configured to acquire image data;

[0074] An image exposure correction architecture construction unit is configured to construct an image exposure correction architecture by using an improved Laplacian pyramid. The image exposure correction architecture includes four subnetworks, wherein the first subnetwork is a four-layer encoder-decoder network, the second subnetwork and the third subnetwork are both three-layer encoder-decoder networks, and the fourth subnetwork is a three-layer encoder-decoder network.

[0075] The image enhancement unit is used to perform color correction and detail enhancement on the image data through an image exposure correction architecture.

[0076] Implementation method 9. A computer device described in this implementation method includes a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a polar unmanned boat image enhancement method based on multi-scale learning according to any one of implementation methods 1 to 7.

[0077] Implementation method 10. A computer-readable storage medium described in this implementation method stores a computer program, and when the computer program is executed by a processor, the steps of a polar unmanned boat image enhancement method based on multi-scale learning as described in any one of implementation methods 1 to 7 are executed.

[0078] Implementation method 11, see Figures 2 to 5 This embodiment provides a specific example of the polar unmanned vehicle image enhancement method based on multi-scale learning described in embodiment 1, and is also used to explain embodiments 2 to 7. Specifically:

[0079] Given an 8-bit sRGB input image I, rendered with incorrect exposure settings, this method aims to generate an output image Y with an exposure error less than I. When both overexposure and underexposure errors are considered, the input image I may contain areas that are nearly oversaturated or undersaturated, resulting in a loss of color and detail information. The color and detail errors of I are corrected sequentially. Specifically, a multiresolution representation of I is processed rather than directly processing the original input image. A Laplacian pyramid derived from the Gaussian pyramid of I is used as the multiresolution decomposition framework.

[0080] The image exposure correction architecture is summarized as follows: a deep network, working from coarse to fine, progressively corrects exposure errors in 8-bit sRGB images. The network first corrects the global color information captured in the last level of the Laplacian pyramid, and then processes the subsequent frequency levels. Specifically, let X be a Laplacian pyramid of an input image I with n layers, where X(l) is the lth level. The last level of the pyramid (i.e., X(n)) captures the low-frequency information of I, while the first level (i.e., X(1)) captures the high-frequency information. These frequency levels are divided into low-frequency layers that store global color information and medium- and high-frequency layers that store image details from coarse to fine. These layers can then be used to reconstruct the full-color image I.

[0081] During Laplacian pyramid reconstruction, exposure correction is significantly achieved by utilizing only the low-frequency layers (global color information) of the target image. By correcting the Laplacian pyramid layer by layer, detail enhancement is performed sequentially, improving the quality of the final image. Because the correctly exposed image is inaccessible during inference, the method predicts the missing color and detail information in each Laplacian pyramid layer.

[0082] The image exposure correction architecture sequentially processes the 4-layer Laplacian pyramid X of the input image I to generate the final corrected image Y. The model consists of 4 sub-networks, each of which has independent weights similar to the UNet structure. The network capacity is allocated according to the contribution of each sub-problem (global color correction and detail enhancement) to the result. The network structure is as follows Figure 2 and Figure 3 shown.

[0083] like Figure 2 As shown, the largest subnetwork (by weight) focuses on processing global color information in I (i.e., X(n)). This subnetwork (shown in yellow) processes the low-frequency layer X(n) and produces an upscaled image Y(n). Upsampling amplifies the output by a factor of two through transposed convolutions with trainable weights. Subsequently, the first mid-frequency layer X(n-1) is added to Y(n) and processed by the second subnetwork. This subnetwork enhances the details of the current layer, generates a residual layer, and adds it to Y(n)+X(n-1) to reconstruct the image Y(n-1), which is equivalent to layer n-1 in a Gaussian pyramid. This process continues until the final image Y is generated. The network is fully reliable and supports end-to-end training.

[0084] The main network consists of four subnets and is trained end-to-end with approximately 7M parameters. The largest capacity is assigned to the first subnet, and the capacity gradually decreases as the layers are refined. Each subnet processes a different image representation extracted from the Laplacian pyramid. The first subnet is a four-layer encoder-decoder network similar to the UNet structure, processing the low-frequency band output X(4). Its output is upsampled by a 2×2×3 transposed convolution with a stride of 2, and the generated image is added to the first mid-frequency band output X(3) of the Laplacian pyramid and fed into the second subnet.

[0085] The second subnet is a three-layer encoder-decoder network with skip connections, which processes the upsampled input from the first subnet, generates a residual layer, and upsamples it through a 2×2×3 transposed convolution with a stride of 2. The result is added to the second mid-band output X(2) of the Laplacian pyramid and passed to the third subnet. The third subnet design is the same as the second subnet. The final result is added to the high-band output X(1) of the Laplacian pyramid and fed into the fourth subnet to generate the final processed image. The last subnet is a three-layer encoder-decoder network with skip connections, and the output of the first conversion layer in the encoder has 16 channels.

[0086] The input is concatenated with each sub-network, and high-frequency information is removed. When the network depth is too deep, the original input information is easily lost, so it needs to be supplemented later. At the same time, it can be seen as a mechanism: machine learning progresses from easy to difficult, so the small-scale image is enhanced first, and then the enhanced image is used to help guide the enhancement of the larger-scale image. The network structure diagram is as follows Figure 3 (Contains four sub-networks with UNet-like structures):

[0087] Based on the original improvement, the second and third sub-networks (excluding upsampling deconvolution) share weights, and the upsampling deconvolution layers after the second and third sub-networks also share weights. All neurons in the first hidden layer detect the same features at different positions in the image (because the weights are shared, the detected features are the same). Therefore, this mapping from the input layer to the hidden layer is called a feature map (or filters, kernels). The weight of this feature map is called a shared weight, and its bias is called a shared bias. Shared weights have the following characteristics:

[0088] (1) Parameter sharing: Using the same weights across neurons within a layer means that these neurons update the same parameters during training. This design can significantly reduce the number of parameters required to learn, lower model complexity, and mitigate the risk of overfitting.

[0089] (2) Local receptive field: In convolutional neural networks, through weight sharing, each neuron only focuses on a small area of ​​the input data, which is called the local receptive field. This enables the network to better capture the local patterns of the input data, thereby enhancing translation invariance.

[0090] (3) Translation invariance: Weight sharing enhances the model’s translation invariance to input data. Since neurons with shared weights have the same response to inputs at different positions, the model is more likely to learn features that are independent of position.

[0091] (4) Application of parameter sharing: Weight sharing is often applied to convolutional layers, where the parameters of the convolution kernel are shared across the entire input image. This weight sharing design makes CNN particularly suitable for image processing tasks.

[0092] (5) Attention and efficiency: Weight sharing can be viewed as an attention mechanism that allows the model to focus more on important areas of shared parameters. In addition, weight sharing also improves the computational efficiency of the model because parameter sharing reduces the number of parameters that need to be trained and stored.

[0093] After the above operations, due to the change in input content, consider adjusting the depth of each sub-network. It is believed that the enhancement of small-scale images does not require a very deep network at the beginning, while a relatively deep network is required later, but the effect is poor. The reason is that the input of the first sub-network is unguided, so although its scale is small, it is still a "difficult task". Therefore, the effect is poor. Subsequently, the first sub-network is connected to a pre-processed image as a "guiding image", such as Figure 3 shown.

[0094] LIME first uses the maximum RGB value of each pixel as the illumination of that pixel. It also preserves the overall structure and smoothes texture details. The Retinex degradation model is shown in the following formula:

[0095] L=R×T

[0096] Among them, L is the dark image, R is the restored image, and T is the illumination map;

[0097] Therefore, the key is to solve T. The preprocessing method is to refer to LIME. First, the maximum RGB value is used as the illumination T of the corresponding pixel of the image, and then the average illumination T of the image is calculated. mean Next, use R = L × 0.5 / T mean The image is pre-processed. The reason for selecting 0.5 here is that the average illumination of GT in the training set is 0.48, which is rounded to 0.5. The purpose is to pre-correct the image illumination based on the retinex theory and LIME method, and use it as a guidance image.

[0098] In this embodiment, the Structural Similarity Index (SSIM) and Peak Signal-to-Noise Ratio (PSNR) are used as primary performance indicators for detecting the accuracy of the image enhancement algorithm.

[0099] Mean Square Error (MSE) is a commonly used indicator to measure the difference between the current image X and the reference image Y. Its formula is:

[0100]

[0101] Where H and W represent the height and width of the image, respectively; n is the number of bits per pixel, which is generally 8, meaning the number of pixel grayscales is 256. The smaller the MSE value, the higher the similarity between the image and the reference image.

[0102] Peak signal-to-noise ratio (PSNR) is a commonly used metric in image quality assessment, often used to measure image compression or reconstruction quality. The unit of PSNR is dB, with higher values ​​indicating lower distortion. Its calculation is based on mean square error (MSE), as shown in the following formula:

[0103]

[0104] The structural similarity index is used to evaluate the structural similarity of two images. The structural similarity calculation formula of two images X and Y is as follows:

[0105]

[0106] where μ is the mean value, σ x is the variance of image X, σ y is the variance of image Y, σ xy is the covariance.

[0107] The structural similarity index measures image similarity using three factors: brightness, contrast, and structure. Specifically, brightness is estimated using the mean, contrast is estimated using the standard deviation, and structural similarity is measured using the covariance. This approach aims to capture structural information in images rather than simply differences in pixel values, providing a distortion metric that is more consistent with human perception.

[0108] The public MIT-Adobe dataset was selected. It contains 5,000 raw RGB images and their corresponding sRGB images, manually rendered by five professional photographers. Using the Adobe Camera Raw SDK, each Raw-RGB image was simulated at different exposure values ​​(EV), including -1.5, -1, 0, +1, and +1.5, simulating underexposure, normal exposure, and overexposure. A relative EV of 0 represents the original exposure setting at the time of capture. The ground truth images were based on images manually retouched by expert photographers, rather than relying solely on rendered images at EV 0. This is because many images contain backlighting or exposure errors. The expert-adjusted images were processed in the ProPhoto RGB color space and then converted to the 8-bit sRGB color space. The dataset consists of 24,330 8-bit sRGB images, divided into a training set (17,675 images), a validation set (750 images), and a test set (5,905 images), ensuring no image duplication between datasets.

[0109] First, the algorithm in this paper is compared with the multi-scale image exposure correction algorithm to illustrate the intuitive improvement of the exposure correction effect of the improvement of the multi-scale image exposure correction algorithm. The results are shown in Table 1. Next, the algorithm in this paper is compared with the multi-scale image exposure correction algorithm to show the improvement effect. It will also be compared qualitatively and quantitatively with traditional image enhancement methods (such as MSRCR algorithm and fusion-based low-light image enhancement algorithm) to evaluate the advantages and disadvantages of the exposure correction effect. The specific comparison results are shown in Figure 4 、 Figure 5 .

[0110] Table 1 Image evaluation indicators

[0111]

[0112] picture".

[0113] As shown in Table 1, the multi-scale image exposure correction algorithm achieves a PSNR of 19.76 and an SSIM of 0.7485. By concatenating the input to each subnetwork and removing high-frequency information, the PSNR increases to 19.95 and the SSIM to 0.7722, showing significant improvements (+0.19 PSNR, +0.0237 SSIM). Furthermore, weight sharing (the second and third subnetworks) further improves the PSNR to 20.02 and the SSIM to 0.7703, a further improvement in PSNR (+0.26) but a slight decrease in SSIM (-0.0019). Continuing to share weights in the deconvolutional layer, the PSNR reaches 20.03 and the SSIM drops slightly to 0.7682, a slight increase in PSNR (+0.27) but a further decrease in SSIM (-0.0021). Finally, a preprocessed image is added as a “guiding image” to the first sub-network, and the PSNR rises to 20.16 and the SSIM is 0.7688, with a significant improvement in PSNR (+0.40) and some recovery in SSIM (+0.0006).

[0114] As the number of refinement operations increases, PSNR shows an overall upward trend, indicating continuous improvement in image quality. SSIM fluctuates more, initially improving, then slightly decreasing, and finally recovering after the addition of the "guidance image," but never returning to its peak value. These results demonstrate that algorithmic improvements not only improve image quality (via PSNR increases) but also suggest that both PSNR and SSIM changes should be considered comprehensively when evaluating image quality.

[0115] See also Figure 4 and Figure 5 From the perspective of qualitative analysis, from left to right are the original input image, the output of the Fusion-based (low brightness image enhancement based on fusion) algorithm, the output of the Retinex algorithm, the output of the MSRCR algorithm, and the output of the improved algorithm of the present invention. Compared with the MSRCR algorithm, the algorithm proposed in the present invention significantly reduces the appearance of target artifacts, and the restoration of edge contours is clearer and more accurate. At the same time, the background restoration effect for polar and cold regions is also better. It can be observed that although the MSRCR algorithm is Figure 5The overall exposure restoration effect of the first pair of underexposed and overexposed images is more obvious, but some details are over-processed, resulting in differences in color restoration, which is inconsistent with the original image information. In contrast, the algorithm proposed in the present invention significantly reduces the haze phenomenon in the background, and is closer to the real scene in terms of the restoration effect of the target color contrast. Although the Retinex algorithm performs well in the dehazing effect in the first four columns of images, it has low accuracy in restoring the target contour, and the grayscale edge changes are not smooth enough, and the contrast is also insufficient. Compared with these algorithms, the improved algorithm of the present invention performs better in edge contour restoration, image brightness adjustment and detail information extraction. Whether it is processing underexposed or overexposed images, it can provide a more balanced and natural output effect.

[0116] In the images output by our improved algorithm, the colors and brightness of snow-covered areas are more natural, with no noticeable distortion or over-enhancement compared to the original images in the first column. The algorithm excels in avoiding the excessive contrast caused by the fusion-based enhancement algorithm (second column). Our improved algorithm demonstrates strong detail preservation in both bright and dark areas. Compared to Retinex (third column) and MSRCR (fourth column), our proposed algorithm uniformly restores more image detail across various lighting conditions, whether in glacier textures or the fine structure of mountain ranges. This performance is particularly pronounced in areas with extreme brightness or darkness. Our improved algorithm not only addresses local exposure issues but also maintains balanced exposure across the entire image, avoiding the local overexposure or underexposure that can occur with other methods, further enhancing the overall image quality. Results from various scenes show that our algorithm demonstrates strong adaptability to complex natural environments (such as glaciers and mountains), producing visually stable output in both low-light and high-light scenarios. Our improved exposure correction algorithm demonstrates significant advantages in color fidelity, detail restoration, and global exposure balance when processing images from extreme environments.

[0117] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Thus, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including the instruction device, which implements the function specified in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure and are not intended to limit its scope of protection. Although the present disclosure has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that after reading the present disclosure, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the invention, but these changes, modifications or equivalent substitutions are all within the scope of protection of the disclosed claims.

Claims

1. A polar unmanned vehicle image enhancement method based on multi-scale learning, characterized in that: The method comprises: collecting image data; An image exposure correction architecture is constructed by improving the Laplacian pyramid. The image exposure correction architecture includes four subnetworks, the first subnetwork is a four-layer encoder-decoder network, the second and third subnetworks are both three-layer encoder-decoder networks, and the fourth subnetwork is a three-layer encoder-decoder network. Perform color correction and detail enhancement on image data through image exposure correction architecture; The first subnet is used to process the low-frequency layer X(4), which is upsampled by a 2×2×3 transposed convolution with a stride of 2. The generated image is added to the intermediate frequency layer X(3) output by the first intermediate frequency band of the Laplacian pyramid and fed into the second subnet. The second subnet is used to process the upsampled input from the first subnet, generate a residual layer, and upsample through a 2×2×3 transposed convolution with a stride of 2; the generated image is added to the intermediate frequency layer X(2) output of the second intermediate frequency band of the Laplacian pyramid and passed to the third subnet.

2. The polar unmanned vehicle image enhancement method based on multi-scale learning according to claim 1, characterized in that: The first transformation layer output in the fourth subnet encoder has 16 channels.

3. The polar unmanned vehicle image enhancement method based on multi-scale learning according to claim 1, characterized in that: The method further includes connecting a pre-processed image as a guidance image to the first sub-network.

4. The polar unmanned vehicle image enhancement method based on multi-scale learning according to claim 3 is characterized in that: The method for obtaining the preprocessed image is: pre-correcting the image illumination based on the retinex theory and the LIME method.

5. The polar unmanned vehicle image enhancement method based on multi-scale learning according to claim 4 is characterized in that: The method based on retinex theory and LIME performs pre-correction on image illumination, including: The Retinex-based degradation model is: in, It is a dark image. is the restored image, For the lighting enhancement map; The maximum RGB value of each pixel is used as the illumination enhancement map T of the corresponding pixel in the image; Calculate the average illumination of the image ; According to the average light Calculate the restored image : 。 6. A polar unmanned vehicle image enhancement system based on multi-scale learning, characterized by: The system comprises: A data acquisition unit, configured to acquire image data; An image exposure correction architecture construction unit is configured to construct an image exposure correction architecture by using an improved Laplacian pyramid. The image exposure correction architecture includes four subnetworks, wherein the first subnetwork is a four-layer encoder-decoder network, the second subnetwork and the third subnetwork are both three-layer encoder-decoder networks, and the fourth subnetwork is a three-layer encoder-decoder network. An image enhancement unit, configured to perform color correction and detail enhancement on image data through an image exposure correction architecture; The first subnet is used to process the low-frequency layer X(4), which is upsampled by a 2×2×3 transposed convolution with a stride of 2. The generated image is added to the intermediate frequency layer X(3) output by the first intermediate frequency band of the Laplacian pyramid and fed into the second subnet. The second subnet is used to process the upsampled input from the first subnet, generate a residual layer, and upsample through a 2×2×3 transposed convolution with a stride of 2; the generated image is added to the intermediate frequency layer X(2) output of the second intermediate frequency band of the Laplacian pyramid and passed to the third subnet.

7. A computer device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a polar unmanned boat image enhancement method based on multi-scale learning according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the polar unmanned vehicle image enhancement method based on multi-scale learning as described in any one of claims 1 to 5.

Images