Polar unmanned ship image enhancement method and system based on multi-scale learning

Abstract

The invention discloses a polar region unmanned ship image enhancement method and system based on multi-scale learning, and relates to the field of image exposure correction. The problem that the existing exposure correction and contrast enhancement method can only correct the error of insufficient exposure and cannot improve the image quality is solved. The method comprises the steps of collecting image data; an image exposure correction architecture is constructed by improving a Laplacian pyramid, the image exposure correction architecture comprises four subnets, the first subnet is a four-layer encoder-decoder network, and the second subnet and the third subnet are both three-layer encoder-decoder networks; the fourth subnet is a three-layer encoder-decoder network; color correction and detail enhancement are performed on image data through an image exposure correction architecture. The method is applied to the field of unmanned surface vehicle target identification.

Description

Technical Field

[0001] The present invention relates to the field of image exposure correction, and in particular to a polar unmanned boat image enhancement method based on multi-scale learning. Background Art

[0002] In the field of marine equipment, surface unmanned boats are a vital piece of equipment. They have the ability to plan and navigate autonomously, and can autonomously complete tasks such as environmental monitoring, target detection, and patrol. Unmanned boats have extremely high strategic significance in both military and civilian fields. With the continuous development of technology, the autonomy, intelligence, modularity, and integration of unmanned boats are constantly improving, enabling them to replace humans in complex marine environments to complete various dangerous and challenging tasks, so they have extremely high application value.

[0003] On the other hand, against the backdrop of global warming and increased human activities, the polar ecological environment faces unprecedented challenges. Affected by the warm current of the Atlantic Ocean, the climate environment of the Arctic is relatively suitable for the survival of organisms. However, since the 1950s, the rapid melting of the polar ice sheets caused by global warming has had a huge impact on the natural habitats of wild animals. If the speed of climate change exceeds the speed at which wild animals can adapt to new climate conditions or migrate, they will face the risk of species extinction. The rapid melting of the ice sheets not only poses a threat to wild animals and plants, but also affects the global climate, bringing severe challenges to humans. As the global population continues to grow, the impact of human activities on the polar regions will become increasingly greater, so polar scientific expeditions and research are particularly important.

[0004] Currently, in polar environments, perception equipment usually relies on optoelectronic cameras. However, due to the harsh polar environment, and even extremely harsh conditions of polar night and polar day, the images obtained by optoelectronic equipment usually suffer from underexposure and overexposure, which has a great impact on target recognition of surface unmanned boats in polar scientific expeditions.

[0005] Traditional exposure correction and contrast enhancement methods usually rebalance image intensity values ​​through image histogram. Tone curve adjustment is also used to correct image exposure errors. This process can be achieved by directly using input image information or relying on a trained deep learning model. Most previous works are based on Retinex theory, assuming that an image with incorrect exposure can be regarded as the result of pixel-by-pixel multiplication of an image with correct exposure settings and an illumination map. Therefore, these methods aim to predict the illumination map to restore the target image with normal exposure. However, these methods can usually only correct underexposure errors and cannot improve image quality. Summary of the invention

[0006] Aiming at the problem that the existing exposure correction and contrast enhancement methods can usually only correct underexposure errors but cannot improve image quality, the present invention proposes a polar unmanned boat image enhancement method based on multi-scale learning, the method comprising: collecting image data;

[0007] An image exposure correction architecture is constructed by improving the Laplacian pyramid, wherein the image exposure correction architecture includes: four subnets, the first subnet is a four-layer encoder-decoder network, the second subnet and the third subnet are both three-layer encoder-decoder networks; and the fourth subnet is a three-layer encoder-decoder network;

[0008] Image data is color corrected and detail enhanced via an image exposure correction framework.

[0009] Furthermore, a preferred method is proposed, wherein the first subnetwork is used to process the low-frequency layer X(4), the low-frequency layer X(4) is upsampled by a 2×2×3 transposed convolution with a stride of 2, and the generated image is added to the intermediate frequency layer X(3) output by the first intermediate frequency band of the Laplacian pyramid and sent to the second subnetwork.

[0010] Furthermore, a preferred method is proposed, wherein the second subnet is used to process the upsampled input from the first subnet, generate a residual layer, and upsample through a 2×2×3 transposed convolution with a stride of 2; the generated image is added to the intermediate frequency layer X(2) output by the second intermediate frequency band of the Laplacian pyramid and passed to the third subnet.

[0011] Furthermore, a preferred embodiment is proposed, in which the first conversion layer in the fourth subnet encoder outputs 16 channels.

[0012] Furthermore, a preferred embodiment is proposed, wherein the method further comprises connecting a preprocessed image as a guidance image to the first sub-network.

[0013] Furthermore, a preferred manner is proposed, in which the method for obtaining the preprocessed image is: pre-correcting the image illumination based on the retinex theory and the LIME method.

[0014] Furthermore, a preferred method is proposed, wherein the method based on retinex theory and LIME performs pre-correction on image illumination, including:

[0015] The Retinex-based degradation model is:

[0016] L=R×T

[0017] Among them, L is the dark image, R is the restored image, and T is the illumination enhancement image;

[0018] The maximum RGB value of each pixel is used as the illumination enhancement map T of the corresponding pixel in the image;

[0019] Calculate the average illumination T of the image mean ;

[0020] According to the average light T mean Calculate the restored image R:

[0021] R=L×0.5 / T mean .

[0022] Based on the same inventive concept, the present invention also proposes a polar unmanned boat image enhancement system based on multi-scale learning, the system comprising:

[0023] A data acquisition unit, used for acquiring image data;

[0024] An image exposure correction architecture construction unit is used to construct an image exposure correction architecture by improving the Laplacian pyramid, wherein the image exposure correction architecture includes: four subnets, the first subnet is a four-layer encoder-decoder network, the second subnet and the third subnet are both three-layer encoder-decoder networks; the fourth subnet is a three-layer encoder-decoder network;

[0025] The image enhancement unit is used to perform color correction and detail enhancement on the image data through an image exposure correction framework.

[0026] Based on the same inventive concept, the present invention also proposes a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes a polar unmanned boat image enhancement method based on multi-scale learning according to any one of the above.

[0027] Based on the same inventive concept, the present invention also proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a polar unmanned boat image enhancement method based on multi-scale learning as described in any of the above items are executed.

[0028] The present invention is beneficial in that:

[0029] Most of the existing exposure correction methods only focus on the single underexposure problem, and usually can only improve the brightness of the image, making the details in the dark area more visible, but often ignore the color restoration and contrast improvement of the image. In practical applications, especially for images taken by unmanned boats in polar environments, underexposure is often accompanied by color distortion, loss of details and low image contrast. The method proposed in the present invention constructs an image exposure correction architecture by improving the Laplace pyramid, and uses a multi-scale learning method to process the image from multiple levels. It can not only solve the underexposure problem, but also effectively improve the color, details and contrast of the image, thereby significantly improving the image quality.

[0030] The role of the Laplacian pyramid in image processing is to help better adjust image information at different scales by extracting different frequency components of the image in layers. The method proposed in the present invention uses an improved Laplacian pyramid structure to design an exposure correction architecture consisting of four subnetworks, including multiple encoder-decoder structures (four-layer and three-layer encoder-decoder networks, respectively). This multi-scale structure can process images at different levels: the four-layer encoder-decoder network is mainly responsible for capturing global image information and adjusting the overall brightness, color and details of the image; the three-layer encoder-decoder network (the second and third subnetworks): focus on the enhancement of medium-frequency and high-frequency details, improve the local details and texture performance of the image, and enhance the contrast and layering of the image; the fourth subnetwork (also a three-layer encoder-decoder network): further refine the detail level, fine-tune the exposure and contrast, and avoid image distortion caused by excessive processing. This multi-scale processing method enables the system to refine the image layer by layer at different spatial scales, ensuring that the details of different frequency bands are appropriately enhanced, avoiding local overexposure or underexposure, and improving the visual quality of the entire image.

[0031] Images taken in polar environments usually have serious color deviation and detail loss, especially in low-light or backlit conditions. The changes in polar lighting are complex, making it difficult for traditional image processing methods to achieve ideal results. Traditional methods usually have certain deficiencies in color and detail enhancement. They can only deal with exposure problems but cannot restore details and restore colors naturally. Through the image exposure correction architecture of the present invention, color correction and detail enhancement can be performed while performing exposure correction. In particular, by utilizing a multi-scale structure, the color information of the image can be optimized at different levels, restoring the true color, enhancing the local details of the image, and improving the visual effect of the entire image.

[0032] The method proposed in the present invention not only effectively corrects underexposure, but also enhances the contrast of the image, making the image more natural and clear in terms of brightness, detail and color restoration, significantly improving the image quality. Through the multi-scale Laplacian pyramid architecture, image processing at different scales can be more accurately controlled, avoiding the loss of details or over-processing that may be caused by simple global processing methods. Through the adaptive adjustment of the deep network model, it can effectively cope with polar images taken in different environments, especially images in complex lighting and low-light environments, greatly improving the usability and quality of the image.

[0033] The present invention is applied to the field of surface unmanned boat target recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a processing flow chart of a polar unmanned boat image enhancement method based on multi-scale learning as described in implementation mode 1;

[0035] Figure 2 This is a schematic diagram of an image exposure correction architecture according to Embodiment 11;

[0036] Figure 3 This is a diagram of the MSPEC network structure described in Embodiment 11;

[0037] Figure 4 This is a comparison diagram of the exposure correction effect under the haze condition described in the eleventh embodiment;

[0038] Figure 5 This is a comparison diagram of the snow exposure correction effect described in Implementation Example 11. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0040] Implementation method 1, see Figure 1 The present embodiment describes a polar unmanned boat image enhancement method based on multi-scale learning, the method comprising:

[0041] Collecting image data;

[0042] An image exposure correction architecture is constructed by improving the Laplacian pyramid, wherein the image exposure correction architecture includes: four subnets, the first subnet is a four-layer encoder-decoder network, the second subnet and the third subnet are both three-layer encoder-decoder networks; and the fourth subnet is a three-layer encoder-decoder network;

[0043] Image data is color corrected and detail enhanced via an image exposure correction framework.

[0044] Most of the existing exposure correction methods only focus on the single underexposure problem, and usually can only improve the brightness of the image, making the details in the dark area more visible, but often ignore the color restoration and contrast improvement of the image. In practical applications, especially for images taken by unmanned boats in polar environments, underexposure is often accompanied by color distortion, loss of details and low image contrast. The present invention constructs an image exposure correction architecture by improving the Laplacian pyramid, adopts a multi-scale learning method, and processes the image from multiple levels, which can not only solve the underexposure problem, but also effectively improve the color, details and contrast of the image, thereby significantly improving the image quality.

[0045] The role of the Laplacian pyramid in image processing is to extract different frequency components of the image in layers, helping to better adjust the image information at different scales. The present invention uses an improved Laplacian pyramid structure to design an exposure correction architecture consisting of four subnetworks, including multiple encoder-decoder structures (four-layer and three-layer encoder-decoder networks, respectively). This multi-scale structure can process images at different levels: the four-layer encoder-decoder network is mainly responsible for capturing global image information and adjusting the overall brightness, color and details of the image; the three-layer encoder-decoder network (the second subnetwork and the third subnetwork): focus on the enhancement of medium-frequency and high-frequency details, improve the local details and texture performance of the image, and enhance the contrast and layering of the image; the fourth subnetwork (also a three-layer encoder-decoder network): further refine the detail level, fine-tune the exposure and contrast, and avoid image distortion caused by excessive processing. This multi-scale processing method enables the system to refine the image layer by layer at different spatial scales, ensuring that the details of different frequency bands are appropriately enhanced, avoiding local overexposure or underexposure, and improving the visual quality of the entire image.

[0046] Images taken in polar environments usually have serious color deviation and detail loss, especially in low-light or backlit conditions. The changes in polar lighting are complex, making it difficult for traditional image processing methods to achieve ideal results. Traditional methods usually have certain deficiencies in color and detail enhancement. They can only deal with exposure problems but cannot restore details and restore colors naturally. Through the image exposure correction architecture of the present invention, color correction and detail enhancement can be performed while performing exposure correction. In particular, by utilizing a multi-scale structure, the color information of the image can be optimized at different levels, restoring the true color, enhancing the local details of the image, and improving the visual effect of the entire image.

[0047] The present invention not only effectively corrects underexposure, but also enhances the contrast of the image, making the image more natural and clear in terms of brightness, detail and color reproduction, significantly improving the image quality. Through the multi-scale Laplacian pyramid architecture, image processing at different scales can be more accurately controlled, avoiding the loss of details or over-processing that may be caused by simple global processing methods. Through the adaptive adjustment of the deep network model, it can effectively cope with polar images taken in different environments, especially images in complex lighting and low-light environments, greatly improving the usability and quality of the image.

[0048] Implementation method 2. This implementation method is a further limitation of the polar unmanned boat image enhancement method based on multi-scale learning described in implementation method 1. The first subnet is used to process the low-frequency layer X(4). The low-frequency layer X(4) is upsampled by a 2×2×3 transposed convolution with a stride of 2. The generated image is added to the intermediate frequency layer X(3) output by the first intermediate frequency band of the Laplacian pyramid and sent to the second subnet.

[0049] In this implementation, by using the Laplacian pyramid to extract information of different frequency bands such as low frequency and medium frequency in layers, it can be ensured that the features of the image at different scales are fully processed. While upsampling the low frequency layer, the details of the medium frequency layer are integrated to enhance the detail layering of the image, making the final output image visually clearer and more natural.

[0050] After the low-frequency layer is upsampled, it is added to the mid-frequency layer to effectively make up for the lack of details in the image. In the low-frequency part, the overall outline and color information of the image are relatively smooth, while the mid-frequency part usually contains the texture and detail information of the image. By fusing these two parts of information, a more realistic and richer image output can be obtained.

[0051] Through transposed convolution upsampling, low-resolution images can be restored to higher-resolution images, thereby refining local features. This is especially important for image processing of polar unmanned vessels, because polar environments usually have high-contrast backgrounds, complex textures, and lighting changes. The enhancement of local details can effectively help image understanding and subsequent processing.

[0052] Multi-scale learning methods can effectively enhance the robustness of the model, especially in complex environments, such as low light, snow, ice, etc. in polar environments. The fusion of features at different scales helps the model handle various environmental factors and improve the stability and effect of image enhancement.

[0053] Implementation method three. This implementation method is a further limitation of the polar unmanned boat image enhancement method based on multi-scale learning described in implementation method one. The second subnet is used to process the upsampled input from the first subnet, generate a residual layer, and upsample through a 2×2×3 transposed convolution with a stride of 2; the generated image is added to the intermediate frequency layer X(2) output by the second intermediate frequency band of the Laplacian pyramid and passed to the third subnet.

[0054] The residual layer generated by the second subnet is added to the output of the second intermediate frequency band of the Laplacian pyramid (intermediate frequency layer X(2)). This process can effectively combine low-frequency and high-frequency information to enhance the details and edges of the image. The second subnet performs upsampling through a 2×2×3 transposed convolution with a stride of 2. Transposed convolution (also called deconvolution) can effectively restore low-resolution images to higher resolutions while maintaining the coherence of image structure and edges. A stride of 2 means that the upsampling scale is 2 times, which can effectively restore the spatial dimension of the image. By adding to the intermediate frequency band of the Laplacian pyramid, the image details can be further refined and restored, reducing possible blur and noise. By introducing residual learning, the residual layer generated by the second subnet helps reduce the complexity of model training and accelerates convergence. Residual learning can optimize image quality by learning errors while retaining the original image structure. The residual layer provides a more refined correction, thereby improving the enhancement effect of image details.

[0055] Implementation method 4: This implementation method is a further limitation of the polar unmanned boat image enhancement method based on multi-scale learning described in implementation method 1, and the first conversion layer output in the fourth subnet encoder has 16 channels.

[0056] The output of the first conversion layer has 16 channels, which means that this layer can extract more feature information. Generally speaking, an increase in the number of channels allows the network to capture more features in the spatial and channel dimensions. This is particularly important for image enhancement tasks, especially in images of polar unmanned boats, where there may be problems such as low contrast, insufficient lighting, and noise interference. More channels can help the network learn richer image features, thereby improving the quality of image enhancement.

[0057] In the multi-scale learning framework, image information of different scales is usually processed through multiple subnets and fused at the end. By specifying that the first conversion layer outputs 16 channels in the fourth subnet encoder, the network's ability to transfer features between different scales can be enhanced, ensuring that low-level features and high-level features can be better integrated during multi-scale learning. This is crucial for image detail recovery and global information enhancement, especially when processing images in complex environments (such as low contrast in polar regions or images under special weather conditions).

[0058] Implementation method 5: This implementation method further limits the polar unmanned boat image enhancement method based on multi-scale learning described in implementation method 1, and the method also includes connecting a preprocessed image as a guidance image to the first sub-network.

[0059] Implementation method six: This implementation method is a further limitation of the polar unmanned boat image enhancement method based on multi-scale learning described in implementation method five. The method for obtaining the preprocessed image is: pre-correcting the image illumination based on retinex theory and LIME method.

[0060] Combined with Implementation Example 5, the advantages of this implementation example are explained. In polar environments, image quality may be greatly affected by factors such as strong lighting changes, shadows, and reflections. Using Retinex theory for lighting pre-correction can effectively eliminate the effects caused by uneven lighting, making the details and colors of the image more realistic and balanced, thereby improving the visual quality of the image. The LIME (Local Interpretable Model-Independent Explanation) method makes the enhancement process more transparent and easy to debug by providing interpretability for the image processing model. LIME can help understand and optimize the contribution of different factors in the image enhancement process, for example, which parts of the lighting correction are most important for the enhancement effect. This can better adjust the image processing strategy for unmanned boats in polar environments, especially in low-light and complex environments, thereby improving the robustness and effectiveness of the algorithm.

[0061] Multi-scale learning is of great significance in image processing. In polar environments, images may have different scale features, such as distant terrain and close-up details. Through multi-scale learning, the model can focus on both the detailed information and global information of the image, which helps to enhance the various levels of the image more comprehensively. In this method, combining multi-scale learning and illumination pre-correction can make the enhancement results have good performance at different scales, which can not only restore details but also improve the overall quality of the image.

[0062] By connecting the preprocessed image as a guidance image to the first sub-network, corrections and optimizations can be made beforehand during the enhancement process. The introduction of this method can reduce the complexity of subsequent network processing, thereby improving the efficiency and accuracy of the entire image enhancement process. The role of the guidance image is similar to providing "prior knowledge" to help the neural network learn and optimize the enhancement strategy faster, especially in complex or harsh environments (such as polar environments), and can better cope with low-quality original images.

[0063] By combining illumination pre-correction and multi-scale learning, this method can effectively reduce the impact of noise on image quality. Especially in polar environments, where weather and illumination change dramatically, it is particularly important to enhance the stability of the method. By using the Retinex theory to correct illumination, the negative impact of uneven illumination on the image can be significantly reduced, improving the robustness of image processing.

[0064] Embodiment 7: This embodiment further limits the polar unmanned boat image enhancement method based on multi-scale learning described in Embodiment 6. The method based on retinex theory and LIME performs pre-correction on image illumination, including:

[0065] The Retinex-based degradation model is:

[0066] L=R×T

[0067] Among them, L is the dark image, R is the restored image, and T is the illumination enhancement image;

[0068] The maximum RGB value of each pixel is used as the illumination enhancement map T of the corresponding pixel in the image;

[0069] Calculate the average illumination T of the image mean ;

[0070] According to the average light T mean Calculate the restored image R:

[0071] R=L×0.5 / T mean .

[0072] Embodiment 8: A polar unmanned boat image enhancement system based on multi-scale learning described in this embodiment, the system includes:

[0073] A data acquisition unit, used for acquiring image data;

[0074] An image exposure correction architecture construction unit is used to construct an image exposure correction architecture by improving the Laplacian pyramid, wherein the image exposure correction architecture includes: four subnets, the first subnet is a four-layer encoder-decoder network, the second subnet and the third subnet are both three-layer encoder-decoder networks; the fourth subnet is a three-layer encoder-decoder network;

[0075] The image enhancement unit is used to perform color correction and detail enhancement on the image data through an image exposure correction framework.

[0076] Embodiment 9. A computer device described in this embodiment includes a memory and a processor, wherein the memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes a polar unmanned boat image enhancement method based on multi-scale learning according to any one of embodiments 1 to 7.

[0077] Implementation method ten, a computer-readable storage medium described in this implementation method, on which a computer program is stored, and when the computer program is executed by a processor, the steps of a polar unmanned boat image enhancement method based on multi-scale learning as described in any one of implementation methods one to seven are executed.

[0078] Implementation Method 11: See Figures 2 to 5 This embodiment provides a specific example of the polar unmanned boat image enhancement method based on multi-scale learning described in the first embodiment, and is also used to explain the second to seventh embodiments, specifically:

[0079] For an 8-bit sRGB input image I, after being rendered with incorrect exposure settings, this method aims to generate an output image Y with an exposure error less than I. When both overexposure and underexposure errors are considered, the input image I may contain almost oversaturated or undersaturated areas, resulting in the corruption of color and detail information. The color and detail errors of I are corrected sequentially. Specifically, a multi-resolution representation of I is processed instead of its original input image directly. The Laplacian pyramid derived from the Gaussian pyramid of I is adopted as the multi-resolution decomposition framework.

[0080] The image exposure correction architecture is summarized as follows: A deep network from coarse to fine progressively corrects the exposure error in 8-bit sRGB images. The network first corrects the global color information captured in the last level of the Laplacian pyramid and then processes the subsequent frequency layers. The specific implementation is as follows: Let X be a Laplacian pyramid of the input image I with n layers, where X(l) is the lth level, the last level of the pyramid (i.e., X(n)) captures the low-frequency information of I, and the first level (i.e., X(1)) captures the high-frequency information. These frequency layers are divided into: low-frequency layers store global color information, and medium and high-frequency layers store image details from coarse to fine. These layers can then be used to reconstruct the full-color image I.

[0081] During the Laplacian pyramid reconstruction process, the exposure can be significantly corrected by only using the low-frequency layers (global color information) of the target image. By correcting the Laplacian pyramid layer by layer, detail enhancement is performed sequentially to improve the quality of the final image. Since the correctly exposed image cannot be accessed during the inference phase, the method chooses to predict the missing color and detail information in each layer of the Laplacian pyramid.

[0082] The image exposure correction architecture processes the 4-layer Laplacian pyramid X of the input image I in sequence to generate the final corrected image Y. The model consists of 4 sub-networks, each of which is similar to the UNet structure with independent weights. The network capacity is allocated according to the contribution of each sub-problem (global color correction and detail enhancement) to the result. The network structure is as follows Figure 2 and Figure 3 shown.

[0083] like Figure 2 As shown, the largest subnet (by weight) focuses on processing the global color information in I (i.e., X(n)). This subnet (yellow in the figure) processes the low-frequency layer X(n) and generates an upscaled image Y(n). Upsampling amplifies the output by a factor of two through transposed convolutions with trainable weights. Subsequently, the first intermediate-frequency layer X(n-1) is added to Y(n) and processed by the second subnet. This subnet enhances the details of the current layer, generates a residual layer, and adds it to Y(n)+X(n-1) to reconstruct the image Y(n-1), which is equivalent to the n-1 layer in the Gaussian pyramid. This process continues until the final image Y is generated. The network is fully reliable and supports end-to-end training.

[0084] The main network consists of four subnets and is trained end-to-end with about 7M parameters. The largest capacity is assigned to the first subnet, and the capacity decreases as the layers are refined. Each subnet processes a different image representation extracted from the Laplacian pyramid. The first subnet is a four-layer encoder-decoder network, similar to the UNet structure, which processes the low-band output X(4). Its output is upsampled by a 2×2×3 transposed convolution with a stride of 2, and the generated image is added to the first mid-band output X(3) of the Laplacian pyramid and fed into the second subnet.

[0085] The second subnet is a three-layer encoder-decoder network with skip connections, which processes the upsampled input from the first subnet, generates a residual layer, and upsamples it through a 2×2×3 transposed convolution with a stride of 2. The result is added to the second mid-band output X(2) of the Laplacian pyramid and passed to the third subnet. The third subnet design is the same as the second subnet. The final result is added to the high-band output X(1) of the Laplacian pyramid and fed into the fourth subnet to generate the final processed image. The last subnet is a three-layer encoder-decoder network with skip connections, and the output of the first conversion layer in the encoder has 16 channels.

[0086] The input is concatenated with each sub-network, and high-frequency information is removed. When the network is too deep, the original input information is easily lost, so it needs to be supplemented later. At the same time, it can be seen as a mechanism: machine learning goes from easy to difficult, so the small-scale image is enhanced first, and then the enhanced image is used to assist in guiding the enhancement of the larger-scale image. The network structure diagram is as follows Figure 3 (Contains four sub-networks with UNet-like structures):

[0087] Based on the original improvement, the second and third sub-networks (excluding upsampling deconvolution) share weights, and the upsampling deconvolution layers after the second and third sub-networks also share weights. All neurons in the first hidden layer detect the same features at different positions of the image (because the weights are shared, the detected features are the same). Therefore, this mapping from the input layer to the hidden layer is called a feature map (or filters, kernels). The weight of the feature map is called a shared weight, and its bias is called a shared bias. Shared weights have the following characteristics:

[0088] (1) Parameter sharing: Using the same weights between neurons in a layer means that these neurons update the same parameters during training. This design can significantly reduce the number of parameters that need to be learned, reduce model complexity, and mitigate the risk of overfitting.

[0089] (2) Local receptive field: In convolutional neural networks, through weight sharing, each neuron only focuses on a small area of ​​the input data, which is called the local receptive field. This enables the network to better capture the local patterns of the input data, thereby enhancing translation invariance.

[0090] (3) Translation invariance: Weight sharing enhances the model’s translation invariance to input data. Since neurons with shared weights have the same response to inputs at different positions, the model is more likely to learn features that are independent of position.

[0091] (4) Application of parameter sharing: Weight sharing is usually applied to convolutional layers, where the parameters of the convolution kernel are shared across the entire input image. This weight sharing design makes CNN particularly suitable for image processing tasks.

[0092] (5) Attention and efficiency: Weight sharing can be viewed as an attention mechanism that allows the model to focus more on important areas of shared parameters. In addition, weight sharing also improves the computational efficiency of the model because parameter sharing reduces the number of parameters that need to be trained and stored.

[0093] After the above operations, since the input content has changed, consider adjusting the depth of each sub-network. It is believed that the enhancement of small-scale images does not require a too deep network at the beginning, but a relatively deep network is required later, but the effect is poor. The reason is that the input of the first sub-network is unguided, so although its scale is small, it is still a "difficult task". Therefore, the effect is poor. Then the first sub-network is connected to a pre-processed image as a "guiding image", such as Figure 3 shown.

[0094] LIME first uses the maximum RGB value of each pixel as the illumination of the pixel. At the same time, it retains the overall structure and smoothes the texture details. The Retinex degradation model is shown in the following formula:

[0095] L=R×T

[0096] Among them, L is the dark image, R is the restored image, and T is the illumination map;

[0097] Therefore, the key is to solve T. The preprocessing method is to refer to LIME. First, the maximum RGB value is used as the illumination T of the corresponding pixel of the image, and then the average illumination T of the image is calculated. mean Next, use R = L × 0.5 / T mean It is preprocessed. The reason for selecting 0.5 here is that the average illumination of GT in the training set is found to be 0.48, which is rounded to 0.5. Its purpose is to pre-correct the image illumination based on the retinex theory and LIME method, and use it as a guide image.

[0098] In this embodiment, the structural similarity index (SSIM) and the peak signal-to-noise ratio (PSNR) are used as the primary performance indicators for detecting the accuracy of the image enhancement algorithm.

[0099] Mean Square Error (MSE) is a common indicator for measuring the difference between the current image X and the reference image Y. Its formula is:

[0100]

[0101] In the formula, H and W represent the height and width of the image respectively; n is the number of bits per pixel, which is generally 8, that is, the number of pixel grayscales is 256. The smaller the MSE value, the higher the similarity between the image and the reference image.

[0102] Peak signal-to-noise ratio is a common indicator in image quality evaluation, usually used to measure image compression or reconstruction quality. The unit of peak signal-to-noise ratio is dB, and the larger the value, the smaller the distortion. Its calculation is based on mean square error, as shown in the following formula:

[0103]

[0104] The structural similarity index is used to evaluate the structural similarity of two images. The structural similarity calculation formula of two images X and Y is as follows:

[0105]

[0106] where μ is the mean value, σ x is the variance of image X, σ y is the variance of image Y, σ xy is the covariance.

[0107] The structural similarity index measures the similarity of images by three factors: brightness, contrast, and structure. Specifically, brightness is estimated by the mean, contrast is estimated by the standard deviation, and structural similarity is measured by covariance. This method aims to capture the structural information in the image rather than the simple difference in pixel values, thereby providing a distortion measure that is more consistent with human perception.

[0108] The public dataset MIT-Adobe was selected. MIT-Adobe contains 5,000 raw RGB images and their corresponding sRGB images, which were manually rendered by five professional photographers. Adobe Camera Raw SDK was used to simulate different exposure values ​​(EV) for each Raw-RGB image, covering EV values ​​of -1.5, -1, 0, +1, and +1.5, simulating underexposure, normal exposure, and overexposure. The relative EV value of 0 represents the original exposure setting when shooting. Images manually modified by expert photographers are used as true value images instead of relying solely on rendered images with EV 0. This is because many images contain backlight or exposure errors. The images adjusted by experts are processed in the ProPhoto RGB color space and then converted to the 8-bit sRGB color space. The dataset includes 24,330 8-bit sRGB images, divided into training sets (17,675 images), validation sets (750 images), and test sets (5,905 images) to ensure that there is no image duplication between different datasets.

[0109] First, the proposed algorithm is compared with the multi-scale image exposure correction algorithm to illustrate the intuitive improvement of the exposure correction effect of the improvement of the multi-scale image exposure correction algorithm. The results are shown in Table 1. Next, the proposed algorithm is compared with the multi-scale image exposure correction algorithm to show the improvement effect. It will also be qualitatively and quantitatively compared with traditional image enhancement methods (such as the MSRCR algorithm and the fusion-based low-light image enhancement algorithm) to evaluate the advantages and disadvantages of the exposure correction effect. The specific comparison results are shown in Figure 4 , Figure 5 .

[0110] Table 1 Image evaluation indicators

[0111]

[0112] picture".

[0113] As shown in Table 1, the PSNR of the multi-scale image exposure correction algorithm is 19.76 and the SSIM is 0.7485. By connecting the input to each sub-network and removing high-frequency information, the PSNR is increased to 19.95 and the SSIM is increased to 0.7722, showing a significant improvement (+0.19PSNR, +0.0237SSIM). On this basis, the weight sharing operation (second and third sub-networks) further improves the PSNR to 20.02 and the SSIM to 0.7703. The PSNR is further improved (+0.26) but the SSIM is slightly reduced (-0.0019). Continuing to share the weights of the deconvolution layer, the PSNR reaches 20.03 and the SSIM is slightly reduced to 0.7682. The PSNR increases slightly (+0.27), but the SSIM continues to decrease (-0.0021). Finally, a preprocessed image is added as a “guiding image” to the first subnetwork, and the PSNR rises to 20.16 and the SSIM is 0.7688, with a significant improvement in PSNR (+0.40) and some recovery in SSIM (+0.0006).

[0114] As the number of improved operations increases, PSNR shows an overall upward trend, indicating that image quality is constantly improving. The change in SSIM is relatively fluctuating, with an increase in the early stage, a slight decrease in the later stage, and finally a recovery after adding the "guidance image", but it failed to return to the highest value. These results show that algorithm improvements not only improve image quality (through the increase in PSNR), but also suggest that the changes in PSNR and SSIM need to be considered comprehensively when evaluating image quality.

[0115] See also Figure 4 and Figure 5 From the perspective of qualitative analysis, from left to right are the original input image, the output of the Fusion-based (low brightness image enhancement based on fusion) algorithm, the output of the Retinex algorithm, the output of the MSRCR algorithm, and the output of the improved algorithm of the present invention. Compared with the MSRCR algorithm, the algorithm proposed in the present invention significantly reduces the appearance of target artifacts, and the restoration of edge contours is clearer and more accurate. At the same time, the background restoration effect in polar and cold regions is also better. It can be observed that although the MSRCR algorithm is Figure 5The overall exposure restoration effect of the first pair of underexposed and overexposed images is more obvious, but some details are over-processed, resulting in differences in color restoration, which is inconsistent with the original image information. In contrast, the algorithm proposed in the present invention significantly reduces the haze phenomenon in the background, and is closer to the real scene in the restoration effect of the target color contrast. Although the Retinex algorithm has excellent defogging effect in the first four columns of images, it has low recovery accuracy of the target contour, and the grayscale edge changes are not smooth enough, and the contrast is also insufficient. Compared with these algorithms, the improved algorithm of the present invention performs better in edge contour restoration, image brightness adjustment and detail information extraction. Whether it is processing underexposed or overexposed images, it can provide a more balanced and natural output effect.

[0116] In the images output by the improved algorithm in this paper, the color and brightness of the snow-covered area are more natural, and there is no obvious distortion or over-enhancement compared with the original images in the first column, especially in avoiding the excessive contrast caused by the Fusion-based enhancement algorithm (the second column). The improved algorithm of the present invention shows a strong ability to retain details when processing high-light and low-light areas. Whether it is the glacier texture or the fine structure of the mountain, compared with Retinex (the third column) and MSRCR (the fourth column), the algorithm proposed in this invention can restore more image details in a balanced manner under different lighting conditions, especially in extremely bright or dark areas. The improved algorithm of the present invention can not only deal with local exposure problems, but also maintain the balance of exposure on the whole image, avoiding the local overexposure or underexposure problems that may be caused by other methods, and further enhancing the overall perception of the image. From the results of different scenes, the algorithm of the present invention shows a strong adaptability in complex natural environments (such as glaciers, mountains, etc.), and can generate output with stable visual effects in both low-light and high-light scenes. The improved exposure correction algorithm of the present invention shows obvious advantages in color fidelity, detail recovery and global exposure balance when processing images in extreme environments.

[0117] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure rather than to limit its protection scope. Although the present disclosure has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that after reading the present disclosure, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the invention, but these changes, modifications or equivalent substitutions are all within the protection scope of the disclosed claims to be approved.

Claims

1. A polar unmanned boat image enhancement method based on multi-scale learning, characterized in that: The method comprises: collecting image data; An image exposure correction architecture is constructed by improving the Laplacian pyramid, wherein the image exposure correction architecture includes: four subnets, the first subnet is a four-layer encoder-decoder network, the second subnet and the third subnet are both three-layer encoder-decoder networks; and the fourth subnet is a three-layer encoder-decoder network; Image data is color corrected and detail enhanced via an image exposure correction framework.

2. The polar unmanned boat image enhancement method based on multi-scale learning according to claim 1 is characterized in that: The first subnet is used to process the low-frequency layer X(4), which is upsampled by a 2×2×3 transposed convolution with a stride of 2. The generated image is added to the intermediate frequency layer X(3) output by the first intermediate frequency band of the Laplacian pyramid and sent to the second subnet.

3. The polar unmanned boat image enhancement method based on multi-scale learning according to claim 1 is characterized in that: The second subnet is used to process the upsampled input from the first subnet, generate a residual layer, and upsample through a 2×2×3 transposed convolution with a stride of 2; the generated image is added to the intermediate frequency layer X(2) output by the second intermediate frequency band of the Laplacian pyramid and passed to the third subnet.

4. The polar unmanned boat image enhancement method based on multi-scale learning according to claim 1 is characterized in that: The first transformation layer output in the fourth subnet encoder has 16 channels.

5. The polar unmanned boat image enhancement method based on multi-scale learning according to claim 1 is characterized in that: The method also includes connecting a preprocessed image as a guidance image to the first sub-network.

6. The polar unmanned boat image enhancement method based on multi-scale learning according to claim 5 is characterized in that: The method for obtaining the preprocessed image is: pre-correcting the image illumination based on the retinex theory and the LIME method.

7. The polar unmanned boat image enhancement method based on multi-scale learning according to claim 6 is characterized in that: The method based on retinex theory and LIME performs pre-correction on image illumination, including: The Retinex-based degradation model is: L=R×T Among them, L is the dark image, R is the restored image, and T is the illumination enhancement image; The maximum RGB value of each pixel is used as the illumination enhancement map T of the corresponding pixel in the image; Calculate the average illumination T of the image mean ; According to the average light T mean Calculate the restored image R: R=L×0.5 / T mean 。 8. A polar unmanned boat image enhancement system based on multi-scale learning, characterized in that: The system comprises: A data acquisition unit, used for acquiring image data; An image exposure correction architecture construction unit is used to construct an image exposure correction architecture by improving the Laplacian pyramid, wherein the image exposure correction architecture includes: four subnets, the first subnet is a four-layer encoder-decoder network, the second subnet and the third subnet are both three-layer encoder-decoder networks; the fourth subnet is a three-layer encoder-decoder network; The image enhancement unit is used to perform color correction and detail enhancement on the image data through an image exposure correction framework.

9. A computer device, characterized in that: The invention comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor runs the computer program stored in the memory, the processor executes a polar unmanned boat image enhancement method based on multi-scale learning according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of a polar unmanned boat image enhancement method based on multi-scale learning as described in any one of claims 1 to 7.

Images