A method, system and device for enhancing low-light color polarization images

Abstract

The present invention discloses a low-illumination color polarization image enhancement method, system and device, comprising the following steps: extracting a low-illumination color polarization image I 0 The polarization state images in four different directions are obtained to obtain the feature representation in different color spaces and polarization information; a network model including an initial convolution module, a three-dimensional polarization residual attention module and a polarization color information recovery module is constructed; the correlation and feature representation between the color polarization sub-images of different channels are obtained through the initial convolution module; the spatial information and texture details in the color polarization image are obtained through the three-dimensional polarization residual attention module; the enhanced color space polarization information image I is generated through the polarization color information recovery module 2 ; Design loss function; Enhance the color space polarization information image I 2 The polarization state images with the same polarization direction are synthesized to generate an enhanced color polarization result image I 3 The technical solution of the present invention can improve the visibility and quality of color polarization images.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a low-illumination color polarization image enhancement method, system and device. Background Art

[0002] Polarization technology is a method of capturing a set of images with recorded intensity responses at different polarization angles. As a new type of information collection method, it can provide clear information such as boundaries and surface features for the target of the image. Color polarization imaging technology has both spectral and polarization characteristics, and can obtain more abundant red, green and blue three-band polarization information than grayscale polarization images. It has potential application value in detecting stealth and camouflage in harsh environments such as haze, smoke, and complex backgrounds. However, in real life, color polarization imaging is often affected by insufficient light and noise, which leads to a serious decline in its imaging quality, which is not conducive to the subsequent processing of the image. Therefore, it is very important to study how to restore the contrast and polarization information of color polarization images in low-light environments.

[0003] Currently, low-illumination polarization image enhancement is divided into two categories. The first is based on traditional enhancement methods, which can suppress image noise and correct color deviation in dark environments. However, the effectiveness of this method is limited by the model capacity of reflectivity and illumination decomposition, and it is very difficult to design effective constraints suitable for various scenarios. The second is the enhancement method based on deep learning. This type of method achieves good results in denoising and color deviation correction, but does not fully consider the differences between color polarization images and traditional intensity images. It is relatively difficult to design an enhancement algorithm that does not lose polarization information and can perform denoising and color deviation correction. Summary of the invention

[0004] The main purpose of the embodiments of the present invention is to propose a low-illumination color polarization image enhancement method, system and device, aiming to design a low-illumination color polarization image enhancement method that improves the visibility and quality of the color polarization image.

[0005] The technical solution of the present invention to solve the above technical problem is to provide a low-illumination color polarization image enhancement method, comprising the following steps:

[0006] (1) Preprocessing of low-illumination color polarization images:

[0007] Extracting low-light color polarization images The polarization state images in four different directions are obtained to obtain the feature representation of different color spaces and polarization information, that is, the color space polarization information image ;

[0008] (2) Building a network model:

[0009] Construct a network model including an initial convolution module, a three-dimensional polarization residual attention module, and a polarization color information recovery module; obtain the correlation and feature representation between color polarization sub-images of different channels through the initial convolution module; obtain the spatial information and texture details in the color polarization image through the three-dimensional polarization residual attention module; generate an enhanced color space polarization information image through the polarization color information recovery module ;

[0010] Design loss function;

[0011] (3) Image fusion:

[0012] The enhanced color space polarization information image Synthesize polarization state images in the same polarization direction to generate an enhanced color polarization result image .

[0013] Furthermore, in the step of extracting polarization state images in four different directions of the low-illuminance color polarization image and obtaining feature representations of different color spaces and polarization information,

[0014] Different polarization states refer to Four directional polarization states, namely , four directional polarization states are used to obtain twelve color space polarization information images in the color model space; wherein the color model space is the HIS color model space, so the twelve color space polarization information images ,Right now

[0015] ;

[0016] In the formula, is a set of color space polarization information images, is the H channel image with polarization 0°, is the I channel image with polarization 0°, It is the S channel image with polarization of 135°.

[0017] Furthermore, the step of obtaining the correlation and feature representation between the color polarization sub-images of different channels through the initial convolution module includes:

[0018] Transform the polarization information of twelve color spaces into images Enter the initial convolution module;

[0019] The initial convolution module performs convolution operations on the color channel through the series connection of three-dimensional convolution layers, fuses the information of different channels, captures the spatial information, color information and polarization information in the color space polarization information image, and generates a more characteristic correlation and feature representation between different channels in the color polarization image, that is, a multi-channel correlation feature image. .

[0020] Furthermore, in the step of obtaining spatial information and texture details in the color polarization image through the three-dimensional polarization residual attention module,

[0021] The three-dimensional polarization residual attention module includes a three-dimensional convolution layer, a three-dimensional attention module and an activation function component.

[0022] The three-dimensional convolution layer extracts features on the width, height and channel of the color polarization image to capture the texture, shape and structure information in the color polarization image space;

[0023] The residual structure is used to emphasize or suppress the information of specific directions or channels in the color polarization image, and the network model's attention to different positions or channels is dynamically adjusted;

[0024] The activation function introduces nonlinear features into the network model, activating the details and edge operations of the color polarization image.

[0025] Furthermore, the calculation process of the three-dimensional attention module is:

[0026] Compute spatial and directional information in color polarization images via the attention mechanism:

[0027] 3D query, key, and value conversion: Convert the output of the 3D convolution into 3D query (Q), key (K), and value (V);

[0028] 3D attention calculation: By calculating the similarity between the 3D query (Q) and the key (K), the 3D attention weight of the color polarization image is calculated using softmax normalization;

[0029] ;

[0030] in, is the 3D attention weight of the color polarization image; T is the transpose of the matrix; is the dimension of the key; weighted sum: weighted sum of the value (K) feature map using the calculated three-dimensional attention weights;

[0031] ;

[0032] in, Represents the output value of the weighted sum.

[0033] Furthermore, the polarization color information recovery module generates an enhanced color space polarization information image The steps include:

[0034] The polarization color information recovery module includes a three-dimensional convolution layer, a batch normalization layer and a ReLu activation function;

[0035] The three-dimensional convolution layer, batch normalization layer and ReLu activation function are connected in series to obtain twelve enhanced color space polarization information images. ;Right now

[0036] ;

[0037] in, is the set of enhanced color space polarization information images, is the H channel image with polarization 0°, is the S channel image with polarization of 135°;

[0038] Furthermore, the enhanced color space polarization information image Synthesize polarization state images in the same polarization direction to generate an enhanced color polarization result image In the steps,

[0039] Different color feature representations of the enhanced color space polarization information image are learned through three-dimensional convolution, and the polarization color channel information is retained in the spatial dimension;

[0040] The complex relationships in the data are learned through nonlinear activation functions, which increases the expressive power of the color synthesis module.

[0041] The batch normalization layer is used to speed up the training process of the network model and improve the stability of the network model.

[0042] Furthermore, in the step of designing a loss function, the loss function includes:

[0043] Strength-Space Loss :

[0044] ;

[0045] in, is a color polarization image at normal illumination; is the enhanced color polarization result image; is the number of training samples; is a pixel of the color polarization image, is the pixel width, is the pixel length; is the L2 norm; ; .

[0046] Polarization information recovery loss :

[0047] ;

[0048] Color information loss :

[0049] ;

[0050] in, It is the total intensity image of the color polarization image, obtained by Stokes vector calculation.

[0051] Total loss function:

[0052] ;

[0053] in, are the hyperparameters in the network, .

[0054] In order to solve the above technical problems, the present invention also provides a low-illumination color polarization image enhancement system based on color space polarization information, comprising:

[0055] A color decomposition module is used to process the input low-illumination color polarization image to obtain a color space polarization information image;

[0056] A low-illumination color polarization image enhancement module is used to process the color space polarization information image to generate an enhanced color space polarization information image;

[0057] The color synthesis module is used to process the enhanced color space polarization information image to obtain an enhanced color polarization result image.

[0058] Furthermore, the low-illumination color polarization image enhancement module includes: an initial convolution module, a three-dimensional polarization residual attention module and a polarization color information recovery module;

[0059] Among them, the initial convolution module, the three-dimensional polarization residual attention module and the polarization color information recovery module are sequentially connected in series to obtain the enhanced color space polarization information image;

[0060] The initial convolution module is used to generate a multi-channel associated feature image;

[0061] The three-dimensional polarization residual attention module is used to generate a spatial texture image;

[0062] The polarization color information recovery module is used to process the complex relationship between polarization and color information and generate an enhanced color space polarization information image.

[0063] To solve the above technical problems, the present invention also provides a device, comprising: a processor, used to execute a computer program of the low-illumination color polarization image enhancement method as described above; and a memory, used to store the computer program and related data.

[0064] Compared with the prior art, this application has the following advantages:

[0065] (1) This invention combines color decomposition and synthesis with deep learning technology for the first time to enhance low-illumination color polarization images. It adopts a relatively novel algorithm and reduces the shortcomings of current low-illumination color polarization images.

[0066] (2) The present invention effectively utilizes color channel, polarization and low-light information to provide a clearer and more colorful image enhancement effect, opening up a new direction to fill the technical gap in this field at home and abroad. The present invention can be used in different environments and scenes, and can be applied to various types of color polarization images, effectively making up for the current technical deficiencies in low-light color polarization image processing.

[0067] (3) Traditional color polarization image enhancement algorithms require the design of relevant specific prior conditions to obtain better results, and are inefficient and cannot be applied in practical applications. The present invention effectively improves the visual quality of color polarization images under low light conditions by combining polarization, color channel information and deep learning technology.

[0068] (4) The present invention uses diverse data sets for training and evaluation to ensure that the algorithm has generalization capabilities in different scenarios and environments. In addition, when training data, factors that may cause technical bias, such as lighting deviation in a specific environment, are removed. The descriptions of various aspects in the embodiments have different emphases. If a part of an embodiment is not described or recorded in detail, you can refer to the relevant descriptions in other embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0070] Figure 1 This is a method flow chart of the low-illumination color polarization image enhancement method of the present invention;

[0071] Figure 2 It is a schematic diagram of the low-illumination color polarization image enhancement system of the present invention;

[0072] Figure 3 It is a schematic diagram of the structural principle of the three-dimensional polarization residual attention module of the present invention;

[0073] Figure 4 It is a schematic diagram of the structural principle of the polarization color information recovery module of the present invention;

[0074] Figure 5 It is a schematic diagram of the structural principle of the color synthesis module of the present invention;

[0075] Figure 6 It is a comparison chart of the processing results of the low-illumination color polarization image enhancement method described in the present invention and other methods.

[0076] Description of Figure Numbers:

[0077] DETAILED DESCRIPTION

[0078] The following will be combined with the drawings in the accompanying drawings of the present invention specification to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0080] In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "several" or "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0081] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0082] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0083] The present invention provides a low-illumination color polarization image enhancement method, system and device, aiming to improve the visibility and quality of the color polarization image.

[0084] The low-illumination color polarization image enhancement method proposed by the present invention will be described in the following specific embodiments:

[0085] In the technical solution of this embodiment, Figure 1 As shown, a low-illumination color polarization image enhancement method comprises the following steps:

[0086] Step 1: Preprocess the low-light color polarization image:

[0087] Extract polarization state images in four different directions of low-illumination color polarization images to obtain feature representations in different color spaces and polarization information;

[0088] Step 2: Build the network model:

[0089] Construct a network model including an initial convolution module, a three-dimensional polarization residual attention module, and a polarization color information recovery module; obtain the correlation and feature representation between color polarization sub-images of different channels through the initial convolution module; obtain the spatial information and texture details in the color polarization image through the three-dimensional polarization residual attention module; generate an enhanced color space polarization information image through the polarization color information recovery module ;

[0090] Design loss function;

[0091] Step 3: Image fusion:

[0092] The enhanced color space polarization information image Synthesize polarization state images in the same polarization direction to generate an enhanced color polarization result image .

[0093] It can be understood that the present application extracts polarization state images in four different directions of low-light color polarization images to obtain their features in different spatial and polarization information. Through this process, specific information in the color polarization image is highlighted or adjusted, which helps the model to better understand the image; and the initial convolution module is used to capture the spatial information on different color channels in the low-light color polarization image, so as to facilitate the extraction of key features in the image, and at the same time, the correlation and spatial distribution between color channels can be learned to more effectively represent the features of the image; the three-dimensional polarization residual attention module uses residual connection, attention mechanism and polarization information integration to more effectively capture low-light The important task of color polarization images in low-light color is to make the network more capable of learning specific representations in polarization images and improve the performance of the model in the entire algorithm; the polarization color information recovery module aims to enhance the color restoration effect under low-light conditions by learning the relationship between polarization and color, and is committed to restoring the color polarization images that may be lost or damaged in low-light color polarization images, thereby improving the visual quality of low-light color polarization images; the color synthesis module fuses the enhanced color space polarization information images with the same polarization direction from different color channels, and synthesizes this image according to the polarization state image with the same polarization direction, so as to produce a richer and more realistic enhanced color polarization result image.

[0094] Furthermore, in the step of extracting polarization state images in four different directions of the low-illuminance color polarization image and obtaining feature representations of different color spaces and polarization information,

[0095] Different polarization states refer to Four directional polarization states, namely , four directional polarization states obtain twelve color space polarization information images in the color model space;

[0096] Among them, the color model space is the HIS color model space, so the twelve color space polarization information images ,Right now

[0097] ;

[0098] In the formula, is a set of color space polarization information images, is the H channel image with polarization 0°, is the I channel image with polarization 0°, It is the S channel image with polarization of 135°.

[0099] It can be understood that different spatial and polarization information feature representations of the color polarization image are extracted so as to analyze the low-illuminance color polarization image represented by different color channels and polarization angles.

[0100] Furthermore, the step of obtaining the correlation and feature representation between the color polarization sub-images of different channels through the initial convolution module includes:

[0101] Transform the polarization information of twelve color spaces into images Enter the initial convolution module;

[0102] The initial convolution module performs convolution operations on the color channel through the series connection of three-dimensional convolution layers, fuses the information of different channels, captures the spatial information, color information and polarization information in the color space polarization information image, and generates the characteristic correlation and feature representation between different channels in the color polarization image, that is, the multi-channel correlation feature image. .

[0103] It can be understood that the initial convolution module is used to perform convolution operations on the color channel through the series connection of three-dimensional convolution layers to fuse information from different channels, which is conducive to capturing the spatial information, color information and polarization information in the color space polarization information image, and generating more characteristic correlations and feature representations between different channels in the color polarization image.

[0104] Specifically, the twelve color space polarization information sub-images ( ) is input to the initial convolution module to generate the correlation and feature representation between the color polarization sub-images of different channels, that is, the multi-channel correlation feature image ( ). The initial convolution module consists of three-dimensional convolutions in series, which is conducive to extracting key features in the image, and can learn the correlation and spatial distribution between color channels to better represent image features. In this initial convolution module instance, a total of 2 three-dimensional convolutions are configured, and the step size is 1×1.

[0105] Furthermore, in the step of obtaining spatial information and texture details in the color polarization image through the three-dimensional polarization residual attention module,

[0106] like Figure 3 As shown, the three-dimensional polarization residual attention module includes a three-dimensional convolution layer, a three-dimensional attention module and an activation function component.

[0107] The three-dimensional convolution layer extracts features on the width, height and channel of the color polarization image to capture the texture, shape and structure information in the color polarization image space;

[0108] The residual structure is used to emphasize or suppress the information of specific directions or channels in the color polarization image, and the network model's attention to different positions or channels is dynamically adjusted;

[0109] The activation function introduces nonlinear features into the network model, activating the details and edge operations of the color polarization image.

[0110] Understandably, the three-dimensional polarization residual attention module is used to better capture the important tasks in low-light color polarization images through residual connection, attention mechanism and polarization information integration, so that the network can better learn specific representations in polarization images and improve the performance of the model in the entire algorithm.

[0111] Specifically, the multi-channel correlation feature image Input to the 3D polarization residual attention module to obtain the spatial information and texture details in the color polarization image, i.e., the spatial texture image ( ). The three-dimensional polarization residual attention module captures the texture, shape and structural information of the color polarization image space through the combination of three-dimensional convolutional layers, three-dimensional attention modules and activation function components. It uses residual structures to emphasize or suppress information in specific directions or channels in color polarization images, and dynamically adjusts the attention of the low-illumination color polarization image enhancement module to different positions or channels. It can more effectively extract key features, focus on important information, and retain original information, which comprehensively improves the performance of the low-illumination color polarization image enhancement module, making the model more capable of adapting to and optimizing images under low-illumination conditions, and achieving effective enhancement of low-illumination color polarization images. The three-dimensional convolution layer is used to process the three-dimensional color polarization image data output by the initial convolution module, that is, the multi-channel associated feature image. The three-dimensional convolution layer extracts features on the width, height and channel of the color polarization image, captures the texture, shape and structure information in the color polarization image space, and provides a basis for three-dimensional convolution and residual connection; the three-dimensional residual attention module dynamically adjusts the attention of the low-illumination color polarization image enhancement module to different positions or channels, more effectively captures important information, and makes it more focused on the part of the image that is relevant to the low-illumination color polarization image enhancement module; the activation function is the ReLu activation function. In the low-illumination color polarization image enhancement module, by introducing nonlinear features and activating the details and edge operations of the color polarization image, the computational efficiency is improved and the problem of gradient disappearance is alleviated.

[0112] Furthermore, the calculation process of the three-dimensional attention module is:

[0113] Compute spatial and directional information in color polarization images via the attention mechanism:

[0114] 3D query, key, and value conversion: Convert the output of the 3D convolution into 3D query (Q), key (K), and value (V);

[0115] 3D attention calculation: By calculating the similarity between the 3D query (Q) and the key (K), the 3D attention weight of the color polarization image is calculated using softmax normalization;

[0116]

[0117] in, is the 3D attention weight of the color polarization image; T is the transpose of the matrix; is the dimension of the key;

[0118] Weighted summation: Use the calculated 3D attention weights to perform weighted summation on the value (K) feature map;

[0119]

[0120] in, Represents the output value of the weighted sum.

[0121] It can be understood that the ReLu activation function is used to introduce nonlinearity and extract features in color polarization images. The three-dimensional polarization residual attention module can simultaneously obtain the height, width, polarization dimension and color dimension information of the low-illuminance color polarization image, capture the spatial texture, shape and structure information of the color polarization image, and dynamically adjust the low-illuminance color polarization image module's attention to different positions or channels, effectively extract key features, focus on important information, retain the original information, and achieve effective enhancement of the low-illuminance color polarization image; the three-dimensional polarization residual attention module is configured with a total of 6 three-dimensional convolutions, 3 three-dimensional attention modules and ReLu nonlinear activation functions.

[0122] Furthermore, the polarization color information recovery module generates an enhanced color space polarization information image The steps include:

[0123] like Figure 4 As shown, the polarization color information recovery module includes a three-dimensional convolution layer, a batch normalization layer and a ReLu activation function;

[0124] The three-dimensional convolution layer, batch normalization layer and ReLu activation function are connected in series to obtain twelve enhanced color space polarization information images. ;Right now

[0125]

[0126] in, is the set of enhanced color space polarization information images, is the H channel image with polarization 0°, It is the S channel image with polarization of 135°;

[0127] It can be understood that the polarization color information recovery module is used to improve the color restoration effect under low illumination conditions by learning the relationship between polarization and color, restore the lost or even damaged color polarization image in the low illumination color polarization image, and improve the visual quality of the low illumination color polarization image.

[0128] Specifically, the three-dimensional convolution layer captures different scale information of the spatial and directional information in the color polarization image, effectively extracts the texture and shape features in the color polarization image, and helps the polarization color information recovery module learn the spatial structure, color information and polarization information of the color polarization image; the batch normalization layer standardizes the output of the three-dimensional convolution layer, standardizes the input of the middle layer of the module, and ensures that its distribution remains consistent during the training process, accelerates the model training speed, and improves the stability of the model; the ReLu activation function extracts the feature information of the color polarization image in the polarization color information recovery module, making the low-light color polarization image enhancement module more suitable for processing the complex relationship between polarization information and color information. The polarization color information recovery module is configured with a total of 3 three-dimensional convolution, batch normalization layers and ReLu nonlinear activation functions.

[0129] Furthermore, the enhanced color space polarization information image Synthesize polarization state images in the same polarization direction to generate an enhanced color polarization result image In the steps,

[0130] Different color feature representations of the enhanced color space polarization information image are learned through three-dimensional convolution, and the polarization color channel information is retained in the spatial dimension;

[0131] The complex relationships in the data are learned through nonlinear activation functions, which increases the expressive power of the color synthesis module.

[0132] The batch normalization layer is used to speed up the training process of the network model and improve the stability of the network model.

[0133] Understandably, if Figure 5 As shown, the color synthesis module includes three-dimensional convolution, nonlinear activation function, batch normalization layer, color channel attention module and 1×1 convolution. By integrating polarization information from different color channels, it learns to more comprehensively express the characteristics of the enhanced color space polarization information image, thereby improving the visual quality of the low-light color polarization image; the three-dimensional convolution can learn the different color feature representations of the enhanced color space polarization information image, and retain the more important polarization color channel information in the spatial dimension; the nonlinear activation function is intended to better learn the complex relationships in the data and increase the expressive power of the color synthesis module; the batch normalization layer is intended to accelerate the training process of the model and improve the stability of the model; the color channel attention module is intended to attach importance to the information from the H, S, and I color channels for more effective fusion; the 1×1 convolution is intended to effectively adjust the relationship between the channels and integrate the information between different channels in the color polarization image.

[0134] Furthermore, in the step of designing a loss function, the loss function includes:

[0135] Strength-Space Loss :

[0136]

[0137] in, is a color polarization image at normal illumination; is the enhanced color polarization result image; is the number of training samples; is a pixel of the color polarization image, is the pixel width, is the pixel length; is the L2 norm; ; .

[0138] Polarization information recovery loss :

[0139]

[0140] Color information loss :

[0141]

[0142] in, is the total light intensity image of the color polarization image, obtained by Stokes vector calculation;

[0143] Total loss function:

[0144]

[0145] in, are the hyperparameters in the network, .

[0146] It can be understood that the intensity-space loss function is used to enhance the intensity of the low-light color polarization image and minimize the loss of image spatial information. Intensity refers to the brightness or grayscale level of the image, and space represents the information at different positions in the image. The brightness and spatial details of the color polarization image are balanced by combining the intensity and space to improve the visual quality of the image and make the details of the generated result image clearer.

[0147] The polarization information recovery loss function is used to recover or enhance the polarization information loss in the image caused by the weakening of light and the influence of noise in the low-light environment. It helps to fuse the information of different bands and highlight the polarization information blurred by the influence of the low-light environment.

[0148] The color information loss function is used to reduce or compensate for the weakening of color information and the influence of noise in low-light environments. By adjusting the brightness and contrast of different channels in low-light color polarization images, a better color balance is achieved, which helps to compensate for the distortion of color information under low-light conditions and improve the color reproduction and recognizability of the image.

[0149] The total loss function assigns weights to each sub-loss function, which can flexibly adjust the relative importance of different types of losses in the training process and balance each sub-loss function to enable the model to achieve better performance on multiple objectives.

[0150] Embodiment 2: A low-illumination color polarization image enhancement system based on color space polarization information, such as Figure 2 As shown, including:

[0151] The color decomposition module 1 is used to process the input low-illumination color polarization image to obtain a color space polarization information image;

[0152] Low illumination color polarization image enhancement module 2, used to process the color space polarization information image to generate an enhanced color space polarization information image;

[0153] The color synthesis module 3 is used to process the enhanced color space polarization information image to obtain an enhanced color polarization result image.

[0154] Furthermore, the low-illumination color polarization image enhancement module 2 includes: an initial convolution module 201, a three-dimensional polarization residual attention module 202 and a polarization color information recovery module 203;

[0155] Among them, the initial convolution module 201, the three-dimensional polarization residual attention module 202 and the polarization color information recovery module 203 are sequentially connected in series to obtain the enhanced color space polarization information image;

[0156] The initial convolution module 201 is used to generate a multi-channel associated feature image;

[0157] The three-dimensional polarization residual attention module 202 is used to generate a spatial texture image;

[0158] The polarization color information recovery module 203 is used to process the complex relationship between polarization and color information and generate an enhanced color space polarization information image.

[0159] Embodiment 3: A device comprises: a processor for executing a computer program of the low-illumination color polarization image enhancement method as described in Embodiment 1; and a memory for storing the computer program and related data.

[0160] Embodiment 4: A computer-readable storage medium, which contains a computer program. When the computer program is executed by a processor, each step described in Embodiment 1 can be implemented. The computer-readable storage medium can take a variety of forms, including but not limited to a disk, an optical disk, a flash memory, etc. In addition, it can be used on different computing devices. By loading and executing the program code in the storage medium, the user can make full use of the functions provided by the embodiment of the present invention to complete the corresponding tasks. This flexibility makes the method of the present invention applicable to different computing devices and application scenarios.

[0161] Embodiment 5: An information data processing terminal for executing on an electronic device. The terminal provides a user input interface to implement the various steps described in Embodiment 1. This information data processing terminal can be various types of devices, such as mobile phones, computers, switches, etc. By using this information data processing terminal, users can easily input and operate to achieve the functions described in the embodiments of the method of the present invention. This information data processing terminal can be used in conjunction with computer equipment to provide users with more efficient task execution and corresponding technical effects. By executing the method of the present invention on an electronic device, users can more conveniently operate and process color polarization images to achieve better quality image enhancement effects.

[0162] Embodiment 6: A server is executed on an electronic device and provides an input interface for a user to implement the steps in the above embodiment 1. This server design allows operation in a distributed computing environment, providing users with efficient and convenient remote access and processing of color polarization images. By executing the method of the present invention on the server, the user can process the image through a network connection without transmitting the image data to a local device. This server-based implementation provides users with greater flexibility and convenience, allowing image enhancement functions to be easily embedded in a variety of different applications and services.

[0163] Embodiment 7: A computer program product, when the computer program product is run on an electronic device, enables the electronic device to perform the steps described in the above embodiment 1. This computer program product can exist in the form of software, and by loading and executing the corresponding program code on the electronic device, the user can easily use the low-light color polarization image enhancement function of the present invention. The implementation of this computer program product enables the method of the present invention to be easily embedded in various computing devices and systems, providing users with a more flexible and efficient image processing solution.

[0164] Embodiment 8: A method for enhancing a low-illumination color polarization image is implemented according to the method of Embodiment 1;

[0165] Specifically, the image data of this embodiment selects the 2023 public data sets LLCP and PLIE as training and test sets, and uses a color decomposition module to further extract different spatial and polarization information feature representations in the low-illuminance color polarization image to obtain a color space polarization information image; a low-illuminance color polarization image enhancement module is used to sequentially input the color space polarization information image into the initial convolution module, the three-dimensional polarization residual attention module and the polarization color information recovery module to obtain an enhanced color space polarization information image; a color synthesis module is used to synthesize the enhanced color space polarization information image according to the polarization state image in the same polarization direction to obtain an enhanced color polarization result image;

[0166] To verify the performance of this embodiment, the enhancement effect of this embodiment is compared with the enhancement algorithm based on deep learning in recent years. Figure 6 FIG. 1 is a diagram comparing the processing results of a low-illumination color polarization image enhancement method of the present invention and other methods.

[0167] It can be clearly seen from the result graph that the algorithm of this embodiment retains polarization information while improving the brightness of the low-light color polarization image. At the same time, the use of three-dimensional convolution can obtain more dimensional color polarization information, suppress the disordered noise distribution in the dark environment, and make the enhanced color polarization image result clearer. For example, the S0 image of the U-Retinex algorithm is darker overall, and it can be seen from the DoLP graph that the polarization information is significantly reduced, and the font on the beverage bottle is already blurred; the DIE and ERR algorithms have better overall brightness, but the AoP images are poor, and the font on the beverage bottle is almost covered. This is due to the presence of a large amount of disordered noise in the dark environment that has not been eliminated; the LIME algorithm is better overall, but it is slightly overexposed, causing the upper part of the sea buckthorn beverage bottle to become too transparent; the S0 images in LoG and RetinexNet are too overexposed and too bright, which does not meet the human eye's senses; the Zero-DCE algorithm is darker overall and has more noise.

[0168] In the following example, the enhancement algorithm of the present invention is qualitatively evaluated, using PSNR, SSIM, and information entropy as evaluation indicators, and 50 groups of low-light images as tests. The comparison results are shown in Table 1.

[0169] The experimental results show that the low-illumination color polarization image enhancement algorithm based on color space polarization information of this embodiment performs better in most aspects of PSNR, SSIM and information entropy for the enhanced color polarization image. For example, the S0 image performs best in both PSNR and information entropy, and SSIM ranks second best; in DoLP and AoP images, it performs best in PSNR and SSIM indicators.

[0170] Table 1 Comparison results of different algorithms

[0171]

[0172] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A low-illumination color polarization image enhancement method, characterized in that: The following steps are involved: (1) Preprocessing of low-illumination color polarization images: Extracting low-light color polarization images The polarization state images in four different directions are obtained to obtain the feature representation of different color spaces and polarization information, that is, the color space polarization information image ; (2) Building a network model: Construct a network model including an initial convolution module, a three-dimensional polarization residual attention module, and a polarization color information recovery module; obtain the correlation and feature representation between color polarization sub-images of different channels through the initial convolution module; obtain the spatial information and texture details in the color polarization image through the three-dimensional polarization residual attention module; generate an enhanced color space polarization information image through the polarization color information recovery module ; Design loss function; (3) Image fusion: The enhanced color space polarization information image Synthesize polarization state images in the same polarization direction to generate an enhanced color polarization result image ; In the step of designing a loss function, the loss function includes: Strength-Space Loss : ;in, is a color polarization image at normal illumination; is the enhanced color polarization result image; is the number of training samples; is a pixel of the color polarization image, is the pixel width, is the pixel length; is the L2 norm; ; ; Polarization information recovery loss : ； Color information loss : ； in, is the total light intensity image of the color polarization image, obtained by Stokes vector calculation; Total loss function: ； in, are the hyperparameters in the network, ; In the step of extracting polarization state images in four different directions of the low-illumination color polarization image and obtaining feature representations of information in different color spaces and polarization information, Different polarization states refer to Four directional polarization states, namely , four directional polarization states obtain twelve color space polarization information images in the color model space; Among them, the color model space is the HIS color model space, so the twelve color space polarization information images ,Right now ； In the formula, is a set of polarization information images in color space, is the H channel image with polarization 0°, is the I channel image with polarization 0°, It is the S channel image with polarization of 135°.

2. The low-illumination color polarization image enhancement method according to claim 1, characterized in that: The step of obtaining the correlation and feature representation between the color polarization sub-images of different channels through the initial convolution module includes: Transform the polarization information of twelve color spaces into images Enter the initial convolution module; The initial convolution module performs convolution operations on the color channel through the series connection of three-dimensional convolution layers, fuses the information of different channels, captures the spatial information, color information and polarization information in the color space polarization information image, and generates a more characteristic correlation and feature representation between different channels in the color polarization image, that is, a multi-channel correlation feature image. .

3. The low-illumination color polarization image enhancement method according to claim 1, characterized in that: In the step of obtaining spatial information and texture details in the color polarization image through the three-dimensional polarization residual attention module, The three-dimensional polarization residual attention module includes a three-dimensional convolution layer, a three-dimensional attention module and an activation function component. The three-dimensional convolution layer extracts features on the width, height and channel of the color polarization image to capture the texture, shape and structure information in the color polarization image space; The residual structure is used to emphasize or suppress the information of specific directions or channels in the color polarization image, and the network model's attention to different positions or channels is dynamically adjusted; The activation function introduces nonlinear features into the network model, activating the details and edge operations of the color polarization image.

4. The low-illumination color polarization image enhancement method according to claim 3, characterized in that: The calculation process of the three-dimensional attention module is: Compute spatial and directional information in color polarization images via the attention mechanism: 3D query, key, and value conversion: Convert the output of the 3D convolution into 3D query (Q), key (K), and value (V); 3D attention calculation: By calculating the similarity between the 3D query (Q) and the key (K), the 3D attention weight of the color polarization image is calculated using softmax normalization; ； in, is the 3D attention weight of the color polarization image; T is the transpose of the matrix; is the dimension of the key; Weighted summation: Use the calculated 3D attention weights to perform weighted summation on the value (K) feature map; ;in, Represents the output value of the weighted sum.

5. The low-illumination color polarization image enhancement method according to claim 1, characterized in that: The polarization color information recovery module generates an enhanced color space polarization information image The steps include: The polarization color information recovery module includes a three-dimensional convolution layer, a batch normalization layer and a ReLu activation function; The three-dimensional convolution layer, batch normalization layer and ReLu activation function are connected in series to obtain twelve enhanced color space polarization information images. ;Right now ； in, is the set of enhanced color space polarization information images, is the H channel image with polarization 0°, It is the S channel image with polarization of 135°.

6. The low-illumination color polarization image enhancement method according to claim 1, characterized in that: The enhanced color space polarization information image Synthesize polarization state images in the same polarization direction to generate an enhanced color polarization result image In the steps, Different color feature representations of the enhanced color space polarization information image are learned through three-dimensional convolution, and the polarization color channel information is retained in the spatial dimension; The complex relationships in the data are learned through nonlinear activation functions, which increases the expressive power of the color synthesis module. The batch normalization layer is used to speed up the training process of the network model and improve the stability of the network model.

7. A low-illumination color polarization image enhancement system based on color space polarization information, used to implement the low-illumination color polarization image enhancement method according to any one of claims 1 to 6, characterized in that: include: A color decomposition module is used to process the input low-illumination color polarization image to obtain a color space polarization information image; A low-illumination color polarization image enhancement module is used to process the color space polarization information image to generate an enhanced color space polarization information image; A color synthesis module is used to process the enhanced color space polarization information image to obtain an enhanced color polarization result image; The low-illumination color polarization image enhancement module includes: an initial convolution module, a three-dimensional polarization residual attention module and a polarization color information recovery module; Among them, the initial convolution module, the three-dimensional polarization residual attention module and the polarization color information recovery module are sequentially connected in series to obtain the enhanced color space polarization information image; The initial convolution module is used to generate a multi-channel associated feature image; The three-dimensional polarization residual attention module is used to generate a spatial texture image; The polarization color information recovery module is used to process the complex relationship between polarization and color information and generate an enhanced color space polarization information image.

8. A device, characterized in that include: A processor, configured to execute a computer program of the low-illumination color polarization image enhancement method according to any one of claims 1 to 6; Memory is used to store computer programs and related data.

Images