Method and system for enhancing low-light color polarized images

Abstract

The application provides an enhancement method and system for low-illumination color polarization images, and relates to the technical field of artificial intelligence, wherein the enhancement method for low-illumination color polarization images comprises the following steps: acquiring a first key parameter corresponding to a low-illumination color polarization image to be enhanced; inputting the low-illumination color polarization image to be enhanced and the first key parameter into a trained deep learning model; and generating, by the trained deep learning model, a plurality of color polarization images of the low-illumination color polarization image to be enhanced under normal illumination according to the first key parameter, so as to enhance the low-illumination color polarization image. The method has strong applicability and improves the overall quality of the generated images.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and particularly relates to a low-illumination color polarization image enhancement method, system, medium and equipment. BACKGROUND

[0002] As an advanced imaging technology, color polarization image plays an important role in many fields due to its unique advantages. Color polarization image can not only reveal the fine texture and structure of the target object surface, but also provide valuable information about the material and light source direction, so it has an irreplaceable position in security monitoring, deep sea exploration, astronomical observation and biomedical research. However, in low-illumination environment (low light intensity), the color polarization image collected is called low-illumination color polarization image, and the low-illumination color polarization image often has some defects, such as the decrease of signal-to-noise ratio and the loss of polarization information, which leads to low usability and reliability of the low-illumination color polarization image. In order to improve the usability and reliability of the low-illumination color polarization image, the low-illumination color polarization image needs to be enhanced.

[0003] The existing schemes for color polarization image enhancement in low-illumination environment mainly include a scheme based on non-local block matching and a scheme based on Retinex theory. The scheme based on non-local block matching can better restore image details, but it is powerless to correct color deviation, resulting in low applicability. The scheme based on Retinex theory can suppress noise generated in dark environment and correct color deviation, but it is only suitable for color deviation correction, resulting in its applicability being limited to a certain extent. SUMMARY

[0004] The purpose of the present application is to overcome the defects of the prior art.

[0005] In order to achieve the above purpose, the present application provides a low-illumination color polarization image enhancement method, system, medium and equipment, and the present application adopts the following technical scheme:

[0006] In a first aspect, the low-illumination color polarization image enhancement method provided by the present application comprises:

[0007] An acquisition step is provided for acquiring a first key parameter corresponding to a low-illumination color polarization image to be enhanced, wherein the first key parameter at least includes first position information and first angle information of a polarization camera;

[0008] An input step is provided for inputting the low-illumination color polarization image to be enhanced and the first key parameter into a trained deep learning model;

[0009] The generating step: according to the first key parameter, the trained deep learning model generates a plurality of color polarization images of the low-illumination color polarization image to be enhanced under normal illumination, and enhances the low-illumination color polarization image, wherein the plurality of color polarization images include light intensity images, polarization angle images and polarization degree images.

[0010] In some examples, the enhancement method further comprises:

[0011] According to the first position information and the first angle information, the trained deep learning model performs the enhancement step to generate a light intensity image of the low-illumination color polarization image to be enhanced under normal illumination.

[0012] In some examples, the enhancement method further comprises:

[0013] According to the light intensity image under normal illumination, the trained deep learning model performs the enhancement step to generate a polarization angle image of the low-illumination color polarization image to be enhanced under normal illumination.

[0014] In some examples, the enhancement method further comprises:

[0015] According to the light intensity image under normal illumination, the trained deep learning model performs the enhancement step to generate a polarization degree image of the low-illumination color polarization image to be enhanced under normal illumination.

[0016] In some examples, the training process of the trained deep learning model comprises:

[0017] Under different illuminations and angles, a plurality of target objects are collected to obtain a plurality of color polarization images corresponding to the plurality of target objects, respectively;

[0018] Second key parameters corresponding to the plurality of color polarization images are obtained, respectively, wherein the second key parameters at least include second position information of a polarization camera, second angle information, and second polarization angle images and second polarization degree images corresponding to the plurality of color polarization images;

[0019] The plurality of color polarization images and the second key parameters are input into a deep learning model as a training data set, the deep learning model is trained to obtain a trained deep learning model.

[0020] In some examples, after obtaining the trained deep learning model, the method further comprises:

[0021] A corresponding loss function is established for the light intensity image, the polarization angle image and the polarization degree image corresponding to each color polarization image generated by the trained deep learning model, respectively, to obtain a first loss function, a second loss function and a third loss function.

[0022] Based on the first loss function, the second loss function and the third loss function, the trained deep learning model is optimized in real time.

[0023] In some examples, before the inputting step is performed, the method further comprises:

[0024] The first key parameter is denoised.

[0025] The denoised first key parameter is normalized.

[0026] In a second aspect, the low-illumination color polarized image enhancement system provided by the present application comprises:

[0027] An acquisition module is configured to acquire a first key parameter corresponding to a low-illumination color polarized image to be enhanced, wherein the first key parameter at least includes first position information and first angle information of a polarized camera;

[0028] An input module is configured to input the low-illumination color polarized image to be enhanced and the first key parameter into a trained deep learning model;

[0029] A generation module is configured to generate, according to the first key parameter, a plurality of color polarized images of the low-illumination color polarized image to be enhanced under normal illumination by the trained deep learning model, so as to enhance the low-illumination color polarized image, wherein the plurality of color polarized images include an intensity image, a polarization angle image and a degree of polarization image.

[0030] Compared with the prior art, the present application has the following advantages:

[0031] The deep learning model is trained based on second key parameters corresponding to color polarized images under different illuminations and viewing angles, the trained deep learning model can simultaneously enhance a plurality of polarization format polarized images (intensity image, polarization angle image and degree of polarization image) corresponding to a low-illumination color polarized image, has strong applicability, and improves the overall quality of the image. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 Fig. 1 shows a low-illumination color polarized image enhancement method flowchart provided by an embodiment of the present application.

[0033] Figure 2 Fig. 2 shows a structure block diagram of a low-illumination color polarized image enhancement system provided by an embodiment of the present application.

[0034] Figure 3 Fig. 3 shows a principle block diagram of an electronic device as a classical computing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0036] Embodiment 1

[0037] As shown in the Figure 1 The low-illumination color polarization image enhancement method provided by the embodiments of the present application comprises:

[0038] S101, obtaining a first key parameter corresponding to a low-illumination color polarization image to be enhanced, wherein the first key parameter comprises first position information and first angle information of a polarization camera.

[0039] Specifically, the low-illumination color polarization image is a light intensity image, and can also be an image of other polarization formats. The first position information is the spatial coordinate information of the photographed target object in the scene relative to the polarization camera, that is, the specific position of the photographed target object in the three-dimensional space, which is specifically represented by three-dimensional coordinates (x, y, z); The first angle information refers to the viewing angle information of the camera relative to the photographed target object, which is specifically represented by Euler angles, including three angle values (pitch angle, yaw angle and roll angle). The position information and the angle information are calculated by the built-in parameter calculation module of the polarization camera, which refers to obtaining the corresponding camera parameters from the polarization image by the COLMAP simulation software.

[0040] S102, inputting the low-illumination color polarization image to be enhanced and the first key parameter into a trained deep learning model.

[0041] Specifically, the deep learning model can be a convolutional neural network model CNN.

[0042] S103, generating a plurality of color polarization images of the low-illumination color polarization image to be enhanced under normal illumination according to the first key parameter and the trained deep learning model, enhancing the low-illumination color polarization image, wherein the plurality of color polarization images comprise a light intensity image, a polarization angle image and a polarization degree image.

[0043] The polarization angle image can represent the vibration direction of linearly polarized light, and the polarization degree image can represent the polarization degree of linearly polarized light.

[0044] In some examples, the enhancement method further comprises:

[0045] According to the first position information and the first angle information, the trained deep learning model performs the enhancement step to generate a light intensity image of the low-illumination color polarized image to be enhanced under normal illumination.

[0046] Specifically, the position information, the angle information, the illumination information and the light intensity image are in one-to-one correspondence. Under different position information, angle information and illumination information, the obtained light intensity image is also different. The trained deep learning model can generate a light intensity image under normal illumination based on the first position information and the first angle information, thereby ensuring that the enhanced color polarized image is consistent in color with the original image.

[0047] In some examples, the enhancement method further includes:

[0048] According to the light intensity image under normal illumination, the trained deep learning model performs the enhancement step to generate a polarization angle image of the low-illumination color polarized image to be enhanced under normal illumination, which helps to maintain the polarization properties of the color polarized image.

[0049] Specifically, the calculation formula of the polarization angle image AoP is:

[0050]

[0051] In some examples, the enhancement method further includes:

[0052] According to the light intensity image under normal illumination, the trained deep learning model performs the enhancement step to generate a polarization degree image of the low-illumination color polarized image to be enhanced under normal illumination, which helps to maintain the authenticity and physical meaning of the original image.

[0053] Specifically, the calculation formula of the polarization degree image DoLP is:

[0054]

[0055] In formula (1) and (2), I0 represents the pixel information of the intensity image in the 0° direction, I 45 represents the pixel information of the intensity image in the 45° direction, I 90 represents the pixel information of the intensity image in the 90° direction, and I 135 represents the pixel information of the intensity image in the 135° direction.

[0056] In some examples, the training process of the trained deep learning model includes:

[0057] A plurality of target objects are collected under different illuminations and viewing angles to obtain a plurality of color polarization images corresponding to the target objects, wherein the different illuminations include low illumination and normal illumination, and the different viewing angles include upward view, horizontal view and downward view.

[0058] Second key parameters corresponding to the plurality of color polarization images are obtained respectively, wherein the second key parameters at least include second position information of the polarization camera, second angle information and second polarization angle images and second polarization degree images corresponding to the plurality of color polarization images;

[0059] The plurality of color polarization images and the corresponding second key parameters are input into a deep learning model as a training data set, the deep learning model is trained to obtain a trained deep learning model.

[0060] The deep learning model is trained based on the second key parameters corresponding to the color polarization images under different illuminations and viewing angles, and the trained deep learning model can simultaneously enhance a plurality of polarization format polarization images (intensity images, polarization angle images and polarization degree images) corresponding to low-illumination color polarization images, has strong applicability and high efficiency.

[0061] The polarization images from each different category of target objects are integrated to create a comprehensive polarization feature representation and fuse the polarization features, so that the model can deeply understand the polarization mode corresponding to the low-illumination color polarization image, can improve the overall quality of the low-illumination color polarization image, and can accurately capture and reproduce the polarization properties of the low-illumination color polarization image.

[0062] In some examples, after obtaining the trained deep learning model, the method further includes:

[0063] A corresponding loss function is established for the intensity images, the polarization angle images and the polarization degree images corresponding to each color polarization image generated by the trained deep learning model to obtain a first loss function, a second loss function and a third loss function;

[0064] The trained deep learning model is optimized in real time based on the first loss function, the second loss function and the third loss function.

[0065] Specifically, the first loss function is used to measure the difference between the color information of the enhanced color polarized image and the color information of the color polarized image under normal illumination, so that the color of the color polarized image generated by the optimized deep learning model is as close to the true color as possible, ensuring that the enhanced image is both beautiful and true. To ensure that the enhanced color polarized image is consistent with the original image in terms of color. The second loss function is used to measure the difference in polarization angle between the enhanced polarization angle image and the polarization angle image obtained under normal illumination, which can maintain the polarization properties of the color polarized image, so that the polarization angle image generated by the optimized deep learning model matches the polarization angle image obtained under normal illumination. The second loss function focuses on the direction information of the polarized light in the image. In many application scenarios, especially tasks involving optical property analysis, the polarization angle provides additional physical attribute information, which can be used to distinguish different types of substances or detect specific surface characteristics. By minimizing the difference between the predicted polarization angle and the actual polarization angle, it is ensured that the enhanced image correctly reflects the polarization characteristics in the scene, which is particularly important for applications that rely on polarization information. The third loss function is used to measure the difference in degree of polarization between the enhanced degree of polarization image and the degree of polarization image obtained under normal illumination, which is also an important indicator of the degree of polarization of light waves, and helps to maintain the realism and physical meaning of the image. The third loss function focuses on the degree of polarization of light in the image, i.e. the uniformity of the direction of light wave vibration. High polarization degree regions often correspond to surfaces with high reflectivity, while low polarization degree regions may represent scattering or diffuse reflection surfaces. By controlling the polarization loss, the deep learning model can learn how to maintain the authenticity of these physical phenomena during the enhancement of the image, which is particularly valuable for tasks that require fine characterization of surface properties.

[0066] Specifically,

[0067] The first loss function is:

[0068]

[0069] In formula (1), N represents the number of training samples, represents the color information of the color polarized image under normal illumination, represents the color information of the enhanced color polarized image.

[0070] The second loss function is:

[0071]

[0072] In formula (2), N represents the number of training samples, represents the polarization angle of the color polarized image under normal illumination, represents the polarization angle of the enhanced color polarized image.

[0073] The third loss function is:

[0074]

[0075] In formula (3), N represents the number of training samples; represents the degree of polarization of the color polarized image under normal illumination, represents the enhanced color polarized image degree of polarization.

[0076] In particular, the total loss function L total :

[0077]

[0078] In particular, the total loss function L total The common guidance model makes reasonable choices when enhancing images, both preserving the basic visual features of the image (such as color) and following the laws of the physical world (such as polarization angle and degree of polarization).

[0079] In formula (4), α, β and γ are weight coefficients, indicating the importance of different loss functions.

[0080] In some examples, before performing the input step, the method further includes:

[0081] The first key parameter is denoised;

[0082] The denoised first key parameter is normalized.

[0083] Specifically, the denoising process includes discarding outliers in the first key parameter. The normalization process includes using a sine or cosine function to encode the position information in the first key parameter, converting the continuous spatial position into a format suitable for deep learning model processing, and then using a sine or cosine function to encode the camera angle information in the first key parameter, so that the trained deep learning model can pay attention to both local and global features of the polarized image, which helps to maintain the consistency and continuity of the image during image enhancement, and enables the deep learning model to better understand and establish the spatial relationship in the scene.

[0084] Embodiment 2

[0085] As Figure 2 shown, the low-illumination color polarized image enhancement system provided by the present application includes:

[0086] The acquisition module is configured to acquire a first key parameter corresponding to a low-illumination color polarization image to be enhanced, wherein the first key parameter at least includes first position information and first angle information of a polarization camera.

[0087] The input module is configured to input the low-illumination color polarization image to be enhanced and the first key parameter into a trained deep learning model.

[0088] The generation module is configured to generate a plurality of color polarization images of the low-illumination color polarization image to be enhanced under normal illumination according to the first key parameter and the trained deep learning model, so as to enhance the low-illumination color polarization image, wherein the plurality of color polarization images include an intensity image, a polarization angle image and a degree of polarization image.

[0089] The low-illumination color polarization image enhancement method, system, medium and device provided by the application can be applied to the following scenarios:

[0090] Autonomous driving: In bad weather (such as rain, snow and fog, etc.), the visible distance of the vehicle is greatly shortened, which is a great challenge for autonomous driving cars. Through the application, the visibility of road target signs, pedestrians and obstacles can be enhanced under low-illumination and adverse weather conditions, and the driving safety can be improved.

[0091] Underwater operation: In deep sea exploration or diving activities, water will absorb most of the visible light, causing serious visual obstacles. Through the application, the clarity of underwater images can be enhanced to help researchers or divers better observe the seafloor topography and biology.

[0092] Military reconnaissance: In night or smog-filled battlefield environment, clear visual intelligence is crucial for designated tactical plans. The unmanned aerial vehicle or other reconnaissance equipment can still provide valuable image information under adverse conditions.

[0093] Astrophysics: When studying distant galaxies or planets, scientists often face weak signals. By enhancing weak signals from the depths of the universe, the application can help researchers observe and understand cosmic phenomena more clearly.

[0094] To verify the effect of the application, the images generated by the application are compared with existing solutions, and peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are used as evaluation indexes, 50 groups of images are used as test, and the comparison results are shown in Table 1. The experimental results show that the application performs best in the indexes PSNR and SSIM, and the overall quality of the images generated by the application is higher.

[0095] Table 1

[0096] Algorithm name PSNR SSIM URetinex-Net 25.730 0.748 ColorPolarNet 25.385 0.799 AwareNet 26.496 0.803 DStokes-CGCP 26.038 0.800 The present application 28.394 0.833

[0097] Embodiment 3

[0098] Figure 3 A hardware structure schematic block diagram of one embodiment of the electronic device provided in the present application is shown. The electronic device includes a processor 601 and a memory 602 storing computer program instructions. The processor implements the method disclosed in the above embodiments of the present application when executing the computer program instructions.

[0099] Specifically, the processor 601 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or can be configured as one or more integrated circuits that implement the embodiments of the present application.

[0100] The memory 602 can include a mass storage for data or instructions. By way of example and not limitation, the memory 602 can include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 602 can include removable or non-removable (or fixed) media. Where appropriate, the memory 602 can be internal or external to the integrated gateway disaster recovery device. In some embodiments, the memory 602 is non-volatile solid-state memory.

[0101] The memory can include read-only memory (ROM), random-access memory (RAM), magnetic disk storage mediums, optical storage mediums, flash memory devices, electrical, optical, or other physical / tangible memory storage devices. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software that, when executed (by one or more processors), is operable to perform operations described with reference to the methods according to an aspect of the present disclosure.

[0102] In some examples, as shown, the electronic device can further include a communication interface 603 and a bus 610. Wherein, as shown, the processor 601, the memory 602, the communication interface 603 are connected through the bus 610 and complete the communication between each other. The electronic device in the embodiments of the present application can be a local server or other computing device, or a cloud server. Figure 3 Figure 3

[0103] ​​The communication interface 603 is mainly used to realize the communication between the modules, devices, units and / or equipment in the embodiments of the present application.

[0104] Bus 610 includes hardware, software, or both, that couples components of the online data traffic billing device to each other in a known manner. For example, but not by way of limitation, bus can include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand (IB) interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-E) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or a combination of two or more of these. Where appropriate, bus 610 can include one or more buses. Although this application describes and shows a particular bus, this application contemplates any suitable bus or interconnect.

[0105] The application also provides a computer program product, which includes computer program instructions, and the computer program instructions are executed by a processor to implement the method provided by the embodiments of the application. The computer program product is, for example, a software installation package, a plug-in compatible with a related software system, etc.

[0106] It should be noted that the application is not limited to the specific configuration and process described above and shown in the drawings. For the sake of brevity, detailed descriptions of well-known methods are omitted. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after understanding the spirit of the application.

[0107] In addition, "first", "second", and similar words used in the present disclosure do not represent any order, number or importance, but are only used to distinguish different parts.

[0108] The above is only a specific implementation of the application, and those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, module and unit can refer to the process in the foregoing method embodiments, which will not be described here. It should be understood that the protection scope of the application is not limited thereto, and any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the application, and these modifications or replacements should be covered within the protection scope of the application.

Claims

1. A method for enhancing low-illumination color polarized images, comprising: obtaining a first key parameter corresponding to a low-illumination color polarized image to be enhanced, wherein the first key parameter comprises at least first position information and first angle information of a polarized camera; inputting the low-illumination color polarized image to be enhanced and the first key parameter into a trained deep learning model; generating, by the trained deep learning model, a plurality of color polarized images of the low-illumination color polarized image to be enhanced under normal illumination according to the first key parameter, wherein the plurality of color polarized images comprises an intensity image, a polarization angle image, and a degree of polarization image, and enhancing the low-illumination color polarized image; a training process of the trained deep learning model comprises: collecting a plurality of target objects under different illuminations and angles to obtain a plurality of color polarized images corresponding to the plurality of target objects, respectively; obtaining a second key parameter corresponding to each of the plurality of color polarized images, wherein the second key parameter comprises at least second position information and second angle information of the polarized camera, and a second polarization angle image and a second degree of polarization image corresponding to the plurality of color polarized images; inputting the plurality of color polarized images and the corresponding second key parameters into the deep learning model as a training data set to train the deep learning model, and obtaining the trained deep learning model.

2. The enhancement method of claim 1, wherein, further comprising: generating, by the trained deep learning model, an intensity image of the low-illumination color polarized image to be enhanced under normal illumination according to the first position information and the first angle information.

3. The enhancement method of claim 2, wherein, further comprising: generating, by the trained deep learning model, a polarization angle image of the low-illumination color polarized image to be enhanced under normal illumination according to the intensity image under normal illumination.

4. The enhancement method of claim 1, wherein, further comprising: generating, by the trained deep learning model, a degree of polarization image of the low-illumination color polarized image to be enhanced under normal illumination according to the intensity image under normal illumination.

5. The method of enhancing of claim 1, wherein, after obtaining the trained deep learning model, the method further comprises: establishing a corresponding loss function for the intensity image, the polarization angle image, and the degree of polarization image corresponding to each color polarized image generated by the trained deep learning model, respectively, to obtain a first loss function, a second loss function, and a third loss function; optimizing the trained deep learning model in real time based on the first loss function, the second loss function, and the third loss function.

6. The enhancement method of claim 1, wherein, before performing the inputting step, the method further comprises: performing denoising processing on the first key parameter; performing normalization processing on the denoised first key parameter.

7. An enhancement system for low-illumination color polarized images, configured to perform the method of claim 1, comprising: an obtaining module configured to obtain a first key parameter corresponding to a low-illumination color polarized image to be enhanced, wherein the first key parameter comprises at least first position information and first angle information of a polarized camera. The input module is configured to input the low-illumination color polarization image to be enhanced and the first key parameter into a trained deep learning model; The generation module is configured to generate, according to the first key parameter, a plurality of color polarization images of the low-illumination color polarization image to be enhanced under normal illumination by the trained deep learning model, and enhance the low-illumination color polarization image, wherein the plurality of color polarization images include an intensity image, a polarization angle image, and a degree of polarization image.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the method of any one of claims 1-6.

9. An electronic device, comprising: The electronic device includes: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method of any one of claims 1-6.

Images