Image enhancement method and device, chip, electronic equipment and readable storage medium

Abstract

The invention relates to the technical field of images, in particular to an image enhancement method and device, a chip, electronic equipment and a readable storage medium, and the image processing method comprises the steps: decomposing a to-be-processed image into a to-be-processed illumination image and a to-be-processed reflection image through a decomposition network algorithm; the to-be-processed illumination diagram represents the illumination intensity of the to-be-processed image; inputting the illuminance diagram to be processed into a joint optimization model, and outputting a first optimized illuminance diagram; inputting the first optimized illuminance diagram into an attention mechanism model, and outputting a second optimized illuminance diagram; inputting the reflectogram to be processed into a hybrid residual network model, and outputting a first optimized reflectogram, the hybrid residual network model being used for improving the similarity between the reflectogram to be processed and a real image; and performing noise reduction processing on the first optimized reflection image, and outputting a second optimized reflection image. According to the method, the enhancement effect on the low-brightness image can be improved, details are improved, and noisy points are reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image enhancement method, device, chip, electronic device and readable storage medium. Background Art

[0002] In the power transmission system, transmission lines are the arteries of power transmission, and their safe and stable operation is crucial. However, since transmission lines are widely distributed in outdoor environments and exposed to natural conditions for a long time, they are easily disturbed by various external factors, especially the attachment or hanging of foreign objects. These hidden dangers may not only cause faults such as short circuits and line breaks, but also pose a threat to the surrounding environment and personnel safety. Therefore, it is necessary to monitor the transmission lines day and night.

[0003] In nighttime environments, poor lighting conditions often lead to image quality degradation, including low contrast, high noise, and color cast. Traditional image processing methods usually use algorithms such as histogram equalization, gamma correction, and Retinex. Histogram equalization improves the contrast of the image by adjusting the grayscale distribution of pixels, while gamma correction enhances dark details by changing the brightness relationship of pixels. However, these methods often have problems of over-enhancement, loss of details, or inability to effectively suppress noise when enhancing low-brightness images. Therefore, an image processing method that can enhance low-brightness images with high performance is needed to achieve high-quality imaging in low-light environments. Summary of the invention

[0004] In order to solve the problems in the related art, the embodiments of the present disclosure provide an image enhancement method, device, chip, electronic device and readable storage medium.

[0005] In a first aspect, an image enhancement method is provided in an embodiment of the present disclosure, comprising:

[0006] Get the image to be processed;

[0007] Decomposing the image to be processed into an illumination map to be processed and a reflection map to be processed by a decomposition network algorithm; the illumination map to be processed represents the illumination intensity of the image to be processed, and the reflection map to be processed contains texture, color and detail information in the image to be processed;

[0008] Inputting the illumination map to be processed into a joint optimization model and outputting a first optimized illumination map, wherein the joint optimization model is used to enhance the contrast of the image to be processed;

[0009] Inputting the first optimized illumination map into an attention mechanism model and outputting a second optimized illumination map, wherein the attention mechanism model is used to restore details of the first optimized illumination map;

[0010] Inputting the reflection image to be processed into a hybrid residual network model and outputting a first optimized reflection image, wherein the hybrid residual network model is used to improve the similarity between the reflection image to be processed and a real image;

[0011] After the first optimized reflection map is subjected to dilated convolution processing, the first optimized reflection map is input into a plurality of first residual modules for noise reduction processing, and the second optimized reflection map is output; the dilated convolution processing is used to expand the receptive field of the first optimized reflection map; the first residual module includes a plurality of stride convolution layers, and jump connections are set between the plurality of first residual modules;

[0012] An enhanced image is output according to the second optimized illumination map and the second optimized reflectance map.

[0013] According to an embodiment of the present disclosure, the step of inputting the illumination map to be processed into a joint optimization model and outputting a first optimized illumination map includes:

[0014] Obtaining the brightness grayscale of the pixel in the illumination map to be processed, the minimum grayscale value of the illumination map to be processed, and the maximum grayscale value of the illumination map to be processed, and calculating the probability density function of the pixel according to the brightness grayscale;

[0015] Calculating a weighted probability density function of the brightness grayscale according to the probability density function of the pixel;

[0016] Normalizing the weighted probability density function to calculate the smoothed cumulative distribution function of the brightness grayscale;

[0017] Performing adaptive gamma correction on the smoothed cumulative distribution function to calculate the mapping function of the brightness grayscale;

[0018] According to the mapping function, the brightness values ​​of the pixels are mapped and a first optimized illumination map is output.

[0019] According to an embodiment of the present disclosure, the probability density function PDF(l) is expressed as follows:

[0020]

[0021] Among them, n l is the number of pixels with brightness l, MN is the total number of pixels in the image;

[0022] The weighted probability density function PDF ω (l) is expressed as follows:

[0023]

[0024] Where α is a constant and 0.5≤α≤1, max(PDF) is the maximum value of the probability density function, and min(PDF) is the minimum value of the probability density function;

[0025] The smoothed cumulative distribution function CDF s (l) is expressed as follows:

[0026]

[0027] Among them, l max is the maximum pixel brightness, l min is the minimum pixel brightness;

[0028] The mapping function T(l) of the brightness grayscale is expressed as follows:

[0029]

[0030] γ=1-CDF s (l)*P;

[0031] Wherein, P is an adaptive function and 0.5≤P≤1.

[0032] According to an embodiment of the present disclosure, the joint optimization model is implemented by a reconfigurable circuit, and the reconfigurable circuit includes a first input terminal, a second input terminal, a third input terminal, a fourth input terminal, a first parameter unit, a second parameter unit, a first calculation unit, a second calculation unit, a third calculation unit, and an output terminal, wherein:

[0033] The first parameter unit is connected to the first input terminal and the second input terminal;

[0034] A first parameter unit is connected to the first input terminal and the second input terminal respectively through a first selector; the first parameter unit is set to log2k, where k is an input value of the first input terminal or the second input terminal;

[0035] The first calculation unit is connected to the first parameter unit through a second selector; the first calculation unit is configured as a subtraction unit;

[0036] The second calculation unit is connected to the first calculation unit, the third input terminal, and the second parameter unit respectively; the second calculation unit is set as a multiplication unit, and the second parameter unit is set as 2 m , m is the output value of the second computing unit;

[0037] The third calculation unit is connected to the second parameter unit, the fourth input terminal and the output terminal respectively; the third calculation unit is a multiplication unit.

[0038] According to an embodiment of the present disclosure, when the reconfigurable circuit is configured as a module for calculating the weighted probability density function, the first input terminal is configured as PDF(l)-min(PDF), the second input terminal is configured as max(PDF)-min(PDF), the third input terminal is configured as α, and the fourth input terminal is configured as max(PDF);

[0039] When the reconfigurable circuit is configured to calculate the smoothed cumulative distribution function, the first input terminal is configured as The second input terminal is configured as ∑PDF ω ;

[0040] When the reconfigurable circuit is configured as a module for calculating the mapping function of the brightness grayscale, the first input terminal is configured as 1, and the second input terminal is configured as 1. max -l min , the third input terminal is configured as γ, and the fourth input terminal is configured as l max -l min .

[0041] According to an embodiment of the present disclosure, the step of inputting the first optimized illumination map into an attention mechanism model and outputting the second optimized illumination map includes:

[0042] Performing Fourier transformation on the first optimized illumination map to convert the first optimized illumination map from the frequency domain to the spatial domain;

[0043] enhancing the high-frequency signal in the spatial domain to restore details of the first optimized illumination map;

[0044] Performing an inverse Fourier transform on the spatial domain to convert from the spatial domain to the frequency domain to generate a first converted illumination map;

[0045] The first converted illumination map is input into a dual attention module, and a second optimized illumination map is output; the dual attention module includes an activation function convolution block, a spatial attention unit and a channel attention unit.

[0046] According to an embodiment of the present disclosure, the hybrid residual network model includes a plurality of second residual modules, the second residual modules include 5 convolutional layers, and the jump connections are set between the second residual modules;

[0047] The training steps of the hybrid residual network model are as follows:

[0048] Obtain a pair of low brightness image low and normal brightness image normal from the low brightness image training set and the corresponding normal brightness image training set;

[0049] Decomposing the low brightness image into a low brightness illumination image and a low brightness reflection image, and decomposing the normal brightness image into a normal brightness illumination image and a normal brightness reflection image;

[0050] Generate a reconstruction loss function according to the low-brightness illumination map, the low-brightness reflection map, the normal-brightness illumination map, the normal-brightness reflection map, the low-brightness image, and the normal-brightness image;

[0051] Generate a constant reflection loss function according to the low brightness reflection map and the normal brightness reflection map;

[0052] Generate an illumination smoothness loss function according to the low brightness illumination map and the normal brightness illumination map;

[0053] Repeat the above steps to generate a loss function according to the reconstruction loss function, the constant reflection loss function, and the illumination smoothness loss function, and minimize the value of the loss function.

[0054] According to an embodiment of the present disclosure, the reconstruction loss function It is expressed as follows:

[0055]

[0056] Among them, low is a low brightness image, normal is a normal brightness image, R i is the reflection map, I i is the illumination diagram, S i is the image before decomposition, α is the first weight coefficient, L1 is the absolute value loss function, SSIM is the structural similarity loss function, Multiply the reflectance map and the illumination map;

[0057] The constant reflection loss function L r It is expressed as follows:

[0058] L r =|R low -R normal |

[0059] Among them, R low For low brightness reflection image R normal It is a normal brightness reflection map;

[0060] The lighting smoothness loss function It is expressed as follows:

[0061]

[0062] in, is the illumination image gradient, exp() is the exponential function, λ tv is a constant, and 0 is matrix multiplication.

[0063] According to an embodiment of the present disclosure, before inputting the first optimized illumination map into the attention mechanism model and outputting the second optimized illumination map, the method further includes:

[0064] Performing a dilated convolution process on the first optimized illumination map, wherein the dilated convolution process is used to expand the receptive field of the first optimized illumination map;

[0065] The first optimized illumination map is input into a plurality of second residual modules for noise reduction processing; and jump connections are set between the plurality of second residual modules.

[0066] In a second aspect, an image enhancement device is provided in an embodiment of the present disclosure, including:

[0067] An acquisition module, configured to acquire an image to be processed;

[0068] A decomposition module is configured to decompose the image to be processed into an illumination map to be processed and a reflection map to be processed by a decomposition network algorithm; the illumination map to be processed represents the light intensity of the image to be processed, and the reflection map to be processed contains texture, color and detail information in the image to be processed;

[0069] A first optimization module is configured to input the illumination map to be processed into a joint optimization model and output a first optimized illumination map, wherein the joint optimization model is used to enhance the contrast of the image to be processed;

[0070] A second optimization module is configured to input the first optimized illumination map into an attention mechanism model and output a second optimized illumination map, wherein the attention mechanism model is used to restore details of the first optimized illumination map;

[0071] A third optimization module is configured to input the reflection image to be processed into a hybrid residual network model and output a first optimized reflection image, wherein the hybrid residual network model is used to improve the similarity between the reflection image to be processed and the real image;

[0072] A fourth optimization module is configured to perform dilated convolution processing on the first optimized reflection map, input multiple first residual modules for noise reduction processing, and output the second optimized reflection map; the dilated convolution processing is used to expand the receptive field of the first optimized reflection map; the first residual module includes multiple stride convolution layers, and jump connections are set between the multiple first residual modules;

[0073] An output module is configured to output an enhanced image according to the second optimized illumination map and the second optimized reflectance map.

[0074] According to an embodiment of the present disclosure, the first optimization module includes:

[0075] A first acquisition submodule is configured to acquire the brightness grayscale of the pixel in the illumination map to be processed, the minimum grayscale value of the illumination map to be processed, and the maximum grayscale value of the illumination map to be processed, and calculate the probability density function of the pixel according to the brightness grayscale;

[0076] A first calculation submodule is configured to calculate a weighted probability density function of the brightness grayscale according to the probability density function of the pixel;

[0077] A second calculation submodule is configured to perform normalization processing on the weighted probability density function to calculate the smoothed cumulative distribution function of the brightness grayscale;

[0078] A third calculation submodule is configured to perform adaptive gamma correction on the smoothed cumulative distribution function to calculate the brightness grayscale mapping function;

[0079] The first output submodule is configured to map the brightness values ​​of the pixels according to the mapping function and output a first optimized illumination map.

[0080] According to an embodiment of the present disclosure, the second optimization module includes:

[0081] A first conversion submodule is configured to perform Fourier transform on the first optimized illumination map to convert the first optimized illumination map from the frequency domain to the spatial domain;

[0082] an enhancement submodule, configured to enhance the high-frequency signal in the spatial domain to restore details of the first optimized illumination map;

[0083] A second conversion submodule is configured to perform an inverse Fourier transform on the spatial domain, convert from the spatial domain to the frequency domain, and generate a first converted illumination map;

[0084] The second output submodule is configured to input the first converted illumination map into a dual attention module and output a second optimized illumination map; the dual attention module includes an activation function convolution block, a spatial attention unit and a channel attention unit.

[0085] According to an embodiment of the present disclosure, before the second optimization module, the device further includes:

[0086] a second convolution submodule, configured to perform a dilated convolution process on the first optimized illumination map, wherein the dilated convolution process is used to expand the receptive field of the first optimized illumination map;

[0087] The second denoising submodule is configured to input the first optimized illumination map into a plurality of second residual modules for denoising; and a jump connection is set between the plurality of second residual modules.

[0088] In a third aspect, an embodiment of the present disclosure provides a chip, comprising a processor and an interface; the processor is used to read instructions to execute any one of the methods described in the first aspect.

[0089] In a fourth aspect, an embodiment of the present disclosure provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, and wherein the one or more computer instructions are executed by the processor to implement a method as described in any one of the first aspects.

[0090] In a fifth aspect, an embodiment of the present disclosure provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, the method described in the first aspect is implemented.

[0091] According to the technical solution provided by the embodiment of the present disclosure, an image enhancement method is disclosed, including: obtaining an image to be processed; decomposing the image to be processed into an illumination map to be processed and a reflection map to be processed by a decomposition network algorithm; the illumination map to be processed represents the light intensity of the image to be processed, and the reflection map to be processed contains texture, color and detail information in the image to be processed; inputting the illumination map to be processed into a joint optimization model, outputting a first optimized illumination map, and the joint optimization model is used to enhance the contrast of the image to be processed; inputting the first optimized illumination map into an attention mechanism model, outputting a second optimized illumination map, and the attention mechanism model is used to Restore the details of the first optimized illumination map; input the reflection map to be processed into a hybrid residual network model, and output a first optimized reflection map, wherein the hybrid residual network model is used to improve the similarity between the reflection map to be processed and the real image; after the first optimized reflection map is subjected to dilated convolution processing, it is input into multiple first residual modules for denoising processing, and the second optimized reflection map is output; the dilated convolution processing is used to expand the receptive field of the first optimized reflection map; the first residual module includes multiple stride convolution layers, and jump connections are set between the multiple first residual modules; according to the second optimized illumination map and the second optimized reflection map, an enhanced image is output.

[0092] According to the implementation manner of the present disclosure, the method decomposes a low-brightness image into an illumination map and a reflection map. For the illumination map, a joint optimization model and an attention mechanism model are successively used to enhance contrast and restore details. For the reflection map, a hybrid residual network model and denoising processing are successively used to improve the similarity between the reflection map and the real image and suppress noise. Finally, the optimized illumination map and reflection map are combined to form an enhanced image, which can improve the enhancement effect of the low-brightness image, enhance details and reduce noise.

[0093] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Other features, objectives and advantages of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0095] Figure 1 A flowchart of an image enhancement method according to an embodiment of the present disclosure is shown.

[0096] Figure 2 A flowchart of a joint optimization model in an image enhancement method according to an embodiment of the present disclosure is shown.

[0097] Figure 3 A structural diagram of a reconfigurable circuit in an image enhancement method according to an embodiment of the present disclosure is shown.

[0098] Figure 4 A structural schematic diagram of a reconfigurable circuit configuration scheme in an image enhancement method according to an embodiment of the present disclosure is shown.

[0099] Figure 5 A flowchart of an attention mechanism model in an image enhancement method according to an embodiment of the present disclosure is shown.

[0100] Figure 6 A structural block diagram of an image enhancement device according to an embodiment of the present disclosure is shown.

[0101] Figure 7 A structural block diagram of a joint optimization model in an image enhancement device according to an embodiment of the present disclosure is shown.

[0102] Figure 8 A structural block diagram of an attention mechanism model in an image enhancement device according to an embodiment of the present disclosure is shown.

[0103] Fig. 9 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0104] Fig.10 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0105] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0106] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, behaviors, components, parts, or a combination thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or a combination thereof exist or are added.

[0107] It should also be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0108] In the present disclosure, if it involves operations of obtaining user information or user data or displaying user information or user data to others, the operations are all authorized and confirmed by the user, or actively selected by the user.

[0109] In the power transmission system, transmission lines are the arteries of power transmission, and their safe and stable operation is crucial. However, since transmission lines are widely distributed in outdoor environments and exposed to natural conditions for a long time, they are easily disturbed by various external factors, especially the attachment or hanging of foreign objects. These hidden dangers may not only cause faults such as short circuits and line breaks, but also pose a threat to the surrounding environment and personnel safety. Therefore, it is necessary to monitor the transmission lines day and night.

[0110] In nighttime environments, poor lighting conditions often lead to image quality degradation, including low contrast, high noise, and color cast. Traditional image processing methods usually use algorithms such as histogram equalization, gamma correction, and Retinex. Histogram equalization improves the contrast of the image by adjusting the grayscale distribution of pixels, while gamma correction enhances dark details by changing the brightness relationship of pixels. However, these methods often have problems of over-enhancement, loss of details, or inability to effectively suppress noise when enhancing low-brightness images. Therefore, an image processing method that can enhance low-brightness images at a high level is needed to achieve high-quality imaging in low-light environments.

[0111] In order to solve the above technical problems, the present invention discloses an image enhancement method, comprising: obtaining an image to be processed; decomposing the image to be processed into an illumination map to be processed and a reflection map to be processed by a decomposition network algorithm; the illumination map to be processed represents the light intensity of the image to be processed, and the reflection map to be processed contains texture, color and detail information in the image to be processed; inputting the illumination map to be processed into a joint optimization model, outputting a first optimized illumination map, and the joint optimization model is used to enhance the contrast of the image to be processed; inputting the first optimized illumination map into an attention mechanism model, outputting a second optimized illumination map, and the attention mechanism model is used to restore the image to be processed. The invention discloses a method for reproducing the details of the first optimized illumination map; inputting the reflection map to be processed into a hybrid residual network model, and outputting a first optimized reflection map, wherein the hybrid residual network model is used to improve the similarity between the reflection map to be processed and the real image; after performing dilated convolution processing on the first optimized reflection map, inputting multiple first residual modules for denoising processing, and outputting the second optimized reflection map; the dilated convolution processing is used to expand the receptive field of the first optimized reflection map; the first residual module includes multiple stride convolution layers, and jump connections are set between the multiple first residual modules; outputting an enhanced image according to the second optimized illumination map and the second optimized reflection map.

[0112] According to the implementation manner of the present disclosure, the method decomposes a low-brightness image into an illumination map and a reflection map. For the illumination map, a joint optimization model and an attention mechanism model are successively used to enhance contrast and restore details. For the reflection map, a hybrid residual network model and noise reduction processing are successively used to improve the similarity between the reflection map and the real image and suppress noise. Finally, the optimized illumination map and reflection map are combined to form an enhanced image, which can improve the enhancement effect of the low-brightness image, enhance details and reduce noise.

[0113] Figure 1 FIG. 1 is a flow chart showing an image enhancement method according to an embodiment of the present disclosure. Figure 1 As shown, the image enhancement method includes the following steps S101-S107:

[0114] In step S101, an image to be processed is obtained, wherein the image to be processed is a low-brightness image captured in a low-brightness environment.

[0115] In step S102, the image to be processed is decomposed into an illumination map to be processed and a reflection map to be processed by a decomposition network algorithm; the illumination map to be processed represents the light intensity of the image to be processed, and the reflection map to be processed contains texture, color and detail information in the image to be processed.

[0116] In step S103, the illumination map to be processed is input into a joint optimization model, and a first optimized illumination map is output. The joint optimization model is used to enhance the contrast of the image to be processed.

[0117] In step S104, the first optimized illumination map is input into an attention mechanism model, and a second optimized illumination map is output. The attention mechanism model is used to restore the details of the first optimized illumination map.

[0118] In step S105, the reflection image to be processed is input into a hybrid residual network model to output a first optimized reflection image, and the hybrid residual network model is used to improve the similarity between the reflection image to be processed and the real image.

[0119] In step S106, after the first optimized reflection map is subjected to dilated convolution processing, multiple first residual modules are input for noise reduction processing, and the second optimized reflection map is output; the dilated convolution processing is used to expand the receptive field of the first optimized reflection map; the first residual module includes multiple stride convolution layers, and jump connections are set between the multiple first residual modules.

[0120] In step S107, an enhanced image is output according to the second optimized illumination map and the second optimized reflectance map.

[0121] According to the embodiments of the present disclosure, the first optimized reflection map is subjected to dilated convolution processing, which can improve the receptive field. In addition, the first optimized reflection map is subjected to convolution processing using multiple residual modules, the residual modules include a stride convolution layer, and skip connections are set between the residual modules, which can improve the performance of processing the first optimized reflection map, reduce the loss of feature information caused by denoising, and can stack more layers to obtain better nonlinear modeling capabilities with fewer parameters.

[0122] Figure 2 FIG. 1 is a flow chart showing a joint optimization model in an image enhancement method according to an embodiment of the present disclosure. Figure 2 As shown, the step S103 includes:

[0123] In step S1031, the brightness grayscale of the pixel in the illumination map to be processed, the minimum grayscale value of the illumination map to be processed, and the maximum grayscale value of the illumination map to be processed are obtained, and the probability density function of the pixel is calculated according to the brightness grayscale;

[0124] In step S1032, a weighted probability density function of the brightness grayscale is calculated according to the probability density function of the pixel;

[0125] In step S1033, the weighted probability density function is normalized to calculate the smoothed cumulative distribution function of the brightness grayscale;

[0126] In step S1034, the smoothed cumulative distribution function is subjected to adaptive gamma correction to calculate the mapping function of the brightness grayscale;

[0127] In step S1035, the brightness values ​​of the pixels are mapped according to the mapping function, and a first optimized illumination map is output.

[0128] According to an embodiment of the present disclosure, the joint optimization model is a contrast enhancement algorithm based on histogram equalization, and its main goal is to enhance the contrast of an image without introducing too much noise, especially for images that lack visual contrast, such as pictures taken under dim or low-light conditions. The joint optimization model first calculates the histogram of the input image and assigns weights to each pixel so that the dark area of ​​the image accounts for a higher proportion in the weight distribution, and then uses the weight distribution to adaptively adjust the gamma value to perform gamma correction on the image. In this way, the brightness and contrast of the image are balanced and improved, and finally the image is remapped so that the contrast-enhanced result still maintains a natural visual effect.

[0129] According to an embodiment of the present disclosure, the probability density function PDF(l) is expressed as follows:

[0130]

[0131] Among them, n l is the number of pixels with brightness l, MN is the total number of pixels in the image;

[0132] The weighted probability density function PDF ω (l) is expressed as follows:

[0133]

[0134] Where α is a constant and 0.5≤α≤1, max(PDF) is the maximum value of the probability density function, and min(PDF) is the minimum value of the probability density function;

[0135] The smoothed cumulative distribution function CDF s (l) is expressed as follows:

[0136]

[0137] Among them, l max is the maximum pixel brightness, l min is the minimum pixel brightness;

[0138] The mapping function T(l) of the brightness grayscale is expressed as follows:

[0139]

[0140] γ=1-CDF s (l)*P;

[0141] Wherein, P is an adaptive function and 0.5≤P≤1.

[0142] Figure 3 FIG. 2 shows a structural diagram of a reconfigurable circuit in an image enhancement method according to an embodiment of the present disclosure. Figure 3 As shown, the joint optimization model is implemented by a reconfigurable circuit, and the reconfigurable circuit includes a first input terminal, a second input terminal, a third input terminal, a fourth input terminal, a first parameter unit, a second parameter unit, a first calculation unit, a second calculation unit, a third calculation unit, and an output terminal, wherein:

[0143] The first parameter unit is connected to the first input terminal and the second input terminal;

[0144] A first parameter unit is connected to the first input terminal and the second input terminal respectively through a first selector; the first parameter unit is set to log2k, where k is an input value of the first input terminal or the second input terminal;

[0145] The first calculation unit is connected to the first parameter unit through a second selector; the first calculation unit is configured as a subtraction unit;

[0146] The second calculation unit is connected to the first calculation unit, the third input terminal, and the second parameter unit respectively; the second calculation unit is set as a multiplication unit, and the second parameter unit is set as 2 m , m is the output value of the second computing unit;

[0147] The third calculation unit is connected to the second parameter unit, the fourth input terminal and the output terminal respectively; the third calculation unit is a multiplication unit.

[0148] According to the embodiments of the present disclosure, the reconfigurable circuit can be dynamically configured to calculate the weighted probability density function, the smoothed cumulative distribution function and the brightness grayscale mapping function, so that different algorithms can share the same hardware resources and flexibly allocate computing resources according to the computing load at different stages, thereby improving resource utilization efficiency and reducing computing costs and energy consumption.

[0149] Figure 4 FIG. 1 is a schematic diagram showing a structural diagram of a reconfigurable circuit configuration scheme in an image enhancement method according to an embodiment of the present disclosure. Figure 4As shown, when the reconfigurable circuit is configured as a module for calculating the weighted probability density function, the first input terminal is configured as PDF(l)-min(PDF), the second input terminal is configured as max(PDF)-min(PDF), the third input terminal is configured as α, and the fourth input terminal is configured as max(PDF). The calculation process is as follows: PDF(l)-min(PDF) and max(PDF)-min(PDF) are input into log2k respectively, then the two are subtracted, the result is multiplied by α of the third input terminal, and the result is brought into 2 m The m in is then multiplied by the max(PDF) at the fourth input to output the result.

[0150] When the reconfigurable circuit is configured to calculate the smoothed cumulative distribution function, the first input terminal is configured as The second input terminal is configured as ∑PDF ω The third input terminal, the fourth input terminal, the second calculation unit and the third calculation unit are all closed, and the calculation of the smoothed cumulative distribution function can be realized according to the reconfigurable circuit. The calculation process is as follows: and ∑PDF ω Input log2k, and we get and log2∑PDF ω , input the result of subtracting the two into 2m and then output the result.

[0151] When the reconfigurable circuit is configured as a module for calculating the mapping function of the brightness grayscale, the first input terminal is configured as 1, and the second input terminal is configured as 1. max -l min , the third input terminal is configured as γ, and the fourth input terminal is configured as l max -l min The calculation process is as follows: l and l max -l min Input log2k, and get log2l and log2(l max -l min ), subtract the two, multiply the result by the γ of the third input, and bring the result into 2 m m in the fourth input terminal and then max -l min Multiply and output the result.

[0152] Figure 5 FIG. 1 is a flow chart showing an attention mechanism model in an image enhancement method according to an embodiment of the present disclosure. Figure 5 As shown, the step S104 includes:

[0153] In step S1041, Fourier transform is performed on the first optimized illumination map to convert the first optimized illumination map from the frequency domain to the spatial domain.

[0154] In step S1042, the high-frequency signal in the spatial domain is enhanced to restore the details of the first optimized illumination map.

[0155] In step S1043, the spatial domain is subjected to an inverse Fourier transform, and is converted from the spatial domain to the frequency domain to generate a first converted illumination map. Extracting frequency information based on Fourier transform can restore more image details. The purpose of Fourier transform is to obtain the distribution of signals in the frequency domain. It can convert images from the spatial domain to the frequency domain, while the inverse Fourier transform restores the image from the frequency domain back to the spatial domain. Therefore, we can obtain the spectrum information of the image through Fourier transform. In the frequency domain, high-frequency signals represent the drastically changing parts of the image, such as details or noise; low-frequency signals represent the smooth parts that do not change frequently, such as the background. By enhancing the high-frequency signals in the image, more details can be restored, making the degraded image clearer, thereby restoring more edge information of objects, etc. Subsequently, the enhanced frequency domain information is restored to the image in the spatial domain through inverse Fourier transform.

[0156] In step S1044, the first converted illumination map is input into a dual attention module, and a second optimized illumination map is output; the dual attention module includes an activation function convolution block, a spatial attention unit, and a channel attention unit.

[0157] According to the embodiments of the present disclosure, the dual attention module includes a spatial attention unit and a channel attention unit. The spatial attention unit can focus on important local areas in low-brightness images (such as the edges of people and objects, etc.), so that key positions can obtain more brightness and contrast enhancement, and improve the clarity of important objects or scene areas. The channel attention unit can adjust the characteristics of each channel according to the importance of different channels, highlight the characteristic channels related to brightness, and ensure that the overall brightness of the image is evenly enhanced, so that the details of the low-brightness area can be displayed.

[0158] According to an embodiment of the present disclosure, the hybrid residual network model includes multiple second residual modules, the second residual modules include 5 convolutional layers, and the jump connections are set between the second residual modules.

[0159] The training steps of the hybrid residual network model are as follows:

[0160] A pair of low-brightness images and normal-brightness images are obtained from a low-brightness image training set and a corresponding normal-brightness image training set.

[0161] The low brightness image is decomposed into a low brightness illumination map and a low brightness reflection map, and the normal brightness image is decomposed into a normal brightness illumination map and a normal brightness reflection map.

[0162] A reconstruction loss function is generated according to the low-brightness illumination map, the low-brightness reflection map, the normal-brightness illumination map, the normal-brightness reflection map, the low-brightness image, and the normal-brightness image. The reconstruction loss function includes an absolute value loss function and a structural similarity loss function. The illumination map and the reflection map after decomposition of the normal-brightness image are used as references to better decompose the low-light image.

[0163] A constant reflection loss function is generated according to the low brightness reflection map and the normal brightness reflection map.

[0164] According to the low brightness illumination map and the normal brightness illumination map, an illumination smoothness loss function is generated.

[0165] Repeat the above steps, generate a loss function according to the reconstruction loss function, the constant reflection loss function, and the illumination smoothness loss function, and minimize the value of the loss function. By training the hybrid residual network model, the difference between the reflectance map of the low-brightness image decomposition and the reflectance map of the normal-brightness image can be improved, thereby improving the brightness and contrast of the reflectance map of the image to be processed, reducing noise and enhancing details.

[0166] According to an embodiment of the present disclosure, the reconstruction loss function It is expressed as follows:

[0167]

[0168] Among them, low is a low brightness image, normal is a normal brightness image, R i is the reflection map, I i is the illumination diagram, S i is the image before decomposition, α is the first weight coefficient, L1 is the absolute value loss function, SSIM is the structural similarity loss function, Multiply the reflectance map and the illumination map;

[0169] The constant reflection loss function L r It is expressed as follows:

[0170] L r =|R low -R normal |

[0171] Among them, R low For low brightness reflection image R normal It is a normal brightness reflection map;

[0172] The lighting smoothness loss function It is expressed as follows:

[0173]

[0174] in, is the illumination image gradient, exp() is the exponential function, λ tv is a constant, and 0 is matrix multiplication.

[0175] According to an embodiment of the present disclosure, before inputting the first optimized illumination map into the attention mechanism model and outputting the second optimized illumination map, the method further includes:

[0176] Performing a dilated convolution process on the first optimized illumination map, wherein the dilated convolution process is used to expand the receptive field of the first optimized illumination map;

[0177] The first optimized illumination map is input into a plurality of second residual modules for denoising; and a jump connection is set between the plurality of second residual modules. Before the first optimized illumination map is input into the attention mechanism model, denoising the first optimized illumination map can reduce the noise points in the first optimized illumination map, which is beneficial to subsequent image processing.

[0178] Figure 6 The structural block diagram of the image enhancement device according to the embodiment of the present disclosure is shown. The device can be implemented as part or all of the electronic device through software, hardware or a combination of both.

[0179] like Figure 6 As shown, the image enhancement device 600 includes an acquisition module 610 , a decomposition module 620 , a first optimization module 630 , a second optimization module 640 , a third optimization module 650 , a fourth optimization module 660 , and an output module 670 .

[0180] The acquisition module is configured to acquire the image to be processed.

[0181] The decomposition module is configured to decompose the image to be processed into an illumination map to be processed and a reflection map to be processed through a decomposition network algorithm; the illumination map to be processed represents the light intensity of the image to be processed, and the reflection map to be processed contains texture, color and detail information in the image to be processed.

[0182] The first optimization module is configured to input the illumination map to be processed into a joint optimization model and output a first optimized illumination map, wherein the joint optimization model is used to enhance the contrast of the image to be processed.

[0183] The second optimization module is configured to input the first optimized illumination map into an attention mechanism model and output a second optimized illumination map, wherein the attention mechanism model is used to restore details of the first optimized illumination map.

[0184] The third optimization module is configured to input the reflection image to be processed into a hybrid residual network model and output a first optimized reflection image, wherein the hybrid residual network model is used to improve the similarity between the reflection image to be processed and the real image.

[0185] The fourth optimization module is configured to perform dilated convolution processing on the first optimized reflection map, input multiple first residual modules for noise reduction processing, and output the second optimized reflection map; the dilated convolution processing is used to expand the receptive field of the first optimized reflection map; the first residual module includes multiple stride convolution layers, and jump connections are set between the multiple first residual modules.

[0186] An output module is configured to output an enhanced image according to the second optimized illumination map and the second optimized reflectance map.

[0187] According to an embodiment of the present disclosure, the fourth optimization module performs dilated convolution processing on the first optimized reflection map, which can improve the receptive field. In addition, multiple residual modules are used to perform convolution processing on the first optimized reflection map, the residual module includes a step convolution layer, and jump connections are set between the residual modules, which can improve the performance of processing the first optimized reflection map, reduce the loss of feature information caused by denoising, and can stack more layers to obtain better nonlinear modeling capabilities with fewer parameters.

[0188] Figure 7 FIG. 2 shows a structural block diagram of a joint optimization model in an image enhancement device according to an embodiment of the present disclosure. Figure 7 As shown, the first optimization module 630 includes:

[0189] The first acquisition submodule 631 is configured to acquire the brightness grayscale of the pixel in the illumination map to be processed, the minimum grayscale value of the illumination map to be processed, and the maximum grayscale value of the illumination map to be processed, and calculate the probability density function of the pixel according to the brightness grayscale;

[0190] A first calculation submodule 632 is configured to calculate a weighted probability density function of the brightness grayscale according to the probability density function of the pixel;

[0191] The second calculation submodule 633 is configured to perform normalization processing on the weighted probability density function to calculate the smoothed cumulative distribution function of the brightness grayscale;

[0192] A third calculation submodule 634 is configured to perform adaptive gamma correction on the smoothed cumulative distribution function to calculate the brightness grayscale mapping function;

[0193] The first output submodule 635 is configured to map the brightness values ​​of the pixels according to the mapping function and output a first optimized illumination map.

[0194] According to an embodiment of the present disclosure, the joint optimization model is a contrast enhancement algorithm based on histogram equalization, and its main goal is to enhance the contrast of an image without introducing too much noise, especially for images that lack visual contrast, such as pictures taken under dim or low-light conditions. The joint optimization model first calculates the histogram of the input image and assigns weights to each pixel so that the dark area of ​​the image accounts for a higher proportion in the weight distribution, and then uses the weight distribution to adaptively adjust the gamma value to perform gamma correction on the image. In this way, the brightness and contrast of the image are balanced and improved, and finally the image is remapped so that the contrast-enhanced result still maintains a natural visual effect.

[0195] According to an embodiment of the present disclosure, the probability density function PDF(l) is expressed as follows:

[0196]

[0197] Among them, n l is the number of pixels with brightness l, MN is the total number of pixels in the image;

[0198] The weighted probability density function PDF ω (l) is expressed as follows:

[0199]

[0200] Where α is a constant and 0.5≤α≤1, max(PDF) is the maximum value of the probability density function, and min(PDF) is the minimum value of the probability density function;

[0201] The smoothed cumulative distribution function CDF s (l) is expressed as follows:

[0202]

[0203] Among them, l max is the maximum pixel brightness, l min is the minimum pixel brightness;

[0204] The mapping function T(l) of the brightness grayscale is expressed as follows:

[0205]

[0206] γ=1-CDF s (l)*P;

[0207] Wherein, P is an adaptive function and 0.5≤P≤1.

[0208] Figure 8FIG. 1 shows a structural block diagram of an attention mechanism model in an image enhancement device according to an embodiment of the present disclosure. Figure 8 As shown, the second optimization module 640 includes:

[0209] A first conversion submodule 641 is configured to perform Fourier transform on the first optimized illumination map to convert the first optimized illumination map from the frequency domain to the spatial domain;

[0210] an enhancement submodule 642, configured to enhance the high-frequency signal in the spatial domain to restore details of the first optimized illumination map;

[0211] A second conversion submodule 643 is configured to perform an inverse Fourier transform on the spatial domain, convert from the spatial domain to the frequency domain, and generate a first converted illumination map;

[0212] The second output submodule 644 is configured to input the first converted illumination map into a dual attention module and output a second optimized illumination map; the dual attention module includes an activation function convolution block, a spatial attention unit and a channel attention unit.

[0213] According to an embodiment of the present disclosure, before the second optimization module, the device further includes:

[0214] a second convolution submodule, configured to perform a dilated convolution process on the first optimized illumination map, wherein the dilated convolution process is used to expand the receptive field of the first optimized illumination map;

[0215] The second denoising submodule is configured to input the first optimized illumination map into a plurality of second residual modules for denoising; and a jump connection is set between the plurality of second residual modules.

[0216] The present disclosure also discloses a chip, including a processor and an interface; the processor is used to read instructions to execute the method according to the embodiment of the present disclosure.

[0217] The present disclosure also discloses an electronic device, Fig. 9 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0218] like Fig. 9 As shown, the electronic device includes a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to an embodiment of the present disclosure.

[0219] An image enhancement method, characterized by comprising:

[0220] Get the image to be processed;

[0221] Decomposing the image to be processed into an illumination map to be processed and a reflection map to be processed by a decomposition network algorithm; the illumination map to be processed represents the illumination intensity of the image to be processed, and the reflection map to be processed contains texture, color and detail information in the image to be processed;

[0222] Inputting the illumination map to be processed into a joint optimization model and outputting a first optimized illumination map, wherein the joint optimization model is used to enhance the contrast of the image to be processed;

[0223] Inputting the first optimized illumination map into an attention mechanism model and outputting a second optimized illumination map, wherein the attention mechanism model is used to restore details of the first optimized illumination map;

[0224] Inputting the reflection image to be processed into a hybrid residual network model and outputting a first optimized reflection image, wherein the hybrid residual network model is used to improve the similarity between the reflection image to be processed and a real image;

[0225] After the first optimized reflection map is subjected to dilated convolution processing, the first optimized reflection map is input into a plurality of first residual modules for noise reduction processing, and the second optimized reflection map is output; the dilated convolution processing is used to expand the receptive field of the first optimized reflection map; the first residual module includes a plurality of stride convolution layers, and jump connections are set between the plurality of first residual modules;

[0226] An enhanced image is output according to the second optimized illumination map and the second optimized reflectance map.

[0227] The step of inputting the illumination map to be processed into a joint optimization model and outputting a first optimized illumination map comprises:

[0228] Obtaining the brightness grayscale of the pixel in the illumination map to be processed, the minimum grayscale value of the illumination map to be processed, and the maximum grayscale value of the illumination map to be processed, and calculating the probability density function of the pixel according to the brightness grayscale;

[0229] Calculating the weighted probability density function of the brightness grayscale according to the probability density function of the pixel;

[0230] Normalizing the weighted probability density function to calculate the smoothed cumulative distribution function of the brightness grayscale;

[0231] Performing adaptive gamma correction on the smoothed cumulative distribution function to calculate the mapping function of the brightness grayscale;

[0232] According to the mapping function, the brightness values ​​of the pixels are mapped and a first optimized illumination map is output.

[0233] The probability density function PDF(l) is expressed as follows:

[0234]

[0235] Among them, n l is the number of pixels with brightness l, MN is the total number of pixels in the image;

[0236] The weighted probability density function PDF ω (l) is expressed as follows:

[0237]

[0238] Where α is a constant and 0.5≤α≤1, max(PDF) is the maximum value of the probability density function, and min(PDF) is the minimum value of the probability density function;

[0239] The smoothed cumulative distribution function CDF s (l) is expressed as follows:

[0240]

[0241] Among them, l max is the maximum pixel brightness, l min is the minimum pixel brightness;

[0242] The mapping function T(l) of the brightness grayscale is expressed as follows:

[0243]

[0244] γ=1-CDF s (l)*P;

[0245] Wherein, P is an adaptive function and 0.5≤P≤1.

[0246] The joint optimization model is implemented by a reconfigurable circuit, which includes a first input terminal, a second input terminal, a third input terminal, a fourth input terminal, a first parameter unit, a second parameter unit, a first calculation unit, a second calculation unit, a third calculation unit, and an output terminal, wherein:

[0247] The first parameter unit is connected to the first input terminal and the second input terminal;

[0248] A first parameter unit is connected to the first input terminal and the second input terminal respectively through a first selector; the first parameter unit is set to log2k, where k is an input value of the first input terminal or the second input terminal;

[0249] The first calculation unit is connected to the first parameter unit through a second selector; the first calculation unit is configured as a subtraction unit;

[0250] The second calculation unit is connected to the first calculation unit, the third input terminal, and the second parameter unit respectively; the second calculation unit is set as a multiplication unit, and the second parameter unit is set as 2 m , m is the output value of the second computing unit;

[0251] The third calculation unit is connected to the second parameter unit, the fourth input terminal and the output terminal respectively; the third calculation unit is a multiplication unit.

[0252] When the reconfigurable circuit is configured as a module for calculating the weighted probability density function, the first input terminal is configured as PDF(l)-min(PDF), the second input terminal is configured as max(PDF)-min(PDF), the third input terminal is configured as α, and the fourth input terminal is configured as max(PDF);

[0253] When the reconfigurable circuit is configured to calculate the smoothed cumulative distribution function, the first input terminal is configured as The second input terminal is configured as ∑PDF ω ;

[0254] When the reconfigurable circuit is configured as a module for calculating the mapping function of the brightness grayscale, the first input terminal is configured as 1, and the second input terminal is configured as 1. max -l min , the third input terminal is configured as γ, and the fourth input terminal is configured as l max -l min .

[0255] The step of inputting the first optimized illumination map into the attention mechanism model and outputting the second optimized illumination map comprises:

[0256] Performing Fourier transformation on the first optimized illumination map to convert the first optimized illumination map from the frequency domain to the spatial domain;

[0257] enhancing the high-frequency signal in the spatial domain to restore details of the first optimized illumination map;

[0258] Performing an inverse Fourier transform on the spatial domain to convert from the spatial domain to the frequency domain to generate a first converted illumination map;

[0259] The first converted illumination map is input into a dual attention module, and a second optimized illumination map is output; the dual attention module includes an activation function convolution block, a spatial attention unit and a channel attention unit.

[0260] The hybrid residual network model includes a plurality of second residual modules, the second residual modules include 5 convolutional layers, and the jump connections are set between the second residual modules;

[0261] The training steps of the hybrid residual network model are as follows:

[0262] Obtain a pair of low brightness image low and normal brightness image normal from the low brightness image training set and the corresponding normal brightness image training set;

[0263] Decomposing the low brightness image into a low brightness illumination image and a low brightness reflection image, and decomposing the normal brightness image into a normal brightness illumination image and a normal brightness reflection image;

[0264] Generate a reconstruction loss function according to the low-brightness illumination map, the low-brightness reflection map, the normal-brightness illumination map, the normal-brightness reflection map, the low-brightness image, and the normal-brightness image;

[0265] Generate a constant reflection loss function according to the low brightness reflection map and the normal brightness reflection map;

[0266] Generate an illumination smoothness loss function according to the low brightness illumination map and the normal brightness illumination map;

[0267] Repeat the above steps to generate a loss function according to the reconstruction loss function, the constant reflection loss function, and the illumination smoothness loss function, and minimize the value of the loss function.

[0268] The reconstruction loss function It is expressed as follows:

[0269]

[0270] Among them, low is a low brightness image, normal is a normal brightness image, R i is the reflection map, I i is the illumination diagram, S i is the image before decomposition, α is the first weight coefficient, L1 is the absolute value loss function, SSIM is the structural similarity loss function, Multiply the reflectance map and the illumination map;

[0271] The constant reflection loss function L r It is expressed as follows:

[0272] L r =|R low -R normal |

[0273] Among them, R low For low brightness reflection image R normal It is a normal brightness reflection map;

[0274] The lighting smoothness loss function It is expressed as follows:

[0275]

[0276] in, is the illumination image gradient, exp() is the exponential function, λ tv is a constant, and 0 is matrix multiplication.

[0277] Before inputting the first optimized illumination map into the attention mechanism model and outputting the second optimized illumination map, the method further includes:

[0278] Performing a dilated convolution process on the first optimized illumination map, wherein the dilated convolution process is used to expand the receptive field of the first optimized illumination map;

[0279] The first optimized illumination map is input into a plurality of second residual modules for noise reduction processing; and jump connections are set between the plurality of second residual modules. Fig.10 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown.

[0280] like Fig.10 As shown, the computer system includes a processing unit, which can perform the various methods in the above-mentioned embodiments according to the program stored in the read-only memory (ROM) or the program loaded from the storage part into the random access memory (RAM). In the RAM, various programs and data required for the operation of the computer system are also stored. The processing unit, ROM and RAM are connected to each other through a bus. The input / output (I / O) interface is also connected to the bus.

[0281] The following components are connected to the I / O interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a LAN card, a modem, etc. The communication part performs a communication process via a network such as the Internet. The drive is also connected to the I / O interface as needed. Removable media, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed so that the computer program read therefrom is installed into the storage part as needed. Among them, the processing unit can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0282] In particular, according to an embodiment of the present disclosure, the method described above can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a machine-readable medium, and the computer program includes a program code for executing the above method. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium.

[0283] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0284] The units or modules involved in the embodiments described in the present disclosure may be implemented by software or programmable hardware. The units or modules described may also be set in a processor, and the names of these units or modules do not constitute limitations on the units or modules themselves in some cases.

[0285] As another aspect, the present disclosure further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system in the above embodiment; or a computer-readable storage medium that exists independently and is not assembled into a device. The computer-readable storage medium stores one or more programs, and the programs are used by one or more processors to execute the method described in the present disclosure.

[0286] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

Claims

1. An image enhancement method, characterized in that: include: Get the image to be processed; Decomposing the image to be processed into an illumination map to be processed and a reflection map to be processed by a decomposition network algorithm; The illumination map to be processed represents the illumination intensity of the image to be processed, and the reflection map to be processed contains texture, color and detail information in the image to be processed; Inputting the illumination map to be processed into a joint optimization model and outputting a first optimized illumination map, wherein the joint optimization model is used to enhance the contrast of the image to be processed; Inputting the first optimized illumination map into an attention mechanism model and outputting a second optimized illumination map, wherein the attention mechanism model is used to restore details of the first optimized illumination map; Inputting the reflection image to be processed into a hybrid residual network model and outputting a first optimized reflection image, wherein the hybrid residual network model is used to improve the similarity between the reflection image to be processed and a real image; After performing dilation convolution processing on the first optimized reflection image, the first optimized reflection image is input into a plurality of first residual modules for noise reduction processing, and the second optimized reflection image is output; The dilated convolution process is used to expand the receptive field of the first optimized reflection map; the first residual module includes a plurality of stride convolution layers, and jump connections are set between the plurality of first residual modules; An enhanced image is output according to the second optimized illumination map and the second optimized reflectance map.

2. The method according to claim 1, characterized in that: The step of inputting the illumination map to be processed into a joint optimization model and outputting a first optimized illumination map comprises: Obtaining the brightness grayscale of the pixel in the illumination map to be processed, the minimum grayscale value of the illumination map to be processed, and the maximum grayscale value of the illumination map to be processed, and calculating the probability density function of the pixel according to the brightness grayscale; Calculating a weighted probability density function of the brightness grayscale according to the probability density function of the pixel; Normalizing the weighted probability density function to calculate the smoothed cumulative distribution function of the brightness grayscale; Performing adaptive gamma correction on the smoothed cumulative distribution function to calculate the mapping function of the brightness grayscale; According to the mapping function, the brightness values ​​of the pixels are mapped and a first optimized illumination map is output.

3. The method according to claim 2, characterized in that: The probability density function PDF(l) is expressed as follows: Among them, n l is the number of pixels with brightness l, MN is the total number of pixels in the image; The weighted probability density function PDF ω (l) is expressed as follows: Where α is a constant and 0.5≤α≤1, max(PDF) is the maximum value of the probability density function, and min(PDF) is the minimum value of the probability density function; The smoothed cumulative distribution function CDF s (l) is expressed as follows: Among them, l max is the maximum pixel brightness, l min is the minimum pixel brightness; The mapping function T(l) of the brightness grayscale is expressed as follows: γ=1-CDF s (l)*P; Wherein, P is an adaptive function and 0.5≤P≤1.

4. The method according to claim 3, characterized in that: The joint optimization model is implemented by a reconfigurable circuit, which includes a first input terminal, a second input terminal, a third input terminal, a fourth input terminal, a first parameter unit, a second parameter unit, a first calculation unit, a second calculation unit, a third calculation unit, and an output terminal, wherein: The first parameter unit is connected to the first input terminal and the second input terminal; A first parameter unit is connected to the first input terminal and the second input terminal respectively through a first selector; the first parameter unit is set to log2k, where k is an input value of the first input terminal or the second input terminal; The first calculation unit is connected to the first parameter unit through a second selector; the first calculation unit is configured as a subtraction unit; The second calculation unit is connected to the first calculation unit, the third input terminal, and the second parameter unit respectively; the second calculation unit is set as a multiplication unit, and the second parameter unit is set as 2 m , m is the output value of the second computing unit; The third calculation unit is connected to the second parameter unit, the fourth input terminal and the output terminal respectively; the third calculation unit is a multiplication unit.

5. The method according to claim 4, characterized in that: When the reconfigurable circuit is configured as a module for calculating the weighted probability density function, the first input terminal is configured as PDF(l)-min(PDF), the second input terminal is configured as max(PDF)-min(PDF), the third input terminal is configured as α, and the fourth input terminal is configured as max(PDF); When the reconfigurable circuit is configured to calculate the smoothed cumulative distribution function, the first input terminal is configured as The second input terminal is configured as ∑PDF ω ; When the reconfigurable circuit is configured as a module for calculating the mapping function of the brightness grayscale, the first input terminal is configured as 1, and the second input terminal is configured as 1. max -l min , the third input terminal is configured as γ, and the fourth input terminal is configured as l max -l min .

6. The method according to claim 1, characterized in that The step of inputting the first optimized illumination map into the attention mechanism model and outputting the second optimized illumination map comprises: Performing Fourier transformation on the first optimized illumination map to convert the first optimized illumination map from the frequency domain to the spatial domain; enhancing the high-frequency signal in the spatial domain to restore details of the first optimized illumination map; Performing an inverse Fourier transform on the spatial domain to convert from the spatial domain to the frequency domain to generate a first converted illumination map; The first converted illumination map is input into a dual attention module, and a second optimized illumination map is output; the dual attention module includes an activation function convolution block, a spatial attention unit and a channel attention unit.

7. The method according to claim 1, characterized in that The hybrid residual network model includes a plurality of second residual modules, the second residual modules include 5 convolutional layers, and the jump connections are set between the second residual modules; The training steps of the hybrid residual network model are as follows: Obtain a pair of low-brightness images and normal-brightness images from a low-brightness image training set and a corresponding normal-brightness image training set; Decomposing the low brightness image into a low brightness illumination image and a low brightness reflection image, and decomposing the normal brightness image into a normal brightness illumination image and a normal brightness reflection image; Generate a reconstruction loss function according to the low-brightness illumination map, the low-brightness reflection map, the normal-brightness illumination map, the normal-brightness reflection map, the low-brightness image, and the normal-brightness image; Generate a constant reflection loss function according to the low brightness reflection map and the normal brightness reflection map; Generate an illumination smoothness loss function according to the low brightness illumination map and the normal brightness illumination map; Repeat the above steps to generate a loss function according to the reconstruction loss function, the constant reflection loss function, and the illumination smoothness loss function, and minimize the value of the loss function.

8. The method according to claim 7, characterized in that The reconstruction loss function It is expressed as follows: Among them, low is a low brightness image, normal is a normal brightness image, R i is the reflection map, I i is the illumination diagram, S i is the image before decomposition, α is the first weight coefficient, L1 is the absolute value loss function, SSIM is the structural similarity loss function, Multiply the reflectance map and the illumination map; The constant reflection loss function L r It is expressed as follows: L r =|R low -R normal | Among them, R low For low brightness reflection image R normal It is a normal brightness reflection map; The lighting smoothness loss function It is expressed as follows: in, is the illumination image gradient, exp() is the exponential function, λ tv is a constant, and 0 is matrix multiplication.

9. The method according to claim 6, characterized in that Before inputting the first optimized illumination map into the attention mechanism model and outputting the second optimized illumination map, the method further includes: Performing a dilated convolution process on the first optimized illumination map, wherein the dilated convolution process is used to expand the receptive field of the first optimized illumination map; The first optimized illumination map is input into a plurality of second residual modules for noise reduction processing; and jump connections are set between the plurality of second residual modules.

10. An image enhancement device, characterized in that: The device comprises: An acquisition module, configured to acquire an image to be processed; A decomposition module is configured to decompose the image to be processed into an illumination map to be processed and a reflection map to be processed by a decomposition network algorithm; the illumination map to be processed represents the light intensity of the image to be processed, and the reflection map to be processed contains texture, color and detail information in the image to be processed; A first optimization module is configured to input the illumination map to be processed into a joint optimization model and output a first optimized illumination map, wherein the joint optimization model is used to enhance the contrast of the image to be processed; A second optimization module is configured to input the first optimized illumination map into an attention mechanism model and output a second optimized illumination map, wherein the attention mechanism model is used to restore details of the first optimized illumination map; A third optimization module is configured to input the reflection image to be processed into a hybrid residual network model and output a first optimized reflection image, wherein the hybrid residual network model is used to improve the similarity between the reflection image to be processed and the real image; A fourth optimization module is configured to perform dilated convolution processing on the first optimized reflection map, input multiple first residual modules for noise reduction processing, and output the second optimized reflection map; the dilated convolution processing is used to expand the receptive field of the first optimized reflection map; the first residual module includes multiple stride convolution layers, and jump connections are set between the multiple first residual modules; An output module is configured to output an enhanced image according to the second optimized illumination map and the second optimized reflectance map.

11. The device according to claim 10, characterized in that The first optimization module comprises: A first acquisition submodule is configured to acquire the brightness grayscale of the pixel in the illumination map to be processed, the minimum grayscale value of the illumination map to be processed, and the maximum grayscale value of the illumination map to be processed, and calculate the probability density function of the pixel according to the brightness grayscale; A first calculation submodule is configured to calculate a weighted probability density function of the brightness grayscale according to the probability density function of the pixel; A second calculation submodule is configured to perform normalization processing on the weighted probability density function to calculate the smoothed cumulative distribution function of the brightness grayscale; A third calculation submodule is configured to perform adaptive gamma correction on the smoothed cumulative distribution function to calculate the brightness grayscale mapping function; The first output submodule is configured to map the brightness values ​​of the pixels according to the mapping function and output a first optimized illumination map.

12. The device according to claim 10, characterized in that The second optimization module comprises: A first conversion submodule is configured to perform Fourier transform on the first optimized illumination map to convert the first optimized illumination map from the frequency domain to the spatial domain; an enhancement submodule, configured to enhance the high-frequency signal in the spatial domain to restore details of the first optimized illumination map; A second conversion submodule is configured to perform an inverse Fourier transform on the spatial domain, convert from the spatial domain to the frequency domain, and generate a first converted illumination map; The second output submodule is configured to input the first converted illumination map into a dual attention module and output a second optimized illumination map; the dual attention module includes an activation function convolution block, a spatial attention unit and a channel attention unit.

13. The device according to claim 12, characterized in that Before the second optimization module, the device further includes: a second convolution submodule, configured to perform a dilated convolution process on the first optimized illumination map, wherein the dilated convolution process is used to expand the receptive field of the first optimized illumination map; The second denoising submodule is configured to input the first optimized illumination map into a plurality of second residual modules for denoising; and a jump connection is set between the plurality of second residual modules.

14. A chip, characterized in that: The method comprises a processor and an interface; the processor is used to read instructions to execute the method according to any one of claims 1 to 9.

15. An electronic device, characterized in that: The method comprises a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to any one of claims 1 to 9.

16. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images