Image enhancement method and device, equipment and medium

Abstract

The invention relates to the technical field of image enhancement, in particular to an image enhancement method and device, equipment and a medium. Decomposing the reflection component and the illumination component of the to-be-enhanced image to obtain a first image representing the characteristics of the reflection component and a second image representing the characteristics of the illumination component, and performing denoising and enhancement processing on the first image and the second image respectively, and obtaining a denoised first image, a denoised second image, an enhanced first image and an enhanced second image, and fusing the image representing the reflection component characteristics and the image representing the illumination component characteristics to obtain an enhanced image of the to-be-enhanced image. In the application, the decomposed reflection component image and the illumination component image are respectively subjected to denoising and enhancement processing, the denoising processing can suppress noise in the image, the enhancement processing can improve the contrast ratio of the image, and the denoised image and the enhanced image are subjected to channel fusion, so that the effect of the enhanced image can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image enhancement technology, and in particular to an image enhancement method, device, equipment and medium. Background Art

[0002] In low-light conditions, the brightness and contrast of images are usually limited, resulting in loss of image details and reduced visibility. For example, almost invisible details and relatively low contrast, as well as dull colors. Existing low-light image enhancement methods are generally based on deep learning methods. This type of method mainly uses the powerful modeling ability of neural networks to learn the mapping from low-light images to normal-light images. This type of method has problems such as over-enhancement of image details, sensitivity to noise, or insufficient global processing of images, resulting in poor enhanced image effects. Therefore, in the process of low-light image enhancement, how to improve the effect of image enhancement has become an urgent problem to be solved. Summary of the invention

[0003] Based on this, it is necessary to provide an image enhancement method, device, equipment and medium to address the above technical problems, so as to solve the problem of poor image enhancement effect during low-light image enhancement.

[0004] A first aspect of an embodiment of the present application provides an image enhancement method, the image enhancement method comprising: Acquire an image to be enhanced for pedestrian detection taken at night, perform a decomposition process on the image to be enhanced based on a reflection component to obtain a first image representing characteristics of the reflection component, and perform a decomposition process on the image to be enhanced based on an illumination component to obtain a second image representing characteristics of the illumination component; Performing denoising processing on the first image and the second image respectively to obtain a denoised first image and a denoised second image, and performing brightness enhancement processing on the first image and the second image respectively to obtain an enhanced first image and an enhanced second image; Adding the denoised first image and the reflection component in the corresponding channel of the enhanced first image to obtain an added first image, and adding the denoised second image and the illumination component in the corresponding channel of the enhanced second image to obtain an added second image; The reflection component and the illumination component of the added first image and the added second image are fused to obtain a fused image, and the fused image is determined as the enhanced image of the image to be enhanced.

[0005] A second aspect of an embodiment of the present application provides an image enhancement device, the image enhancement device comprising: an acquisition module, used to acquire an image to be enhanced for pedestrian detection taken at night, decompose the image to be enhanced by a reflection component to obtain a first image representing the characteristics of the reflection component, and decompose the image to be enhanced by an illumination component to obtain a second image representing the characteristics of the illumination component; a denoising and enhancement module, configured to perform denoising processing on the first image and the second image respectively to obtain a denoised first image and a denoised second image, and perform brightness enhancement processing on the first image and the second image respectively to obtain an enhanced first image and an enhanced second image; an adding module, configured to add the denoised first image and the reflection component in the corresponding channel of the enhanced first image to obtain the added first image, and to add the denoised second image and the illumination component in the corresponding channel of the enhanced second image to obtain the added second image; A fusion module is used to fuse the reflection component and the illumination component of the added first image and the added second image to obtain a fused image, and determine the fused image as the enhanced image of the image to be enhanced.

[0006] In a third aspect, an embodiment of the present invention provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image enhancement method as described in the first aspect when executing the computer program.

[0007] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image enhancement method as described in the first aspect is implemented.

[0008] Compared with the prior art, the present invention has the following beneficial effects: Acquire an image to be enhanced for pedestrian detection taken at night, decompose the image to be enhanced by a reflection component to obtain a first image characterizing the characteristics of the reflection component, decompose the image to be enhanced by an illumination component to obtain a second image characterizing the characteristics of the illumination component, denoise the first image and the second image respectively to obtain a denoised first image and a denoised second image, perform brightness enhancement on the first image and the second image respectively to obtain an enhanced first image and an enhanced second image, add the reflection components in corresponding channels of the denoised first image and the enhanced first image to obtain an added first image, add the illumination components in corresponding channels of the denoised second image and the enhanced second image to obtain an added second image, fuse the reflection components and illumination components of the added first image and the added second image to obtain a fused image, and determine the fused image as the enhanced image of the image to be enhanced. In the present application, the decomposed reflection component map and the illumination component map are denoised and enhanced respectively to obtain a denoised smooth reflection image and a smooth illumination image, as well as an enhanced reflection image and an enhanced illumination image. The denoising process can suppress the noise in the image, and the enhancement process can improve the contrast of the image. The denoised image and the enhanced image are channel-fused, thereby improving the enhanced image effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0010] Figure 1 is a schematic diagram of an application environment of an image enhancement method provided by an embodiment of the present invention; Figure 2 is a flowchart of an image enhancement method provided by an embodiment of the present invention; Figure 3 is a structural schematic diagram of an image enhancement device provided by an embodiment of the present invention; Figure 4 It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0011] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0013] It should also be understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0014] As used in the present specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]", depending on the context.

[0015] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0016] References to "one embodiment" or "some embodiments" etc. described in the specification of the present invention mean that one or more embodiments of the present invention include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0017] The embodiments of the present invention can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0018] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0019] It should be understood that the order of execution of the steps in the following embodiments does not imply a precedence of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0020] In order to illustrate the technical solution of the present invention, specific embodiments are provided below for illustration.

[0021] An image enhancement method provided by an embodiment of the present invention can be applied in Figure 1 In an application environment, a client communicates with a server. The client includes, but is not limited to, a PDA, a desktop computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and other computer devices. The server can be implemented with an independent server or a server cluster consisting of multiple servers.

[0022] See also Figure 2 , is a flow chart of an image enhancement method provided by an embodiment of the present invention. The above-mentioned image enhancement processing method can be applied to Figure 1 The server in the above connects to the corresponding client, such as Figure 2 As shown, the image enhancement method may include the following steps.

[0023] S201: Acquire an image to be enhanced for pedestrian detection taken at night, decompose the reflection component of the image to be enhanced to obtain a first image representing the characteristics of the reflection component, and decompose the illumination component of the image to be enhanced to obtain a second image representing the characteristics of the illumination component.

[0024] In step S201, an image to be enhanced for pedestrian detection taken at night is obtained, wherein the image to be enhanced is an image with low illumination intensity, and the image to be enhanced is decomposed into a reflection component and an illumination component to obtain a first image representing the characteristics of the reflection component and a second image representing the characteristics of the illumination component. The illumination component image represents the incident light image, which directly determines the dynamic range that can be achieved by the pixels in the image, and the reflection component image is an image of the reflection properties of the object, which represents the intrinsic properties of the image.

[0025] In this embodiment, when detecting pedestrians at night, it is necessary to collect nighttime pedestrian images. Due to insufficient light at night, the details and visibility of the collected pedestrian images are relatively weak. Before detecting pedestrians, in order to improve the detection accuracy, the collected pedestrian images need to be enhanced.

[0026] In this embodiment, the image is enhanced based on the Retinex theory, and the image to be enhanced is decomposed into a reflection component and an illumination component to obtain a first image representing the characteristics of the reflection component and a second image representing the characteristics of the illumination component. In this way, the first image representing the characteristics of the reflection component and the second image representing the characteristics of the illumination component are enhanced respectively, thereby improving the enhancement effect in the image.

[0027] It should be noted that before decomposing the image to be enhanced, the image to be enhanced can be preprocessed. The preprocessing can be to denoise the image and adjust the size of the image. When collecting the image to be enhanced, due to the influence of factors such as illumination, the images collected may often have different image resolutions, brightness and contrast. These different image parameters often interfere with standardized recognition systems. The scale of the image series can be roughly normalized by estimating the field of view size, and then the background brightness in the image to be enhanced can be estimated using a Gaussian filter, and then the brightness balance and contrast enhancement of the overall image can be completed by background subtraction. In order to make all images have consistent length and width, the extracted images need to be filled around. By using the image aspect ratio for normalization, this method can better retain the geometric structure information in the original image and prevent it from distorting the information in the distorted image.

[0028] Optionally, performing a decomposition process on a reflection component of the image to be enhanced to obtain a first image representing characteristics of the reflection component, and performing a decomposition process on an illumination component of the image to be enhanced to obtain a second image representing characteristics of the illumination component, comprises: Extracting the reflection component in the image to be enhanced using a preset reflection component extraction network to obtain a first image representing the characteristics of the reflection component; According to the reflection component of each pixel in the first image and the pixel value of each pixel in the image to be enhanced, the illumination component of each pixel is calculated to obtain a second image representing the characteristics of the illumination component.

[0029] In this embodiment, when decomposing the image to be enhanced, a preset reflection component extraction network is first used to extract the reflection component in the image to be enhanced to obtain a first image characterizing the characteristics of the reflection component. Then, based on the relationship between the reflection component and the illumination component and the image to be enhanced, a second image corresponding to the characteristics of the illumination component can be obtained.

[0030] The preset reflection component extraction network in this embodiment can be composed of 8 convolutional layers (Conv), 6 activation layers (ReLU), 2 pooling layers (Pooling), 2 deconvolution layers (Deconv), and 2 activation layers. Since the network structure designed by the present invention includes several layers of the same type, the function of each type of layer remains unchanged, but the parameters are different. In order to avoid redundancy, the function and operation of each layer are described below.

[0031] The function of the convolution layer is to extract image features, and through the convolution operation, the input image to be enhanced is converted into a feature map of the output channel number. The parameters generally included include: convolution kernel size, input channel number, output channel number, step size, filling method, etc. The filling methods used in all operations in the present invention are all SAME. The function of the activation layer is to convert the original linear mapping into a nonlinear mapping, and usually a nonlinear function calculation is performed on the input feature map. The activation function used in the present invention is all ReLU, and its function is to make it equal to 0 if the calculated value is less than 0, otherwise keep the original value unchanged. The function of the pooling layer is to downsample the image, which can extract multi-scale features on the one hand and reduce the amount of calculation on the other hand. The pooling methods used in this invention are all maximum pooling, that is, the maximum value of the feature points in the neighborhood is obtained as the output of the neighborhood. The parameters generally included include: convolution kernel size, step size, filling method, etc. The function of deconvolution is to upsample the image, that is, to enlarge the size of the feature map, which is conducive to the subsequent merging operation. The parameters generally included include: convolution kernel size, step size, filling method, etc. The activation layer refers to the calculation of an activation function on the feature maps of several input channels. Its function is to convert the input value to between 0 and 1 so as to conform to the value range of the reflection map.

[0032] The reflection component in the image to be enhanced is extracted according to the preset reflection component extraction network to obtain a first image representing the characteristics of the reflection component. After the first image is obtained, each pixel value in the first image is the reflection component value of the corresponding pixel. Since the reflection component and the illumination component in the image to be enhanced satisfy the following relationship, as shown in the following formula: in, is the pixel in the image to be enhanced The pixel value at is the pixel in the image to be enhanced The amount of light at is the pixel in the image to be enhanced The reflected component at .

[0033] Therefore, according to the reflection component of each pixel in the first image and the pixel value of each pixel in the image to be enhanced, the illumination component of each pixel is calculated to obtain the second image representing the characteristics of the illumination component.

[0034] In another embodiment, the corresponding preset decomposition network can also be used to decompose the enhanced image. For example, the decomposition part of the illumination component can be a three-layer U-shaped network, consisting of an upsampling layer, a pooling layer, and a convolution layer. The decomposition of the reflection component includes more detailed textures. Compared with the smooth illumination component, the decomposition part of the reflection component is more complex to extract richer features. We downsample the input to form three resolutions for parallel processing. After extracting features through the convolution layer, the features of the three resolution branches are fused. The fusion process consists of sampling and convolution operations.

[0035] The image to be enhanced is decomposed into reflection components to obtain a first image characterizing the characteristics of the reflection components, and the image to be enhanced is decomposed into illumination components to obtain a second image characterizing the characteristics of the illumination components. The decomposition process may also use other decomposition methods, which are not limited in this embodiment.

[0036] S202: performing denoising processing on the first image and the second image respectively to obtain a denoised first image and a denoised second image, and performing brightness enhancement processing on the first image and the second image respectively to obtain an enhanced first image and an enhanced second image.

[0037] In step S202, the decomposed emission component and illumination component are subjected to denoising to obtain a smooth reflection component map and a smooth illumination component map. The decomposed reflection component and illumination component are subjected to brightness enhancement processing to enhance the brightness in the first image and the second image to reduce the influence of noise.

[0038] In this embodiment, when the first image and the second image are subjected to denoising processing, high-frequency noise in the first image and the second image is removed.

[0039] In this embodiment, the decomposed emission component and illumination component often contain a lot of noise, which seriously affects the display of details. De-noising is performed on the decomposed emission component and illumination component to obtain a smooth reflection component map and a smooth illumination component map, which is convenient for subsequent fusion to obtain more texture details.

[0040] Optionally, performing denoising on the first image and the second image respectively to obtain a denoised first image and a denoised second image includes: Performing denoising on the first image using Fourier transform to obtain a denoised first image; The second image is denoised using Fourier transform to obtain a denoised second image.

[0041] In this embodiment, Fourier transform is used for denoising, Fourier transform is performed on the first image and the second image to obtain the first image after Fourier transform and the second image after Fourier transform, and filtering denoising is performed on the first image after Fourier transform and the second image after Fourier transform to obtain the first image after noise and the second image after denoising.

[0042] In another embodiment, a denoising network may be used for denoising. For example, the denoising network may be composed of 41 convolutional layers, 44 activation layers, and 3 deconvolutional layers. The denoising network may also use multiple residual modules, each of which contains 5 convolutional layers, and uses jump connections at the head and tail to prevent the denoising network from gradient disappearance or explosion during the training process. The activation function of the activation layer of the denoising network may use a linear rectification function. In another embodiment, a bilateral filtering denoising method may be used to denoise the first image and the second image to improve edge details of the image. Different denoising methods may also be used, which are not limited in this embodiment.

[0043] In this embodiment, when brightness enhancement processing is performed on the first image and the second image respectively, a cascaded multi-convolutional layer can be used for the enhancement processing, so that the obtained enhanced first image and second image can improve the corresponding visual effects.

[0044] Optionally, performing brightness enhancement processing on the first image and the second image respectively to obtain an enhanced first image and an enhanced second image includes: Using a convolution layer in a preset fully convolutional neural network, convolution operations are performed on the first image and the second image respectively to obtain a first convolution feature map of the first image and a second convolution feature map of the second image; Using a normalization layer in a preset fully convolutional neural network, respectively normalize the first convolutional feature map and the second convolutional feature map to obtain a first normalized feature map of the first convolutional feature map and a second normalized feature map of the second convolutional feature map; Using the activation layer in the preset fully convolutional neural network, the first normalized feature map and the second normalized feature map are activated respectively to obtain a first activated feature map of the first normalized feature map and a second activated feature map of the second normalized feature map, wherein the first activated feature map is an enhanced first image, and the second activated feature map is an enhanced second image.

[0045] In this embodiment, a preset full convolutional neural network is used to perform brightness enhancement processing, wherein the preset full convolutional neural network includes a convolutional layer, a normalization layer and an activation layer. The convolutional layer in the preset full convolutional neural network is used to perform convolution operations on the first image and the second image, respectively, to obtain a first convolutional feature map of the first image and a second convolutional feature map of the second image. The normalization layer in the preset full convolutional neural network is used to perform normalization operations on the first convolutional feature map and the second convolutional feature map, respectively, to obtain a first normalized feature map of the first convolutional feature map and a second normalized feature map of the second convolutional feature map. The activation layer in the preset full convolutional neural network is used to perform activation operations on the first normalized feature map and the second normalized feature map, respectively, to obtain a first activated feature map of the first normalized feature map and a second activated feature map of the second normalized feature map. The first activated feature map is an enhanced first image, and the second activated feature map is an enhanced second image.

[0046] In another embodiment, a U-type network can also be used for brightness enhancement processing. The U-type network may include 4 pairs of convolutional layer units and deconvolution layer units that are connected in pairs and symmetrically for brightness extraction, a brightness splicing and a 3-convolutional layer reconstruction unit, the convolutional layer unit includes two convolutional layers and a maximum pooling layer, the deconvolution layer unit includes a deconvolution layer and two convolution layers, and the deconvolution layer is suitable for further enhancement of brightness.

[0047] In another embodiment, gamma correction technology can also be used to adjust the brightness of the first image and the second image, and the gamma correction technology performs brightness enhancement according to the input adjustment scale factor. When performing brightness adjustment, the gamma correction technology can include an adjustment factor expansion unit, a splicing unit, and a brightness adjustment unit. The adjustment factor expansion unit expands the preset adjustment scale factor into a matrix of the same size as the first image or the second image. The splicing unit is used to splice the matrix expanded by the whole scale factor expansion unit with the first image or the second image, and the brightness adjustment unit is used to adjust the brightness based on the splicing result. Among them, the brightness adjustment unit can be a fully convolutional neural network including 4 convolutional layers. In order to better train the fully convolutional neural network in the brightness adjustment unit, the loss function used in the training of the fully convolutional neural network includes a gradient level fidelity term to ensure the training effect of the fully convolutional neural network. Among them, the gradient level fidelity term is used to measure the horizontal or vertical gradient distance between the predicted result in the training sample and the true label.

[0048] S203: Add the denoised first image and the reflection component in the corresponding channel in the enhanced first image to obtain the added first image, and add the denoised second image and the illumination component in the corresponding channel in the enhanced second image to obtain the added second image.

[0049] In step S203, the denoised reflection component and the enhanced reflection component are added and fused, and the denoised illumination component and the brightness-enhanced illumination image are added and fused, so that the obtained reflection component and illumination component are visible.

[0050] In this embodiment, before adding the reflected components in the corresponding channels of the denoised first image and the enhanced first image, the denoised first image and the enhanced first image are converted into HSV images. After the conversion, the denoised first image and the enhanced first image are three-channel images, the H channel is used as the first channel, the S channel is used as the second channel, and the V channel is used as the third channel. The pixel values ​​of the pixels at the same position in the first channel of the denoised first image and the first channel of the enhanced first image are added and the average value is taken as the pixel value of the pixels at the same position in the first channel of the added first image, the pixel values ​​of the pixels at the same position in the second channel of the denoised first image and the second channel of the enhanced first image are added and the average value is taken as the pixel value of the pixels at the same position in the second channel of the added first image, the pixel values ​​of the pixels at the same position in the third channel of the denoised first image and the third channel of the enhanced first image are added and the average value is taken as the pixel value of the pixels at the same position in the third channel of the added first image, and the finally obtained added first image is a three-channel image.

[0051] Before adding the illumination components in the corresponding channels of the denoised second image and the enhanced second image, the denoised second image and the enhanced second image are converted into HSV images. After the conversion, the denoised second image and the enhanced second image are 3-channel images, the H channel is used as the first channel, the S channel is used as the second channel, and the V channel is used as the third channel. The denoised second image includes the first channel, the second channel and the third channel, and the enhanced second image also includes the first channel, the second channel and the third channel. The pixel values ​​of the pixels at the same position of the first channel in the denoised second image and the first channel in the enhanced second image are added and the average value is taken as the pixel value of the pixels at the same position of the first channel in the added second image, the pixel values ​​of the pixels at the same position of the second channel in the denoised second image and the second channel in the enhanced second image are added and the average value is taken as the pixel value of the pixels at the same position of the second channel in the added second image, the pixel values ​​of the pixels at the same position of the third channel in the denoised second image and the third channel in the enhanced second image are added and the average value is taken as the pixel value of the pixels at the same position of the third channel in the added second image, and the finally obtained added second image is a 3-channel image.

[0052] S204: fusing the reflection component and the illumination component of the added first image and the added second image to obtain a fused image, and determining the fused image as an enhanced image of the image to be enhanced.

[0053] In step S204, the reflection component and the illumination component of the added first image and the added second image are fused to obtain an image including the reflection component and the illumination component.

[0054] In this embodiment, the reflection component and the illumination component of the added first image and the added second image are fused to obtain a fused image. When fusion is performed, the reflection component and the illumination component may be multiplied to obtain a fused image. The brightness and smoothness of the fused image are improved, and the visibility of the enhanced image is improved.

[0055] Optionally, fusing the reflection component and the illumination component of the added first image and the added second image to obtain a fused image includes: Add the reflection component and the illumination component in the corresponding channel of the added first image and the added second image to obtain an initial channel fusion feature map; According to the attention mechanism, the attention value of each channel in the initial channel fusion feature map is extracted; According to the attention value of each channel, the attention image of each channel in the initial channel fusion feature map is calculated, and the attention image of each channel is merged to obtain the fused image.

[0056] In this embodiment, the corresponding reflection components and illumination components in the corresponding channels of the added first image and the added second image are added to obtain an initial channel fusion feature map, for example, the pixel values ​​of the first channel in the added first image and the first channel in the added second image at the same position are multiplied to obtain the pixel values ​​of the first channel in the initial channel fusion feature map, the second channel in the added first image and the pixel values ​​of the second channel in the added second image at the same position are multiplied to obtain the pixel values ​​of the second channel in the initial channel fusion feature map, and the third channel in the added first image and the pixel values ​​of the third channel in the added second image at the same position are multiplied to obtain the pixel values ​​of the third channel in the initial channel fusion feature map. The obtained initial channel fusion feature map contains 3 channels, and the weight value corresponding to each channel is calculated according to the pixel value of the pixel position in each channel, and the attention image of each channel in the initial channel fusion feature map is calculated according to the attention value of each channel, and the attention image of each channel is channel-merged to obtain the fused image.

[0057] Optionally, according to the attention mechanism, the attention value of each channel in the initial channel fusion feature map is extracted, including: Perform a global average pooling operation on each channel in the initial channel fusion feature map to obtain a pooled feature map; Through the preset fully connected layer, the pooled feature map is fully connected to obtain the attention value of each channel in the pooled feature map.

[0058] In this embodiment, according to the attention mechanism, when extracting the attention value of each channel in the initial channel fusion feature map, a global average pooling operation is performed on each channel in the initial channel fusion feature map to obtain a pooled feature map, and a full connection operation is performed on the pooled feature map through a preset full connection layer to obtain the attention value of each channel in the pooled feature map. Among them, the size of the pooled feature map is , through the preset fully connected layer, the pooled feature map is fully connected, and the global average pooling value of each channel is mapped to between 0 and 1 as the attention value of each channel, and the sum of the attention values ​​of each channel is 1.

[0059] In another embodiment, the attention value corresponding to each pixel position can be calculated. The same pixel position includes three channels, and the attention calculation is performed on each channel of the pixel so that each pixel can display relatively important information. For example, for any pixel position, the pixel value in the first channel is a, the pixel value in the second channel is b, and the pixel value in the third channel is c. It should be noted that a, b, and c are in the same dimensional unit, and the attention value in the first channel can be calculated as , the attention value in the second channel is , the attention value in the third channel is , according to the corresponding attention value, calculate the pixel value corresponding to each channel in the corresponding pixel position, traverse each pixel position, and get the final fused image.

[0060] Acquire an image to be enhanced for pedestrian detection taken at night, decompose the image to be enhanced by a reflection component to obtain a first image characterizing the characteristics of the reflection component, decompose the image to be enhanced by an illumination component to obtain a second image characterizing the characteristics of the illumination component, denoise the first image and the second image respectively to obtain a denoised first image and a denoised second image, perform brightness enhancement on the first image and the second image respectively to obtain an enhanced first image and an enhanced second image, add the reflection components in corresponding channels of the denoised first image and the enhanced first image to obtain an added first image, add the illumination components in corresponding channels of the denoised second image and the enhanced second image to obtain an added second image, fuse the reflection components and illumination components of the added first image and the added second image to obtain a fused image, and determine the fused image as the enhanced image of the image to be enhanced. In the present application, the decomposed reflection component map and the illumination component map are denoised and enhanced respectively to obtain a denoised smooth reflection image and a smooth illumination image, as well as an enhanced reflection image and an enhanced illumination image. The denoising process can suppress the noise in the image, and the enhancement process can improve the contrast of the image. The denoised image and the enhanced image are channel-fused, thereby improving the enhanced image effect.

[0061] See also Figure 3 , Figure 3 Schematic diagram of the structure of an image enhancement device provided by an embodiment of the present invention. In this embodiment, the terminal includes various units for executing Figure 2 For details, please refer to the steps in the corresponding embodiment. Figure 2 as well as Figure 2 For the convenience of explanation, only the parts related to this embodiment are shown. Figure 3 As shown, the image enhancement device 30 includes: an acquisition module 31, a denoising and enhancement module 32, an addition module 33, and a fusion module 34.

[0062] The acquisition module 31 is used to acquire the image to be enhanced taken at night for pedestrian detection, decompose the reflection component of the image to be enhanced to obtain a first image representing the characteristics of the reflection component, and decompose the illumination component of the image to be enhanced to obtain a second image representing the characteristics of the illumination component.

[0063] The denoising and enhancement module 32 is used to perform denoising processing on the first image and the second image respectively to obtain the denoised first image and the denoised second image, and perform brightness enhancement processing on the first image and the second image respectively to obtain the enhanced first image and the enhanced second image.

[0064] The adding module 33 is used to add the denoised first image and the reflection component in the corresponding channel in the enhanced first image to obtain the added first image, and to add the denoised second image and the illumination component in the corresponding channel in the enhanced second image to obtain the added second image.

[0065] The fusion module 34 is used to fuse the reflection component and the illumination component of the added first image and the added second image to obtain a fused image, and determine the fused image as the enhanced image of the image to be enhanced.

[0066] Optionally, the acquisition module 31 includes: The extraction unit is used to extract the reflection component in the image to be enhanced by using a preset reflection component extraction network to obtain a first image representing the characteristics of the reflection component.

[0067] The removal unit is used to calculate the illumination component of each pixel according to the reflection component of each pixel in the first image and the pixel value of each pixel in the image to be enhanced, so as to obtain a second image representing the characteristics of the illumination component.

[0068] Optionally, the denoising and enhancement module 32 includes: The first denoising unit is used to perform denoising on the first image by using Fourier transform to obtain a denoised first image.

[0069] The second denoising unit is used to perform denoising on the second image by using Fourier transform to obtain a denoised second image.

[0070] Optionally, the denoising and enhancement module 32 includes: The convolution unit is used to use the convolution layer in the preset fully convolutional neural network to perform convolution operations on the first image and the second image respectively, so as to obtain a first convolution feature map of the first image and a second convolution feature map of the second image.

[0071] A normalization unit is used to use a normalization layer in a preset fully convolutional neural network to perform normalization operations on the first convolutional feature map and the second convolutional feature map, respectively, to obtain a first normalized feature map of the first convolutional feature map and a second normalized feature map of the second convolutional feature map.

[0072] An activation unit is used to use an activation layer in a preset fully convolutional neural network to perform activation operations on the first normalized feature map and the second normalized feature map, respectively, to obtain a first activated feature map of the first normalized feature map and a second activated feature map of the second normalized feature map, the first activated feature map is an enhanced first image, and the second activated feature map is an enhanced second image.

[0073] Optionally, the fusion module 34 includes: The first adding unit is used to add the reflection component and the illumination component in the corresponding channel of the added first image and the added second image to obtain an initial channel fusion feature map.

[0074] The extraction unit is used to extract the attention value of each channel in the initial channel fusion feature map according to the attention mechanism.

[0075] The calculation unit is used to calculate the attention image of each channel in the initial channel fusion feature map according to the attention value of each channel, and merge the attention images of each channel to obtain a fused image.

[0076] Optionally, the extraction unit comprises: The pooling unit is used to perform a global average pooling operation on each channel in the initial channel fusion feature map to obtain a pooled feature map.

[0077] The unit is used to perform a full connection operation on the pooled feature map through a preset full connection layer to obtain the attention value of each channel in the pooled feature map.

[0078] It should be noted that the information interaction, execution process and other contents between the above-mentioned units are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0079] Figure 4 Schematic diagram of the structure of a computer device provided by an embodiment of the present invention. Figure 4 As shown, the computer device of this embodiment includes: at least one processor ( Figure 4 Only one is shown in the figure), a memory, and a computer program stored in the memory and executable on at least one processor, wherein when the processor executes the computer program, any of the above-mentioned steps of the image enhancement method are implemented.

[0080] The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that Figure 4 These are merely examples of computer devices and do not constitute limitations on the computer devices. The computer devices may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, they may also include a network interface, a display screen, and an input device.

[0081] The processor may be a CPU, or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0082] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory may be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device, and in other embodiments, it may also be an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Further, the memory may also include both an internal storage unit of the computer device and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of a computer program, etc. The memory may also be used to temporarily store data that has been output or is to be output.

[0083] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiment can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0084] The present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiment when executing.

[0085] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0086] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0087] In the embodiments provided by the present invention, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0088] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0089] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. An image enhancement method, characterized in that: The image enhancement method comprises: Acquire an image to be enhanced for pedestrian detection taken at night, perform a decomposition process on the image to be enhanced in terms of a reflection component to obtain a first image representing characteristics of the reflection component, and perform a decomposition process on the image to be enhanced in terms of an illumination component to obtain a second image representing characteristics of the illumination component; Performing denoising processing on the first image and the second image respectively to obtain a denoised first image and a denoised second image, and performing brightness enhancement processing on the first image and the second image respectively to obtain an enhanced first image and an enhanced second image; Adding the denoised first image and the reflection component in the corresponding channel of the enhanced first image to obtain an added first image, and adding the denoised second image and the illumination component in the corresponding channel of the enhanced second image to obtain an added second image; The reflection component and the illumination component of the added first image and the added second image are fused to obtain a fused image, and the fused image is determined as the enhanced image of the image to be enhanced.

2. The image enhancement method according to claim 1, characterized in that: The step of performing a decomposition process on the reflection component of the image to be enhanced to obtain a first image representing the characteristics of the reflection component, and performing a decomposition process on the illumination component of the image to be enhanced to obtain a second image representing the characteristics of the illumination component includes: Extracting the reflection component in the image to be enhanced using a preset reflection component extraction network to obtain a first image representing the characteristics of the reflection component; According to the reflection component of each pixel in the first image and the pixel value of each pixel in the image to be enhanced, the illumination component of each pixel is calculated to obtain a second image representing the characteristics of the illumination component.

3. The image enhancement method according to claim 1, characterized in that: The denoising process is performed on the first image and the second image respectively to obtain a denoised first image and a denoised second image, including: Performing denoising on the first image using Fourier transform to obtain a denoised first image; The second image is denoised using the Fourier transform to obtain a denoised second image.

4. The image enhancement method according to claim 1, characterized in that: The performing brightness enhancement processing on the first image and the second image respectively to obtain an enhanced first image and an enhanced second image comprises: Using a convolution layer in a preset fully convolutional neural network, respectively perform a convolution operation on the first image and the second image to obtain a first convolution feature map of the first image and a second convolution feature map of the second image; Using a normalization layer in a preset fully convolutional neural network, respectively normalize the first convolutional feature map and the second convolutional feature map to obtain a first normalized feature map of the first convolutional feature map and a second normalized feature map of the second convolutional feature map; Using the activation layer in a preset fully convolutional neural network, activation operations are performed on the first normalized feature map and the second normalized feature map, respectively, to obtain a first activated feature map of the first normalized feature map and a second activated feature map of the second normalized feature map, wherein the first activated feature map is the enhanced first image, and the second activated feature map is the enhanced second image.

5. The image enhancement method according to claim 1, characterized in that: The step of fusing the reflection component and the illumination component of the added first image and the added second image to obtain a fused image includes: Adding the reflection component and the illumination component in the corresponding channel of the added first image and the added second image to obtain an initial channel fusion feature map; According to the attention mechanism, extracting the attention value of each channel in the initial channel fusion feature map; According to the attention value of each channel, the attention image of each channel in the initial channel fusion feature map is calculated, and the attention image of each channel is channel-merged to obtain a fused image.

6. The image enhancement method according to claim 5, characterized in that: The step of extracting the attention value of each channel in the initial channel fusion feature map according to the attention mechanism includes: Perform a global average pooling operation on each channel in the initial channel fusion feature map to obtain a pooled feature map; A full connection operation is performed on the pooled feature map through a preset full connection layer to obtain the attention value of each channel in the pooled feature map.

7. An image enhancement device, characterized in that: The image enhancement device comprises: an acquisition module, used to acquire an image to be enhanced for pedestrian detection taken at night, decompose the image to be enhanced by a reflection component to obtain a first image representing the characteristics of the reflection component, and decompose the image to be enhanced by an illumination component to obtain a second image representing the characteristics of the illumination component; a denoising and enhancement module, configured to perform denoising processing on the first image and the second image respectively to obtain a denoised first image and a denoised second image, and perform brightness enhancement processing on the first image and the second image respectively to obtain an enhanced first image and an enhanced second image; an adding module, configured to add the denoised first image and the reflection component in the corresponding channel of the enhanced first image to obtain the added first image, and to add the denoised second image and the illumination component in the corresponding channel of the enhanced second image to obtain the added second image; A fusion module is used to fuse the reflection component and the illumination component of the added first image and the added second image to obtain a fused image, and determine the fused image as the enhanced image of the image to be enhanced.

8. The image enhancement device according to claim 7, characterized in that: The acquisition module comprises: An extraction unit, configured to extract the reflection component in the image to be enhanced by using a preset reflection component extraction network, and obtain a first image representing the characteristics of the reflection component; The removal unit is used to calculate the illumination component of each pixel according to the reflection component of each pixel in the first image and the pixel value of each pixel in the image to be enhanced, so as to obtain a second image representing the characteristics of the illumination component.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the image enhancement method according to any one of claims 1 to 6 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image enhancement method according to any one of claims 1 to 6 is implemented.

Images