Image enhancement method and device

Abstract

The embodiment of the invention provides an image enhancement method and device. The method comprises the following steps: acquiring a first image acquired in a preset exposure value range; inputting the first image into the model, and outputting a first illumination image and a first reflection image corresponding to the first image; inputting the first illumination image into a model, outputting a second illumination image corresponding to the first illumination image, inputting the first reflection image into the model, and outputting a second reflection image corresponding to the first reflection image; determining a difference illumination image according to the first illumination image and the second illumination image, and further determining a first spliced image; determining a second spliced image according to the first spliced image, the second illumination image and the first reflection image; and inputting the second spliced image into the model to obtain a second image corresponding to the first image. According to the technical scheme provided by the embodiment of the invention, the illumination image and the reflection image are separated and then spliced and reconstructed, and the brightness enhanced image is output, so that the effect of improving vision is achieved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of image processing technology, and in particular to an image enhancement method and device. Background Art

[0002] With the rapid development of smart terminals, security monitoring, and autonomous driving, the demand for image quality in low-light environments has become increasingly prominent.

[0003] At present, in order to improve the quality of low-brightness images, curve transformation or histogram equalization is used to change the pixel distribution of the image to increase the brightness of the low-brightness image. However, this not only brings about greater noise performance and reduces image quality, but also causes problems such as detail loss and color distortion. Summary of the Invention

[0004] The embodiments of the present disclosure provide an image enhancement method and apparatus to output a brightness-enhanced image, thereby achieving an improved visual effect.

[0005] In a first aspect, an embodiment of the present disclosure provides an image enhancement method, the method comprising:

[0006] Acquire a first image captured within a preset exposure value range;

[0007] Inputting the first image into a pre-trained first image processing model, and outputting a first illumination image and a first reflection image corresponding to the first image;

[0008] Inputting the first illumination pattern into a pre-trained first illumination pattern processing model, and outputting a second illumination pattern corresponding to the first illumination pattern; inputting the first reflection pattern into a pre-trained first reflection pattern processing model, and outputting a second reflection pattern corresponding to the first reflection pattern;

[0009] Determining a difference illumination map according to the first illumination map and the second illumination map, and determining a first stitched image based on the difference illumination map and the second reflection map;

[0010] determining a second stitched image based on the first stitched image, the second illumination image, and the first reflection image;

[0011] The second stitched image is input into a pre-trained stitched image processing model to obtain a second image corresponding to the first image; wherein the exposure value of the second image is the target exposure value.

[0012] In a second aspect, an embodiment of the present invention further provides an image enhancement device, the device comprising:

[0013] A first image acquisition module is used to acquire a first image captured within a preset exposure value range;

[0014] a first illumination pattern output module, configured to input the first image into a pre-trained first image processing model, and output a first illumination pattern and a first reflection pattern corresponding to the first image;

[0015] a second illumination pattern output module, configured to input the first illumination pattern into a pre-trained first illumination pattern processing model and output a second illumination pattern corresponding to the first illumination pattern, and input the first reflection pattern into a pre-trained first reflection pattern processing model and output a second reflection pattern corresponding to the first reflection pattern;

[0016] a first stitched image determining module, configured to determine a difference illumination image according to the first illumination image and the second illumination image, and determine a first stitched image based on the difference illumination image and the second reflection image;

[0017] A second stitched image determining module, configured to determine a second stitched image based on the first stitched image, the second illumination image, and the first reflection image;

[0018] The second image output module is used to input the second stitched image into a pre-trained stitched image processing model to obtain a second image corresponding to the first image; wherein the exposure value of the second image is the target exposure value.

[0019] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising:

[0020] one or more processors;

[0021] a storage device for storing one or more programs,

[0022] When the one or more programs are executed by the one or more processors, the one or more processors implement the image enhancement method as described in any one of the embodiments of the present invention.

[0023] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to perform the image enhancement method as described in any one of the embodiments of the present invention.

[0024] In a fifth aspect, an embodiment of the present invention further provides a computer program product, comprising a computer program, characterized in that when the computer program is executed by a processor, the image enhancement method as described in any one of the embodiments of the present invention is implemented.

[0025] The technical solution of the embodiment of the present disclosure is to obtain a first image captured under a preset exposure value range, input the first image into a pre-trained first image processing model, and output a first illumination map and a first reflection map corresponding to the first image. Then, the first illumination map is input into a pre-trained first illumination map processing model, and output a second illumination map corresponding to the first illumination map. The first reflection map is input into a pre-trained first reflection map processing model, and output a second reflection map corresponding to the first reflection map. Then, a difference illumination map is determined based on the first illumination map and the second illumination map, and a first stitched image is determined based on the difference illumination map and the second reflection map. Further, a second stitched image is determined based on the first stitched image, the second illumination map and the first reflection map. Finally, the second stitched image is input into a pre-trained stitched image processing model to obtain a second image corresponding to the first image, which solves the problems of large noise performance, reduced image quality, loss of details and color distortion when using curve transformation or histogram equalization to change the pixel distribution of the image to improve the brightness of low-brightness images. The embodiment of the present invention separates the illumination image and the reflection image and then reconstructs them to output a brightness-enhanced image, thereby achieving an effect of improving vision. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings introduced here only illustrate some of the embodiments to be described by the present invention, and are not exhaustive. A person skilled in the art can derive other drawings based on these drawings without inventive effort.

[0027] Figure 1 is a schematic diagram of an image enhancement process provided by an embodiment of the present disclosure;

[0028] Figure 2 is a schematic diagram of image enhancement provided by an embodiment of the present disclosure;

[0029] Figure 3 is a schematic diagram of a first image processing model provided by an embodiment of the present disclosure;

[0030] Figure 4 is a schematic diagram of a first illumination image processing model and a first reflection image processing model provided by an embodiment of the present disclosure;

[0031] Figure 5 is a schematic diagram of a splicing image processing model provided by an embodiment of the present disclosure;

[0032] Figure 6 is a schematic diagram of an image enhancement process provided by an embodiment of the present disclosure;

[0033] Figure 7 is a schematic structural diagram of an image enhancement device provided by an embodiment of the present invention;

[0034] Figure 8 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0036] Before introducing the technical solutions provided by the embodiments of the present disclosure, an exemplary application scenario can be first described. The technical solutions provided by the embodiments of the present disclosure can be applied to scenarios where a low-brightness image is processed to obtain a corresponding high-brightness image.

[0037] It should be noted that the exposure value (EV) is used to indicate the level of exposure and is a standardized parameter for measuring the image brightness under a combination of shutter speed and aperture. When EV is zero, it corresponds to standard exposure under specific lighting conditions. For example, the aperture is f / 1.0 and the shutter speed is 1s. For every decrease of 1 in the exposure value, the corresponding exposure is halved, and the image will become one level darker; for every increase of 1 in the exposure value, the corresponding exposure is doubled, and the image will become one level brighter. Based on the technical solution of the embodiment of the present disclosure, the illumination map and the reflection map are separated and processed, and then spliced ​​and reconstructed to output a brightness-enhanced image, thereby achieving the effect of improving vision.

[0038] Example 1

[0039] Figure 1 This is a flow chart of an image enhancement method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to situations where a low-brightness image is processed to obtain a corresponding high-brightness image. The method can be executed by an image enhancement device, which can be implemented in the form of software and / or hardware. The hardware can be a mobile electronic device, which can execute the vehicle control method provided by this technical solution.

[0040] like Figure 1 As shown, the method includes:

[0041] S110: Acquire a first image captured within a preset exposure value range.

[0042] The preset exposure value range is from EV-3 to EV-1, which is 1 to 3 stops lower than the standard exposure value of EV0. The first image is typically captured in a low-light environment, such as a night scene or in the shadows, and the exposure value of the first image is between EV-3 and EV-1.

[0043] It should be noted that when the exposure value of the first image is EV-1, the exposure of the first image is half of EV0, and the first image is darker at this time; when the exposure value of the first image is EV-2, the exposure of the first image is one-quarter of EV0, and the first image is even darker at this time; when the exposure value of the first image is EV-3, the exposure of the first image is one-eighth of EV0, and the first image is extremely dark.

[0044] Specifically, a first image with an exposure value between EV-3 and EV-1 is acquired.

[0045] S120: Input the first image into a pre-trained first image processing model, and output a first illumination image and a first reflection image corresponding to the first image.

[0046] The exposure value of the first illumination image is a first exposure value, and the exposure value of the first reflection image is a second exposure value. The first image processing model is a pre-trained deep learning model that decomposes the input first image into two components: a first illumination image and a first reflection image. The first image processing model includes an input layer, a maximization layer, a normalization layer, and at least six convolutional layers.

[0047] It should be noted that, see Figure 2 The first illumination map represents the light intensity distribution in the first image, that is, the degree to which different areas in the scene are illuminated by light. The first illumination map contains low-frequency information in the first image, reflecting the intensity of the ambient light or light source. The first reflection map represents the surface reflection properties of the object in the first image, that is, the object's ability to reflect light, which is independent of the lighting. The first reflection map contains high-frequency information in the first image, reflecting the inherent properties of the object such as texture and color. The first reflection map corresponding to the first image may contain noise or have lost details.

[0048] It should also be noted that the first exposure value refers to the exposure value of the first illumination map, which is typically consistent with or approximately consistent with the exposure value of the first image. The first illumination map reflects the light intensity distribution in the first image, and its first exposure value is directly inherited from the lighting conditions of the first image. For example, if the exposure value of the first image is EV-3, the exposure value of the first illumination map is typically also EV-3 or close to EV-3. The second exposure value refers to the exposure value of the first reflectance map, which is typically different from the first exposure value. The first reflectance map reflects the surface reflectance properties of an object and is independent of lighting conditions. The second exposure value is typically implicitly normalized to be independent of the lighting conditions of the first image. The first reflectance map is similar to the "object itself without lighting effects," similar to an object photographed under uniform lighting. Therefore, the second exposure value is typically normalized rather than inheriting the low exposure value of the first image. For example, the first reflectance map can be adjusted to EV0 or a neutral exposure. If the exposure value of the first image is EV-3, the first reflectance map may have an exposure value of EV0 because it needs to preserve the object's inherent color and texture without being affected by insufficient lighting.

[0049] Optionally, for at least one pixel point in the first image, after obtaining the first pixel value corresponding to the first component, the second pixel value corresponding to the second component, and the third pixel value corresponding to the third component in the pixel point, the maximum value of the first pixel value, the second pixel value, and the third pixel value is used as the pixel value of the pixel point, thereby obtaining the first image to be used; after the first image to be used passes through at least three convolution layers, a first illumination map corresponding to the first image is obtained, and after the second image to be used passes through at least three convolution layers, a first reflection map corresponding to the first image is obtained.

[0050] The first component is a red component, the second component is a green component, the third component is a blue component, and the second image to be used is an image obtained by normalizing the first image.

[0051] It should be noted that, see Figure 3 After the first image is input through the input layer, for each pixel in the first image, the values ​​of the three RGB color components are extracted through the maximization layer. The first pixel value refers to the pixel value corresponding to the red component. The second pixel value refers to the pixel value corresponding to the green component. The third pixel value refers to the pixel value corresponding to the blue component. For each pixel, the first image to be used refers to an image in which the pixel value of each pixel is the largest of the three pixel values ​​in each pixel. Normalization processing refers to scaling the pixel values ​​of each pixel of the first image to the range of [0,1] or [-1,1] in the normalization layer. Normalizing the first image can eliminate uneven illumination or color deviation, making it easier for subsequent model processing.

[0052] It should also be noted that, see Figure 3 , the first illumination image is the output image after the first image to be used is input into three convolution layers. The first reflection image is the output image after the second image to be used is input into three convolution layers. The convolution layer is a special neural network layer that can be used to process two-dimensional images. In the convolution layer, input data, such as images, are convolved with several learnable convolution kernels. Each convolution kernel slides on the input data, calculates the weighted sum at each position, and outputs a feature map. Through multiple sets of convolution kernels, different spatial features of the input data can be extracted. The convolution layer has the characteristics of parameter sharing and local connection, so it is computationally efficient and suitable for processing large-scale data.

[0053] Specifically, the first image is input into a pre-trained first image processing model, and after being processed by an input layer, a maximization layer, a normalization layer, and at least six convolutional layers, a first illumination image and a first reflection image corresponding to the first image are output.

[0054] S130: Input the first illumination image into a pre-trained first illumination image processing model, and output a second illumination image corresponding to the first illumination image; input the first reflection image into a pre-trained first reflection image processing model, and output a second reflection image corresponding to the first reflection image.

[0055] The exposure value of the second illumination image is the third exposure value, and the exposure value of the second reflection image is the fourth exposure value.

[0056] It should be noted that the first illumination image processing model is a deep learning model that converts the input first illumination image into the second illumination image. The first illumination image processing model learns the rules for converting illumination images from low-light conditions to high-light conditions. The first reflection image processing model is a deep learning model that converts the input first reflection image into the second reflection image. The first reflection image processing model can maintain the inherent properties of the reflection image, such as color and texture, while adapting to high-brightness conditions.

[0057] Optionally, according to the processing order of the input image, the model structures of the first illumination image processing model and the first reflection image processing model are respectively a first convolution layer, a first spatial attention layer, a second convolution layer, and a second spatial attention layer.

[0058] It should be noted that, see Figure 4The first convolutional layer is the first layer of the first illumination image processing model and the first reflectance image processing model, directly processing the input first illumination image or first reflectance image. The first convolutional layer uses a set of convolution kernels to extract basic low-level features from the input image, such as edges, lines, and color, laying the foundation for subsequent more complex feature extraction. The first spatial attention layer is the first spatial attention mechanism layer after the first convolutional layer. By learning the importance of different spatial locations, the first spatial attention layer automatically adjusts the weight of each location in the feature map, allowing the model to focus more on the most useful areas for the task and reduce attention to irrelevant areas. The second convolutional layer is the second convolutional layer after the first spatial attention layer. Building on the basic features extracted by the first convolutional layer, the second convolutional layer further extracts more complex and abstract feature combinations in the image, such as texture and shape, to enhance the network's expressive power. The second spatial attention layer is the second spatial attention layer after the second convolutional layer. The second spatial attention layer further weights the spatial locations in the feature map, further highlighting the feature expressions of key areas and suppressing information from irrelevant areas, thereby improving the network's focus on objects and its discriminative ability.

[0059] It should also be noted that the second illumination image is the result of converting the first illumination image using the first illumination image processing model, and corresponds to the illumination component of the high-brightness image. The second reflection image is the result of converting the first reflection image using the first reflection image processing model, and corresponds to the reflection component of the high-brightness image. The third exposure value corresponds to EV0 of the high-brightness image. The third exposure value is typically consistent with the second exposure value.

[0060] Specifically, after obtaining the first illumination pattern and the first reflection pattern, the first illumination pattern is input into a pre-trained first illumination pattern processing model, and a second illumination pattern corresponding to the first illumination pattern is output, with the exposure value of the output second illumination pattern being the third exposure value. The first reflection pattern is input into a pre-trained first reflection pattern processing model, and a second reflection pattern corresponding to the first reflection pattern is output, with the exposure value of the output second reflection pattern being the fourth exposure value.

[0061] S140: Determine a difference illumination map according to the first illumination map and the second illumination map, and determine a first stitched image based on the difference illumination map and the second reflection map.

[0062] The difference illumination map is an image obtained by performing a pixel-by-pixel subtraction operation on the second illumination map from the first illumination map. The difference illumination map reflects the amount of illumination change from low-brightness illumination conditions to high-brightness illumination conditions, that is, the amount of exposure change from a low exposure value to a high exposure value. The values ​​of pixels in the difference illumination map may be positive, negative, or zero, depending on the pixel values ​​of the second illumination map and the first illumination map. A positive value indicates an increase in illumination, a negative value indicates a decrease in illumination, and a zero value indicates no change in illumination. The difference illumination map can be used to quantify the increase in brightness. See [1]. Figure 2 The first stitched image is created by stitching the difference illumination map and the second reflection map along the channel dimension. This stitched image combines information about both illumination variations and reflection properties. Assuming both the difference illumination map and the second reflection map are three-channel images, the stitched first image will be a six-channel image. This stitched image preserves information about illumination variations and reflection properties, and the channel-dimensional stitching does not change the spatial dimensions of the image, making it easier for subsequent deep learning models to process.

[0063] Specifically, first, a pixel-by-pixel subtraction operation is performed on the second illumination image and the first illumination image to obtain a difference illumination image. Then, the obtained difference illumination image and the second reflection image are spliced ​​in dimension to obtain a first spliced ​​image.

[0064] S150: Determine a second stitched image according to the first stitched image, the second illumination image, and the first reflection image.

[0065] It should be noted that the second stitched image is obtained by stitching the first stitched image, the second illumination map, and the first reflection map along the channel dimension. This stitched second image combines the illumination variation information from the difference illumination map, the high-brightness illumination information from the second illumination map, and the low-brightness reflection information from the first reflection map, providing a richer feature representation for subsequent deep learning models.

[0066] Specifically, ensure that the spatial dimensions of the first stitched image, the second illumination image, and the first reflection image are consistent to facilitate channel stitching. Before stitching, it may be necessary to normalize all input images to ensure that their numerical ranges are consistent. Finally, stitch the first stitched image, the second illumination image, and the first reflection image together along the channels to produce the second stitched image.

[0067] S160: Input the second stitched image into a pre-trained stitched image processing model to obtain a second image corresponding to the first image.

[0068] The exposure value of the second image is the target exposure value.

[0069] It should be noted that the stitching image processing model is a deep learning model that converts the input second stitched image, which contains information about illumination variations, high-brightness illumination, and low-brightness reflections, into a high-brightness, high-quality second image. The second image is the result of the stitching image processing model converting the second stitched image, corresponding to a high-brightness, high-quality image. The exposure value of the second image is the target exposure value, i.e., EV0.

[0070] Optionally, according to the order of processing the input image, the model structure of the spliced ​​image processing model is the third convolution layer, the first mixed attention layer, the fourth convolution layer and the second mixed attention layer.

[0071] It should be noted that, see Figure 5 The third convolutional layer is a fundamental component of CNNs. It extracts features by sliding convolution kernels across the second stitched image. The third convolutional layer typically receives the second stitched image directly and performs preliminary feature extraction on it. It detects simple, low-level features such as edges and textures. The first hybrid attention layer typically refers to a network layer that combines different types of attention mechanisms. For example, the first hybrid attention layer can combine spatial and channel attention mechanisms. The first hybrid attention layer is typically used in shallow networks to enhance the expressive power of preliminary features. Through this attention mechanism, the stitched image processing model learns to focus on more important regions or channels, improving feature discrimination. The fourth convolutional layer is the second convolutional layer in the stitched image processing model. Its input is typically the feature map output by the first hybrid attention layer. This convolutional layer extracts more complex and abstract features. The fourth convolutional layer typically uses more convolution kernels than the third convolutional layer, outputting new, higher-level feature maps. The second hybrid attention layer is the hybrid attention layer located after the fourth convolutional layer. The second hybrid attention layer can further apply attention to higher-level features, strengthening the stitching image processing model's capture of important features and helping to model more complex global relationships or structural information. This second hybrid attention layer can further improve the performance and generalization capabilities of the stitching image processing model.

[0072] Specifically, after the second stitched image is input into the pre-trained stitched image processing model, a second image with a brightness value of EV0 is output, which has uniform brightness, rich details, less noise and artifacts, and more natural colors and textures.

[0073] The technical solution of the embodiment of the present disclosure is to obtain a first image captured under a preset exposure value range, input the first image into a pre-trained first image processing model, and output a first illumination map and a first reflection map corresponding to the first image. Then, the first illumination map is input into a pre-trained first illumination map processing model, and output a second illumination map corresponding to the first illumination map. The first reflection map is input into a pre-trained first reflection map processing model, and output a second reflection map corresponding to the first reflection map. Then, a difference illumination map is determined based on the first illumination map and the second illumination map, and a first stitched image is determined based on the difference illumination map and the second reflection map. Further, a second stitched image is determined based on the first stitched image, the second illumination map and the first reflection map. Finally, the second stitched image is input into a pre-trained stitched image processing model to obtain a second image corresponding to the first image, which solves the problems of large noise performance, reduced image quality, loss of details and color distortion when using curve transformation or histogram equalization to change the pixel distribution of the image to improve the brightness of low-brightness images. The embodiment of the present invention separates the illumination image and the reflection image and then reconstructs them to output a brightness-enhanced image, thereby achieving an effect of improving vision.

[0074] Example 2

[0075] Figure 6 This is a flow chart of an image enhancement method provided by an embodiment of the present disclosure. Based on the aforementioned embodiments, the training samples and loss functions of each model are detailed. For specific implementations, please refer to the technical solution of this embodiment. Technical terms that are identical or corresponding to those in the aforementioned embodiments are not repeated here.

[0076] like Figure 6 As shown, the method specifically includes the following steps:

[0077] S210: Acquire multiple first samples.

[0078] The first sample includes at least one first sample image whose exposure value is within a preset exposure range, and at least one second sample image whose exposure value is a target exposure value.

[0079] It should be noted that the first sample is a data unit for training or evaluating the first image processing model, which includes an image under a preset exposure range, i.e., low exposure conditions, and an image under a target exposure value, i.e., standard exposure conditions. The first sample image refers to a sample with an exposure value within the preset exposure range, i.e., low exposure conditions. The second sample image refers to a sample with an exposure value at the target exposure value, i.e., standard exposure conditions. When obtaining multiple first samples, a camera device can be used to capture images of the same scene under different exposure settings. For example, a first sample image is obtained by capturing at EV-3, EV-2, and EV-1, and a second sample image is obtained by capturing at EV0. The obtained first sample images and second sample images are formed into multiple first samples. The first sample can also be obtained from an existing low-light image dataset.

[0080] Specifically, a plurality of first sample images and second sample images are obtained, and a pair of the first sample image and the second sample image may be from the same scene. The plurality of pairs of the first sample images and the second sample images are aggregated to obtain a first sample.

[0081] S220 , processing the plurality of first sample images and the plurality of second sample images respectively based on a Gaussian blur algorithm to obtain first illumination label images corresponding to the first sample images and second illumination label images corresponding to the second sample images.

[0082] The image obtained by processing the first sample image with the Gaussian blur algorithm is called the first lighting label image, and the image obtained by processing the second sample image with the Gaussian blur algorithm is called the second lighting label image.

[0083] It should be noted that before performing Gaussian blur processing on the multiple first sample images and the multiple second sample images based on the Gaussian blur algorithm, the first sample images and the second sample images may be preprocessed. Preprocessing refers to a series of operations performed on the first sample images and the second sample images before the primary analysis or processing of the first sample images or the second sample images. The purpose of preprocessing is to improve the quality of the first sample images and the second sample images, extract useful information, or provide better input conditions for subsequent processing steps. In embodiments of the present invention, the preprocessing performed on the first sample images and the second sample images may include operations such as denoising, contrast enhancement, normalization, and geometric correction.

[0084] It should also be noted that the formula for obtaining the first illuminated label image corresponding to the first sample image by processing the first sample image based on the Gaussian blur algorithm is:

[0085] L low =Gaussian(S low (x,y));

[0086] Among them, S low (x,y) refers to the first sample image. L low Refers to the first lighting label image corresponding to the first sample image. The second sample image is processed based on the Gaussian blur algorithm, and the formula for obtaining the second lighting label image corresponding to the second sample image is:

[0087] L high =Gaussian(S high (x,y));

[0088] Among them, S high (x,y) refers to the second sample image. L high Refers to the second illuminated label image corresponding to the second sample image.

[0089] Specifically, after preprocessing the multiple first sample images and the multiple second sample images, the preprocessed multiple first sample images are subjected to a Gaussian blur algorithm to obtain multiple first lighting label images corresponding to the multiple first sample images; and the preprocessed multiple second sample images are subjected to a Gaussian blur algorithm to obtain multiple second lighting label images corresponding to the multiple second sample images.

[0090] S230: Process the first sample image and the first illuminated label image based on a logarithmic algorithm to obtain a first reflective label image corresponding to the first sample image; process the second sample image and the second illuminated label image based on a logarithmic algorithm to obtain a second reflective label image corresponding to the second sample image.

[0091] Among them, the first reflective label image refers to the reflective image corresponding to the first sample image obtained after the first sample image and the first illuminated label image are logarithmically calculated; the second reflective label image refers to the reflective image corresponding to the second sample image obtained after the second sample image and the second illuminated label image are logarithmically calculated.

[0092] It should be noted that the formula for obtaining the first reflective label image corresponding to the first sample image by processing the first sample image and the first illuminated label image based on the logarithmic algorithm is:

[0093] R low =logS low (x,y)-logL low (x,y);

[0094] Among them, R low Refers to the first reflective label image. The second sample image and the second illuminated label image are processed based on the logarithmic algorithm to obtain the formula for the second reflective label image corresponding to the second sample image:

[0095] R high =logS high (x,y)-logL high (x,y);

[0096] Among them, R high Refers to the second reflective label image.

[0097] Specifically, after obtaining the first illuminated label image corresponding to the first sample image and the second illuminated label image corresponding to the second sample image, for the first sample image and the first illuminated label image, after calculation based on the logarithmic algorithm, the first reflective label image corresponding to the first sample image can be obtained; for the second sample image and the second illuminated label image, after calculation based on the logarithmic algorithm, the second reflective label image corresponding to the second sample image can be obtained.

[0098] S240: Determine first target sample data for training a first image processing model based on each first illumination tag image, each second illumination tag image, each first reflection tag image, and each second reflection tag image.

[0099] It should be noted that for the first image processing model, when the input image during training is the first sample image, the corresponding first target sample data is each first illumination label image and each first reflection label image corresponding to each first sample image. When the input image during training is the second sample image, the corresponding first target sample data is each second illumination label image and each second reflection label image corresponding to each second sample image.

[0100] It should also be noted that for the first illumination image processing model, the input images for training the first illumination image processing model can be the first illumination label image and the first illumination image. When the input image for training the first illumination image processing model is the first illumination label image, the corresponding label for training the first illumination image processing model can be the second illumination label image; when the input image for training the first illumination image processing model is the first illumination image, the corresponding label for training the first illumination image processing model can be the second illumination image. In this case, the second illumination image refers to the illumination image output after the second sample image is input into the first image processing model. For the first reflection image processing model, the input images for training the first reflection image processing model can be the first reflection label image and the first reflection image. When the input image for training the first reflection image processing model is the first reflection label image, the corresponding label for training the first reflection image processing model can be the second reflection label image; when the input image for training the first reflection image processing model is the first reflection image, the corresponding label for training the first reflection image processing model can be the second reflection image. In this case, the second reflection image refers to the reflection image output after the second sample image is input into the first image processing model. For the stitching image processing model, the image input during training of the stitching image processing model may be the second stitching image, and the corresponding label of the training stitching image processing model may be the second sample image.

[0101] During training of the first image processing model, to improve model training accuracy, multiple first sample images and multiple second sample images captured from different camera perspectives may be acquired, and the acquired multiple first sample images and multiple second sample images may be used as first samples. The sum of the first samples and each corresponding first illuminated label image, each second illuminated label image, each first reflected label image, and each second reflected label image constitutes first target sample data.

[0102] For each first target sample data, the first sample in the current first target sample data is input into the first image processing model to be trained to obtain the actual illumination image and the actual reflection image corresponding to the current first sample.

[0103] The first image processing model to be trained is a model whose model parameters are initial parameters or default parameters. The actual illumination image and the actual reflection image are the illumination image and the reflection image output after the first sample in the current first target sample data is input into the first image processing model to be trained.

[0104] It should be noted that the model parameters in the first image processing model to be trained do not meet the expected requirements. Therefore, there are certain differences between the actual illumination image, actual reflection image and the corresponding label image output based on the model parameters at this time. Therefore, the corresponding error loss value can be determined based on the actual illumination image, actual reflection image and corresponding label image corresponding to each first target sample data.

[0105] Based on the first preset loss function in the first image processing model to be trained, loss processing is performed on the current actual illumination image, the actual reflection image and the corresponding label image, so as to correct the model parameters in the first image processing model to be trained according to the obtained loss value.

[0106] It should be noted that the training parameters can be set to default values ​​before training the first image processing model to be trained. During training, the training parameters in the model can be modified based on the output of the first image processing model to be trained. In other words, the target first image processing model can be obtained by modifying the loss function in the first image processing model to be trained. Each first sample image or second sample image has a corresponding loss value, which is determined based on the actual illumination image, the actual reflection image, and the corresponding label image.

[0107] The convergence of the first preset loss function is used as a training goal to obtain a target first image processing model, wherein the first image processing model is a model obtained by final training and is used to determine the illumination image and the reflection image of the first sample.

[0108] Specifically, the training error of the loss function, that is, the loss parameter, can be used as a condition for detecting whether the loss function has reached convergence, such as whether the training error is less than the preset error or whether the error change trend tends to be stable, or whether the current number of iterations is equal to the preset number. If the detection reaches the convergence condition, such as the training error of the loss function is less than the preset error or the error change tends to be stable, it indicates that the training of the first image processing model to be trained is completed, and the iterative training can be stopped at this time. If it is detected that the convergence condition is not reached at present, multiple first samples can be further obtained to train the first image processing model to be trained until the training error of the loss function is within the preset range. When the training error of the loss function reaches convergence, the first image processing model to be trained can be used as the target first image processing model.

[0109] It should also be noted that the training process of the first illumination image processing model, the first reflection image processing model and the spliced ​​image processing model is similar to the training process of the first image processing model and will not be repeated here.

[0110] Optionally, the loss functions in the first image processing model, the first illumination image processing model, and the stitching image processing model include one or more of a first sub-loss function, a second sub-loss function, and a third sub-loss function.

[0111] In this embodiment, the first sub-loss function is:

[0112]

[0113] Among them, X' (i,j) is the predicted pixel value of each pixel in the first predicted image; X' 1(i,j) is the label pixel value of each pixel in the first label image;

[0114] It should be noted that the first predicted image refers to the predicted image output by the first image processing model, the first illumination image processing model or the spliced ​​image processing model during model training. The predicted pixel value refers to the pixel value corresponding to each pixel in the first predicted image. The first label image refers to the label image corresponding to the image input by the first image processing model, the first illumination image processing model or the spliced ​​image processing model. The label pixel value refers to the pixel value corresponding to each pixel in the first label image. For the first sub-loss function, the data required to calculate the loss value is the predicted pixel value of each pixel in the first predicted image and the label pixel value of each pixel in the first label image. After the predicted pixel value of each pixel in the first predicted image and the label pixel value of each pixel in the first label image are operated by the first sub-loss function, the first loss value corresponding to the first sub-loss function can be output.

[0115] The second sub-loss function is:

[0116]

[0117] Among them, λ1 is the weight, β1 is the smoothness strength;

[0118] It should be noted that after the predicted pixel value of each pixel point in the first predicted image is operated by the second sub-loss function, a second loss value corresponding to the second sub-loss function can be output.

[0119] The third sub-loss function is:

[0120]

[0121] Among them, X' is the first predicted image; X1' is the first label image; λ2 is the weight; Vgg k (·) represents the network that inputs the first predicted image or the first label image to extract the feature map of the kth layer; C represents the number of channels of the feature map; H represents the height of the feature map; W represents the width of the feature map.

[0122] It should be noted that after the predicted pixel value of each pixel in the first predicted image and the label pixel value of each pixel in the first label image are operated by the third sub-loss function, a third loss value corresponding to the third sub-loss function can be output.

[0123] Optionally, the loss function in the first reflection image processing model is:

[0124]

[0125] in, Represents the pixel value of the second reflected label image under the RGB channel; Represents the pixel value of the first reflected label image under the RGB channel; R' t Processing the pixel values ​​of the second predicted image of the first reflection image model under the RGB channel; Represents the mean of the pixel values ​​of the second reflective label image under the R channel.

[0126] It should be noted that the second predicted image refers to the predicted image output by the first reflectance image processing model during model training. Based on the pixel values ​​of the second reflective label image in the RGB channels, the pixel values ​​of the first reflective label image in the RGB channels, and the pixel values ​​of the second predicted image of the first reflectance image processing model in the RGB channels, the input data for the loss function of the first reflectance image processing model is calculated. Furthermore, based on the input data, after the loss function of the first reflectance image processing model is run, a loss value corresponding to the loss function of the first reflectance image processing model can be output.

[0127] The technical solution of the embodiment of the present disclosure obtains multiple first samples, then processes the multiple first sample images and the multiple second sample images based on a Gaussian blur algorithm to obtain a first illumination label image corresponding to the first sample image and a second illumination label image corresponding to the second sample image. Furthermore, the first sample image and the first illumination label image are processed based on a logarithmic algorithm to obtain a first reflection label image corresponding to the first sample image, and the second sample image and the second illumination label image are processed based on a logarithmic algorithm to obtain a second reflection label image corresponding to the second sample image. Finally, based on each first illumination label image, each second illumination label image, each first reflection label image, and each second reflection label image, first target sample data for training the first image processing model is determined. This in turn determines sample data for training the first illumination image processing model, the first reflection image processing model, and the stitched image processing model. The first image processing model, the first illumination pattern processing model, the first reflection pattern processing model, and the stitched image processing model are trained to obtain the trained first image processing model, the first illumination pattern processing model, the first reflection pattern processing model, and the stitched image processing model. This increases the diversity of the training data, thereby improving the generalization ability of each model and enhancing the accuracy and robustness of image processing. Furthermore, loss values ​​are calculated based on different loss functions, which is more flexible and adaptable to the needs of different tasks, improving the adaptability of each model and the stability of model training.

[0128] Example 3

[0129] Figure 7 is a structural diagram of an image enhancement device provided by an embodiment of the present disclosure, such as Figure 7 As shown, the apparatus includes: a first image acquisition module 310 , a first illumination pattern output module 320 , a second illumination pattern output module 330 , a first stitched image determination module 340 , a second stitched image determination module 350 and a second image output module 360 ​​.

[0130] A first image acquisition module is configured to acquire a first image captured within a preset exposure value range; a first illumination pattern output module is configured to input the first image into a pre-trained first image processing model and output a first illumination pattern and a first reflection pattern corresponding to the first image; a second illumination pattern output module is configured to input the first illumination pattern into a pre-trained first illumination pattern processing model and output a second illumination pattern corresponding to the first illumination pattern, and input the first reflection pattern into a pre-trained first reflection pattern processing model and output a second reflection pattern corresponding to the first reflection pattern; a first stitched image determination module is configured to determine a difference illumination pattern based on the first illumination pattern and the second illumination pattern, and determine a first stitched image based on the difference illumination pattern and the second reflection pattern; a second stitched image determination module is configured to determine a second stitched image based on the first stitched image, the second illumination pattern, and the first reflection pattern; a second image output module is configured to input the second stitched image into a pre-trained stitched image processing model to obtain a second image corresponding to the first image; wherein the exposure value of the second image is a target exposure value.

[0131] The technical solution of the embodiment of the present disclosure is to obtain a first image captured under a preset exposure value range, input the first image into a pre-trained first image processing model, and output a first illumination map and a first reflection map corresponding to the first image. Then, the first illumination map is input into a pre-trained first illumination map processing model, and output a second illumination map corresponding to the first illumination map. The first reflection map is input into a pre-trained first reflection map processing model, and output a second reflection map corresponding to the first reflection map. Then, a difference illumination map is determined based on the first illumination map and the second illumination map, and a first stitched image is determined based on the difference illumination map and the second reflection map. Further, a second stitched image is determined based on the first stitched image, the second illumination map and the first reflection map. Finally, the second stitched image is input into a pre-trained stitched image processing model to obtain a second image corresponding to the first image, which solves the problems of large noise performance, reduced image quality, loss of details and color distortion when using curve transformation or histogram equalization to change the pixel distribution of the image to improve the brightness of low-brightness images. The embodiment of the present invention separates the illumination image and the reflection image and then reconstructs them to output a brightness-enhanced image, thereby achieving an effect of improving vision.

[0132] Based on the above technical solutions, the exposure value of the first illumination image is a first exposure value, the exposure value of the first reflection image is a second exposure value, the exposure value of the second illumination image is a third exposure value, and the exposure value of the second reflection image is a fourth exposure value.

[0133] On the basis of the above technical solutions, the first lighting pattern output module includes: a first image to be used determination submodule and a convolution processing submodule.

[0134] a first image to be used determining submodule configured to, for at least one pixel point in the first image, obtain a first pixel value corresponding to a first component, a second pixel value corresponding to a second component, and a third pixel value corresponding to a third component in the pixel point, and use the maximum value of the first pixel value, the second pixel value, and the third pixel value as the pixel value of the pixel point, thereby obtaining the first image to be used; wherein the first component is a red component, the second component is a green component, and the third component is a blue component;

[0135] The convolution processing submodule is used to obtain a first illumination image corresponding to the first image after passing the first image to be used through at least three convolution layers, and to obtain a first reflection image corresponding to the first image after passing the second image to be used through at least three convolution layers; wherein the second image to be used is an image obtained by normalizing the first image.

[0136] Based on the above technical solutions, according to the processing order of the input image, the model structures of the first illumination image processing model and the first reflection image processing model are respectively the first convolution layer, the first spatial attention layer, the second convolution layer and the second spatial attention layer.

[0137] On the basis of the above technical solutions, according to the processing order of the input image, the model structure of the spliced ​​image processing model is the third convolution layer, the first mixed attention layer, the fourth convolution layer and the second mixed attention layer.

[0138] On the basis of the above technical solutions, the device further includes a first sample acquisition module, a Gaussian fuzzy algorithm processing module, a logarithmic algorithm processing module and a first target sample data determination module.

[0139] A first sample acquisition module is configured to acquire a plurality of first samples, wherein the first sample includes at least one first sample image having an exposure value within the preset exposure range, and at least one second sample image having an exposure value of the target exposure value;

[0140] a Gaussian blur algorithm processing module, configured to process the plurality of first sample images and the plurality of second sample images respectively based on the Gaussian blur algorithm to obtain a first illuminated label image corresponding to the first sample image and a second illuminated label image corresponding to the second sample image;

[0141] a logarithmic algorithm processing module, configured to process the first sample image and the first illuminated label image based on a logarithmic algorithm to obtain a first reflective label image corresponding to the first sample image, and to process the second sample image and the second illuminated label image based on a logarithmic algorithm to obtain a second reflective label image corresponding to the second sample image;

[0142] The first target sample data determination module is used to determine first target sample data for training the first image processing model based on each first illuminated label image, each second illuminated label image, each first reflected label image, and each second reflected label image.

[0143] Based on the above technical solutions, the loss functions in the first image processing model, the first lighting image processing model and the stitching image processing model include one or more of a first sub-loss function, a second sub-loss function and a third sub-loss function.

[0144] Based on the above technical solutions, the first sub-loss function is:

[0145]

[0146] Among them, X' (i,j) is the predicted pixel value of each pixel in the first predicted image; X' 1(i,j) is the label pixel value of each pixel in the first label image; the second sub-loss function is:

[0147]

[0148] Among them, λ1 is the weight, β1 is the smoothness strength; the third sub-loss function is:

[0149]

[0150] Where X' is the first predicted image; X1' is the first label image; λ2 is the weight; Vgg k (·) represents a network that inputs the first predicted image or the first label image to extract the feature map of the kth layer; C represents the number of channels of the feature map; H represents the height of the feature map; and W represents the width of the feature map.

[0151] Based on the above technical solutions, the loss function in the first reflection image processing model is:

[0152]

[0153] in, Represents the pixel value of the second reflected label image under the RGB channel; Represents the pixel value of the first reflected label image under the RGB channel; R' t Pixel values ​​of the second predicted image of the first reflection image processing model under RGB channels; Represents the mean of the pixel values ​​of the second reflective label image under the R channel.

[0154] The image enhancement device provided in the embodiments of the present disclosure can execute the image enhancement method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0155] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0156] Example 4

[0157] Figure 8 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 8 , which shows an electronic device (eg Figure 8 The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a laptop computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), an in-vehicle terminal (such as an in-vehicle navigation terminal), and the like. Figure 8 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0158] like Figure 8 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the electronic device 500 are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An edit / output (I / O) interface 505 is also connected to the bus 504.

[0159] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0160] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0161] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0162] The electronic device provided by the embodiment of the present disclosure and the image enhancement method provided by the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0163] Example 5

[0164] An embodiment of the present disclosure provides a computer storage medium having a computer program stored thereon. When the program is executed by a processor, the image enhancement method provided in the above embodiment is implemented.

[0165] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0166] In some embodiments, the server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0167] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0168] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0169] Acquire a first image captured within a preset exposure value range;

[0170] Inputting the first image into a pre-trained first image processing model, and outputting a first illumination image and a first reflection image corresponding to the first image;

[0171] Inputting the first illumination pattern into a pre-trained first illumination pattern processing model, and outputting a second illumination pattern corresponding to the first illumination pattern; inputting the first reflection pattern into a pre-trained first reflection pattern processing model, and outputting a second reflection pattern corresponding to the first reflection pattern;

[0172] Determining a difference illumination map according to the first illumination map and the second illumination map, and determining a first stitched image based on the difference illumination map and the second reflection map;

[0173] A second stitched image determining module, configured to determine a second stitched image based on the first stitched image, the second illumination image, and the first reflection image;

[0174] The second stitched image is input into a pre-trained stitched image processing model to obtain a second image corresponding to the first image; wherein the exposure value of the second image is the target exposure value.

[0175] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0176] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0177] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0178] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0179] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0180] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0181] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0182] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. An image enhancement method, characterized in that: include: Acquire a first image captured within a preset exposure value range; Inputting the first image into a pre-trained first image processing model, and outputting a first illumination image and a first reflection image corresponding to the first image; Inputting the first illumination pattern into a pre-trained first illumination pattern processing model, and outputting a second illumination pattern corresponding to the first illumination pattern; inputting the first reflection pattern into a pre-trained first reflection pattern processing model, and outputting a second reflection pattern corresponding to the first reflection pattern; Determining a difference illumination map according to the first illumination map and the second illumination map, and determining a first stitched image based on the difference illumination map and the second reflection map; determining a second stitched image based on the first stitched image, the second illumination image, and the first reflection image; The second stitched image is input into a pre-trained stitched image processing model to obtain a second image corresponding to the first image; wherein the exposure value of the second image is the target exposure value.

2. The method according to claim 1, characterized in that The exposure value of the first illumination image is a first exposure value, the exposure value of the first reflection image is a second exposure value, the exposure value of the second illumination image is a third exposure value, and the exposure value of the second reflection image is a fourth exposure value.

3. The method according to claim 1, characterized in that Inputting the first image into a pre-trained first image processing model and outputting a first illumination image and a first reflection image corresponding to the first image includes: For at least one pixel point in the first image, after obtaining a first pixel value corresponding to a first component, a second pixel value corresponding to a second component, and a third pixel value corresponding to a third component in the pixel point, a maximum value among the first pixel value, the second pixel value, and the third pixel value is used as the pixel value of the pixel point, thereby obtaining a first image to be used; wherein the first component is a red component, the second component is a green component, and the third component is a blue component; After the first image to be used passes through at least three convolution layers, a first illumination image corresponding to the first image is obtained, and after the second image to be used passes through at least three convolution layers, a first reflection image corresponding to the first image is obtained; wherein, the second image to be used is an image obtained after the first image is normalized.

4. The method according to claim 1, wherein The first illumination image processing model and the first reflection image processing model include: According to the processing order of the input image, the model structures of the first illumination image processing model and the first reflection image processing model are respectively a first convolution layer, a first spatial attention layer, a second convolution layer and a second spatial attention layer.

5. The method according to claim 1, characterized in that The splicing image processing model includes: According to the processing order of the input image, the model structure of the spliced ​​image processing model is the third convolution layer, the first mixed attention layer, the fourth convolution layer and the second mixed attention layer.

6. The method according to claim 1, wherein The method further comprises: Acquire a plurality of first samples, wherein the first samples include at least one first sample image having an exposure value within the preset exposure range, and at least one second sample image having an exposure value of the target exposure value; Processing the plurality of first sample images and the plurality of second sample images respectively based on a Gaussian blur algorithm to obtain a first illuminated label image corresponding to the first sample image and a second illuminated label image corresponding to the second sample image; Processing the first sample image and the first illuminated label image based on a logarithmic algorithm to obtain a first reflective label image corresponding to the first sample image, and processing the second sample image and the second illuminated label image based on a logarithmic algorithm to obtain a second reflective label image corresponding to the second sample image; First target sample data for training the first image processing model is determined based on each of the first illuminated tag images, each of the second illuminated tag images, each of the first reflected tag images, and each of the second reflected tag images.

7. The method according to claim 1, characterized in that The loss functions in the first image processing model, the first lighting image processing model and the stitching image processing model include one or more of a first sub-loss function, a second sub-loss function and a third sub-loss function.

8. The method according to claim 7, characterized in that The first sub-loss function is: Among them, X' (i,j) is the predicted pixel value of each pixel in the first predicted image; X' 1(i,j) is the label pixel value of each pixel in the first label image; the second sub-loss function is: Among them, λ1 is the weight, β1 is the smoothness strength; the third sub-loss function is: Where X' is the first predicted image; X1' is the first label image; λ2 is the weight; Vgg k (·) represents a network that inputs the first predicted image or the first label image to extract the feature map of the kth layer; C represents the number of channels of the feature map; H represents the height of the feature map; and W represents the width of the feature map.

9. The method according to claim 1, characterized in that The loss function in the first reflection image processing model is: in, Represents the pixel value of the second reflected label image under the RGB channel; Represents the pixel value of the first reflected label image under the RGB channel; R' t Pixel values ​​of the second predicted image of the first reflection image processing model under RGB channels; Represents the mean of the pixel values ​​of the second reflective label image under the R channel.

10. An image enhancement device, characterized in that: include: A first image acquisition module is used to acquire a first image captured within a preset exposure value range; a first illumination pattern output module, configured to input the first image into a pre-trained first image processing model, and output a first illumination pattern and a first reflection pattern corresponding to the first image; a second illumination pattern output module, configured to input the first illumination pattern into a pre-trained first illumination pattern processing model and output a second illumination pattern corresponding to the first illumination pattern, and input the first reflection pattern into a pre-trained first reflection pattern processing model and output a second reflection pattern corresponding to the first reflection pattern; a first stitched image determining module, configured to determine a difference illumination image according to the first illumination image and the second illumination image, and determine a first stitched image based on the difference illumination image and the second reflection image; A second stitched image determining module, configured to determine a second stitched image based on the first stitched image, the second illumination image, and the first reflection image; The second image output module is used to input the second stitched image into a pre-trained stitched image processing model to obtain a second image corresponding to the first image; wherein the exposure value of the second image is the target exposure value.

Images