Image processing method, apparatus, device, and storage medium

Abstract

The application relates to an image processing method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a raw image to be processed, performing noise reduction processing on the raw image to be processed, then performing conversion processing to obtain an RGB image, performing brightness correction processing on the RGB image to obtain a target brightness correction image, and performing pixel dynamic range mapping processing on the target brightness correction image to obtain a target dynamic image, wherein the pixel dynamic range of the target dynamic image is smaller than the pixel dynamic range of the RGB image. Through the above process, noise reduction processing is first performed on the raw image, then high dynamic range image to low dynamic range pixel dynamic range mapping processing is performed, noise of the output image is ensured to be low, more image details are reserved on a limited dynamic range display device, and the image quality of the output image is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image processing method, device, equipment and storage medium. BACKGROUND

[0002] Raw format pictures are the most original image data information, and the JPEG pictures in our daily life are photos obtained by image compression from Raw format pictures, which loses a lot of important information.

[0003] In the prior art, noise reduction processing is generally performed on JPEG images, but due to the limitations of the existing noise reduction algorithm of JPEG and the information loss of JPEG images due to compression processing, the output JPEG images still have the problem of large noise.

[0004] With the development of image processing technology, high dynamic images are widely used because they can provide more dynamic range and image details. However, in some applications, the dynamic range of the display device for displaying images is limited dynamic range or low dynamic range, which is relative to high dynamic range, for example, CRT (Cathode Ray Tube) display, LCD display or projector, etc., which only have limited dynamic range. There are often situations where high dynamic images need to be displayed on the above limited dynamic range devices, at this time, it is necessary to process the high dynamic images into images that can be displayed on limited dynamic range devices and can retain image details, so that users can display the same effect as high dynamic images on limited dynamic range display devices.

[0005] In view of the above limitations in image processing, the output JPEG images currently have the problems of large noise and cannot well retain image details on limited dynamic range display devices. SUMMARY

[0006] Therefore, it is necessary to provide an image processing method, device, computer equipment and storage medium which can reduce the noise of the output image and retain more image details on limited dynamic range display devices to solve the above technical problems.

[0007] In a first aspect, the present application provides an image processing method, which comprises:

[0008] obtaining a to-be-processed Raw image;

[0009] performing noise reduction processing on the to-be-processed Raw image to obtain a noise reduction Raw image;

[0010] performing conversion processing on the noise reduction Raw image to obtain an RGB image;

[0011] performing brightness correction processing on the RGB image to obtain a target brightness correction image;

[0012] performing pixel dynamic range mapping processing on the target brightness correction image to obtain a target dynamic image, a pixel dynamic range of the target dynamic image being less than a pixel dynamic range of the RGB image.

[0013] In one of the embodiments, the denoising processing on the to-be-processed Raw image to obtain a denoised Raw image specifically includes:

[0014] inputting the to-be-processed Raw image into a preset image denoising model to perform denoising processing to obtain a denoised Raw image.

[0015] In one of the embodiments, the conversion processing on the denoised Raw image to obtain an RGB image specifically includes:

[0016] performing processing on the denoised Raw image by using an interpolation algorithm to obtain an RGB image.

[0017] In one of the embodiments, the brightness correction processing on the RGB image to obtain a brightness correction image specifically includes:

[0018] obtaining an image brightness statistical value corresponding to the RGB image;

[0019] obtaining an ambient light intensity statistical value, and obtaining a target brightness statistical value corresponding to a shooting environment according to the ambient light intensity statistical value; the ambient light intensity statistical value is a light intensity statistical value of a shooting environment where the RGB image is located;

[0020] determining a target brightness correction coefficient according to the target brightness statistical value and the image brightness statistical value;

[0021] performing brightness correction on the RGB image according to the target brightness correction coefficient to obtain a target brightness correction image.

[0022] In one of the embodiments, the determination of the target brightness correction coefficient according to the target brightness statistical value and the image brightness statistical value includes at least one of the following steps:

[0023] when the target brightness statistical value is greater than the image brightness statistical value, obtaining a brightness enhancement coefficient as the target brightness correction coefficient;

[0024] when the target brightness statistical value is less than the image brightness statistical value, obtaining a brightness attenuation coefficient as the target brightness correction coefficient.

[0025] In one of the embodiments, the target brightness correction coefficient comprises a first brightness correction coefficient, and the determining the target brightness correction coefficient according to the target brightness statistical value and the image brightness statistical value comprises:

[0026] calculating a brightness ratio of the target brightness statistical value and the image brightness statistical value;

[0027] logarithmically calculating the brightness ratio as a logarithm in a logarithm function to obtain the first brightness correction coefficient, wherein an index of the logarithm function is greater than 1.

[0028] In one of the embodiments, the target brightness correction coefficient further comprises a second brightness correction coefficient and a third brightness correction coefficient, and the determining the target brightness correction coefficient according to the target brightness statistical value and the image brightness statistical value comprises:

[0029] decreasing the first brightness correction coefficient to obtain the second brightness correction coefficient;

[0030] increasing the second brightness correction coefficient to obtain the third brightness correction coefficient;

[0031] the brightness correction of the RGB image according to the target brightness correction coefficient to obtain a target brightness correction image comprises:

[0032] respectively correcting the RGB image according to the first brightness correction coefficient, the second brightness correction coefficient and the third brightness correction coefficient to obtain a first brightness correction image corrected by the first brightness correction coefficient, a second brightness correction image corrected by the second brightness correction coefficient and a third brightness correction image corrected by the third brightness correction coefficient;

[0033] the pixel dynamic range mapping of the target brightness correction image to obtain a target dynamic image comprises:

[0034] respectively performing pixel dynamic range mapping on the first brightness correction image, the second brightness correction image and the third brightness correction image to obtain a first mapping dynamic image corresponding to the first brightness correction image, a second mapping dynamic image corresponding to the second brightness correction image and a third mapping dynamic image corresponding to the third brightness correction image;

[0035] performing fusion processing on the first mapping dynamic image, the second mapping dynamic image and the third mapping dynamic image to obtain a target dynamic image.

[0036] In one of the embodiments, the fusion processing on the first mapping dynamic image, the second mapping dynamic image and the third mapping dynamic image to obtain a target dynamic image comprises:

[0037] fusing the first mapping dynamic image, the second mapping dynamic image and the third mapping dynamic image to obtain a fused image;

[0038] obtaining an image region of the fused image;

[0039] obtaining a reference image region;

[0040] calculating a local mapping gain value of the image region of the fused image relative to the reference image region;

[0041] performing tone mapping processing on the RGB image according to the local mapping gain value to obtain a target dynamic image.

[0042] In a second aspect, the present application provides an image processing device, which comprises:

[0043] an image obtaining module, configured to obtain a to-be-processed Raw image;

[0044] a noise reduction module, configured to perform noise reduction processing on the to-be-processed Raw image to obtain a noise-reduced Raw image;

[0045] a conversion processing module, configured to perform conversion processing on the noise-reduced Raw image to obtain an RGB image;

[0046] a luminance correction module, configured to perform luminance correction processing on the RGB image to obtain a target luminance correction image;

[0047] a dynamic mapping module, configured to perform pixel dynamic range mapping on the target luminance correction image to obtain a target dynamic image.

[0048] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0049] obtaining a to-be-processed Raw image;

[0050] performing noise reduction processing on the to-be-processed Raw image to obtain a noise-reduced Raw image;

[0051] performing conversion processing on the noise-reduced Raw image to obtain an RGB image;

[0052] performing luminance correction processing on the RGB image to obtain a target luminance correction image;

[0053] performing pixel dynamic range mapping processing on the target luminance correction image to obtain a target dynamic image, a pixel dynamic range of the target dynamic image being less than a pixel dynamic range of the RGB image.

[0054] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0055] obtaining a to-be-processed Raw image;

[0056] performing noise reduction processing on the to-be-processed Raw image to obtain a noise-reduced Raw image;

[0057] performing conversion processing on the noise-reduced Raw image to obtain an RGB image;

[0058] performing luminance correction processing on the RGB image to obtain a target luminance correction image;

[0059] performing pixel dynamic range mapping processing on the target luminance correction image to obtain a target dynamic image, a pixel dynamic range of the target dynamic image being less than a pixel dynamic range of the RGB image.

[0060] The image processing method, device, computer device and storage medium described above, by obtaining a to-be-processed Raw image, performing noise reduction processing on the to-be-processed Raw image, and then performing conversion processing to obtain an RGB image, performing luminance correction processing on the RGB image to obtain a target luminance correction image, and performing pixel dynamic range mapping processing on the target luminance correction image to obtain a target dynamic image, wherein a pixel dynamic range of the target dynamic image is less than a pixel dynamic range of the RGB image. Through the above process, the Raw image is first processed by noise reduction, and then the pixel dynamic range mapping processing from a high dynamic range image to a low dynamic range is performed, so as to ensure that the output image has low noise and retains more image details on a limited dynamic range display device, thereby improving the image quality of the output image. BRIEF DESCRIPTION OF DRAWINGS

[0061] Figure 1 An application environment diagram of the image processing method in one embodiment;

[0062] Figure 2 A flowchart of the image processing method in one embodiment;

[0063] Figure 3 A flowchart of the image noise reduction model generation method in another embodiment;

[0064] Figure 4A flowchart of a method for determining a target brightness correction coefficient according to a target brightness statistical value and an image brightness statistical value in another embodiment;

[0065] Figure 5 A flowchart of a method for fusing a first mapped dynamic image, a second mapped dynamic image and a third mapped dynamic image to obtain a target dynamic image in another embodiment;

[0066] Figure 6 A flowchart of a method for obtaining an ambient light intensity statistical value in another embodiment;

[0067] Figure 7 A block diagram of a structure of an image processing device in an embodiment;

[0068] Figure 8 An internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0069] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not used to limit the present application.

[0070] The image processing method provided by the present application can be applied to, for example, Figure 1The application environment includes an image acquisition device 102 and a terminal 104, wherein the image acquisition device 102 and the terminal 104 are in communication connection. After the image acquisition device 102 acquires a dynamic image, the dynamic image is transmitted to the terminal 104. The terminal 104 obtains a Raw image to be processed. The terminal 104 can first perform noise reduction and conversion processing on the Raw image to be processed to obtain an RGB image. Then, the terminal 104 obtains image brightness statistical values and environmental light intensity statistical values corresponding to the RGB image from the obtained Raw image. The terminal 104 obtains target brightness statistical values corresponding to a shooting environment according to the environmental light intensity statistical values. The environmental light intensity statistical values are statistical values of light intensity of the shooting environment where the RGB image is located. The terminal 104 determines a target brightness correction coefficient according to the target brightness statistical values and the image brightness statistical values. The terminal 104 performs brightness correction on the RGB image according to the target brightness correction coefficient to obtain a target brightness correction image. The terminal 104 performs pixel dynamic range mapping on the target brightness correction image to obtain a target dynamic image. The pixel dynamic range of the target dynamic image is smaller than the pixel dynamic range of the RGB image. The image acquisition device 102 can be, but is not limited to, various image acquisition devices, and can be distributed outside the terminal 104 or inside the terminal 104. For example, various cameras, scanners, various cameras, and image acquisition cards distributed outside the terminal 104. The terminal 104 can be, but is not limited to, various cameras, personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices.

[0071] It can be understood that the method provided by the embodiments of the present application can also be executed by a server.

[0072] In one embodiment, as shown in Figure 2 , an image processing method is provided. The method is applied to a terminal in Figure 1 for example, and includes the following steps:

[0073] S201, obtaining a Raw image to be processed.

[0074] The Raw image is original data converted by a CMOS or CCD image sensor from a light source signal into a digital signal, is lossless, and contains original color information of an object. A data format of the Raw image generally adopts a Bayer array arrangement. A color filter array (CFA) is generated through a filter. Since the human eye is more sensitive to green bands, the Bayer array data format contains 50% green information and 25% red and blue information. The Bayer array is a 4x4 array, which is composed of 8 green, 4 blue, and 4 red pixels. When converting a grayscale image into a color image, 9 operations are performed in a 2x2 matrix to finally generate a color image. The Bayer array generally has four formats: RGGB (a), BGGR (b), GBRG (c), and GRBG (d).

[0075] Specifically, when the terminal receives the image processing instruction, the image processing instruction carries an image identifier of the image to be processed. Through the image identifier, the terminal can obtain the Raw image to be processed from the stored image. The terminal obtains the Raw image to be processed to facilitate subsequent processing of the Raw image to be processed.

[0076] The Raw image to be processed can be obtained by shooting with any device having a shooting function, such as a digital camera, a panoramic camera, a mobile phone, a tablet computer, a sports camera, or the like. In addition, the Raw image to be processed can also be obtained by any image processing method, such as image transformation, image stitching, image segmentation, image synthesis, image compression, image enhancement, image restoration, or the like. In addition, the Raw image to be processed can be a panoramic image or a normal planar image.

[0077] S202, performing noise reduction processing on the Raw image to be processed to obtain a noise-reduced Raw image.

[0078] The noise reduction processing on the Raw image to be processed can be performed by using various noise reduction methods. In an embodiment, the Raw image to be processed can be input into a preset image noise reduction model to perform noise reduction processing to obtain a noise-reduced Raw image.

[0079] In an embodiment, as shown in FIG. 2, a preset image noise reduction model generation method is provided, including the following steps: Figure 3

[0080] S301, obtaining N first Raw images of different Bayer arrays under the same scene.

[0081] ​The first Raw image can be obtained by photographing through any device with photographing function, such as a digital camera, a panoramic camera, a mobile phone, a tablet computer, an action camera, etc. In addition, the first Raw image can also be a Raw image obtained through any image processing method, such as image transformation, image stitching, image segmentation, image synthesis, image compression, image enhancement, image restoration, etc. In addition, the first Raw image can be a panoramic image or a normal planar image.

[0082] In order to obtain N first Raw images of different Bayer arrays in the same scene, a plurality of methods can be used, for example: in a first method, a plurality of Raw images in the same scene can be continuously photographed by one photographing device, or a plurality of Raw images in the same scene can be photographed by a plurality of photographing devices, and then N first Raw images with consistent Bayer arrays can be selected from the plurality of Raw images; in a second method, N Raw images in the same scene can be continuously photographed by one photographing device, or a plurality of Raw images in the same scene can be photographed by a plurality of photographing devices, and then N first Raw images with consistent Bayer arrays can be obtained through a Bayer array conversion method; in a third method, N first Raw images of different Bayer arrays in the same scene can be continuously photographed by one photographing device or a plurality of photographing devices. It should be understood that different combinations of the above-mentioned methods or any other method can also be used to obtain N first Raw images of different Bayer arrays in the same scene, which will not be listed one by one here.

[0083] S302, the N first Raw images of different Bayer arrays are fused respectively to obtain N second Raw images corresponding thereto.

[0084] In one embodiment, S302 specifically includes the following steps:

[0085] S3021, any one of the N first Raw images is selected as a reference image in sequence;

[0086] S3022, differences between the reference image and other N-1 first Raw images are detected respectively;

[0087] S3023, the reference image is processed according to the differences to obtain a to-be-fused image;

[0088] S3024, the to-be-fused image is weighted and fused pixel by pixel to obtain a corresponding second Raw image.

[0089] In one embodiment, in step S3021, any one of the N first Raw images is selected as a reference image in turn, and one way that can be used is that the N first Raw images are marked as the 1st image, the 2nd image, the 3rd image,..., and the Nth image in turn, and then the 1st image, the 2nd image, the 3rd image,..., and the Nth image are selected as the reference image in turn.

[0090] In one embodiment, in step S3022, the difference points of the reference image and the other N-1 first Raw images are detected respectively, and the specific steps include the following steps:

[0091] The N first Raw images are all divided into G grids, where G is an integer greater than or equal to 2.

[0092] An alignment algorithm is used to calculate the difference points of each grid content of the reference image and the corresponding grid content of the other N-1 first Raw images.

[0093] In one embodiment, the N first Raw images are all divided into G grids, where G is an integer greater than or equal to 2. The image can be divided into multiple grids in multiple ways, and the common grids can be divided into two categories: structured grid and unstructured grid. The connection between the unit nodes of the structured grid has only a limited number of ways, and all internal nodes in the generated grid body area have the same number of adjacent units and the same number of adjacent nodes. For a two-dimensional plane or a three-dimensional curved surface, the grid unit generated by the structured grid is generally a quadrilateral, and the three-dimensional entity space is mainly a hexahedron. In the unstructured grid, the nodes between the grid units can be connected in any form, and do not need to have the same adjacent units. The number of connections between different nodes in the grid area can be different. For a two-dimensional plane or a three-dimensional curved surface, the grid unit generated by the unstructured grid is generally a triangle, and the three-dimensional entity space is mainly a tetrahedron. Common structured grid generation methods generally include mapping method, geometric decomposition method, etc. The unstructured grid generation method generally includes octree method, quadtree method, Delaunay triangulation method, etc.

[0094] It should be understood that the above is only to list some common grid generation methods, and the embodiments of the present application are not limited thereto. The embodiments of the present application can divide the N first Raw images into G grids by using any one or more of the above methods, or can use other grid generation methods in the prior art, which will not be described here.

[0095] Further, it should be understood that the G grids divided can be of any shape, such as an AXA grid or an AXB grid, where A and B are not equal and are both integers greater than or equal to 2.

[0096] In one embodiment, a difference point between each grid content of the reference image and the corresponding grid content of the other N-1 first Raw images is calculated by using an alignment algorithm, wherein the alignment algorithm can be any image alignment algorithm, for example, a method based on pixel value cross-correlation, a sequential similarity detection algorithm, a phase correlation algorithm based on Fourier transform, a mutual information based alignment algorithm, an image alignment method based on optimization, for example, Lucas-Kanade algorithm and its improved algorithm.

[0097] In one embodiment, in step S3023, the reference image is processed according to the difference point to obtain a to-be-fused image, which specifically includes the following steps:

[0098] If it is detected that the difference point is caused by movement in a random direction, the corresponding grid in the N first Raw images is deleted to obtain N to-be-fused images.

[0099] If it is detected that the difference point is caused by movement in a single direction, the corresponding grids of the N-1 first Raw images are translated in the opposite direction of the single direction relative to the reference image, and the N-1 translated images and the reference image are taken as N to-be-fused images.

[0100] In one embodiment, the difference caused by movement in a random direction is relative to the difference caused by movement in a single direction, that is, the difference caused by movement in a random direction contains movement difference in at least two directions. Further, the difference caused by movement refers to the offset difference between two grids in a set direction, for example, grid 1 and grid 2, grid 1 is a grid on the corresponding first Raw image, and grid 2 is a corresponding grid on the reference image, both of which are set to detect the difference point in the X and Y directions. If there is an offset difference between grid 1 and grid 2 in the X and Y directions, it means that the difference point between grid 1 and grid 2 is caused by movement in a random direction, and grid 1 and grid 2 are deleted. Further, as described above, if there is an offset difference between grid 1 and grid 2 in the X or Y direction, it means that the difference point between grid 1 and grid 2 is caused by movement in a single direction, and grid 2 is translated in the opposite direction of the single offset direction relative to grid 1 to ensure that there is no difference point between the translated grid 1 and grid 2 in any direction.

[0101] In one embodiment, in step 3024, the to-be-fused images are subjected to pixel-by-pixel weighted fusion processing to obtain a corresponding second Raw image, wherein the pixel-by-pixel weighted fusion processing is generally direct weighted calculation on the pixel points of the images, and the algorithm for selecting the weight directly affects the effect of the fused image. Based on how to effectively select the fusion weight coefficient, the pixel-by-pixel weighted fusion processing can be generally divided into an average weighted fusion algorithm (directly taking the average of the corresponding pixels of two images for processing), a multi-scale weighted gradient fusion algorithm, principal component analysis (PCA), and the like.

[0102] It should be understood that the pixel-by-pixel weighted fusion processing in step S3024 is not the only way of the fusion processing in step S302, but is only one optional way. The pixel-by-pixel weighted fusion processing can also be replaced by other pixel-level image fusion methods, such as a simple pixel-by-pixel addition fusion processing and a multi-scale transformation-based fusion method. The pixel-by-pixel addition fusion processing generally refers to the addition operation of the pixels at the corresponding positions of two or more images of the same size to generate a new image containing the information of the two or more images. It should be understood that the addition of the pixel values at the corresponding coordinate positions of two or more 255 grayscale level images will inevitably exceed the maximum grayscale representation range 255. Obviously, the result of the image addition operation needs to be processed. There are three basic methods: one is to take the average of the two or more pixel grayscale values after addition as the addition result; one is to make a proportional reduction according to the minimum and maximum values of the addition results of all pixel grayscale values of the two or more images, so that the result grayscale value meets the grayscale value range of 0 to 255; and the other is to take 255 when the value after the addition of the two or more pixel grayscale values exceeds 255.

[0103] Further, in the embodiment of the present application, the multiple to-be-fused images can be subjected to pixel-by-pixel average weighted fusion processing, and then the average of the multiple pixel grayscale values after addition is taken as the addition result, that is, as the pixel grayscale value of the corresponding second Raw image. In this process, because the average of the multiple pixel grayscale values after addition is taken as the pixel grayscale value of the corresponding second Raw image, the random noise between the to-be-fused images can be effectively reduced.

[0104] S303, subtracting the corresponding second Raw image from the first Raw image pixel by pixel to obtain a corresponding residual noise image.

[0105] In an embodiment, the first Raw image is subtracted from the corresponding second Raw image pixel by pixel to obtain a corresponding residual noise image, wherein the pixel-by-pixel subtraction generally refers to a method of subtracting the pixel values of corresponding positions of two images of the same size to generate a new image containing information of the two images. When the result of subtracting the pixel values of corresponding coordinates of two 255 gray scale images is greater than or equal to zero, the gray scale value of the corresponding position of the result image is taken as the result. When the result of subtraction is less than zero, the negative value is generally taken as the result. Of course, for some special application purposes, the absolute value can also be taken as the result. Further, in the embodiment of the present application, the first Raw image is subtracted from the corresponding second Raw image pixel by pixel to obtain a corresponding residual noise image, wherein the residual noise image contains the difference information of the first Raw image and the second Raw image.

[0106] S304, based on the first Raw image and the residual noise image, a sample image database is established.

[0107] In an embodiment, the first Raw image is subtracted from the corresponding second Raw image pixel by pixel to obtain a corresponding residual noise image, wherein the pixel-by-pixel subtraction generally refers to a method of subtracting the pixel values of corresponding positions of two images of the same size to generate a new image containing information of the two images. When the result of subtracting the pixel values of corresponding coordinates of two 255 gray scale images is greater than or equal to zero, the gray scale value of the corresponding position of the result image is taken as the result. When the result of subtraction is less than zero, the negative value is generally taken as the result. Of course, for some special application purposes, the absolute value can also be taken as the result. Further, in the embodiment of the present application, the first Raw image is subtracted from the corresponding second Raw image pixel by pixel to obtain a corresponding residual noise image, wherein the residual noise image contains the difference information of the first Raw image and the second Raw image.

[0108] If the Bayer arrays of the first Raw image and the residual noise image are inconsistent, the first Raw image and the residual noise image are converted to obtain the first Raw image and the residual noise image with consistent Bayer arrays.

[0109] Wherein, because the Raw image Bayer array generally has four formats: RGGB (a), BGGR (b), GBRG (c), and GRBG (d). In an embodiment, the conversion process, i.e., the conversion of the Bayer array format of the first Raw image and the corresponding residual noise image, aims to keep the Bayer array format of the first Raw image and the corresponding residual noise image in the sample image database consistent; further, in order to clearly illustrate how to perform the conversion process, for example, if the Bayer array format of the first Raw image is "RGGB", when the Bayer array format of the corresponding residual noise image of the first Raw image is not "RGGB", the conversion process can be performed in the following manner: when the Bayer array format of the corresponding residual noise image is "GBRG", one pixel is added to the upper and lower boundaries of the corresponding residual noise image; when the Bayer array format of the corresponding residual noise image is "GRBG", one pixel is added to the left and right boundaries of the corresponding residual noise image; and when the Bayer array format of the corresponding residual noise image is "BGGR", one pixel is added to the upper, lower, left, and right boundaries of the corresponding residual noise image.

[0110] Further, in order to establish the sample image database, the image data sets in M groups of scenes can be obtained by the above-mentioned method of obtaining N images in the same scene, and details are not repeated herein, wherein M is an integer greater than or equal to 2.

[0111] S305, based on the sample image database, model training is performed to obtain an image denoising model.

[0112] In an embodiment, based on the sample image database, model training is performed to obtain an image denoising model, specifically including:

[0113] The first Raw image with the consistent Bayer array is taken as input image data, and the residual noise image corresponding to the first Raw image with the consistent Bayer array is taken as corresponding output image data, and model training is performed to obtain an image denoising model.

[0114] In an embodiment, after the construction of the sample image data set is completed, the first Raw image set can be taken as training input, and the corresponding residual noise image set can be taken as target output, and model training is performed according to a pre-set training algorithm to train an image denoising model for image denoising processing.

[0115] The training algorithm is a machine learning algorithm, which can process data through continuous feature learning. The machine learning algorithm can include a decision tree algorithm, a logistic regression algorithm, a Bayesian algorithm, a neural network algorithm (which can include a deep neural network algorithm, a convolutional neural network algorithm, and a recurrent neural network algorithm), a clustering algorithm, and the like.

[0116] It should be noted that the selection of the training algorithm for training the image denoising model can be selected by a person skilled in the art according to actual needs. For example, the embodiment of the present application can select a convolutional neural network algorithm to train the model, so as to obtain the image denoising model.

[0117] It should be understood that the sample image data can include a set of image data under M groups of scenes, each group of scenes including N first Raw images with inconsistent Bayer arrays. The sample image data includes the first Raw images and the corresponding residual noise images under the M groups of scenes. Further, when the values of M and N become larger and larger, the sample image database contains a large amount of training data, and the obtained image denoising model is more and more accurate.

[0118] In one embodiment, it is first determined whether the Bayer array of the to-be-processed Raw image is consistent with the Bayer array of the preset image denoising model obtained in steps S301-S305. If not, the conversion processing in step S304 is used (not described again here) to make the Bayer array of the to-be-processed Raw image consistent with the Bayer array of the image denoising model, and then the to-be-processed Raw image is input into the preset image denoising model obtained in steps S301-S305 to obtain the residual noise image corresponding to the to-be-processed Raw image. The Bayer array of the residual noise image is converted to be consistent with the to-be-processed Raw image, and then the final denoising Raw image is obtained through the pixel-by-pixel weighted fusion processing.

[0119] In an embodiment, after the steps S301-S302, the obtained second Raw image is taken as an output image, and the corresponding first Raw image is taken as an input image, and a preset training algorithm is used for model training, so as to obtain an image denoising model for image denoising processing. It should be understood that the model training herein is the same as that in step S305, and will not be described herein again. It is still determined whether the Bayer array of the to-be-processed Raw image is consistent with the image denoising model obtained in the embodiment. If not, the conversion processing in step S304 (which will not be described herein again) is used to make the Bayer array of the to-be-processed Raw image consistent with the Bayer array of the preset image denoising model, and then the to-be-processed Raw image is input into the preset image denoising model obtained in the embodiment, so that a final denoised Raw image of the to-be-processed Raw image can be directly obtained.

[0120] It should be understood that there are various methods for generating an image denoising model, and the above embodiments only list two image denoising model generation methods and corresponding Raw image denoising methods. However, the preset image denoising model in S202 is not limited to the above embodiments, that is, the Raw image denoising method in S202 is not limited to the denoising method in the above embodiments, and a person skilled in the art can also use other Raw image denoising methods to achieve the purpose of Raw image denoising.

[0121] S203, performing conversion processing on the denoised Raw image to obtain an RGB image.

[0122] In an embodiment, the interpolation algorithm is used to process the denoised Raw image to obtain an RGB image.

[0123] In an embodiment, the interpolation algorithm is used to process the denoised Raw image to obtain an RGB image. The interpolation algorithm can be a nearest neighbor interpolation method, a bilinear interpolation, a bicubic interpolation, a trilinear interpolation, a Demosaic algorithm, etc. Because each pixel in the original Raw image only contains one of the R / G / B components, in order to obtain an RGB image, the other two components missing from each pixel need to be supplemented by an interpolation algorithm, so as to obtain an RGB image.

[0124] S204, performing brightness correction processing on the RGB image to obtain a target brightness correction image.

[0125] In an embodiment, the brightness correction processing is performed on the RGB image to obtain a target brightness correction image. Specifically, the following steps can be used:

[0126] S2041, obtaining an image brightness statistical value of the RGB image.

[0127] S2042, acquire an ambient light intensity statistical value, and acquire a standard brightness statistical value corresponding to a shooting environment according to the ambient light intensity statistical value; the ambient light intensity statistical value is a light intensity statistical value of a shooting environment in which the RGB image is located;

[0128] S2043, determine a target brightness correction coefficient according to the standard brightness statistical value and the image brightness statistical value;

[0129] S2044, correct the brightness of the RGB image according to the target brightness correction coefficient to obtain a target brightness correction image.

[0130] In an embodiment, in step S2041, the image brightness statistical value refers to a comprehensive quantitative performance of image brightness, and the overall situation of image brightness can be obtained from the image brightness statistical value. The statistical value is obtained through statistics, and for example, can be an average value or a median value.

[0131] Specifically, after the terminal acquires the RGB image, the brightness histogram is obtained by converting the RGB image into a grayscale image. The brightness histogram can represent the number of pixels of each brightness level of the image, and the average brightness of the RGB image is obtained through the brightness histogram.

[0132] In an embodiment, in step S2042, the ambient light intensity statistical value refers to a comprehensive quantitative performance of ambient light intensity, and the overall situation of ambient light intensity can be obtained through the ambient light intensity statistical value. The statistical value is obtained through statistics, and for example, can be an average value or a median value.

[0133] Specifically, after the terminal acquires the image brightness statistical value and the ambient light intensity statistical value corresponding to the RGB image, the target brightness statistical value corresponding to the ambient light intensity statistical value can be obtained according to the corresponding relationship between the ambient light intensity statistical value and the target brightness statistical value.

[0134] In an embodiment, the exposure parameter can be obtained according to the sensitivity, the shutter speed, and the aperture value, the image brightness statistical value is divided by the exposure parameter as the true number of the logarithmic function, the logarithmic calculation is performed, and the ambient light intensity statistical value is obtained. The ambient light intensity statistical value is represented as EE, the image brightness statistical value is represented as V0, the target brightness statistical value is represented as V, I represents the sensitivity, s is the shutter speed unit in seconds, a is the aperture value, and the relationship between the ambient light intensity statistical value and the image brightness statistical value can be represented as:

[0135] EE = log2(V0 / (I * s / a 2 )

[0136] The different ambient light intensity statistical values have different target brightness statistical values. Assuming that the target brightness statistical value is V, there is a one-to-one correspondence between the ambient light intensity statistical value EE and the target brightness statistical value V. It can be understood that the image brightness that a user likes is different under different ambient light intensities. For example, people like a picture with a brighter picture under sufficient light on a sunny day, so the target brightness statistical value V corresponding to the ambient light intensity statistical value EE is relatively large, and for an image taken outdoors at night, the target brightness statistical value V corresponding to the ambient light intensity statistical value EE of the image with a suitable brightness is relatively small.

[0137] In an embodiment, the one-to-one correspondence between the ambient light intensity statistical value EE and the target brightness statistical value V can be denoted as f. For example, assuming that the image pixel point brightness value range is between 0 and 1, f can be a mapping table, as shown in Table 1, which is part of the mapping table f:

[0138] EE(cd / m 2 )]]> -12 -10 -8 -4 -2 3 5 [CD / m 2 )]]> 0.025 0.05 0.2 0.25 0.375 0.5 0.5

[0139] Table 1 Ambient light intensity statistical value and target brightness statistical value mapping table

[0140] From Table 1, the numerical relationship between the different ambient light intensity statistical values EE and the corresponding target brightness statistical values V can be obtained, and the numerical relationship is unique. Based on this one-to-one correspondence, the target brightness statistical value V can be expressed as f(EE), and the target brightness statistical value f(EE) under a certain light intensity can be adjusted by adjusting the values in the correspondence, so that the image can achieve a brighter or darker effect.

[0141] In an embodiment, in step S2043, the brightness correction coefficient is a parameter for brightness correction of the image. The image can be adjusted to the appropriate brightness, such as lightening or darkening, by the brightness correction coefficient.

[0142] Specifically, after the terminal has obtained the target brightness statistical value and the image brightness statistical value, the target brightness correction coefficient can be determined through the functional relationship between the target brightness statistical value and the image brightness statistical value. For example, there is a ratio type functional relationship between the target brightness statistical value and the image brightness statistical value.

[0143] In an embodiment, the target brightness statistical value is expressed as f(EE), and the image brightness statistical value is expressed as V0. The target brightness correction coefficient is obtained through the functional relationship between the target brightness correction coefficient, the target brightness statistical value, and the image brightness statistical value. For example, the target brightness correction coefficient is β, and β can be expressed as: β = log2(f(EE) / V0).

[0144] In an embodiment, in step S2044, specifically, the terminal corrects the brightness of the RGB image according to the determined target brightness correction coefficient, and obtains a corrected RGB image as a target brightness correction image. For example, the brightness of each pixel value of the RGB image can be multiplied by the target brightness correction coefficient to obtain a corrected pixel value, and the image composed of the corrected pixel values is the target brightness correction image.

[0145] In an embodiment, the target brightness correction coefficient can be directly used to correct the brightness of the RGB image, so that the image brightness statistical value of the corrected RGB image is close to the target brightness statistical value. For example, assuming that each pixel value of the RGB image is X, and the target brightness correction coefficient e1 is used to correct the brightness of the RGB image, then the pixel value of the corresponding position of the obtained target brightness correction image is X*e1; or the final correction coefficient can be obtained by calculating a function corresponding to the target brightness correction coefficient, and the final correction coefficient is used to correct the brightness of the RGB image. For example, the function corresponding to the target brightness correction coefficient is an exponential function, the target brightness correction coefficient can be used as the index, and a preset value greater than 1 can be used as the base number for exponential calculation. For example, assuming that the preset value is 2, then the final correction coefficient is 2 e1 X*2 e1 .

[0146] S205, performing pixel dynamic range mapping on the target brightness correction image to obtain a target dynamic image, the pixel dynamic range of the target dynamic image being smaller than the pixel dynamic range of the RGB image.

[0147] Specifically, dynamic range mapping refers to mapping an image from one dynamic range to another dynamic range. The pixel dynamic range of the RGB image is greater than the pixel dynamic range of the target dynamic image. In order to adapt the pixel dynamic range of the image to a limited dynamic or low dynamic display device, it is necessary to perform pixel dynamic range mapping on the target brightness correction image obtained by the terminal, and convert the high dynamic image into a low dynamic image, so that the mapped image can adapt to the low dynamic display device.

[0148] In an embodiment, after the target brightness correction image is obtained by correcting the brightness of the RGB image using the brightness correction coefficient, the target brightness correction image at this time is still a high dynamic image, and it is necessary to convert it into a low dynamic image through dynamic range mapping, so that the low dynamic image can be used as the target dynamic image and can be applied to a low dynamic device.

[0149] In one embodiment, a gamma transform can be utilized to implement the conversion from a high dynamic image to a low dynamic image. For example, a high dynamic image having a pixel value range of 0 to 65535 is converted to a low dynamic image having a pixel value range of 0 to 255.

[0150] In the image processing method, the RGB image to be processed is obtained, and the image brightness statistical value of the RGB image is obtained. The ambient light intensity statistical value is obtained, the target brightness statistical value corresponding to the shooting environment is obtained according to the ambient light intensity statistical value, the target brightness correction coefficient is determined according to the target brightness statistical value and the image brightness statistical value, the RGB image is corrected by the target brightness correction coefficient, and the target brightness correction image is obtained. The correction parameter of the image can be determined by the ambient light intensity, and the brightness of the RGB image can be corrected by the correction parameter, so that the image with appropriate brightness and more details can be obtained. The target dynamic image is obtained by performing pixel dynamic range mapping on the target brightness correction image, and the pixel dynamic range of the target dynamic image is smaller than the pixel dynamic range of the RGB image, so that the conversion from the high dynamic image to the low dynamic image is realized. Through the above process, the image correction parameter is determined according to the ambient light intensity and the image brightness statistical value when the image is shot, and the image is corrected by the image correction parameter, so that the image processing process from the high dynamic image to the low dynamic image is realized on the basis of retaining more details and appropriate brightness, and the image processing effect is improved.

[0151] In one embodiment, the target brightness correction coefficient is determined according to the target brightness statistical value and the image brightness statistical value, and includes at least one of the following steps: when the target brightness statistical value is greater than the image brightness statistical value, a brightness enhancement coefficient is obtained as the target brightness correction coefficient; and when the target brightness statistical value is less than the image brightness statistical value, a brightness attenuation coefficient is obtained as the target brightness correction coefficient.

[0152] Specifically, the brightness enhancement coefficient causes brightness enhancement, and when the target brightness statistical value is greater than the image brightness statistical value, the target brightness correction coefficient has brightness enhancement on the image. At this time, the target brightness correction coefficient can be referred to as a brightness enhancement coefficient. The brightness enhancement coefficient can linearly enhance the image brightness, or can nonlinearly enhance the image brightness. The brightness enhancement coefficient and the brightness attenuation coefficient can be preset or calculated by a preset algorithm. For example, the brightness ratio of the target brightness statistical value to the image brightness statistical value can be calculated, the brightness ratio is taken as the logarithm in the logarithmic function to perform logarithmic calculation to obtain a first brightness correction coefficient, and the exponent of the logarithmic function is greater than 1.

[0153] In one embodiment, the target brightness statistical value can be represented as f(EE), the image brightness statistical value is V0, and the target brightness correction coefficient is α. α can be represented as:

[0154] a = 2 e

[0155] wherein e can be expressed as:

[0156] e = log2(f(EE) / V0)

[0157] When the target luminance statistical value is greater than the image luminance statistical value, f(EE) / V0 is a positive number greater than 1, at this time e is a positive number greater than 0, and then a is a positive number greater than 1, multiplying the pixel value of the image by a will increase the luminance, at this time e can be called a luminance enhancement coefficient, and the image luminance can be enhanced and adjusted. When the target luminance statistical value is less than the image luminance statistical value, f(EE) / V0 is a positive number less than 1, at this time e is a negative number less than 0, and then a is a positive number less than 1, at this time e can be called a luminance attenuation coefficient, and the image luminance can be attenuated.

[0158] In this embodiment, the target luminance statistical value and the image luminance statistical value can be used to determine the target luminance parameter, and the target luminance parameter can be used to enhance or attenuate the image luminance.

[0159] In one embodiment, the target luminance correction coefficient includes a first luminance correction coefficient, and determining the target luminance correction coefficient according to the target luminance statistical value and the image luminance statistical value includes: calculating a luminance ratio of the target luminance statistical value and the image luminance statistical value; and performing logarithmic calculation on the luminance ratio as the logarithm in the logarithmic function to obtain the first luminance correction coefficient.

[0160] Specifically, the terminal can obtain the first luminance correction coefficient through the target luminance statistical value and the image luminance statistical value, for example, first calculating the luminance ratio of the target luminance statistical value and the image luminance statistical value. For example, the target luminance statistical value is expressed as f(EE), the image luminance statistical value is V0, and the ratio is calculated as a, then a can be expressed as:

[0161] a = f(EE) / V0

[0162] The first luminance correction coefficient can be expressed as e1, wherein the exponent of the logarithmic function is greater than 1, for example, e1 is an exponential function with a certain integer as the base, and e1 can change with the change of the exponent of the exponential function. For example, e1 is a monotonically increasing logarithmic function with 2 as the base.

[0163] e1 = log2a;

[0164] In the embodiment, the exponent of the logarithmic function is greater than 1, so that the brightness ratio is in a positive correlation with the first brightness correction coefficient, the first brightness correction coefficient is calculated by the brightness ratio of the target brightness statistical value and the image brightness statistical value, so that the brightness ratio reflects the size relationship between the target brightness statistical value and the image brightness statistical value, when the target brightness statistical value is greater than the image brightness statistical value, the first brightness enhancement coefficient is the brightness enhancement coefficient. When the target brightness statistical value is less than the image brightness statistical value, the first brightness enhancement coefficient is the brightness attenuation coefficient. Therefore, the adjusted image is matched with the environment brightness of the shooting environment.

[0165] In one embodiment, as shown in FIG. 7, the target brightness correction coefficient further includes a second brightness correction coefficient and a third brightness correction coefficient; and determining the target brightness correction coefficient according to the target brightness statistical value and the image brightness statistical value includes: Figure 4

[0166] S401, the first brightness correction coefficient is reduced to obtain the second brightness correction coefficient.

[0167] Specifically, after obtaining the first brightness correction coefficient, the first brightness correction coefficient can be reduced based on the first brightness correction coefficient to obtain the second brightness correction coefficient.

[0168] In one embodiment, the first brightness correction coefficient can be reduced by reducing the corresponding percentage. For example, the original correction parameter is b, which can be reduced by 0.1b to obtain the second brightness correction coefficient b1=b-0.1b=0.9b after reduction.

[0169] In one embodiment, the first brightness correction coefficient can be reduced by reducing the corresponding coefficient value. For example, the original correction parameter is b, which can be reduced by a value m to obtain the second brightness correction coefficient b1=b-m, where m can be any positive number, for example, 3.

[0170] S402, the first brightness correction coefficient is increased to obtain the third brightness correction coefficient.

[0171] Specifically, after obtaining the first brightness correction coefficient, the first brightness correction coefficient can be increased based on the first brightness correction coefficient to obtain the third brightness correction coefficient.

[0172] In one embodiment, the first brightness correction coefficient can be increased by increasing the corresponding percentage. For example, the original correction parameter is b, which can be increased by 0.1b to obtain the third brightness correction coefficient b1=b+0.1b=1.1b after reduction. ​

[0173] In one embodiment, the increasing processing of the first brightness correction coefficient can be realized by increasing the corresponding coefficient value. For example, the original correction parameter is b, and the third brightness correction coefficient b1 = b + n can be obtained by increasing the value n, where n is any positive number, for example, 3.

[0174] S403, respectively according to the first brightness correction coefficient, the second brightness correction coefficient, the third brightness correction coefficient, the RGB image is corrected, the first brightness correction coefficient is corrected to obtain the first brightness correction image, the second brightness correction coefficient is corrected to obtain the second brightness correction image and the third brightness correction coefficient is corrected to obtain the third brightness correction image.

[0175] Specifically, after the terminal obtains the first brightness correction coefficient, the second brightness correction coefficient and the third brightness correction coefficient, the first brightness correction image, the second brightness correction image and the third brightness correction image are obtained by processing the RGB image respectively using the first brightness correction coefficient, the second brightness correction coefficient and the third brightness correction coefficient. For example, the first brightness correction coefficient is used as the target correction coefficient, and the corresponding first brightness correction image is used as the target correction image. Since the second brightness correction image is the correction image corresponding to the brightness correction coefficient after the decreasing processing, the second brightness correction image is a darker image. Similarly, the third brightness correction image is a brighter image.

[0176] In one embodiment, when the first brightness correction coefficient is larger, the third brightness correction coefficient can be configured to be closer to the first brightness correction coefficient; when the first brightness correction coefficient is smaller, the second brightness correction coefficient can be configured to be closer to the first brightness correction coefficient. Therefore, the details of each brightness level in the picture can be better balanced, so that the image after correction by the correction coefficient can reflect more details of the target image.

[0177] S404, respectively, the first brightness correction image, the second brightness correction image and the third brightness correction image are pixel dynamic range mapping, the first brightness correction image corresponds to the first mapping dynamic image, the second brightness correction image corresponds to the second mapping dynamic image and the third brightness correction image corresponds to the third mapping dynamic image.

[0178] In one embodiment, the first brightness correction image, the second brightness correction image and the third brightness correction image can be respectively pixel dynamic range mapping by gamma transformation, to obtain the first brightness correction image corresponding to the first mapping dynamic image, the second brightness correction image corresponding to the second mapping dynamic image and the third brightness correction image corresponding to the third mapping dynamic image. Thus, the high dynamic brightness correction image is transformed into the low dynamic range mapping dynamic image, and different brightness low dynamic range mapping dynamic images are obtained.

[0179] In one embodiment, the low dynamic range image can be an eight-bit low dynamic range image, the high dynamic range correction image can be a sixteen-bit high dynamic range image, the high dynamic image is a red-green-blue three-channel image with pixel value ranging between 0 and 65535, and the low dynamic range image is a red-green-blue three-channel image with pixel value ranging between 0 and 255.

[0180] S405, the first mapping dynamic image, the second mapping dynamic image and the third mapping dynamic image are fused to obtain a target dynamic image.

[0181] The fusion processing refers to fusing images of different brightness according to a certain image fusion method, so as to make the processed image have more rich image details.

[0182] Specifically, first, the first mapping dynamic image, the second mapping dynamic image and the third mapping dynamic image are down-sampled; according to the down-sampled first mapping dynamic image, the second mapping dynamic image and the third mapping dynamic image, a first weight map corresponding to the first mapping dynamic image, a second weight map corresponding to the second mapping dynamic image and a third weight map corresponding to the third mapping dynamic image are obtained; the down-sampled first mapping dynamic image, the second mapping dynamic image and the third mapping dynamic image are respectively converted into gray scale images, and the three gray scale images and the first weight map, the second weight map and the third weight map are multi-resolution fused to obtain a multi-resolution fused gray scale image; according to the gray scale image, and the three gray scale images and the first weight map, the second weight map and the third weight map converted from the down-sampled first mapping dynamic image, the second mapping dynamic image and the third mapping dynamic image, new weight maps are obtained through the following formula, which are a fourth weight map, a fifth weight map and a sixth weight map. Assuming that the new weight map is represented as w i ', the first weight map, the second weight map and the third weight map are represented as w i , i∈(1,2,3), the multi-resolution fused gray scale image is represented as I f , the gray scale images converted from the first mapping dynamic image, the second mapping dynamic image and the third mapping dynamic image are represented as I1, I2 and I3 respectively, and the new weight map is represented as w i ' can be obtained through the following formula:

[0183] w i '=kw i

[0184]

[0185] I f '=w1I1+w2I2+w3I3,i∈(1,2,3)

[0186] The new weight maps, the fourth weight map, the fifth weight map and the sixth weight map, are up-sampled respectively to form images with the same size as the first mapped dynamic image, the second mapped dynamic image and the third mapped dynamic image. The fourth weight map, the fifth weight map and the sixth weight map are fused with the first mapped dynamic image, the second mapped dynamic image and the third mapped dynamic image to obtain the final fused target dynamic image. It can be understood that the image fusion method described above can use other fusion methods that can achieve the same effect.

[0187] In one embodiment, the multi-resolution fusion method can also use a biorthogonal wavelet transform multi-resolution fusion method. The redundancy and complementary information of multiple images can be utilized so that the fused image can contain more abundant and comprehensive information.

[0188] In one embodiment, the first mapped dynamic image, the second mapped dynamic image and the third mapped dynamic image can be fused by a Laplacian pyramid weighted fusion method to obtain the target dynamic image.

[0189] In this embodiment, the second brightness correction coefficient and the third brightness correction coefficient are obtained by using the first brightness correction coefficient, and the first brightness correction image, the second brightness correction image and the third brightness correction image are obtained by using the three correction coefficients. The corresponding mapped dynamic images are obtained by using the first brightness correction image, the second brightness correction image and the third brightness correction image, and the target dynamic image is obtained by fusing the three mapped dynamic images. This can achieve the purpose of complementing details between different brightness images, using more values of lower brightness images for higher brightness images, and using more values of higher brightness images for lower brightness images, so that the target dynamic image can retain more image details and improve the image processing effect.

[0190] In one embodiment, as shown in FIG. 5, the fusion processing of the first mapped dynamic image, the second mapped dynamic image and the third mapped dynamic image to obtain the target dynamic image includes: Figure 5

[0191] S501, the first mapped dynamic image, the second mapped dynamic image and the third mapped dynamic image are fused to obtain a fused image.

[0192] Specifically, in order to retain more image details, the first mapped dynamic image, the second mapped dynamic image and the third mapped dynamic image are fused respectively to obtain a fused image.

[0193] ​In one embodiment, the first mapping dynamic image, the second mapping dynamic image and the third mapping dynamic image are all low dynamic images with pixel values between 0 and 255, and the fusion processing image is also a low dynamic image with pixel values between 0 and 255.

[0194] S502, obtaining an image region of the fusion processing image.

[0195] Specifically, the fusion processing image retains the highlights and shadow details in the high dynamic range image, but due to the color information in some highlight regions such as color signboards, the corrected images after different brightness correction will show different colors due to highlight truncation. For example, a brighter image will be overexposed due to brightness correction, resulting in a white picture at this location. Therefore, the image obtained by the fusion processing method often has a color deviation phenomenon in the color of the highlights. The image region can be a partial region in the fusion image or the entire region of the fusion image.

[0196] S503, obtaining a reference image region.

[0197] Specifically, the size of the fusion image is the same as that of the second mapping dynamic image, and the reference image region is obtained in the second mapping dynamic image corresponding to the fusion image. The reference image region can be a partial region in the second mapping dynamic image or the entire region of the second mapping dynamic image.

[0198] S504, calculating a local mapping gain value of the image region of the fusion image relative to the reference image region.

[0199] The local mapping gain value refers to the mapping value corresponding to the mapping gain value of the image region.

[0200] In one embodiment, the local mapping gain value is obtained by performing inverse gamma transformation on the image region of the fusion image and the reference image region of the second mapping dynamic image, and performing ratio calculation on the two brightness values after the inverse gamma transformation to obtain the linear gain value of the brightness value of each pixel of the fusion image relative to the corresponding pixel brightness value of the second mapping dynamic image.

[0201] In one embodiment, the linear gain value of the brightness value of each pixel of the fusion image relative to the corresponding pixel brightness value of the second mapping dynamic image can be obtained by performing difference calculation on the two brightness values of the image region to be processed in the fusion image after inverse gamma transformation and the reference image region of the second mapping dynamic image.

[0202] S505, performing tone mapping processing on the RGB image according to the local mapping gain value to obtain a target dynamic image.

[0203] Specifically, after obtaining the local mapping gain value, the RGB image is subjected to a tone mapping process, and after gamma conversion, the RGB image is converted into an eight-bit low dynamic range image with pixel values between 0 and 255.

[0204] In one embodiment, the product of the pixel value of the RGB image to be processed in the image region and the corresponding second luminance parameter correction coefficient of the corresponding pixel of the second mapping dynamic image is multiplied by the local mapping gain value of the corresponding pixel position, and then the pixel value after the gain is subjected to gamma conversion to convert it into a target dynamic image with pixel values between 0 and 255 in eight-bit low dynamic range. Through the above processing, more details of the image can be preserved, the image processing effect can be improved, and the color in the highlight area can be more accurate.

[0205] In one embodiment, when the pixel value after the gain exceeds the preset pixel value, it is limited to the preset pixel value. For example, when the preset pixel value is limited to 65535, the part of the pixel value after the gain that exceeds 65535 is limited to 65535.

[0206] In this embodiment, the fusion processing image is obtained through the first mapping dynamic image, the second mapping dynamic image, and the third mapping dynamic image, and the local mapping gain value can be used to process the image region to be processed in the fusion processing image. The processed fusion image is subjected to gamma conversion to obtain a target dynamic image, which can achieve a target image with more preserved image details after image processing.

[0207] In one embodiment, as shown in Figure 6 , obtaining the ambient light intensity statistical value includes:

[0208] S601, obtaining the sensitivity, shutter speed, and aperture value corresponding to the RGB image.

[0209] The sensitivity refers to the sensitivity of the camera to light when the RGB image is obtained. High sensitivity can affect image quality. Although the brightness of the obtained image is biased towards brightness, high sensitivity can cause more image noise. The shutter speed refers to the opening time of the shutter when the camera is used to obtain an image. The faster the shutter speed, the shorter the opening time, and the less light enters the camera, resulting in a darker image. Conversely, the slower the shutter speed, the longer the opening time, and the more light enters the camera, resulting in a brighter image. The aperture value refers to the relative value of the camera lens light. The smaller the aperture value, the greater the amount of light entering in the same unit of time. Conversely, the larger the aperture value, the greater the amount of light entering in the same unit of time.

[0210] Specifically, the ambient light intensity statistical value and the sensitivity, shutter speed, and aperture value have a functional relationship. To obtain the ambient light intensity statistical value, the above-mentioned sensitivity, shutter speed, and aperture value parameters need to be obtained first.

[0211] S602, obtaining a first parameter value according to the ISO speed, the shutter speed and the aperture value.

[0212] Specifically, after obtaining the ISO speed, the shutter speed and the aperture value, the first parameter value is obtained according to the ISO speed, the shutter speed and the aperture value. For example, when the shutter speed is doubled, the lens light quantity is reduced by half in the sequence of 1 second, 1 / 2 second, 1 / 4 second and 1 / 8 second; when the aperture value is increased by one step, for example, 1.4, 2.0, 4.0, 5.6 and 8.0, the light quantity is also reduced by half; the shutter speed is increased or reduced by a multiple, the aperture value is increased or reduced by a fixed square root of the value, and the ISO speed is doubled, and the light quantity is reduced by half. When the exposure is insufficient, a larger aperture, a slower shutter speed and a higher ISO value can be set to adjust. When the exposure is excessive, a smaller aperture, a faster shutter speed and a lower ISO value can be set to adjust.

[0213] In an embodiment, the first parameter value can be represented by a formula, which includes the ISO speed, the shutter speed and the aperture value. Wherein the ISO speed is represented as I, the shutter speed is represented as s, the aperture value is represented as a, and the first parameter value is represented as c, c can be represented as:

[0214] c = I * s / a 2

[0215] S603, calculating a parameter ratio of the image brightness statistical value and the first parameter.

[0216] Specifically, the image brightness statistical value is represented as V0, and the parameter ratio of the image brightness statistical value and the first parameter can be calculated. Through the parameter ratio, the functional relationship between the image brightness statistical value and the obtained image parameter value can be preliminarily judged.

[0217] In an embodiment, the parameter ratio of the image brightness statistical value and the first parameter can be represented by b:

[0218] b = V0 / I * s / a 2

[0219] Step S604, performing logarithmic calculation on the parameter ratio as the true number of the logarithmic function to obtain an ambient light intensity statistical value.

[0220] Specifically, the parameter ratio is logarithmically calculated as the true number of the logarithmic function, and the ambient light intensity statistical value can be obtained. The ambient light intensity statistical value can be represented as EE, and EE is represented by a formula as:

[0221] EE = log2b

[0222] The environmental light intensity EE is a value greater than 0, so the b of the true number of the logarithmic function is greater than 1, and the pre-processed image brightness statistical value is greater than the parameter value of the first parameter.

[0223] In this embodiment, the purpose of obtaining the statistical value of the environmental light intensity can be achieved through the function relationship among the sensitivity, shutter speed and aperture value corresponding to the RGB image, and the image brightness statistical value.

[0224] In one embodiment, the terminal first obtains the RGB image to be processed; obtains the target brightness statistical value corresponding to the shooting environment through the image brightness statistical value and the statistical value of the environmental light intensity corresponding to the RGB image, determines the target brightness correction coefficient through the target brightness statistical value and the image brightness statistical value, and takes the target brightness correction coefficient as the first brightness correction coefficient; then increases or decreases the brightness on the basis of the first brightness correction coefficient to form two other brightness correction coefficients, which are the second brightness correction coefficient and the third brightness correction coefficient respectively; processes the high dynamic range image through the three brightness correction coefficients to obtain three images after brightness correction, which are the normal image corrected by the first brightness correction coefficient, the darker image corrected by the second brightness correction coefficient, and the brighter image corrected by the third brightness correction coefficient respectively. After gamma transformation of the three images, the image pixel values are mapped to the red-green-blue three-channel low dynamic image with the pixel value range of 0 to 255. The three low dynamic images after mapping are fused to obtain a single fused image. Although the fused image retains the highlight and shadow details in the original high dynamic range image, the images corrected by different brightness correction coefficients will present different colors due to highlight truncation in some originally color information high light areas, such as color signboards. For example, the brighter image will be overexposed due to brightness correction, resulting in white picture at this place. The color of the fused image at the highlight often has a color deviation. Taking the local brightness information of the fused image as a reference, the local tone mapping gain map is obtained in combination with the corresponding local brightness information of the darker image. The pixel brightness values of the fused image and the darker image in the three low dynamic ranges are extracted. The two brightness values are inversely gamma transformed, and the two brightness values after inverse gamma transformation are operated by ratio, for example, division, to obtain the linear gain value of each pixel of the fused image relative to the corresponding pixel brightness value of the darker image. The pixel value of the RGB image is multiplied by the brightness correction coefficient corresponding to the darker image, and then multiplied by the local tone mapping gain value of the corresponding pixel position to obtain a gain image. When the pixel value in the gain image exceeds the preset pixel value, it is limited to the preset pixel value, and then the pixel value of the gain image is gamma transformed to convert it into an eight-bit low dynamic range image with the red-green-blue three-channel pixel value of 0-255.

[0225] It should be understood that, although Figures 1-6The steps in the flowchart are shown in sequence according to the arrows, but the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of the steps is not strictly limited in sequence, and the steps can be executed in other orders. Moreover, Figures 1-6 At least part of the steps in the flowchart can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the steps or stages is not necessarily sequential, but can be alternately executed with other steps or steps or stages in other steps.

[0226] In one embodiment, as shown in Figure 7 An image processing apparatus 700 is provided, comprising: an image acquisition module 701, a noise reduction module 702, a conversion processing module 703, a brightness correction module 704, a dynamic image mapping module 705, wherein:

[0227] The image acquisition module 701 is configured to acquire a to-be-processed Raw image;

[0228] The noise reduction module 702 is configured to perform noise reduction processing on the to-be-processed Raw image to obtain a noise-reduced Raw image;

[0229] The conversion processing module 703 is configured to perform conversion processing on the noise-reduced Raw image to obtain an RGB image;

[0230] The brightness correction module 704 is configured to perform brightness correction processing on the RGB image to obtain a target brightness correction image;

[0231] The dynamic mapping module 705 is configured to perform pixel dynamic range mapping on the target brightness correction image to obtain a target dynamic image.

[0232] For specific limitations of the image processing apparatus, refer to the limitations of the image processing method described above, which will not be repeated here. Each module in the above image processing apparatus can be realized by software, hardware and their combination in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so as to call and execute the operations corresponding to the above modules by the processor.

[0233] In one embodiment, a computer device is provided, which can be a server, and its internal structure diagram can be as shown in Figure 8As shown in the figure. The computer device includes a processor, a memory and a network interface connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store image processing data. The network interface of the computer device is used to communicate with the external terminal through the network connection. The computer program is executed by the processor to implement an image processing method.

[0234] Those skilled in the art can understand that, Figure 8 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0235] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the following steps:

[0236] Obtain a to-be-processed Raw image;

[0237] Perform denoising processing on the to-be-processed Raw image to obtain a denoised Raw image;

[0238] Perform conversion processing on the denoised Raw image to obtain an RGB image;

[0239] Perform brightness correction processing on the RGB image to obtain a target brightness correction image;

[0240] Perform pixel dynamic range mapping processing on the target brightness correction image to obtain a target dynamic image, and the pixel dynamic range of the target dynamic image is smaller than the pixel dynamic range of the RGB image.

[0241] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0242] Obtain a to-be-processed Raw image;

[0243] Perform denoising processing on the to-be-processed Raw image to obtain a denoised Raw image;

[0244] Perform conversion processing on the denoised Raw image to obtain an RGB image;

[0245] The RGB image is subjected to brightness correction processing to obtain a target brightness correction image.

[0246] The target brightness correction image is subjected to pixel dynamic range mapping processing to obtain a target dynamic image, and a pixel dynamic range of the target dynamic image is smaller than a pixel dynamic range of the RGB image.

[0247] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0248] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0249] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. An image processing method characterized by, The method comprises: obtaining a to-be-processed Raw image; performing noise reduction processing on the to-be-processed Raw image to obtain a noise-reduced Raw image; performing conversion processing on the noise-reduced Raw image to obtain an RGB image; performing brightness correction processing on the RGB image to obtain a target brightness correction image; performing pixel dynamic range mapping processing on the target brightness correction image to obtain a target dynamic image, wherein the pixel dynamic range of the target dynamic image is smaller than the pixel dynamic range of the RGB image; the brightness correction processing on the RGB image to obtain the target brightness correction image comprises: determining a target brightness correction coefficient according to a target brightness statistical value and an image brightness statistical value, and taking the target brightness correction coefficient as a first brightness correction coefficient; performing reduction processing on the first brightness correction coefficient to obtain a second brightness correction coefficient; 2. The method of claim 1, wherein, performing increase processing on the second brightness correction coefficient to obtain a third brightness correction coefficient; performing brightness correction on the RGB image according to the first brightness correction coefficient, the second brightness correction coefficient and the third brightness correction coefficient respectively to obtain a first brightness correction image corrected by the first brightness correction coefficient, a second brightness correction image corrected by the second brightness correction coefficient and a third brightness correction image corrected by the third brightness correction coefficient.

3. The method of claim 1, wherein, The noise reduction processing on the to-be-processed Raw image to obtain the noise-reduced Raw image specifically comprises: inputting the to-be-processed Raw image into a preset image noise reduction model to perform noise reduction processing and obtain the noise-reduced Raw image.

4. The method of claim 1, wherein, The conversion processing on the noise-reduced Raw image to obtain the RGB image specifically comprises: performing processing on the noise-reduced Raw image by using an interpolation algorithm to obtain the RGB image. The brightness correction processing on the RGB image to obtain the brightness correction image specifically comprises: obtaining an image brightness statistical value corresponding to the RGB image; obtaining an ambient light intensity statistical value, and obtaining a target brightness statistical value corresponding to a shooting environment according to the ambient light intensity statistical value; the ambient light intensity statistical value is a light intensity statistical value of a shooting environment where the RGB image is located; 5. The method of claim 4, wherein, determining a target brightness correction coefficient according to the target brightness statistical value and the image brightness statistical value; performing brightness correction on the RGB image according to the target brightness correction coefficient to obtain the target brightness correction image. The determination of the target brightness correction coefficient according to the target brightness statistical value and the image brightness statistical value comprises at least one of the following steps:

6. The method according to claim 4 or 5, characterized in that, when the target brightness statistical value is greater than the image brightness statistical value, obtaining a brightness enhancement coefficient as the target brightness correction coefficient; when the target brightness statistical value is less than the image brightness statistical value, obtaining a brightness attenuation coefficient as the target brightness correction coefficient. The determination of the target brightness correction coefficient according to the target brightness statistical value and the image brightness statistical value comprises: calculating a brightness ratio of the target brightness statistical value and the image brightness statistical value; Logarithmically calculating the luminance ratio as a logarithm in a logarithmic function with an exponent greater than 1, to obtain a first luminance correction coefficient.

7. The method of claim 6, wherein, The pixel dynamic range mapping on the target luminance correction image to obtain a target dynamic image comprises: respectively performing pixel dynamic range mapping on the first luminance correction image, the second luminance correction image and the third luminance correction image to obtain a first mapping dynamic image corresponding to the first luminance correction image, a second mapping dynamic image corresponding to the second luminance correction image and a third mapping dynamic image corresponding to the third luminance correction image; performing fusion processing on the first mapping dynamic image, the second mapping dynamic image and the third mapping dynamic image to obtain a target dynamic image.

8. The method of claim 7, wherein, The fusion processing on the first mapping dynamic image, the second mapping dynamic image and the third mapping dynamic image to obtain a target dynamic image comprises: performing fusion processing on the first mapping dynamic image, the second mapping dynamic image and the third mapping dynamic image to obtain a fusion processing image; obtaining an image region of the fusion processing image; obtaining a reference image region; calculating a local mapping gain value of the image region of the fusion processing image relative to the reference image region; 9. An image processing apparatus characterized by comprising: performing tone mapping processing on the RGB image according to the local mapping gain value to obtain a target dynamic image. The device comprises an image acquisition module configured to acquire a Raw image to be processed; a noise reduction module configured to perform noise reduction processing on the Raw image to obtain a noise-reduced Raw image; a conversion processing module configured to perform conversion processing on the noise-reduced Raw image to obtain an RGB image; a luminance correction module configured to perform luminance correction processing on the RGB image to obtain a target luminance correction image; a dynamic mapping module configured to perform pixel dynamic range mapping on the target luminance correction image to obtain a target dynamic image. The luminance correction processing on the RGB image to obtain a target luminance correction image comprises: determining a target luminance correction coefficient according to a target luminance statistical value and an image luminance statistical value, and taking the target luminance correction coefficient as a first luminance correction coefficient; performing reduction processing on the first luminance correction coefficient to obtain a second luminance correction coefficient; performing increase processing on the second luminance correction coefficient to obtain a third luminance correction coefficient; 10.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-9. respectively performing luminance correction on the RGB image according to the first luminance correction coefficient, the second luminance correction coefficient and the third luminance correction coefficient to obtain a first luminance correction image corrected by the first luminance correction coefficient, a second luminance correction image corrected by the second luminance correction coefficient and a third luminance correction image corrected by the third luminance correction coefficient.

11. A computer readable storage medium having stored thereon a computer program, characterized in that, The processor, when executing the computer program, implements the steps of the method of any one of claims 1 to 8. The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 8.

Images