Image processing method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to an image processing method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: generating an SDR image according to an original image obtained by the electronic equipment in an image shooting process; performing HDR conversion on the SDR image according to the display effect adjustment parameter to obtain an HDR image; and processing the HDR image to obtain a fast thumbnail of the HDR. By adopting the method, the display effect of the quick thumbnail can be consistent with that of the final image, so that the transition from the quick thumbnail to the final image is more natural.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and particularly to an image processing method, apparatus, electronic device, storage medium, and program product. Background Art

[0002] Currently, with the gradual improvement and unification of HDR standard specifications, and the continuous popularization of devices supporting HDR displays, electronic devices supporting HDR technology are increasingly favored by consumers due to their high dynamic range, rich colors, and excellent detail presentation capabilities.

[0003] In related technologies, when generating the final image in HDR format, it takes a relatively long time. Therefore, in order to enable users to quickly preview the content of the photographed image, a fast thumbnail is usually used as a transitional display before the final image is generated.

[0004] However, the current algorithms and software pipelines for generating fast thumbnails are all designed based on SDR, so there are obvious differences between the fast thumbnails generated in this way and the display effects of the final images, resulting in a jump problem in the transition from fast thumbnails to the final images. Summary of the Invention

[0005] Based on this, it is necessary to provide an image processing method, apparatus, electronic device, storage medium, and program product that can make the transition from fast thumbnails to the final images more natural for the above technical problems.

[0006] In a first aspect, this application provides an image processing method, including:

[0007] Generating an SDR image based on the original image obtained by the electronic device during the image shooting process;

[0008] Performing HDR conversion on the SDR image according to the parameters adjusted for the display effect to obtain an HDR image;

[0009] Processing the HDR image to obtain a fast thumbnail of HDR.

[0010] In a second aspect, this application also provides an image processing apparatus, including:

[0011] An SDR image generation module for generating an SDR image based on the original image obtained by the electronic device during the image shooting process;

[0012] An HDR image generation module for performing HDR conversion on the SDR image according to the parameters adjusted for the display effect to obtain an HDR image;

[0013] A fast thumbnail generation module for processing the HDR image to obtain a fast thumbnail of HDR.

[0014] In a third aspect, the present application further provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps described in the first aspect above are implemented.

[0015] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the first aspect above are implemented.

[0016] In a fifth aspect, the present application further provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the method described in the first aspect above are implemented.

[0017] For the above image processing method, device, electronic device, storage medium and program product, an SDR image is generated according to the original image obtained during the image shooting process of the electronic device; parameters are adjusted according to the display effect, and the SDR image is subjected to HDR conversion to obtain an HDR image; the HDR image is processed to obtain a fast thumbnail of the HDR. The image processing method provided by the present application processes the original image collected by the electronic device into an SDR image, and then converts the SDR image into an HDR image, realizing the format conversion of the SDR image, thereby generating a fast thumbnail based on the HDR image, so that the display effect of the fast thumbnail is also HDR, maintaining consistency with the display effect of the final image, thus making the transition from the fast thumbnail to the final image more natural. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 It is a schematic flowchart of an image processing method in an embodiment;

[0020] Figure 2 It is a schematic flowchart of obtaining a fast thumbnail of HDR in an embodiment;

[0021] Figure 3 It is a schematic flowchart of generating a fast thumbnail of HDR in an embodiment;

[0022] Figure 4 It is a schematic flowchart of obtaining a fast thumbnail of HDR in another embodiment;

[0023] Figure 5 Schematic diagram of the process for generating an SDR image in one embodiment;

[0024] Figure 6 Schematic diagram of the process for generating an SDR image in another embodiment;

[0025] Figure 7 Schematic diagram of the post - processing of Quick bitmap and Quick Jpeg in one embodiment;

[0026] Figure 8 Block diagram of the structure of an image processing device in one embodiment;

[0027] Figure 9 Internal structure diagram of an electronic device in one embodiment. Detailed implementation manners

[0028] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0029] A quick thumbnail (quick image) is a thumbnail used for mobile phones and other terminal devices to quickly generate the content of the photographed picture for display in the photo album before generating the full - effect large picture (Final image) of the shot.

[0030] Existing mobile phones and other terminal devices do not yet support shooting UHDR photos containing RGB three - channel color information. For devices that support single - channel HDR shooting, their quick - image processing algorithms and software links are all designed based on SDR, and the generated quick image is still an SDR picture, and its performance in terms of dynamic range and color is not as good as that of an HDR picture. At the same time, after taking a photo and clicking on the photo album, since the SDR quick image is displayed first and then transitions to the HDR Final image display after the Final image is generated, there will be a relatively obvious difference in brightness and dynamic performance between the two display stages, resulting in a jump problem in the transition from the quick image to the final image.

[0031] In view of this, embodiments of the present application provide an image processing method, apparatus, electronic device, storage medium, and program product. In this image processing method, an SDR image can be generated based on the original image obtained by the electronic device during the image capture process; according to the display effect adjustment parameters, perform HDR conversion on the SDR image to obtain an HDR image; process the HDR image to obtain a fast thumbnail of the HDR. The original image collected by the electronic device is processed into an SDR image, and then the SDR image is converted into an HDR image, realizing the format conversion of the SDR image, so as to generate a fast thumbnail based on the HDR image, making the display effect of the fast thumbnail also HDR, maintaining consistency with the display effect of the final image, and thus making the transition from the fast thumbnail to the final image more natural.

[0032] In an exemplary embodiment, as Figure 1 shown, an image processing method is provided. Taking the application of this method to an electronic device as an example for illustration, where the electronic device can be a device with image processing capabilities. On this basis, the electronic device can also be a device with image capture capabilities. Optionally, the electronic device can be a laptop computer, a smart phone, a tablet computer, an Internet of Things device, and a portable wearable device, etc. Among them, the Internet of Things device can be a smart speaker, a smart TV, a smart air conditioner, a smart vehicle-mounted device, a projection device, etc., and the portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, a smart glasses, etc. As Figure 1 shown, the image processing method may include the following steps:

[0033] Step 101, generate an SDR image based on the original image obtained by the electronic device during the image capture process.

[0034] In an optional embodiment of the present application, after the user triggers the image capture function through the electronic device, the electronic device captures the original image, and then the electronic device generates an SDR image based on the original image captured during the image capture process.

[0035] In a possible implementation manner, by performing operations such as color gamut conversion and tone mapping on the original image to generate an SDR image. For example, first convert the color gamut of the original image to the sRGB color gamut of SDR, and the original image can be converted using a color conversion matrix. Then, map the brightness value of the image to the appropriate range of SDR, which can be achieved through a simple linear mapping or a more complex non-linear mapping function. Next, save the processed image in the SDR image format to obtain the generated SDR image.

[0036] In another possible implementation, through operations such as linear RAW processing, dynamic range segmented compression, and local contrast enhancement, an SDR image is generated. For example, first, the image signal processor of the electronic device performs operations such as demosaicing, noise reduction, and white balance correction on the original image. Then, the processed image is divided into highlight, midtone, and shadow regions, and a non-linear gamma curve (such as sRGB) is applied to compress the highlights and enhance the dark parts respectively. Next, the image base layer (low-frequency luminance) and the detail layer are separated, and after dynamic compression of the base layer, they are merged to obtain the SDR image.

[0037] Step 102: Adjust the parameters according to the display effect and perform HDR conversion on the SDR image to obtain an HDR image.

[0038] Among them, the display effect adjustment parameters refer to a series of parameters used to control and optimize the display effect of the image in the high dynamic range. These parameters can adjust aspects such as the brightness, contrast, color saturation, and hue of the image to achieve a more realistic and visually impactful HDR effect. Optionally, the display effect adjustment parameters can include brightness, contrast, color saturation, tone mapping curve, gamma correction, highlight and shadow details, color temperature, local contrast enhancement, etc.

[0039] In one possible implementation, through operations such as brightness separation and mapping, highlight recovery and dark part enhancement, color gamut conversion and saturation matching, etc., the SDR image is converted into an HDR image. For example, the SDR image is decomposed into a base luminance layer and a high-frequency detail layer, and the base layer is processed to expand the luminance range of the SDR to the luminance range of the HDR. Then, the lost highlight details are reconstructed through histogram stretching, and noise suppression is performed on the dark parts to avoid introducing noise when enhancing the shadows. Next, the color space of the aforementioned processed image is converted to increase the color saturation and match the HDR display requirements, thereby obtaining the HDR image. Among them, during the luminance expansion process, the display effect adjustment parameters can be used, such as the peak brightness of the target display, to dynamically adjust the slope of the luminance expansion curve.

[0040] In another possible implementation, an SDR-to-HDR model that has been pre-trained is used to process the SDR image to obtain an HDR image. For example, a paired SDR-HDR image dataset can be used to train a neural network so that the model learns the mapping relationship from SDR to HDR, including brightness expansion, highlight reconstruction, and color restoration, etc. until the model training is completed to obtain a trained SDR-to-HDR model. Then, the SDR image is input into the trained SDR-to-HDR model for HDR conversion of the SDR image, thereby obtaining an HDR image. Among them, when the SDR image is input into the trained model for HDR conversion, the display effect adjustment parameters can be input into the model together to dynamically control the generation process, so as to achieve device self-adaptation or user-defined HDR effects and improve the flexibility of HDR image generation and the quality of HDR images.

[0041] Step 103: Process the HDR image to obtain a fast thumbnail of the HDR.

[0042] In an alternative embodiment of the present application, after obtaining the HDR image, the HDR image is processed to obtain a fast thumbnail of the HDR, so that the brightness, dynamics, and color effects of the fast thumbnail are consistent with the brightness, dynamics, and color effects of the final image, that is, the display effect consistency between the fast thumbnail and the final image is achieved, so that the transition from the fast thumbnail to the final image is more natural.

[0043] The image processing method provided in this embodiment generates an SDR image based on the original image obtained during the image shooting process of the electronic device; performs HDR conversion on the SDR image according to the display effect adjustment parameters to obtain an HDR image; and processes the HDR image to obtain a fast thumbnail of the HDR. The image processing method provided in the present application processes the original image collected by the electronic device into an SDR image, and then converts the SDR image into an HDR image, realizing the format conversion of the SDR image, and thus generating a fast thumbnail based on the HDR image, so that the display effect of the fast thumbnail is also HDR, maintaining consistency with the display effect of the final image, thereby making the transition from the fast thumbnail to the final image more natural.

[0044] Please refer to Figure 2 , in an alternative embodiment of the present application, an optional technical process for processing an HDR image to obtain a fast thumbnail in HDR format is provided, as Figure 2 shown, and this technical process includes the following steps:

[0045] Step 201: Perform SDR conversion on the HDR image according to the display effect adjustment parameters to obtain a candidate SDR image and a gain image.

[0046] Among them, the display effect adjustment parameters are the display adjustment parameters used for HDR conversion of the SDR image as described above, including Hybrid Log-Gamma (HLG Gamma) and Standard Gamma (sRGB Gamma).

[0047] The above conversion of the SDR image to an HDR image using the display effect adjustment parameters is an up-conversion. In the current process, using the display effect adjustment parameters again to convert the HDR image to an SDR image is a down-conversion. When up-conversion and down-conversion are required in the same scene, the Tone Curve of SDR and the Tone Curve of HDR will be designed simultaneously. The Tone is combined with the standard encoding (HLG Gamma, SRGB Gamma) respectively to obtain two final gamma curves with Tone, hlgGammaTone and srgbGammaTone. Among them, srgbGammaTone = sRGB Gamma + sdr_ToneCurve; hlgGammaTone = HLG Gamma + hdr_ToneCurve. Among them, sRGB (standard Red Green Blue) defines the specific values of the three primary colors of red, green, and blue and parameters such as the Gamma curve, and has a relatively balanced setting in terms of the brightness and contrast performance of the image.

[0048] When converting the SDR image to an HDR image, applying hlgGammaTone with adjustments (Tone) such as brightness, color, contrast, and tone performance to the SDR image in P010 format can obtain the converted HDR image.

[0049] When converting the HDR image to an SDR image, inverse hlgGammaTone (restoring back to the P010 image) and applying srgbGammaTone with Tone again can convert to obtain a candidate SDR image and a gain image. Among them, the gain image is an RGB image used to store the brightness and color of the three channels.

[0050] In an optional embodiment of the present application, through the HDR-to-SDR algorithm, not only is the HDR image converted to the color space and dynamic range of the SDR, but also a gain image (Gain map) is generated to record the gain adjustment information of the color and brightness of the three channels. The gain image can be used for subsequent further processing or display optimization of the image. For example, according to the characteristics of different display devices or user requirements, the gain image is used to adjust the brightness, contrast, or color balance of the image.

[0051] Step 202: Generate a fast thumbnail of the HDR based on the candidate SDR image and the gain image.

[0052] The candidate SDR image has an SDR effect, and the gain image records the gain adjustment information for each channel in the three-channel RGB. When the candidate SDR image is converted into a fast thumbnail, the gain image can be used to adjust the image brightness range, contrast, and color, etc., so that the fast thumbnail also has an HDR display effect.

[0053] In the embodiments of the present application, by using display effect adjustment parameters, the HDR image is down-converted to obtain a candidate SDR image and a gain image, and thus a fast thumbnail of HDR is quickly generated based on the candidate SDR image and the gain image. On the one hand, it can be used for fast display, improving the display speed of the thumbnail. On the other hand, the screen excitation brightness is dynamically adjusted based on the screen-perceived ambient brightness, and the Gamma adjustment is supported to match the HDR display ability observed by the human eye, making the fast thumbnail more immersive in the shooting scene.

[0054] In practical applications, fast thumbnails can be divided into two categories. One is the fast thumbnail displayed after directly entering the album from the camera application, which can be called Quick bitmap. It is used when the camera application takes a photo and clicks to evoke the album, and can quickly display the content of the captured picture. The other is the fast thumbnail displayed after exiting the camera application and entering the camera from the desktop, which can be called Quick Jpeg. It is used after continuously taking multiple photos. When exiting the camera and clicking into the album gallery from the system desktop, before the final images (Final images) of these photos are generated, Quick Jpeg can be displayed first to facilitate the user to quickly browse the content of multiple pictures. At the same time, when the algorithm of the final image has not been processed and the final image has not been generated, Quick Jpeg can also be shared as a normal picture to other applications and saved.

[0055] Based on this, it can be known that the fast thumbnail includes Quick bitmap and Quick Jpeg. Therefore, when generating the fast thumbnail of HDR based on the candidate SDR image and the gain image, Quick bitmap and Quick Jpeg can be generated respectively for the user to preview the content of the captured picture in different ways of entering the album.

[0056] The following describes the process of generating fast thumbnails based on the two types of fast thumbnails, Quick bitmap and Quick Jpeg.

[0057] In an exemplary embodiment, generating an HDR fast thumbnail based on the candidate SDR image and the gain image includes: performing bitmap conversion on the candidate SDR image according to the gain image to obtain a first fast thumbnail in HDR bitmap format.

[0058] That is, the candidate SDR image is converted into a bitmap, and the bitmap-converted image is optimized based on the gain image, so as to obtain a first quick thumbnail in the HDR bitmap format, and the first quick thumbnail is the aforementioned Quickbitmap.

[0059] In an alternative embodiment of the present application, when the candidate SDR image is converted into a bitmap Bitmap, the Gainmap gain image is simultaneously set into the Bitmap object to obtain a Quick bitmap, and this Quick bitmap is returned to the album application layer, and the Quick bitmap then has the ability to display HDR.

[0060] In the embodiment of the present application, the candidate SDR image is converted into a bitmap, and the obtained Bitmap has an SDR display effect with a relatively fixed brightness and color range. Therefore, it is easier to perform targeted adjustment and optimization according to the characteristics of the device to achieve a better display effect. In this way, after the gain image is set into the Bitmap object, the first quick thumbnail can achieve a better HDR display effect, thereby improving the display effect of the quick thumbnail.

[0061] Please refer to Figure 3 , in an alternative embodiment of the present application, another optional technical process for generating a quick thumbnail of HDR based on the candidate SDR image and the gain image is provided, as Figure 3 shown, and this technical process includes the following steps:

[0062] Step 301: Upsample the candidate SDR image and the gain image respectively to obtain an upsampled candidate SDR image and an upsampled gain image.

[0063] Step 302: Perform UHDR encoding on the upsampled candidate SDR image and the upsampled gain image to obtain a second quick thumbnail in the JPEGR format that supports HDR.

[0064] In an alternative embodiment of the present application, the candidate SDR image and the gain image are respectively upsampled to obtain the upsampled candidate SDR image and the upsampled gain image, so as to enlarge the candidate SDR image and the gain image to a certain size, laying a foundation for making the size of the second quick thumbnail consistent with the size of the final image, where the second quick thumbnail is the aforementioned Quick Jpeg. That is to say, after the candidate SDR image and the gain image are respectively upsampled, the image sizes of the candidate SDR image and the gain image can be enlarged. Then, the upsampled candidate SDR image and the upsampled gain image are subjected to UHDR encoding processing to obtain the second quick thumbnail in JPEGR format. In this way, the size of the quick thumbnail can be consistent with the size of the final image, so that the Quick Jpeg can be better shared as an alternative image of the final image. Moreover, since both the Quick Jpeg and the final image support HDR display and their sizes are the same, when the generated final image is lost or damaged, the Quick Jpeg can be used as the final image, enabling the application of the electronic device to continue running, avoiding error prompts or functional abnormalities, and maintaining the basic functions and user experience of the application.

[0065] After obtaining the upsampled candidate SDR image and the upsampled gain image, through UHDR encoding, the upsampled candidate SDR image and the upsampled gain image are processed to obtain a UHDR jpegr format Quick Jpeg, which is the second quick thumbnail in JPEGR format that supports HDR. Among them, UHDR encoding is a super high dynamic range encoding method that can process a wider range of brightness and color ranges, and has higher precision and dynamic range than ordinary HDR technology. Through UHDR encoding, more details can be retained in the highlight and shadow parts of the image, and the colors are more vivid and realistic, thus bringing a more immersive visual experience to users.

[0066] In the embodiment of the present application, by respectively upsampling the candidate SDR image and the gain image, the image sizes of the candidate SDR image and the gain image are enlarged. Then, by performing UHDR encoding on the upsampled candidate SDR image and the upsampled gain image, the second quick thumbnail obtained after encoding has a larger dynamic range, richer colors, and higher contrast, improving the display effect of the quick thumbnail, and at the same time making the size of the second quick thumbnail consistent with the size of the final image, providing a better experience for image sharing and the continuous operation of the electronic device application.

[0067] Please refer to Figure 4, in an alternative embodiment of the present application, another alternative technical process for processing an HDR image to obtain a fast thumbnail of the HDR is provided, as Figure 4 shown. This technical process includes the following steps:

[0068] Step 401: Perform image enhancement processing and / or image correction processing on the HDR image to obtain a processed HDR image.

[0069] Among them, the image enhancement processing includes beauty, filter and other processing. The image correction processing includes wide-angle distortion correction, face distortion correction, etc.

[0070] In an alternative embodiment of the present application, before generating the fast thumbnail, the HDR image is first subjected to image enhancement processing and image correction processing to optimize the details of the people in the HDR image and improve the composition accuracy.

[0071] In an alternative embodiment of the present application, the current HDR image is in the P010 format. Therefore, the beauty, distortion correction and other algorithms used at this time are re-adapted when the input format changes from the original NV21 / SDR texture to P010 / HDR Texture. Through such re-adaptation, it can be ensured that the HDR image in the P010 format maintains the effect of the algorithm itself and does not lose the HDR function after passing through these post-processing algorithms.

[0072] Step 402: Process the processed HDR image to obtain a fast thumbnail of the HDR.

[0073] After obtaining the processed HDR image as described above, the processed HDR image is processed again to obtain a fast thumbnail of the HDR.

[0074] Among them, the method of processing the processed HDR image again is the same as the method of directly processing the HDR image to obtain a fast thumbnail of the HDR in the foregoing embodiment.

[0075] That is, the parameters can be adjusted according to the display effect, and the processed HDR image is converted to SDR to obtain the current converted candidate SDR image and the gain image; according to the current converted candidate SDR image and the gain image, a fast thumbnail of the HDR is generated.

[0076] In a possible implementation manner, according to the currently converted gain image, the current candidate SDR image is converted into a bitmap to obtain a first fast thumbnail of the HDR.

[0077] In another possible implementation, the currently transformed candidate SDR image and the gain image are respectively upsampled to obtain an upsampled candidate SDR image and an upsampled gain image; the upsampled candidate SDR image and the upsampled gain image are subjected to UHDR encoding to obtain a second fast thumbnail in JPEG format that supports HDR.

[0078] In the embodiments of the present application, first, the HDR image is subjected to image enhancement processing and / or image correction processing to obtain a processed HDR image. By preprocessing the HDR image, while retaining the advantages of a high dynamic range, problems such as uneven skin color and distortion that are prone to occur can be avoided, so as to optimize facial details and improve the composition accuracy. Then, the optimized HDR image is processed to obtain a fast thumbnail of the HDR, making the fast thumbnail closer to the picture of the real scene and improving the authenticity of the fast thumbnail.

[0079] It can be understood that different processors are usually set in an electronic device to cooperate to complete computing and graphics processing tasks. In actual applications, different electronic devices may use different processors for image enhancement processing and / or image correction processing of HDR images. Based on this, taking an electronic device including a graphics processing unit and a central processing unit as an example, the process of performing image enhancement processing and / or image correction processing on an HDR image will be described below.

[0080] In an exemplary embodiment, performing image enhancement processing and / or image correction processing on an HDR image to obtain a processed HDR image includes: if the electronic device includes a graphics processing unit (GPU) and the image processing algorithm supports GPU processing, then the HDR image is converted into a texture image, and image enhancement processing and / or image correction processing is performed on the texture image to obtain a processed HDR image; if the processor is a central processing unit, then image enhancement processing and / or image correction processing is directly performed on the HDR image to obtain a processed HDR image.

[0081] That is, if the electronic device includes a Graphics Processing Unit (GPU) and the image processing algorithm supports GPU processing, first convert the HDR image into a texture image, and then perform image enhancement processing and / or image correction processing on the converted texture image to obtain a processed texture image. Next, convert the processed texture image back to an HDR format image, thus obtaining the processed HDR image. For example, convert the HDR image into a texture of RGBA_1010102, and then perform image enhancement processing and image correction processing on the RGBA_1010102 texture image. RGBA indicates a color space format that includes four channels: Red, Green, Blue, and Alpha. This format can completely describe the color information of the image and the transparency of each pixel, and is very useful when processing images with transparent effects or complex color mixing operations. 1010102 indicates the precision of each channel, which means that the three color channels of Red, Green, and Blue are all 10-bit precision, while the Alpha channel is 2-bit precision. The higher color channel precision can provide richer color details and can represent the color information in the image more accurately.

[0082] If the electronic device supports processing by a Central Processing Unit (CPU) or a Neural Processing Unit (NPU), directly perform image enhancement processing and / or image correction processing on the HDR image in P010 format. Among them, the method of performing image enhancement processing and / or image correction processing on the HDR image can refer to the foregoing embodiments and will not be elaborated here.

[0083] In the embodiments of the present application, if the algorithm supports GPU processing, convert the HDR image into a texture image and then perform image enhancement processing and image correction processing to make the texture image completely describe the color information and transparency information, thereby improving the image quality of the processed HDR image; if the algorithm supports CPU or NPU processing, use the HDR image in P0101 format as the input for image enhancement processing and image correction processing, and perform fine adjustment and optimization on the dynamic range of the image, thereby improving the image quality of the processed HDR image.

[0084] Please refer to Figure 5 , in an alternative embodiment of the present application, an alternative technical process for generating an SDR image from the original image obtained during the image capture process by the electronic device is provided, as Figure 5 shown. This technical process includes the following steps:

[0085] Step 501: Obtain a reference frame original image and an underexposed frame original image from multiple original images obtained by the electronic device during the image capture process.

[0086] Step 502: Generate an SDR image based on the reference frame original image and the underexposed frame original image.

[0087] Among them, the reference frame original image is a frame image with standard exposure and image quality meeting the preset requirements among multiple original images; the underexposed frame original image is an underexposed image of a frame adjacent to the reference frame among multiple original images. In an optional embodiment of the present application, multiple candidate images with standard exposure are determined from multiple original images, and then, the candidate image with the optimal image quality is selected as the reference frame original image from these multiple candidate images. And, multiple candidate images with underexposure are determined from multiple original images, and then, a candidate image adjacent to the reference frame original image among these multiple candidate images is used as the underexposed frame original image.

[0088] When the electronic device captures images in HDR mode, multiple original images with different exposure degrees, such as overexposure, normal exposure, and underexposure, can be captured during the image capture process, and then merged into a final image that retains the details of the bright and dark parts.

[0089] It can be understood that when generating a fast thumbnail, for quick display, it is not necessary to process multiple original images together. Only two original images need to be captured from multiple original images for processing, so as to quickly generate a fast thumbnail for the user to preview. One of the two original images is an image with normal exposure, and one image is an image with low exposure.

[0090] Optionally, obtain an image with normal exposure as the reference frame original image and an image with low exposure as the underexposed frame original image from multiple original images obtained by the electronic device during the image capture process, and use the reference frame original image and the underexposed original image for generating the fast thumbnail.

[0091] In the embodiment of the present application, a normally exposed image and a low-exposure image are obtained from multiple original images captured by the electronic device as the original images for generating the fast thumbnail, and then, an SDR image is generated based on these two original images. On the one hand, only selecting two original images for processing can significantly reduce the amount of calculation and processing time. On the other hand, using only the image with normal exposure may have problems such as highlight overflow or insufficient dark part details in some scenarios, and using only the low-exposure image will be overall too dark and the details will not be fully displayed. By combining these two images to generate the fast thumbnail in the embodiment of the present application, a relatively stable preview image with better quality can be provided under different lighting conditions.

[0092] Please refer to Figure 6 , in an alternative embodiment of the present application, an alternative technical process for generating an SDR image based on the original reference frame image and the underexposed frame original image is provided, as Figure 6 shown. This technical process includes the following steps:

[0093] Step 601: Perform image format conversion processing on the original reference frame image and the underexposed frame original image respectively to obtain the SDR reference frame original image and the SDR underexposed frame original image.

[0094] In an alternative embodiment of the present application, through the IPE (Image Processing Engine), the original reference frame image and the underexposed frame original image are converted from the RAW format to the YUV format, including steps such as black level correction, demosaicing, color correction, Gamma correction, and RGB to YUV conversion, so as to obtain the SDR reference frame original image in the P0101 format and the SDR underexposed frame original image in the P0101 format. Among them, the RAW format contains the original data directly output by the image sensor. These data are unprocessed and uncompressed, retaining the most original information of the image, including the red, green, and blue (RGB) color values of each pixel point or relevant information in other color modes, as well as metadata such as the shooting parameters of the camera. YUV is a color encoding method, where Y represents luminance and U and V represent chrominance.

[0095] Optionally, the image sensor usually uses a color filter array to obtain RAW data. Each pixel point can usually only obtain the value of one color channel. The complete RGB color value of each pixel point is estimated through an interpolation algorithm. Then, the obtained RGB value is converted to the YUV color space, which can be achieved through a series of matrix operations and mathematical formulas, and there are different conversion coefficients according to different color space standards. During the conversion process, the IPE can also perform white balance adjustment processing to correct the color deviation of the image so that white can present accurate white under different lighting conditions, and perform noise filtering processing to remove random noise in the image and improve the image quality.

[0096] Step 602: Fuse the SDR reference frame original image and the SDR underexposed frame original image to obtain the SDR image.

[0097] After obtaining the original reference frame image in P010 format under SDR and the original underexposed frame image in P010 format under SDR, the original reference frame image in P010 format under SDR and the original underexposed frame image in P010 format under SDR are combined into a P010 format SDR image with a highlight suppression effect. For example, using CHDR (Couple Frame High - Dynamic Range, dual - frame high - dynamic range fusion algorithm), the original reference frame image in P0101 format under SDR and the original underexposed frame image in P0101 format under SDR are fused, processed, and optimized to obtain a P010 format SDR image with a highlight suppression effect, retaining the details of the highlight and shadow parts in the image and balancing the brightness and color in different regions.

[0098] In the embodiment of the present application, the image format of the two original images is converted from the RAW format to the YUV format to obtain two SDR images, providing support for the generation of fast thumbnails of HDR. Then, the two images with different exposure degrees are fused into one image to obtain an SDR image, retaining the details of the highlight and shadow parts in the image, balancing the brightness and color in different regions, and improving the image quality of the SDR image.

[0099] In an exemplary embodiment, parameters are adjusted according to the display effect to perform HDR conversion on the SDR image to obtain an HDR image, including: performing gamma mapping processing on the SDR image according to the hybrid log - gamma to obtain the HDR image.

[0100] In an alternative embodiment of the present application, the display effect adjustment parameters include hybrid log - gamma (HLG Gamma) and standard gamma (SRGB Gamma). Optionally, an SDR - to - HDR algorithm is used, combined with the HLG Gamma configured by the effect debugging terminal, to convert the SDR image into an HDR image. Among them, the HLG Gamma curve has unique characteristics. It can support high - dynamic - range display while being compatible with SDR display devices. In the low - brightness region, it is similar to the traditional SDR Gamma curve, ensuring a reasonable display effect on ordinary display devices; while in the high - brightness region, a logarithmic curve is used, which can effectively expand the dynamic range, so as to present richer highlight details and more vivid colors on display devices supporting HDR.

[0101] In the embodiment of the present application, through the hybrid log - gamma parameter, gamma mapping processing is performed on the SDR image to map the dynamic range brightness and color information of the SDR image into the HDR dynamic range, so as to improve the details and color performance of the image, thereby improving the image quality of the HDR image.

[0102] For ease of understanding, the following uses a complete embodiment to illustrate the image processing method provided by the embodiments of the present application.

[0103] Obtain a reference frame original image and an underexposed frame original image from multiple frames of original images obtained by the electronic device during the image shooting process. Among them, the reference frame original image is a frame of image with standard exposure and image quality meeting the preset requirements among the multiple frames of original images; the underexposed frame original image is a frame of underexposed image adjacent to the reference frame among the multiple frames of original images.

[0104] Perform image format conversion processing on the reference frame original image and the underexposed frame original image respectively to obtain an SDR reference frame original image and an SDR underexposed frame original image; fuse the SDR reference frame original image and the SDR underexposed frame original image to obtain an SDR image.

[0105] Perform gamma processing on the SDR image according to the hybrid logarithmic gamma parameter to obtain an HDR image.

[0106] Perform image enhancement processing and / or image correction processing on the HDR image to obtain a processed HDR image.

[0107] According to the display effect adjustment parameters, perform SDR conversion on the processed HDR image to obtain a candidate SDR image and a gain image.

[0108] According to the gain image, perform bitmap conversion on the candidate SDR image to obtain a first quick thumbnail in HDR bitmap format. And, perform upsampling on the candidate SDR image and the gain image respectively to obtain an upsampled candidate SDR image and an upsampled gain image; perform UHDR encoding on the upsampled candidate SDR image and the upsampled gain image to obtain a second quick thumbnail in JPEG format that supports HDR.

[0109] In the embodiments of the present application, the original image collected by the electronic device is processed into an SDR image, and then the SDR image is converted into an HDR image, realizing the format conversion of the SDR image, so as to generate a quick thumbnail based on the HDR image, making the display effect of the quick thumbnail also HDR, maintaining the same display effect as the final image, and thus making the transition from the quick thumbnail to the final image more natural.

[0110] In an exemplary embodiment, taking the quick thumbnails as Quick bitmap and Quick Jpeg as an example, the post-processing process of the native layer of Quick Bitmap and Quick Jpeg is described.

[0111] Such as Figure 7As shown, two frames of raw images, EV0 and EV-, are processed from raw to YUV by IPE, resulting in two frames of SDR P010 images with a size of 960p. Through the synthesis of a two-frame HDR algorithm (CHDR), a frame of SDR P010 image with a highlight suppression effect is obtained.

[0112] Combined with the HLG and SRGB two Gamma configured by the effect debugging terminal, the SDR p010 is converted to HDR P010. Thereafter, quick image algorithms such as beauty, wide-angle distortion correction, filters, and face distortion correction are all based on HDR processing. If the algorithm supports GPU processing, it is processed on a texture of RGBA_1010102, and the CPU algorithm is processed in the format of HDR P010.

[0113] Finally, the HDR image in the p010 format, combined with HLG and SRGB Gamma, obtains the SDR NV21 and the three-channel RGB Gainmap through the HDR to SDR algorithm.

[0114] When converting NV21 to Bitmap, the Gainmap is set into the Bitmap object at the same time, and this Quick Bitmap is returned to the album application layer, and the Quick Bitmap then has the ability to display HDR.

[0115] The 960p NV21 is upsampled to a size of 12M, and the 480p RGB Gainmap is upsampled to a size of 3M, and then encoded by uhdr to obtain a UHDR jpegr format Quick Jpeg. This process is a parallel process with the above process of converting NV21 to Bitmap. In this way, the generation of the quick Jpeg can be ensured not to affect the generation speed of the subsequent photo quick Bitmap.

[0116] In the embodiments of the present application, by generating a quick image with a three-channel UHDR effect, the display effect of the quick image has a higher dynamic range and richer color performance. At the same time, it makes the transition from the quick image to the Final image more natural, and improves the consistency experience of brightness, dynamics, and color in the process of taking pictures to generate photos. It provides the ability support for the HDR effect consistency of the complete photographing link during preview, shooting, and photo processing when the photo is generated.

[0117] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are sequentially shown according to the indications of the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0118] Based on the same inventive concept, an embodiment of the present application also provides an image processing apparatus for implementing the above-mentioned image processing method. The solution provided by this apparatus to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the following image processing apparatus can refer to the limitations on the image processing method in the above text, and will not be repeated here.

[0119] In an exemplary embodiment, as Figure 8 shown, an image processing apparatus 800 is provided, including: an SDR image generation module 801, an HDR image generation module 802, and a fast thumbnail generation module 803, where:

[0120] The SDR image generation module 801 is configured to generate an SDR image according to the original image obtained by the electronic device during the image shooting process;

[0121] The HDR image generation module 802 is configured to perform HDR conversion on the SDR image according to the parameters adjusted for the display effect to obtain an HDR image;

[0122] The fast thumbnail generation module 803 is configured to process the HDR image to obtain a fast thumbnail of the HDR.

[0123] In an alternative embodiment of the present application, the fast thumbnail generation module 803 is specifically configured to: perform SDR conversion on the HDR image according to the parameters adjusted for the display effect to obtain a candidate SDR image and a gain image; generate a fast thumbnail of the HDR according to the candidate SDR image and the gain image.

[0124] In an alternative embodiment of the present application, the fast thumbnail generation module 803 is specifically configured to: perform bitmap conversion on the candidate SDR image according to the gain image to obtain a first fast thumbnail in HDR bitmap format.

[0125] In an alternative embodiment of the present application, the fast thumbnail generation module 803 is specifically configured to: perform upsampling on the candidate SDR image and the gain image respectively to obtain the upsampled candidate SDR image and the upsampled gain image; perform UHDR encoding on the upsampled candidate SDR image and the upsampled gain image to obtain a second fast thumbnail in JPEG format that supports HDR.

[0126] In an alternative embodiment of the present application, the fast thumbnail generation module 803 is specifically configured to: perform image enhancement processing and / or image correction processing on the HDR image to obtain a processed HDR image; process the processed HDR image to obtain a fast thumbnail of HDR.

[0127] In an alternative embodiment of the present application, the fast thumbnail generation module 803 is specifically configured to: if the electronic device includes a graphics processing unit (GPU) and the image processing algorithm supports GPU processing, convert the HDR image into a texture image, and perform image enhancement processing and / or image correction processing on the texture image to obtain a processed HDR image; if the processor is a central processing unit, directly perform image enhancement processing and / or image correction processing on the HDR image to obtain a processed HDR image.

[0128] In an alternative embodiment of the present application, the SDR image generation module 801 is specifically configured to: obtain a reference frame original image and an underexposed frame original image from multiple frames of original images obtained during the image capture process by the electronic device, where the reference frame original image is a frame of image with standard exposure and image quality meeting a preset requirement among the multiple frames of original images; the underexposed frame original image is a frame of underexposed image adjacent to the reference frame among the multiple frames of original images.

[0129] In an alternative embodiment of the present application, the SDR image generation module 801 is specifically configured to: generate an SDR image according to the reference frame original image and the underexposed frame original image, including: perform image format conversion processing on the reference frame original image and the underexposed frame original image respectively to obtain the reference frame original image in SDR format and the underexposed frame original image in SDR format; fuse the reference frame original image in SDR format and the underexposed frame original image in SDR format to obtain an SDR image.

[0130] In an alternative embodiment of the present application, the HDR image generation module 802 is specifically configured to:

[0131] Perform gamma processing on the SDR image according to the hybrid logarithmic gamma parameter to obtain an HDR image.

[0132] Each module in the above image processing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above modules can be embedded in the processor of the computer device in hardware form or independent of the processor, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0133] In an exemplary embodiment, an electronic device is provided, and its internal structure diagram can be as Figure 9 shown. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the electronic device is used for exchanging information between the processor and external devices. The communication interface of the electronic device is used for communicating with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC), or other technologies. The computer program, when executed by the processor, implements an image processing method. The display unit of the electronic device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the electronic device, or an external keyboard, touchpad, or mouse, etc.

[0134] Those skilled in the art can understand that Figure 9 the structure shown in

[0135] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0136] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the foregoing method embodiments are implemented.

[0137] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the foregoing method embodiments are implemented.

[0138] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0139] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0140] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0141] The above-described embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. An image processing method, characterized in that: The method comprises: Generate an SDR image according to an original image obtained by the electronic device during the image capturing process; Adjust parameters according to display effects, and perform HDR conversion on the SDR image to obtain an HDR image; The HDR image is processed to obtain a quick HDR thumbnail.

2. The method according to claim 1, characterized in that The step of processing the HDR image to obtain a quick thumbnail of the HDR includes: According to the display effect adjustment parameter, the HDR image is converted into an SDR image to obtain a candidate SDR image and a gain image; A quick thumbnail of the HDR is generated based on the candidate SDR image and the gain image.

3. The method according to claim 2, characterized in that The step of generating a quick thumbnail of the HDR according to the candidate SDR image and the gain image comprises: According to the gain image, the candidate SDR image is bitmap converted to obtain a first quick thumbnail in an HDR bitmap format.

4. The method according to claim 2, characterized in that: The step of generating a quick thumbnail of the HDR according to the candidate SDR image and the gain image comprises: Upsampling the candidate SDR image and the gain image respectively to obtain an upsampled candidate SDR image and an upsampled gain image; The upsampled candidate SDR image and the upsampled gain image are encoded using UHDR to obtain a second fast thumbnail in a JPEGR format supporting HDR.

5. The method according to claim 1, characterized in that The step of processing the HDR image to obtain a quick thumbnail of the HDR includes: Performing image enhancement processing and / or image correction processing on the HDR image to obtain a processed HDR image; The processed HDR image is processed to obtain a quick thumbnail of the HDR.

6. The method according to claim 5, characterized in that Performing image enhancement processing and / or image correction processing on the HDR image to obtain a processed HDR image includes: If the electronic device includes a graphics processor (GPU) and the image processing algorithm supports the GPU processing, the HDR image is converted into a texture image, and image enhancement processing and / or image correction processing is performed on the texture image to obtain the processed HDR image; If the processor is a central processing unit, image enhancement processing and / or image correction processing is directly performed on the HDR image to obtain the processed HDR image.

7. The method according to any one of claims 1 to 6, characterized in that: The step of generating an SDR image according to an original image obtained by the electronic device during the image capturing process includes: Acquire a reference frame original image and an underexposed frame original image from multiple frames of original images obtained by the electronic device during the image capturing process, wherein the reference frame original image is a frame of the multiple frames of original images with standard exposure and image quality meeting preset requirements; the underexposed frame original image is an underexposed frame of the multiple frames of original images that is adjacent to the reference frame; The SDR image is generated according to the reference frame original image and the underexposed frame original image.

8. The method according to claim 7, characterized in that The step of generating the SDR image according to the reference frame original image and the underexposed frame original image comprises: Performing image format conversion processing on the reference frame original image and the underexposed frame original image respectively to obtain an SDR reference frame original image and an SDR underexposed frame original image; The SDR reference frame original image and the SDR underexposed frame original image are fused to obtain an SDR image.

9. The method according to any one of claims 1 to 6, characterized in that: The display effect adjustment parameter includes a mixed log gamma; and performing HDR conversion on the SDR image according to the display effect adjustment parameter to obtain an HDR image includes: The SDR image is subjected to gamma mapping processing according to the mixed log-gamma to obtain the HDR image.

10. An image processing device, characterized in that: The device comprises: An SDR image generation module is used to generate an SDR image according to an original image obtained by the electronic device during the image capturing process; An HDR image generation module is used to adjust parameters according to display effects, perform HDR conversion on the SDR image, and obtain an HDR image; The fast thumbnail generation module is used to process the HDR image to obtain a fast thumbnail of the HDR.

11. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images