Image processing method, storage medium and display device

By acquiring the brightness map and high-frequency images of HDR videos and determining the tone mapping factor, the problem of image details loss in HDR videos during tone mapping in traditional display devices is solved, and image quality is improved.

CN114679626BActive Publication Date: 2024-11-22SHENZHEN TCL DIGITAL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011556114.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-24
Publication Date
2024-11-22
Estimated Expiration
2040-12-24

AI Technical Summary

Technical Problem

HDR video often loses image details when tone mapping in traditional display devices, affecting image quality.

Method used

By acquiring the brightness map of the image to be processed, the corresponding high-frequency image and brightness map are determined, and tone mapping is performed based on these images, image detail information is collected to determine the tone mapping factor, and image detail information is fully retained.

Benefits of technology

Improves the image effect of the enhanced image after tone mapping, ensuring the integrity of image details.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114679626B_ABST
    Figure CN114679626B_ABST
Patent Text Reader

Abstract

The present application discloses an image processing method, a storage medium and a display device, the method comprising: obtaining a brightness map of an image to be processed; determining a high-frequency image and a brightness mapping map corresponding to the image to be processed based on the brightness map, wherein the high-frequency image is used to reflect the image detail information of the image to be processed; and determining an enhanced image corresponding to the image to be processed based on the high-frequency image and the brightness mapping map. The present application determines a tone mapping factor for the brightness map and the high-frequency image during tone mapping, thereby collecting a high-frequency image and a brightness map carrying image detail information to determine the tone mapping factor, fully retaining the image detail information of the image, thereby improving the image effect of the enhanced image obtained by tone mapping.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, a storage medium and a display device. Background Art

[0002] In recent years, high dynamic range (HDR) technology and related HDR display devices, HDR video production and transmission industries and technologies have been continuously developing and becoming more mature. Compared with traditional standard dynamic range (SDR) video, HDR video provides richer bright and dark details due to the increase in bit depth. However, the video signals of existing display devices (such as traditional TVs, etc.) generally use standard dynamic range video, which requires HDR video to be tone mapped before traditional TVs and monitors so that the HDR brightness is compressed to the display range of the display device. However, HDR video generally has the problem of losing image details during tone mapping, which affects the image quality after tone mapping. Summary of the invention

[0003] The technical problem to be solved by the present application is to provide an image processing method, a storage medium and a display device in view of the deficiencies in the prior art.

[0004] In order to solve the above technical problems, the first aspect of the embodiments of the present application provides an image processing method, the method comprising:

[0005] Get the brightness map of the image to be processed;

[0006] Determining a high-frequency image and a brightness mapping map corresponding to the image to be processed based on the brightness map, wherein the high-frequency image is used to reflect image detail information of the image to be processed;

[0007] An enhanced image corresponding to the image to be processed is determined based on the high-frequency image and the brightness map.

[0008] In the image processing method, the image to be processed is a high dynamic range image, and the enhanced image is a standard dynamic range image.

[0009] The image processing method, wherein the linear light signal corresponding to the image to be processed belongs to a nonlinear space, and before obtaining the brightness map of the image to be processed, the method further comprises:

[0010] Converting the image to be processed into a linear space to obtain a candidate image;

[0011] Normalizing the candidate image to obtain a normalized image;

[0012] The normalized image is subjected to color gamut conversion, and the normalized image after the color gamut conversion is used as the image to be processed, wherein the conversion matrix for color gamut conversion is determined based on a display device used to display the image to be processed.

[0013] The image processing method, wherein the step of determining the high-frequency image corresponding to the image to be processed based on the brightness map specifically comprises:

[0014] Performing logarithmic domain conversion on the brightness image to obtain a first logarithmic image;

[0015] Performing guided filtering on the brightness image, and converting the brightness image after the guided filtering into a logarithmic domain to obtain a second logarithmic image;

[0016] Based on the first logarithmic image and the second logarithmic image, a high-frequency image corresponding to the image to be processed is determined.

[0017] The image processing method, wherein determining the brightness mapping map corresponding to the image to be processed based on the brightness map specifically comprises:

[0018] converting the brightness map into a coded image;

[0019] For each pixel in the encoded image, determining a candidate mapping value corresponding to the pixel based on a preset threshold;

[0020] For each candidate mapping value, performing photoelectric conversion on the candidate mapping value to obtain a target mapping value corresponding to the candidate mapping value;

[0021] An image formed by all the acquired target mapping values ​​is used as a brightness mapping map corresponding to the brightness map.

[0022] The image processing method, wherein the step of determining the candidate mapping value corresponding to the pixel point based on a preset threshold value specifically comprises:

[0023] When the encoding value corresponding to the pixel point is less than a preset threshold, the encoding value of the pixel point is used as the candidate mapping value corresponding to the pixel point;

[0024] When the encoding value corresponding to the pixel is greater than or equal to a preset threshold, the preset threshold is used as the candidate mapping value corresponding to the pixel.

[0025] The image processing method, wherein the step of determining the enhanced image corresponding to the image to be processed based on the high-frequency image and the brightness map specifically comprises:

[0026] Acquire a weight map corresponding to the high-frequency image, wherein the weight map includes weight coefficients corresponding to each pixel point in the high-frequency image;

[0027] Determining a fusion image corresponding to the high-frequency image based on the weight map and the brightness map;

[0028] Determining a mapping factor of the image to be processed based on the fusion image and the brightness image;

[0029] The to-be-processed image is subjected to tone adjustment based on the mapping factor to obtain an enhanced image corresponding to the to-be-processed image.

[0030] The image processing method, wherein before determining the fusion image corresponding to the high-frequency image based on the brightness map, the method further comprises:

[0031] Based on a preset scale threshold, determining a first image portion and a second image portion corresponding to the high-frequency image, wherein a pixel value of each pixel point in the first image portion is greater than or equal to the preset scale threshold, and a pixel value of each pixel point in the second image portion is less than the preset scale threshold;

[0032] adjusting a pixel value of each pixel in the second image portion to obtain a third image portion;

[0033] An image composed of the first image portion and the third image portion is used as a high-frequency image.

[0034] The image processing method, wherein the step of obtaining a weight map corresponding to the high-frequency image specifically comprises:

[0035] Performing filtering on the image to be processed to obtain a filtered image corresponding to the image to be processed;

[0036] Performing guided wave processing on the filtered image based on the brightness map to obtain a guided wave weight map corresponding to the image to be processed;

[0037] Based on a preset weight threshold, determining a first weight map and a second weight map corresponding to the guided wave weight map, wherein the pixel points in the guided wave weight map whose pixel values ​​are greater than or equal to the preset weight threshold are included in the first weight map, and the pixel points in the guided wave weight map whose pixel values ​​are less than the preset weight threshold are included in the second weight map;

[0038] Based on a preset brightness weight function, determining a first target weight map corresponding to the first weight map, and a second target weight map corresponding to the second weight map;

[0039] The first target weight map and the second target weight map are fused to obtain a weight map corresponding to the high-frequency image.

[0040] The image processing method, wherein the image to be processed is an image that has undergone color gamut conversion, and the conversion matrix used for color gamut conversion is determined based on a display device used to display the image to be processed; and the step of adjusting the hue of the image to be processed based on the mapping factor to obtain an enhanced image corresponding to the image to be processed specifically comprises:

[0041] Mapping the image to be processed based on the mapping factor to obtain a mapped image;

[0042] Performing color adjustment on the mapped image based on a color threshold to obtain an adjusted image;

[0043] Gamma correction is performed on the adjusted image to obtain an enhanced image corresponding to the image to be processed.

[0044] A second aspect of an embodiment of the present application provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in any of the image processing methods described above.

[0045] A third aspect of an embodiment of the present application provides a display device, comprising: a processor, a memory, and a communication bus; the memory stores a computer-readable program that can be executed by the processor;

[0046] The communication bus realizes the connection and communication between the processor and the memory;

[0047] When the processor executes the computer-readable program, the steps in any of the above-mentioned image processing methods are implemented.

[0048] Beneficial effects: Compared with the prior art, the present application provides an image processing method, a storage medium and a display device, the method comprising: obtaining a brightness map of an image to be processed; determining a high-frequency image and a brightness mapping map corresponding to the image to be processed based on the brightness map, wherein the high-frequency image is used to reflect the image detail information of the image to be processed; and determining an enhanced image corresponding to the image to be processed based on the high-frequency image and the brightness mapping map. The present application determines a tone mapping factor for a brightness map and a high-frequency image during tone mapping, thereby collecting a high-frequency image and a brightness map carrying image detail information to determine a tone mapping factor, fully retaining the image detail information of the image, thereby improving the image effect of the enhanced image obtained by tone mapping. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without inventive work.

[0050] Figure 1 A flowchart of the image processing method provided in this application.

[0051] Figure 2 This is an example diagram of the principle of the image processing method provided in this application.

[0052] Figure 3 This is an example flowchart of the weight map acquisition process in the image processing method provided in this application.

[0053] Figure 4 This is a schematic diagram of the structure of the display device provided in this application. DETAILED DESCRIPTION

[0054] The present application provides an image processing method, a storage medium and a display device. To make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0056] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of the ordinary technicians in the field to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have a meaning consistent with the meaning in the context of the prior art, and unless specifically defined as here, will not be interpreted with an idealized or overly formal meaning. In addition, the sequence number and size of each step in the present embodiment do not mean the order of execution, and the execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0057] The inventor has found through research that in recent years, high dynamic range (HDR) technology and related HDR display devices, HDR video production and transmission and other industries and technologies have been continuously developing and becoming more mature. Compared with traditional standard dynamic range (SDR) video, HDR video provides richer bright area details and dark area details due to the increase in bit depth. However, the video signals of existing display devices (e.g., traditional televisions, etc.) generally use standard dynamic range video, which requires tone mapping of HDR video before traditional televisions and monitors so that the brightness of HDR is compressed to the display range of the display device. However, HDR video generally has the problem of losing image details during tone mapping, which affects the image quality after tone mapping.

[0058] In order to solve the above problems, in an embodiment of the present application, a brightness map of an image to be processed is obtained; based on the brightness map, a high-frequency image and a brightness mapping map corresponding to the image to be processed are determined, wherein the high-frequency image is used to reflect the image detail information of the image to be processed; based on the high-frequency image and the brightness mapping map, an enhanced image corresponding to the image to be processed is determined. The present application determines a tone mapping factor for a brightness map and a high-frequency image during tone mapping, thereby collecting a high-frequency image and a brightness map carrying image detail information to determine a tone mapping factor, fully retaining the image detail information of the image, thereby improving the image effect of the enhanced image obtained by tone mapping.

[0059] For example, the embodiment of the present application can be applied to a brightness range of 0.01 cd / m 2 To 100cd / m 2In this scenario, first, the display device can obtain a brightness map of a to-be-processed image with a high dynamic range; determine a high-frequency image and a brightness mapping map corresponding to the to-be-processed image based on the brightness map, wherein the high-frequency image is used to reflect the image detail information of the to-be-processed image; perform tone mapping on the to-be-processed image based on the high-frequency image and the brightness mapping map to obtain an enhanced image with a standard dynamic range corresponding to the to-be-processed image; and finally display the enhanced image.

[0060] It is understandable that in the above application scenario, although the actions of the implementation of the present application are described as being partially performed by the display device, these actions can also be partially performed by the display device and partially performed by the server connected to the display device. For example, the display device is used to obtain a high dynamic range image to be processed, and send the obtained image to be processed to the server; the server receives the image to be processed and obtains a brightness map of the high dynamic range image to be processed; based on the brightness map, a high-frequency image and a brightness mapping map corresponding to the image to be processed are determined, wherein the high-frequency image is used to reflect the image detail information of the image to be processed; based on the high-frequency image and the brightness mapping map, the image to be processed is tone mapped to obtain an enhanced image of a standard dynamic range corresponding to the image to be processed, and the enhanced image is fed back to the display device; finally, the display device displays the enhanced image. Therefore, the present application is not limited in terms of the execution subject, as long as the actions disclosed in the implementation of the present application are performed.

[0061] It should be noted that the above application scenarios are only shown for the purpose of facilitating the understanding of the present application, and the implementation of the present application is not limited in this respect. On the contrary, the implementation of the present application can be applied to any applicable scenario.

[0062] The application content is further explained below through the description of embodiments in conjunction with the accompanying drawings.

[0063] This implementation provides an image processing method, such as Figure 1 and 2 As shown, the method includes:

[0064] S10: Obtain a brightness map of the image to be processed.

[0065] Specifically, the image to be processed may be a video frame sent from an external signal source to a display device (e.g., a conventional television, etc.) equipped with the image processing method, wherein the external signal source is used to send the image to be processed to the display device, and the display device is used to process the image to be processed according to the image processing method provided in this embodiment. In this embodiment, the image to be processed may be a high dynamic range (HDR) image, wherein the dynamic range refers to the ratio of the maximum brightness to the minimum brightness in the video signal.

[0066] The brightness map is used to reflect the brightness of each pixel in the image to be processed. The image size of the brightness map is the same as the image size of the image to be processed. For each pixel in the image to be processed, there is a target pixel corresponding to the pixel in the brightness map. The target pixel is used to reflect the brightness value of the pixel. The target pixel corresponding to the pixel means that the pixel position of the target pixel in the brightness map is the same as the pixel position of the pixel in the image to be processed. For example, if the image size of the image to be processed is 256*256, then the image size of the brightness map is 256*256, and the pixel position of the target pixel corresponding to the pixel (100,100) in the image to be processed is (100,100) in the brightness map.

[0067] In one implementation of this embodiment, the brightness map can be determined based on the R value, G value and B value of the pixel points in the image to be processed, and the determination process may include: for each pixel point in the image to be processed, obtain the R value, G value and B value of the pixel point; select the maximum value among the R value, G value and B value, and use the obtained maximum value as the brightness value corresponding to the pixel point; finally, use the image composed of all the obtained brightness values ​​as the brightness map corresponding to the image to be processed. Of course, it is worth explaining that this embodiment is explained as an RGB image to be processed. When the color space to which the image to be processed belongs is not an RGB space, the color space of the image to be processed can be converted to the RGB space first, and then the brightness map corresponding to the image to be processed is determined. For example, when the color space to which the image to be processed belongs is a YUV space, the color space to which the image to be processed belongs can be converted from the YUV space to the RGB space, and then for each pixel point in the image to be processed, obtain the R value, G value and B value of the pixel point,

[0068] Since HDR images are generally transmitted to display devices via electrical signals, HDR images generally belong to nonlinear space, and thus the image to be processed in this embodiment may belong to nonlinear space. Therefore, before determining the brightness map of the image to be processed, the image to be processed may be preprocessed, and the image to be processed may belong to linear space through preprocessing. Accordingly, in an implementation of this embodiment, the linear light signal corresponding to the image to be processed belongs to nonlinear space, and before obtaining the brightness map of the image to be processed, the method further includes:

[0069] Converting the image to be processed into a linear space to obtain a candidate image;

[0070] Normalizing the candidate image to obtain a normalized image;

[0071] The normalized image is subjected to color gamut conversion, and the normalized image after the color gamut conversion is used as the image to be processed, wherein the conversion matrix for color gamut conversion is determined based on a display device used to display the image to be processed.

[0072] Specifically, the linear space refers to the space where the linear light signal is located; the nonlinear space refers to the space where the linear light signal is transformed using the nonlinear region. The nonlinear curves used by HDR include PQ EOTF -1 curve, HLG OETF curve, etc. It can be understood that the nonlinear space can be considered as a visual linear space. -1 The curve is the inverse curve of PQ EOTF. The PQ EOTF curve is used to convert electrical signals into linear optical signals. Correspondingly, PQEOTF -1 The curve is used to convert a linear optical signal of [0,10000] nits into a PQ-coded electrical signal, where PQ EOTF -1 The curve is a linear light signal with an input of [0,10000] nits and an output of [0,1]. The HLG OETF curve is used to convert the linear scene light into a non-information electrical signal by HLG encoding.

[0073] Further, the image to be processed is an electrical signal belonging to a nonlinear space, so that after the image to be processed is acquired, the image to be processed can be electro-optically converted to convert the image to be processed into a linear space. For example, when the image to be processed is an RGB image, converting the image to be processed into a linear space means converting the image to be processed into a linear RGB space by using the n-bit encoding bit of the nonlinear red, green and blue (RGB) space of the image to be processed. In addition, when the image to be processed is electro-optically converted, an EOTF electro-optical transfer function can be used to convert the image to be processed into a linear space through the EOTF electro-optical transfer function to obtain a candidate image, wherein the EOTF electro-optical transfer function is a conversion relationship from a nonlinear primary color value to a linear primary color value, and the primary color value is a numerical value of a corresponding feature map image color component (for example, R, G, B or Y); the linear primary color value is proportional to the light intensity, and the nonlinear primary color value is a normalized digital expression value of the image information, which is proportional to the digital code (the digital expression value of the image signal, and the digital code value is used to represent the nonlinear primary color value). In a specific implementation of this embodiment, the EOTF electro-optical transfer function adopts the PQ curve developed by Dolby and published by the SMPTE organization in SMPTE ST20843. It is suitable for HDR images with brightness levels higher than 10000cd / m^2, and the color gamut is suitable for BT.2020 standard HDR images, wherein the color gamut is the range of colors contained in the color space, and the BT.2020 standard is the color gamut standard.

[0074] Further, after acquiring the candidate image, the candidate image is normalized, wherein the normalization can be the brightness value of each pixel in the candidate image divided by the maximum brightness value of 10000 nits allowed by Dolby Vision to obtain a normalized image. The conversion matrix for color gamut conversion is determined based on the display device used to display the image to be processed, so that the normalized image after the color gamut conversion meets the color gamut requirements of the display device. In one implementation of this embodiment, the determination process of the conversion matrix can be: first use a color analyzer (e.g., CA410, etc.) to measure the chromaticity coordinate information of the display device displaying a pure red, green, blue and white screen, and determine the 3*3 conversion matrix of the BT.2020 color gamut to which the HDR image belongs to the display color gamut of the display device based on the measured chromaticity coordinate information to obtain a conversion matrix. After acquiring the conversion matrix, it is determined that the HDR image is mapped from the BT.2020 color gamut to the display color gamut of the display device, and the converted normalized image is used as the image to be processed.

[0075] S20: Determine a high-frequency image and a brightness mapping map corresponding to the image to be processed based on the brightness map.

[0076] Specifically, the high-frequency image is used to reflect the image detail information of the image to be processed, and the brightness mapping diagram is an image obtained by brightness mapping the image to be processed, wherein the brightness mapping is used to map the brightness of the image to be processed to the brightness of the target system. In this embodiment, the target system refers to the brightness system of the display device used to display the image to be processed, and the brightness of the image to be processed is mapped to the brightness system of the display device so that the display device can display the image to be processed, so as to avoid display errors or display abnormalities caused by the brightness of the image to be processed being higher than the maximum display brightness in the brightness system of the display device.

[0077] In an implementation of this embodiment, determining the brightness map corresponding to the image to be processed based on the brightness map specifically includes:

[0078] converting the brightness map into a coded image;

[0079] For each pixel in the encoded image, determining a candidate mapping value corresponding to the pixel based on a preset threshold;

[0080] For each candidate mapping value, performing photoelectric conversion on the candidate mapping value to obtain a target mapping value corresponding to the candidate mapping value;

[0081] An image formed by all the acquired target mapping values ​​is used as a brightness mapping map corresponding to the brightness map.

[0082] Specifically, the encoded image is a brightness map represented by a digital coding value, and the value of each pixel in the encoded image corresponds to a digital coding value, which is recorded as a coding value. The preset threshold can be pre-configured, and is used to classify the pixels in the encoded image, and determine the candidate mapping value corresponding to each pixel based on the classification result, wherein the pixels whose digital coding values ​​corresponding to the pixels are greater than or equal to the preset threshold are classified into one category, and the pixels whose coding values ​​corresponding to the pixels are less than the preset threshold are classified into another category. Based on this, the determination of the candidate mapping value corresponding to the pixel based on the preset threshold specifically includes: when the coding value corresponding to the pixel is less than the preset threshold, the coding value of the pixel is used as the candidate mapping value corresponding to the pixel; when the coding value corresponding to the pixel is greater than or equal to the preset threshold, the preset threshold is used as the candidate mapping value corresponding to the pixel.

[0083] In one implementation of this embodiment, the calculation formula of the target mapping value can be expressed as:

[0084] val tp_nor =(val tp -TDB) / TDP

[0085] val tp =EOTF(valencode_tp ×CMDP)

[0086] val encode_tp =val′ encode +b×(1-val′ encode ) 4

[0087]

[0088] Among them, val tp To map the brightness value, val tp_nor val tp Normalized target brightness mapping value, val′ encode is the candidate mapping value, val encode_tp is the intermediate value determined based on the candidate mapping value, val encode is the coding value, K s ,P,b calculation formula is as follows:

[0089] K s =1.5*max_Lum-0.5

[0090] P=(2T3-3T2+1)*KS+(T3-2T2+T)*(1-KS)+(-2T3+3T2)*max_Lum

[0091] b=CTDB / CMDP

[0092] Among them, T,max_Lum is calculated as follows:

[0093] T=(val encode -K s ) / (1-K s )

[0094] Max_Lum=CTDP / CMDP

[0095] Among them, CMDP, CTDP, and CTDB are the nonlinear coding values ​​after MDP, TDP, and TDB photoelectric conversion (OETF in the PQ curve), respectively; MDP Mastering Display Peak) is the peak brightness of the reference display; TDP (Target Display Peak) is the peak brightness of the display device; and TDB (Target Display Black) is the minimum brightness of the display device.

[0096] In an implementation of this embodiment, determining the high-frequency image corresponding to the image to be processed based on the brightness map specifically includes:

[0097] Performing logarithmic domain conversion on the brightness image to obtain a first logarithmic image;

[0098] Performing guided filtering on the brightness image, and converting the brightness image after the guided filtering into a logarithmic domain to obtain a second logarithmic image;

[0099] Based on the first logarithmic image and the second logarithmic image, a high-frequency image corresponding to the image to be processed is determined.

[0100] Specifically, the logarithmic domain conversion of the brightness map refers to performing a logarithmic operation on the pixel values ​​corresponding to each pixel in the brightness map, and using the value obtained by the logarithmic operation as the pixel value corresponding to the pixel, so as to obtain the first logarithmic image corresponding to the brightness map. It can be understood that for each pixel in the first logarithmic image, there is a reference pixel in the brightness map, and the value of the pixel is equal to the logarithm of the pixel value corresponding to the reference pixel. For example, for pixel a in the first logarithmic image, pixel b in the brightness map is the reference pixel corresponding to pixel a, then the pixel value corresponding to pixel a is equal to log(pixel value corresponding to the pixel), wherein the pixel position of pixel b in the brightness map is the same as the pixel position of pixel a in the first logarithmic image, for example, both are (50,50).

[0101] The guided filtering is an image filtering technology, which filters the initial image through a guided image, so that the image content information of the filtered image is similar to that of the initial image, and the texture information of the filtered image is similar to that of the guided image. In this embodiment, when the guided filtering is performed on the brightness map, the guided image and the initial image are both the brightness maps, so that the filtered brightness map retains the image content information and texture information of the brightness map, achieving the effect of edge-preserving smoothing. In addition, after obtaining the second logarithmic image, the brightness map after the guided filtering is converted to the logarithmic domain to obtain the second logarithmic image, wherein the process of converting the second logarithmic image to the logarithmic domain is the same as the process of converting the brightness map to the logarithmic domain, which will not be repeated here, and the specific instructions for converting the brightness map to the logarithmic domain can be referred to.

[0102] After obtaining the first logarithmic image and the second logarithmic image, the first logarithmic image and the second logarithmic image are subtracted to obtain a high-frequency image corresponding to the image to be processed. The calculation formula of the high-frequency image can be expressed as:

[0103] Detail=log(val)-log(guide_filter(I=val,P=val))

[0104] Where Detail represents the high-frequency image, val represents the brightness image, log(val) represents the first logarithmic image, log(guide_filter(I=val,P=val)) represents the second logarithmic image, guide_filter(·) represents the guided filter, I represents the guided image, and P represents the initial image.

[0105] S30: Determine an enhanced image corresponding to the image to be processed based on the high-frequency image and the brightness map.

[0106] Specifically, the enhanced image is a processed image corresponding to the image to be processed, the image size of the enhanced image is the same as the image size of the image to be processed, the dynamic range of the enhanced image is smaller than the dynamic range of the image to be processed, and the dynamic range of the enhanced image is smaller than or equal to the dynamic range of a display device used to display the image to be processed, so that the enhanced image can be displayed on the display device. In an implementation of this embodiment, the enhanced image is a standard dynamic range (SDR) image.

[0107] In an implementation of this embodiment, determining the enhanced image corresponding to the image to be processed based on the high-frequency image and the brightness map specifically includes:

[0108] S31, obtaining a weight map corresponding to the high-frequency image;

[0109] S32, determining a fusion image corresponding to the high-frequency image based on the weight map and the brightness map;

[0110] S33, determining a mapping factor of the image to be processed based on the fusion image and the brightness image;

[0111] S34. Adjust the tone of the image to be processed based on the mapping factor to obtain an enhanced image corresponding to the image to be processed.

[0112] Specifically, in step S31, the weight map includes weight coefficients corresponding to each pixel in the high-frequency image, and the weight coefficients corresponding to the pixel points are used to reflect the importance of the pixel points, wherein the larger the weight coefficient, the higher the importance of the pixel point; conversely, the smaller the weight coefficient, the lower the importance of the pixel point.

[0113] In one implementation of this embodiment, Figure 3 As shown, the obtaining of the weight map corresponding to the high-frequency image specifically includes:

[0114] Performing filtering on the image to be processed to obtain a filtered image corresponding to the image to be processed;

[0115] Performing guided wave processing on the filtered image based on the brightness map to obtain a guided wave weight map corresponding to the image to be processed;

[0116] Based on a preset weight threshold, determining a first weight map and a second weight map corresponding to the guided wave weight map, wherein the weight corresponding to each pixel point in the first weight map is greater than or equal to the preset weight threshold, and the weight corresponding to each pixel point in the second weight map is less than the preset weight threshold;

[0117] Based on a preset brightness weight function, determining a first target weight map corresponding to the first weight map, and a second target weight map corresponding to the second weight map;

[0118] The first target weight map and the second target weight map are fused to obtain a weight map corresponding to the high-frequency image.

[0119] Specifically, the filtered image carries high-frequency detail information of the image to be processed, and the signal-to-noise ratio of the filtered image is higher than that of the image to be processed. For example, the signal-to-noise ratio of the filtered image is 60db, and the signal-to-noise ratio of the image to be processed is 50db. In a specific implementation of this embodiment, the high-frequency detail information of the image to be processed included in the filtered image is obtained by performing Laplace filtering on the image to be processed, and improving the signal-to-noise ratio of the filtered image can be achieved by Gaussian filtering. Based on this, the filtering process can include a Laplace filtering process and a Gaussian filtering process. Correspondingly, the filtering process of the image to be processed to obtain the filtered image corresponding to the image to be processed can be specifically: performing Laplace filtering on the image to be processed to obtain a first filtered image, and then performing Gaussian filtering on the first filtered image to obtain a filtered image, wherein the Laplace filtering uses a high-pass filter and the Gaussian filtering uses a low-pass filter. In this way, after obtaining the first filtered image corresponding to the image to be processed through Laplace filtering, performing Gaussian filtering on the first filtered image can remove noise in the first filtered image and avoid the influence of noise on the fusion image. Of course, it is worth noting that the use of Laplace filtering and Gaussian filtering to process the image to be processed in this application is only one implementation method. Other filtering methods that can improve the high-frequency detail information of the image to be processed, as well as filtering methods that can be used for denoising, can be used in this embodiment and will not be explained one by one here.

[0120] In one implementation of this embodiment, the filter window of the Gaussian filter can be 11×11, the convolution kernel size of the Gaussian filter is 5, the standard deviation is 5, and the filter window of the Laplace filter can be a 3×3 window. In addition, after the first filtered image is obtained by Laplace filtering, the absolute value operation is performed on the first filtered image, so that the negative values ​​generated by the Laplace filter in the x and y direction gradients can be removed, thereby avoiding the problem that the subsequent color gamut mapping is incorrect due to the presence of negative values ​​in the weight map. Of course, it is worth noting that when filtering, the boundary of the image to be filtered can be filled, wherein the filling is performed in an axially symmetrical manner with the most edge pixels, so that the pixel value of the filled pixel point is the same as the pixel value of the pixel point corresponding to it, wherein the edge pixel point between the pixel point and the pixel point symmetrical to it is the symmetry center.

[0121] In one implementation of this embodiment, the guided filtering is an image filtering technology, which filters the initial image through a guiding image, so that the image content information of the filtered image is similar to that of the initial image, and the texture information of the filtered image is similar to that of the guiding image. When the guided wave processing is performed on the filtered image based on the brightness map, the guiding image is the brightness map, and the initial image is the filtered image. In this way, by using the brightness map as the guiding image and the filtered image as the initial image to perform the guided filtering processing on the filtered image, artifacts can be prevented from existing in the fused image. In addition, the guided wave weight map can be expressed as:

[0122] Wg=guide_filter(I=val,P=img_lap_gussi_mask)

[0123] Wherein, Wg represents the guided wave weight map, val represents the brightness map, and img_lap_gussi_mask represents the filtered image.

[0124] In addition, in order to eliminate the influence of noise on the obtained guided wave weight map and enhance the texture and other details of the guided wave weight map, after obtaining the guided filter weight map Wg, the guided filter weight map Wg is normalized and adaptively transformed, wherein the specific formulas of the normalization and adaptive transformation are as follows:

[0125]

[0126] W g ′=a·(sigm(c(W g -b))-sigm(-c(W g +b)))

[0127]

[0128]

[0129] Among them, Wg represents the guided wave weight map, and W’ gg represents the derivative of Wg, 0 < b < 1, and b and c are used to control the amplitude of the weight value.

[0130] In an implementation manner of this embodiment, the preset weight threshold is used as the basis for dividing the guided wave weight map image. Based on this preset weight threshold, the first weight map and the second weight map corresponding to the guided wave weight map can be obtained. The image size of the first weight map is the same as that of the guided wave weight map, and the image size of the second weight map is the same as that of the guided wave weight map. Moreover, the pixel points in the guided wave weight map whose pixel values are greater than or equal to the preset weight threshold are included in the first weight map, and the pixel points in the guided wave weight map whose pixel values are less than the preset weight threshold are included in the second weight map. In other words, as Figure 3 shown, for the pixel points in the guided wave weight map whose pixel values are greater than or equal to the preset weight threshold, there is a corresponding pixel point in the first weight image, and the pixel value of this corresponding pixel point is the same as the pixel value of this pixel point. For the pixel points in the guided wave weight map whose pixel values are less than the preset weight threshold, there is a corresponding pixel point in the first weight image, and the pixel value of this corresponding pixel point is the first preset value; for the pixel points in the guided wave weight map whose pixel values are greater than or equal to the preset weight threshold, there is a corresponding pixel point in the second weight image, and the pixel value of this corresponding pixel point is the second preset value; for the pixel points in the guided wave weight map whose pixel values are less than the preset weight threshold, there is a corresponding pixel point in the first weight image, and the pixel value of this corresponding pixel point is the same as the pixel value of this pixel point. In an implementation manner of this embodiment, the first preset value and the second preset value can be equal. For example, they are both 0.

[0131] The preset weight threshold can be set in advance or determined based on the guided wave weight map. In an implementation manner of this embodiment, the preset weight threshold is determined based on the guided wave weight map. And when the guided wave weight map is the guided wave weight map after normalization and adaptive processing, then the preset weight threshold is based on the guided wave weight map after normalization and adaptive processing. Among them, the preset threshold can be equal to the average value of the pixel values of each pixel point in the guided wave weight map, or determined based on the maximum pixel value corresponding to the guided wave weight map (for example, equal to one-fourth, one-third, etc. of the maximum pixel value), or determined based on the minimum pixel value corresponding to the guided wave weight map (for example, equal to twice, three times, etc. of the maximum pixel value).

[0132] For example, the preset weight threshold is equal to one-fourth of the maximum pixel value corresponding to the guided wave weight map, and the determination formulas corresponding to the first weight map and the second weight map can be:

[0133]

[0134]

[0135] Among them, W′ g_weak Represents the second weight graph, W′ g_strong Denotes the first weight graph, W′ g represents the normalized and adaptively processed guided wave weight map, max(W′ g ) expresses the maximum pixel value corresponding to the guided wave weight map.

[0136] The brightness weight function is used for the low brightness area and high brightness area in the image to be processed, so as to reduce the loss of image details in the low brightness area during the tone mapping process, to ensure the visual effect of the low brightness area, and to prevent overexposure after retaining the details of the high brightness. In this embodiment, the expression of the brightness weight function can be:

[0137]

[0138] Among them, v e Represents the coded image, a1, b1, c1, a2, b2, c2, a3, b3, c3, a4, b4 and c4 are all constants.

[0139] Based on this, the calculation formulas of the first target weight map and the second target weight map may be:

[0140]

[0141]

[0142] Among them, W″ g_weak Represents the second target weight graph, W′ g_weak Represents the second weight graph, W″ g_strong Represents the first target weight graph, W′ g_strong represents the first weight graph, β are adjustment parameters, The larger β is, the greater the detail enhancement weight is, and the larger β is, the smaller the detail enhancement weight is.

[0143] In an implementation of this embodiment, before determining the fusion image corresponding to the high-frequency image based on the brightness map, the method further includes:

[0144] Based on a preset scale threshold, determining a first image portion and a second image portion corresponding to the high-frequency image, wherein a pixel value of each pixel point in the first image portion is greater than or equal to the preset scale threshold, and a pixel value of each pixel point in the second image portion is less than the preset scale threshold;

[0145] adjusting a pixel value of each pixel in the second image portion to obtain a third image portion;

[0146] An image composed of the first image portion and the third image portion is used as a high-frequency image.

[0147] Specifically, the preset scale threshold may be pre-configured and is the basis for dividing the high-frequency image. For example, the preset scale threshold may be 0, etc. The first image portion is included in the high-frequency image, the second image portion is included in the high-frequency image, and the first image portion and the second image portion constitute the high-frequency image. In addition, for any pixel point in the first image portion, the pixel point is not included in the second image portion. Conversely, for any pixel point in the second image portion, the pixel point is not included in the first image portion. In a specific implementation of this embodiment, the pixel value of each pixel point in the first image portion is greater than or equal to the preset scale threshold, and the pixel value of each pixel point in the second image portion is less than the preset scale threshold.

[0148] In one implementation, the preset scale threshold is 0, and the expressions corresponding to the first image part and the second image part can be:

[0149] Detail_pos=Detail(Detail>=0)

[0150] Detail_neg=Detail(Detail<0)

[0151] Here, Detail_pos represents the first image part, and Detail_neg represents the second image part.

[0152] The first image portion is used to reflect the positive scale area in the high-frequency image, and the second image portion is used to reflect the negative scale area in the high-frequency image. After obtaining the second image portion for representing the negative scale area, the second image portion can be compressed to suppress the problem of black edges formed by the negative scale area. In a specific implementation, the process of adjusting the pixel value of each pixel point in the second image portion can be expressed as:

[0153] Detail_neg′→Detail_neg / constant

[0154] Wherein, Detail_neg′ represents the third image portion, Detail_neg represents the second image portion, and constant represents the compression factor, which is an empirical constant.

[0155] After acquiring the third image part, the first image part and the third image part are added, and the added image is used as a high-frequency image, which can improve the dynamic contrast between the positive scale area and the negative scale area, thereby improving the display effect of the image. The high-frequency image can be expressed as Detail_restrain_neg = Detail_pos + Detail_neg'.

[0156] Further, in the step S32, after obtaining the weight map, determining the fusion map corresponding to the high-frequency image based on the weight map and the brightness map may be weighting the brightness map and the high-frequency image to obtain the fusion map, wherein the weight coefficient of each pixel in the high-frequency image is determined based on the weight map. In a specific implementation, the weight coefficient of each pixel in the brightness map is 1, and accordingly, the calculation formula of the fusion map may be:

[0157] Out=log(valtp_nor)+Detail_restrain_neg*Refined_weight_map

[0158] Among them, Refined_weight_ma represents the weight map, log(valtp_nor) represents the brightness map, and Detail_restrain_neg represents the high-frequency image.

[0159] Further, in step S33, the mapping factor is an adjustment parameter for adjusting the hue, and the hue of the image to be processed is adjusted based on the mapping factor to obtain an enhanced image. The calculation formula of the mapping factor can be:

[0160]

[0161] Among them, val represents the brightness image and Out represents the fusion image.

[0162] Further, in the step S34, the tone adjustment is to perform tone mapping on the image to be processed based on the mapping factor to adjust the tone of the image to be processed. Before tone mapping is performed on the image to be processed, the image to be processed needs to be preprocessed, and the image obtained by the preprocessing is used as the image to be processed, wherein the preprocessing includes linear space conversion, normalization processing, and color gamut conversion processing. The preprocessing process is the same as the operation on the image to be processed before obtaining the brightness map. For details, please refer to the above description, and it will not be repeated here.

[0163] In an implementation of this embodiment, the image to be processed is a pre-processed image to be processed; and adjusting the tone of the image to be processed based on the mapping factor to obtain an enhanced image corresponding to the image to be processed specifically includes:

[0164] Mapping the image to be processed based on the mapping factor to obtain a mapped image;

[0165] Performing color adjustment on the mapped image based on a color threshold to obtain an adjusted image;

[0166] Gamma correction is performed on the adjusted image to obtain an enhanced image corresponding to the image to be processed.

[0167] Specifically, mapping the image to be processed based on the mapping factor refers to mapping the three channels of RGB of the image to be processed respectively according to the mapping factor R based on the mapping factor to obtain a mapped image, wherein the mapping process can be expressed as:

[0168] out_R=img_R*β; out_G=img_G*β; out_B=img_B*β

[0169] Among them, img is the image to be processed, out is the mapped image, and β represents the mapping factor.

[0170] After acquiring the mapped image, in order to prevent the color of the image to be processed from being overexposed, the mapped image may be color adjusted based on a color threshold, wherein the color threshold may be preset. In a specific implementation, the color threshold is 1, and accordingly, the color adjustment may be identified as:

[0171]

[0172] Among them, out(x,y) represents the pixel point of the mapped image, out′(x,y) represents the pixel point in the adjusted image, (x,y) represents the pixel coordinates, and max(out(x,y)) represents the maximum value of the three RGB channels in the pixel point with pixel coordinates (x,y).

[0173] The gamma correction is used to adjust the brightness mapping curve by gamma compensation, wherein the gamma adjustment can be expressed as output = (input) γ , where γ is the brightness adjustment factor. When the brightness adjustment factor is 1 / 2.2, the gamma correction can be expressed as:

[0174]

[0175] Wherein, out′ represents the adjusted image, and Out′ represents the adjusted image after gamma correction.

[0176] In addition, after obtaining the gamma-corrected adjusted image, the gamma-corrected adjusted image is multiplied by 255 to enlarge the image color value to the range of 0 to 255, and the pixel values ​​outside the range of 0 to 255 are truncated and rounded down to obtain the enhanced image, wherein the expression of the enhanced image can be:

[0177] img_tp=Out*255

[0178]

[0179] Among them, the floor() function is a floor rounding function, and img_tp is an enhanced image.

[0180] In summary, this embodiment provides an image processing method, which includes: obtaining a brightness map of an image to be processed; determining a high-frequency image and a brightness mapping map corresponding to the image to be processed based on the brightness map, wherein the high-frequency image is used to reflect the image detail information of the image to be processed; and determining an enhanced image corresponding to the image to be processed based on the high-frequency image and the brightness mapping map. This embodiment determines a tone mapping factor for the brightness map and the high-frequency image during tone mapping, thereby collecting a high-frequency image and a brightness map carrying image detail information to determine the tone mapping factor, fully retaining the image detail information of the image, thereby improving the image effect of the enhanced image obtained by tone mapping.

[0181] Based on the above-mentioned image processing method, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the image processing method as described in the above-mentioned embodiment.

[0182] Based on the above image processing method, the present application also provides a display device, such as Figure 4As shown, it includes at least one processor (processor) 20; display screen 21; and memory (memory) 22, and may also include a communications interface (Communications Interface) 23 and a bus 24. Among them, the processor 20, the display screen 21, the memory 22 and the communication interface 23 can communicate with each other through the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setting mode. The communication interface 23 can transmit information. The processor 20 can call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0183] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0184] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0185] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the display device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.

[0186] In addition, the specific process of loading and executing the multiple instruction processors in the above storage medium and the display device has been described in detail in the above method, and will not be described here one by one.

[0187] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Get the brightness map of the image to be processed; Determining a high-frequency image and a brightness mapping map corresponding to the image to be processed based on the brightness map, wherein the high-frequency image is used to reflect image detail information of the image to be processed; Acquire a weight map corresponding to the high-frequency image, wherein the weight map includes weight coefficients corresponding to each pixel point in the high-frequency image; Determining a fusion image corresponding to the high-frequency image based on the weight map and the brightness map; Determining a mapping factor of the image to be processed based on the fusion image and the brightness image; The to-be-processed image is subjected to tone adjustment based on the mapping factor to obtain an enhanced image corresponding to the to-be-processed image.

2. The image processing method according to claim 1, characterized in that: The image to be processed is a high dynamic range image, and the enhanced image is a standard dynamic range image.

3. The image processing method according to claim 1, characterized in that: The linear light signal corresponding to the image to be processed in the brightness map of the image to be processed is obtained and belongs to a nonlinear space; Before obtaining the brightness map of the image to be processed, the method further includes: Converting the image to be processed into a linear space to obtain a candidate image; Normalizing the candidate image to obtain a normalized image; The normalized image is subjected to color gamut conversion, and the normalized image after the color gamut conversion is used as the image to be processed, wherein the conversion matrix for color gamut conversion is determined based on a display device used to display the image to be processed.

4. The image processing method according to claim 1, characterized in that: Determining the high-frequency image corresponding to the image to be processed based on the brightness map specifically includes: Performing logarithmic domain conversion on the brightness image to obtain a first logarithmic image; Performing guided filtering on the brightness image, and converting the brightness image after the guided filtering into a logarithmic domain to obtain a second logarithmic image; Based on the first logarithmic image and the second logarithmic image, a high-frequency image corresponding to the image to be processed is determined.

5. The image processing method according to claim 1, characterized in that: Determining the brightness map corresponding to the image to be processed based on the brightness map specifically includes: converting the brightness map into a coded image; For each pixel in the encoded image, determining a candidate mapping value corresponding to the pixel based on a preset threshold; For each candidate mapping value, performing photoelectric conversion on the candidate mapping value to obtain a target mapping value corresponding to the candidate mapping value; An image formed by all the acquired target mapping values ​​is used as a brightness mapping map corresponding to the brightness map.

6. The image processing method according to claim 5, characterized in that: The step of determining the candidate mapping value corresponding to the pixel point based on the preset threshold specifically includes: When the encoding value corresponding to the pixel point is less than a preset threshold, the encoding value of the pixel point is used as the candidate mapping value corresponding to the pixel point; When the encoding value corresponding to the pixel is greater than or equal to a preset threshold, the preset threshold is used as the candidate mapping value corresponding to the pixel.

7. The image processing method according to claim 1, characterized in that: Before determining the fusion image corresponding to the high-frequency image based on the brightness map, the method further includes: Based on a preset scale threshold, determining a first image portion and a second image portion corresponding to the high-frequency image, wherein a pixel value of each pixel point in the first image portion is greater than or equal to the preset scale threshold, and a pixel value of each pixel point in the second image portion is less than the preset scale threshold; adjusting a pixel value of each pixel in the second image portion to obtain a third image portion; An image composed of the first image portion and the third image portion is used as a high-frequency image.

8. The image processing method according to any one of claims 1 to 7, characterized in that: The obtaining of the weight map corresponding to the high-frequency image specifically includes: Performing filtering on the image to be processed to obtain a filtered image corresponding to the image to be processed; Performing guided wave processing on the filtered image based on the brightness map to obtain a guided wave weight map corresponding to the image to be processed; Based on a preset weight threshold, determining a first weight map and a second weight map corresponding to the guided wave weight map, wherein the pixel points in the guided wave weight map whose pixel values ​​are greater than or equal to the preset weight threshold are included in the first weight map, and the pixel points in the guided wave weight map whose pixel values ​​are less than the preset weight threshold are included in the second weight map; Based on a preset brightness weight function, determining a first target weight map corresponding to the first weight map, and a second target weight map corresponding to the second weight map; The first target weight map and the second target weight map are fused to obtain a weight map corresponding to the high-frequency image.

9. The image processing method according to claim 1, characterized in that: The image to be processed is an image that has undergone color gamut conversion, and a conversion matrix for color gamut conversion is determined based on a display device used to display the image to be processed; and the step of adjusting the hue of the image to be processed based on the mapping factor to obtain an enhanced image corresponding to the image to be processed specifically includes: Mapping the image to be processed based on the mapping factor to obtain a mapped image; Performing color adjustment on the mapped image based on a color threshold to obtain an adjusted image; Gamma correction is performed on the adjusted image to obtain an enhanced image corresponding to the image to be processed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the image processing method according to any one of claims 1 to 9.

11. A display device, characterized in that: include: Processor, memory and communication bus; The memory stores a computer-readable program executable by the processor; The communication bus realizes the connection and communication between the processor and the memory; When the processor executes the computer-readable program, the steps in the image processing method according to any one of claims 1 to 9 are implemented.