Image processing method and device, electronic equipment, chip and storage medium

Abstract

The application provides an image processing method and device, electronic equipment, a chip and a storage medium, and relates to the field of image processing. The method comprises the following steps: acquiring an encoded image; the file structure of the encoded image comprises a first photographed image in SDR format and an additional image; the additional image is used for indicating the color mapping relationship between the SDR format and the HDR format; the encoded image is decoded to obtain the first photographed image and the additional image; and image rendering is performed according to the additional image and the first photographed image to display a second photographed image in the HDR format. Therefore, the additional image is used for indicating the color (such as brightness, hue and saturation) mapping relationship required for conversion from the SDR format to the HDR format, the image in the SDR format is rendered by using the color mapping relationship provided by the additional image, the brightness can be accurately adjusted in the process of converting the image from the SDR format to the HDR format, and the rendered image can exhibit more rich color gradation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image processing method and device, electronic equipment, a chip and a storage medium. BACKGROUND

[0002] The emergence of High Dynamic Range (HDR) image technology marks an important breakthrough in the field of digital image processing. It breaks through the limitations of Standard Dynamic Range (SDR) images in terms of brightness and contrast, and can capture and present a wider range of brightness than SDR images, thereby retaining rich details in both bright and dark parts in the same picture and effectively avoiding overexposure or underexposure. HDR technology has been widely used in film and television production, photography and consumer electronics, significantly improving the visual quality and realism of images and videos. SUMMARY

[0003] The present application proposes an image processing method, device, electronic equipment, chip and storage medium to at least partially solve one of the technical problems in the related art.

[0004] An embodiment of the present application provides an image processing method, comprising:

[0005] obtaining an encoded image; wherein a file structure of the encoded image comprises a first captured image in a Standard Dynamic Range (SDR) format and an additional image, the additional image being used to indicate a color mapping relationship between the SDR format and a High Dynamic Range (HDR) format of the first captured image;

[0006] decoding the encoded image to obtain the first captured image and the additional image;

[0007] performing image rendering according to the additional image and the first captured image to display a second captured image in the HDR format.

[0008] Another embodiment of the present application provides an image processing device, comprising:

[0009] an obtaining module configured to obtain an encoded image; wherein a file structure of the encoded image comprises a first captured image in a Standard Dynamic Range (SDR) format and an additional image, the additional image being used to indicate a color mapping relationship between the SDR format and a High Dynamic Range (HDR) format of the first captured image;

[0010] a decoding module configured to decode the encoded image to obtain the first captured image and the additional image;

[0011] rendering module, configured to perform image rendering according to the additional image and the first captured image to display the second captured image in the HDR format.

[0012] In still another aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the image processing method according to the foregoing aspect when executing the program.

[0013] In yet another aspect, an embodiment of the present application provides a chip, including an interface circuit and a processing circuit coupled with each other, the interface circuit is configured to input or output signals, and the processing circuit is configured to execute the image processing method according to the foregoing aspect.

[0014] In still another aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium, having stored thereon computer program instructions, and the computer program instructions are executed by a processor to implement the image processing method according to the foregoing aspect.

[0015] In yet another aspect, an embodiment of the present application provides a computer program product, having stored thereon a computer program, and the program is executed by a processor to implement the image processing method according to the foregoing aspect.

[0016] The image processing method, device, electronic device, chip and storage medium provided by the present application, the additional image is used to indicate the color mapping relationship (such as the mapping relationship of brightness, hue and saturation) required for conversion from the SDR format to the HDR format, the first captured image in the SDR format is rendered by using the color mapping relationship provided by the additional image, and the second captured image in the HDR format is generated, which not only can ensure that the brightness is accurately adjusted in the process of converting the image from the SDR format to the HDR format, avoid the overexposure or underexposure phenomenon caused by improper brightness adjustment, retain the details of the image, but also make the rendered second captured image in the HDR format can exhibit more rich color layering, thereby enhancing the realism of the second captured image and improving the visual experience of the user.

[0017] The additional aspects and advantages of the present application will be partially given in the following description, partially become obvious from the following description, or be understood by practicing the present application. BRIEF DESCRIPTION OF DRAWINGS

[0018] The above and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

[0019] Figure 1 A flowchart of an image processing method according to an exemplary embodiment of the present application;

[0020] Figure 2 Another flowchart of an image processing method provided by an example embodiment of the present application;

[0021] Figure 3 Another flowchart of an image processing method provided by an example embodiment of the present application;

[0022] Figure 4 Another flowchart of an image processing method provided by an example embodiment of the present application;

[0023] Figure 5 Another flowchart of an image processing method provided by an example embodiment of the present application;

[0024] Figure 6 Another flowchart of an image processing method provided by an example embodiment of the present application;

[0025] Figure 7 An implementation principle diagram of an HDR image provided by an example embodiment of the present application;

[0026] Figure 8 A structural diagram of an image processing device provided by an example embodiment of the present application;

[0027] Figure 9 A structural diagram of an electronic device provided by an example embodiment of the present application;

[0028] Figure 10 A structural diagram of a chip provided by an example embodiment of the present application. DETAILED DESCRIPTION

[0029] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.

[0030] In the field of video, HDR technology has formed a mature standard, ensuring that high dynamic range video content can be accurately transmitted from the recording end to the display device, so as to effectively present the details and colors of high dynamic scenes on various display devices. However, although the HDR technology has made significant progress in the field of video, the image medium has not yet formed a unified high dynamic range standard due to compatibility and standardization issues. Therefore, the high-quality images that consumers are concerned about, especially when shared through social media, often cannot be accurately presented.

[0031] To solve these problems, the related technology proposes a photo-level HDR standard based on "gain map", mainly including the following two schemes, which enables users to control the images of HDR and SDR formats at the same time, while ensuring compatibility, and retaining the details and dynamic range of the image to the greatest extent. Through the gain map, users can share HDR images on different devices and platforms, and normally present SDR images on non-compatible devices, providing a balance between compatibility and effect, making the sharing and display of HDR images more flexible and popular.

[0032] The first one is a single-channel HDR additional map generation scheme. The HDR additional map is a gray map that represents the luminance conversion form required for the conversion of the SDR image to the HDR image. The methods used mainly include the following:

[0033] A. Using a single Luma-based tone mapping curve, combined with a local contrast enhancement algorithm, to output a luminance change result as an additional map saved after the SDR image (hereinafter referred to as the main image);

[0034] B. Using the HDR fusion technology of the image signal processor (Image Signal Processing, ISP for short), directly comparing the HDR image and the SDR image, calculating the luminance ratio of the two, and generating an additional map to record the luminance ratio and store it after the main image;

[0035] C. Calculate the ratio of the SDR image and the content encoded by the perceptual quantizer (Perceptual Quantizer, PQ for short, a color transfer function for HDR content encoding) / hybrid log-gamma (Hybrid Log-Gamma, HLG for short, another color transfer function for HDR content encoding) curve, and store it in the form of an additional map after the main image.

[0036] In this scheme, the main role of the gain map (i.e. the HDR additional map) is to process the luminance information by converting the luminance mapping relationship between the HDR image and the SDR image into an additional map form for storage. This method focuses on luminance conversion and aims to accurately present the dynamic range of the image.

[0037] However, this scheme only focuses on the adjustment of the luminance relationship and fails to fully consider the color perception differences caused by changes in the luminance dynamic range, resulting in potential deficiencies in the color performance of the converted HDR image, such as low color richness and saturation, especially in the highlight area (or high-light area), which affects the details and color depth of the HDR image.

[0038] The second is an International Color Consortium Profile (ICC) managed picture HDR display solution. This solution draws on the method of video color management and uses a wide color gamut + high dynamic range to display normal HDR images. Almost all formats can support this function, and the high-efficiency coding AV1 Image File Format (AVIF) based on H.265 further improves the coding efficiency. This HDR picture implementation technology is closer to "single-frame HDR" and is a kind of imitation of video HDR. For normal versions of pictures (such as PNG / JPG / TIFF), the display of HDR effects mainly depends on the CICP field (Color Primaries (CP, which defines the basic color coordinates of the color space) / Transfer Characteristics (TC, which defines how the image data is converted from linear luminance to non-linear encoding values such as gamma or logarithmic curves) / Matrix Coefficients (MC, which defines how to convert RGB color components into YCbCr or other color space coefficients)) in the ICC file of color management, which follows the H.273 rule of video. AVIF can directly use H.265 encoding to input the above color management information, but in order to obtain better compatibility, ICC information is usually inserted, and the image content can be displayed on the display that supports H.265 single-frame decoding.

[0039] In this solution, the HDR image content is mainly presented through standard encoding formats and color management, but there are still some problems: 1. H.265-based image formats (such as AVIF) cannot be normally decoded on most devices, limiting their wide application; 2. There are differences in the support of ICC files by different manufacturers, and ICC information is easy to lose in the process of streaming media transmission. Once there is an error in color management, the image cannot accurately present the creator's intention, and even the picture quality may be affected, which seriously affects the viewing effect.

[0040] Therefore, in view of at least one problem in the related art described above, the present application provides an image processing method, device, electronic equipment, chip and storage medium.

[0041] The image processing method, device, electronic equipment, chip and storage medium of the embodiments of the present application are described below with reference to the drawings. Before specifically describing the embodiments of the present application, in order to facilitate understanding, first, the commonly used technical terms are introduced:

[0042] RGB color space: A type of additive color model that produces various colors based on three primary colors: red (R), green (G), and blue (B). Each color channel is typically represented by 8 bits, ranging from 0 to 255:

[0043] R (Red channel): Controls the intensity of red in the image, with a value range of 0 to 255. 0 represents no red component, while 255 represents maximum red intensity.

[0044] G (Green channel): Controls the intensity of green in the image, with a value range of 0 to 255. 0 represents no green component, while 255 represents maximum green intensity.

[0045] B (Blue channel): Controls the intensity of blue in the image, with a value range of 0 to 255. 0 represents no blue component, while 255 represents maximum blue intensity.

[0046] YCbCr color space: A color space based on the separation of luminance (Luma) and chrominance (Chroma). It decomposes an image into luminance (Y) and two color components (Cb and Cr), each typically represented by 8 bits, ranging from 0 to 255:

[0047] Y (Luminance component): Represents the luminance information of the image, typically ranging from 0 to 255.

[0048] Cb (Blue-difference component): Represents the blue difference relative to luminance, typically ranging from 0 to 255.

[0049] Cr (Red-difference component): Represents the red difference relative to luminance, typically ranging from 0 to 255.

[0050] XYZ color space: A standard color space that defines colors by simulating the human eye's response to light. The X, Y, and Z components represent different spectral response curves:

[0051] Y component: Mainly used to represent the brightness of colors, it is the only factor that directly affects the lightness and darkness of an image, similar to the Y component in YCbCr.

[0052] X and Z components: Together determine the hue and saturation of colors, without directly involving brightness. They are used to describe specific properties of colors, such as different proportions of red, green, and blue.

[0053] Figure 1 Flowchart of an image processing method provided for exemplary embodiments of the present application.

[0054] It should be noted that the image processing method of the embodiments of the present application can be applied to an image processing device. In some possible embodiments, the image processing device can be configured in an electronic device or a chip, so that the electronic device or the chip can perform an image processing function. In addition, in some possible embodiments, the image processing device can also be software in the electronic device and the like.

[0055] In any one of the embodiments of the present application, the chip can be integrated into an electronic device. The chip includes a central processing unit (CPU), an image signal processor (ISP), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a field-programmable gate array (FPGA), a system on chip (SOC), a reduced instruction set computer (RISC), and the like, which are not listed one by one.

[0056] The electronic device includes, but is not limited to, a terminal, a personal computer, and the like. The terminal is an entity on the user side for receiving or transmitting signals, such as a mobile phone. The terminal can also be referred to as a terminal device (terminal), user equipment (UE), a mobile station (MS), a mobile terminal (MT), and the like. The terminal can be a car, a smart car, a mobile phone, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal, an augmented reality (AR) terminal, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, and the like. Embodiments of the present application do not limit the specific technology and specific device form of the terminal.

[0057] As shown in Figure 1 The image processing method can include the following steps S101-S103:

[0058] In step S101, an encoded image is obtained. The file structure of the encoded image includes a first captured image in SDR format and an additional image. The additional image is used to indicate the color mapping relationship between the SDR format and the HDR format of the first captured image.

[0059] The first captured image, also referred to as an SDR image, can be a picture obtained in response to a shooting operation. The shooting operation includes, but is not limited to, a photographing operation, and the like. The trigger form of the shooting operation includes, but is not limited to, a physical key trigger, a gesture recognition trigger, a voice instruction trigger, a biological feature trigger, and the like. The physical key trigger refers to triggering the camera to shoot by triggering the physical key (such as the side key of the mobile phone, the main key, and the like). The gesture recognition trigger refers to triggering the camera to shoot by recognizing the user's gesture through the camera or sensor. The voice instruction trigger refers to triggering the camera to shoot by voice command (such as "turn on the camera", and the like). The biological feature trigger refers to triggering the camera to shoot by recognizing the biological feature (such as facial features, voice features, fingerprint features, and the like).

[0060] The first photographed image can be an original picture captured or photographed by the camera, or can also be a new picture obtained by image processing on the original picture, and the embodiments of the present application do not limit this. The first photographed image includes but is not limited to a photographed picture and the like. The camera includes but is not limited to a front camera and a rear camera.

[0061] The color includes but is not limited to brightness, hue, saturation and the like.

[0062] The additional picture is generated according to the first photographed image in the SDR format and the second photographed image in the HDR format, and is used to indicate the color mapping relationship between the SDR format and the HDR format.

[0063] The additional picture can be used to indicate the color mapping relationship of the first photographed image in the SDR format and the second photographed image in the HDR format on a plurality of color channels. The plurality of color channels are, for example, three color channels of R, G and B.

[0064] The second photographed image, also referred to as an HDR image, is obtained by image processing on the first photographed image.

[0065] The encoded picture is obtained by encoding the first photographed image and the additional picture by using an image encoding technology, and the file structure of the encoded picture includes the first photographed image (also referred to as a main picture) and the additional picture. The first photographed image and the additional picture can be embedded into the same file by using a multi-picture format (MPF) mechanism of the image encoding technology.

[0066] The image encoding technology is not limited, and the image encoding technology includes but is not limited to a JPG MPF (JPEG Multi-Picture Format) encoding technology and the like. The JPG MPF encoding technology writes the first photographed image in the JPEG format into a file, and embeds the additional picture in the JPEG format into the file.

[0067] As an example, the first photographed image is stored as a base picture (or a main picture) in the starting part of the file, and the additional picture provides the color mapping relationship required for conversion from the first photographed image in the SDR format to the second photographed image in the HDR format. The additional picture is embedded into the same file by using the MPF mechanism of the JPEG, is usually located at a specific position of the file, and is indicated by metadata marking to indicate its existence and attributes.

[0068] It should be noted that the present application does not limit the acquisition manner of the encoded image. For example, the encoded image can be an image generated by the electronic device itself. For example, the electronic device can acquire a first photographed image in response to a photographing operation, perform image processing on the first photographed image to obtain a second photographed image, generate an additional image according to the first photographed image and the second photographed image, and encode the first photographed image and the additional image by using an image encoding technology to obtain the encoded image. Alternatively, the encoded image can be an image shared or sent by another user. Alternatively, the encoded image can be an online collected image, and the like. The present application does not limit the embodiments.

[0069] In step S102, the encoded image is decoded to obtain the first photographed image and the additional image.

[0070] In the embodiments of the present application, in the case that the electronic device has a decoding function for the encoded image, the electronic device can decode the encoded image to obtain the first photographed image and the additional image.

[0071] In step S103, the first photographed image and the additional image are used for image rendering to display the second photographed image in the HDR format.

[0072] In the embodiments of the present application, the electronic device can perform image rendering on the first photographed image in the SDR format according to the additional image to display the second photographed image in the HDR format. That is, the first photographed image is color-mapped based on the color mapping relationship indicated by the additional image to obtain the second photographed image.

[0073] The image processing method of the embodiments of the present application uses the additional image to indicate the color mapping relationship (such as the mapping relationship of luminance, hue and saturation) required for conversion from the SDR format to the HDR format, and uses the color mapping relationship provided by the additional image to render the first photographed image in the SDR format to generate the second photographed image in the HDR format. This can not only ensure that the luminance is accurately adjusted in the process of converting the image from the SDR format to the HDR format, avoid overexposure or underexposure caused by improper luminance adjustment, and retain the details of the image, but also make the rendered second photographed image in the HDR format exhibit more rich color gradation, thereby enhancing the realism of the second photographed image and improving the visual experience of the user.

[0074] As a possible implementation manner, Figure 2 Another flowchart of an image processing method provided by an exemplary embodiment of the present application.

[0075] It should be noted that the image processing method can be executed alone, or can be executed in combination with any of the embodiments or possible implementation manners in the embodiments, or can be executed in combination with any of the technical solutions in the related art, and the embodiments of the present application do not make any limitation in this regard.

[0076] As shown in the Figure 2 image processing method can include the following steps S201 to S204:

[0077] Step S201, obtaining an encoded image; wherein the file structure of the encoded image includes a first photographed image in SDR format and an additional image embedded after the first photographed image.

[0078] The additional image is used to indicate the color mapping relationship between the SDR format and the HDR format of the first photographed image.

[0079] It should be noted that the explanation of step S201 can refer to the related description in any of the embodiments of the present application, which will not be repeated here.

[0080] Step S202, determining whether there is a decoding function for the encoded image, if yes, executing step S203, if not, executing step S204.

[0081] It should be noted that step S203 and step S204 are two parallel implementation manners, and one of them can be executed.

[0082] Step S203, decoding the encoded image to obtain the first photographed image and the additional image, and performing image rendering according to the additional image and the first photographed image to display a second photographed image in HDR format.

[0083] In the embodiments of the present application, in the case that the electronic device has a decoding function for the encoded image, the encoded image can be decoded to obtain the first photographed image in SDR format and the additional image, and image rendering is performed according to the additional image and the first photographed image to display the second photographed image in HDR format. The implementation principle can refer to the related description in any of the embodiments of the present application, which will not be repeated here.

[0084] Step S204, performing image rendering according to the encoded image to display the first photographed image in SDR format.

[0085] In the embodiments of the present application, in the case that the electronic device does not have a decoding function for the encoded image, image rendering can be directly performed according to the encoded image to display the first photographed image in SDR format.

[0086] The image processing method of the embodiment of the present application uses the first captured image in SDR format as the main image, stores the main image in front of the encoded image, and embeds the additional image after the main image, so that the device that does not support decoding the encoding format can still normally display the main image in SDR format, ensuring adaptive presentation of the image on different platforms and devices, and realizing wide compatibility guarantee.

[0087] As a possible implementation manner, Figure 3 A flowchart of another image processing method provided by an exemplary embodiment of the present application is shown in FIG. 6.

[0088] It should be noted that the image processing method can be executed alone, or can be executed in combination with any one of the embodiments or the possible implementation manners of the embodiments, or can be executed in combination with any one of the technical solutions in the related art, and the embodiments of the present application do not limit this.

[0089] As shown in FIG. 6, Figure 3 On the basis of any one of the embodiments of the present application, the additional image can be obtained by the following steps S301 to S302:

[0090] In step S301, the pixel value gain of each pixel point is determined according to the pixel value of each pixel point in the first captured image in SDR format and the pixel value of the corresponding pixel point in the second captured image in HDR format; wherein the pixel value gain includes gain values of multiple color channels, and the gain value is used to indicate the color mapping relationship of the corresponding color channel.

[0091] As an example, taking the first captured image and the second captured image as images in RGB color space, the following formula can be used to calculate the gain value of each pixel point in multiple color channels:

[0092] ; (1)

[0093] wherein, is the gain value of the R channel, is the pixel value (or channel value) of a certain pixel point in the R channel in the second captured image in HDR format, is the pixel value of the corresponding pixel point in the R channel in the first captured image in SDR format, is the preset offset of the R channel in HDR format (a relatively small value), is the preset offset of the R channel in SDR format (a relatively small value).

[0094] Similarly, is the gain value of the G channel, is a pixel value of a certain pixel in the G channel in the second shooting image in the HDR format, is a pixel value of a corresponding pixel in the G channel in the first shooting image in the SDR format, is a preset offset of the G channel in the HDR format (a relatively small value), is a preset offset of the G channel in the SDR format (a relatively small value).

[0095] is a gain value of the B channel, is a pixel value of a certain pixel in the B channel in the second shooting image in the HDR format, is a pixel value of a corresponding pixel in the B channel in the first shooting image in the SDR format, is a preset offset of the B channel in the HDR format (a relatively small value), is a preset offset of the B channel in the SDR format (a relatively small value).

[0096] In step S302, the pixel value gain of each pixel is encoded according to the minimum luminance and the maximum luminance supported by the display to obtain an additional image.

[0097] As an example, the gain value of each pixel in multiple color channels can be converted into a log domain code, and the minimum luminance and the maximum luminance supported by the display are read. According to the log domain code of each pixel in multiple color channels, the minimum luminance and the maximum luminance, the color mapping relationship (i.e. color ratio coefficient) of each pixel in multiple color channels is determined, and the additional image is generated according to the color mapping relationship of each pixel in multiple color channels. Each pixel in the additional image is used to indicate the color mapping relationship of corresponding pixels in multiple color channels in the first shooting image in the SDR format and the second shooting image in the HDR format.

[0098] As a possible implementation, the color mapping relationship (i.e. color ratio coefficient) of each pixel in multiple color channels can also be numerically limited (clamped) in a suitable value range, and scaled to the standard range of an 8-bit image.

[0099] Thus, in the present application, the image encoding technology can be used to embed the additional image into the back of the first shooting image in the SDR format for storage to obtain an encoded image. In this way, on software and display devices supporting decoding functions, the second shooting image in the HDR format can be correctly displayed according to the encoded image, and the dynamic range of the second shooting image is fully exhibited, while on devices without decoding functions, the first shooting image in the SDR format is correctly displayed.

[0100] In any one of the embodiments of the present application, in order to save the occupation of storage resources, the size of the additional image can be smaller than the size of the first captured image in SDR format, for example, the size of the additional image can be compressed to 1 / 4 or even 1 / 16 of the size of the first captured image, greatly saving the storage space.

[0101] As an example, the pixel value gain of each pixel point can be encoded according to the lowest brightness and the highest brightness supported by the display to obtain an initial additional image, and the initial additional image is down-sampled to obtain the additional image.

[0102] Each pixel point in the initial additional image is used to indicate the color mapping relationship of the corresponding pixel points in the first captured image and the second captured image in the plurality of color channels.

[0103] For the above step S103, in the present application, the additional image can be calculated by bilinear interpolation to obtain an additional image with the same size as the first captured image, and based on the color mapping relationship of each pixel point in the additional image in the plurality of color channels, the corresponding pixel points in the first captured image are color mapped to obtain the second captured image in HDR format.

[0104] The image processing method of the present application embodiment, each pixel point in the additional image is used to indicate the color mapping relationship of the corresponding pixel points in the first captured image in SDR format and the second captured image in HDR format in the plurality of color channels (such as R, G, B channels). The color mapping relationship of the plurality of color channels provided by the additional image can realize more delicate and rich color levels in the process of converting the image from SDR format to HDR format, which not only enhances the overall visual effect of the image, but also makes the color transition more natural and smooth. In addition, the color mapping relationship of the plurality of color channels provided by the additional image can accurately adjust the value of the pixel points in each color channel in the first captured image, thereby effectively improving the color performance of the second captured image in HDR format. This accurate adjustment ensures the accuracy and consistency of the color, even in extreme lighting conditions, it can also maintain high-quality color reproduction, so that the adjusted second captured image in HDR format can show real colors in highlight and shadow areas.

[0105] As a possible implementation manner, Figure 4 Another flowchart of an image processing method provided by an exemplary embodiment of the present application.

[0106] It should be noted that the image processing method can be executed alone, or can be executed in combination with any of the embodiments or possible implementation manners in the embodiments, or can be executed in combination with any of the related technical solutions, and the embodiments of the present application do not make any limitation in this regard.

[0107] As Figure 4 indicated, on the basis of any of the embodiments, the second photographed image can be obtained by using the following steps S401 to S403.

[0108] In step S401, in response to the photographing operation, the SDR format first photographed image is obtained, and the first photographed image is converted to the linear domain to obtain the first intermediate image.

[0109] It should be noted that the foregoing embodiment is also applicable to the explanation and description of the photographing operation and the first photographed image, and thus no further description is given herein.

[0110] In the embodiment, the SDR format first photographed image can be converted to the linear domain to obtain the first intermediate image.

[0111] As an example, taking the first photographed image as an image in the RGB color space, assuming that the first photographed image is , and , , is a Gamma encoded color signal, the value range of which is between [0, 1], the following formula can be used to convert it to the linear domain:

[0112] ; (2)

[0113] Wherein, cctf is the abbreviation of Colour Component Transfer Function, cctf_decoding is the Gamma curve dependent on the display device, and the result is usually converted using the standard RGB color space (standard Red Green Blue, abbreviated as sRGB):

[0114] ; (3)

[0115] In step S402, the crosstalk elimination processing is performed on the plurality of color channels of the first intermediate image according to the crosstalk matrix to obtain the second intermediate image, wherein the crosstalk matrix is determined according to the color coupling degree between the plurality of color channels.

[0116] In the embodiments of the present application, the crosstalk matrix can be used to eliminate the color interference between the plurality of color channels of the first intermediate image, ensure the independence of each color channel, and preserve the highlight information in the image. For example, the crosstalk matrix can be multiplied by the first intermediate image to obtain the second intermediate image.

[0117] As an example, it is assumed that the crosstalk between the plurality of color channels (such as the three color channels R, G, and B) is symmetrical, and the degree of color coupling between each color channel is controlled by a parameter , then the crosstalk matrix can be represented as:

[0118] ; (4)

[0119] wherein, is a parameter for controlling the strength of crosstalk, which can be manually adjusted according to the image effect. The element in the i-th row and the j-th column of the crosstalk matrix is used to indicate the color coupling relationship between the i-th color channel and the j-th color channel. Among them, the elements on the diagonal line represent the proportion of color information of each color channel itself (i.e., the "self" information of the color channel), minus the part of crosstalk; the elements on the non-diagonal line represent the degree of color coupling between each color channel and other color channels, i.e., how much color information of each color channel is affected by other color channels.

[0120] In step S403, the second intermediate image is subjected to inverse tone mapping processing to obtain a second captured image in SDR format.

[0121] In the embodiments of the present application, the second intermediate image can be subjected to inverse tone mapping processing to expand the brightness information of the processed image (referred to as the second captured image in the present application) and maintain the image details.

[0122] The image processing method of the embodiments of the present application converts the first captured image in SDR format to a linear space, eliminates the color interference between the plurality of color channels of the image in the linear space through the crosstalk matrix, ensures the independence of each color channel, reduces the saturation of the highlight region while ensuring the invariance of the tone, and avoids the loss of color information in the operation process after the expansion of the highlight part in the image processing process. The application of inverse tone mapping can expand the brightness information of the image and maintain the image details.

[0123] As a possible implementation manner, Figure 5 Fig. 4 is a flowchart of another image processing method provided by the exemplary embodiments of the present application.

[0124] It should be noted that the image processing method can be executed alone, or can be executed in combination with any of the embodiments or possible implementation manners in the present application, or can be executed in combination with any of the related technical solutions, and the present application embodiment does not make any limitation.

[0125] As Figure 5 indicated, on the basis of any of the embodiments of the present application, the second photographed image can be obtained by using the following steps S501 to S506:

[0126] Step S501, in response to a photographing operation, a first photographed image in SDR format is obtained, and the first photographed image is converted to a linear domain to obtain a first intermediate image.

[0127] Step S502, according to the crosstalk matrix, crosstalk elimination processing is performed on a plurality of color channels of the first intermediate image to obtain a second intermediate image; wherein the crosstalk matrix is determined according to the color coupling degree between the plurality of color channels.

[0128] It should be noted that the explanation and description of steps S501 to S502 can refer to the related description in any of the embodiments of the present application, which will not be repeated here.

[0129] Step S503, based on the color gamut of the first photographed image, the second intermediate image is converted from a first color space to a second color space to obtain a third intermediate image.

[0130] The first color space is a color space to which the second intermediate image or the first photographed image belongs. For example, the first color space is, for example, an RGB color space.

[0131] The second color space is a reference color space for tone mapping. For example, the second color space is, for example, an XYZ color space.

[0132] In the embodiments of the present application, the second intermediate image can be converted from the first color space to the second color space based on the color gamut of the first photographed image to obtain the third intermediate image in the second color space.

[0133] As an example, a mapping relationship between different color gamuts in the first color space and conversion matrices can be pre-configured, wherein the conversion matrix is used to convert the image from the first color space to the second color space. Thus, in the present application, the mapping relationship can be queried according to the color gamut of the first photographed image to determine the conversion matrix suitable for the color gamut of the first photographed image, and the second intermediate image is converted from the first color space to the second color space by using the conversion matrix.

[0134] For example, taking the first color space as RGB color space, the color gamut includes but is not limited to sRGB, display P3 (a wide color gamut color space, i.e., P3 color gamut is wider than sRGB color gamut, especially in green and red regions, which makes it able to display more kinds of bright colors) and the like.

[0135] For example, taking the color gamut of the first photographed image as sRGB color gamut and the second color space as XYZ color space, the second intermediate image in SDR format can be converted from RGB color space to XYZ color space by using the following formula:

[0136] (5)

[0137] wherein, is a conversion matrix adapted to sRGB color gamut.

[0138] As a possible implementation, the x and y chrominance components of the third intermediate image in XYZ color space can be expressed as:

[0139] (6)

[0140] Step S504, determining a low-light region and a high-light region from the third intermediate image; wherein the first color component corresponding to luminance in the low-light region is less than or equal to a luminance threshold, and the first color component in the high-light region is greater than the luminance threshold.

[0141] wherein the luminance threshold is determined according to the neutral gray luminance (gray_luminance) of the display. For example, the luminance threshold can be ; wherein, For adjusting the luminance of the image in HDR format, it can be set as the neutral gray luminance (gray_luminance) of the display (such as an HDR display) / c1. Wherein c1 is the first mapping parameter.

[0142] As an example, taking the second color space as XYZ color space, the first color component can be Y component, the Y component of the low-light region is less than or equal to , and the Y component of the high-light region is greater than the luminance threshold .

[0143] Step S505, using the first mapping parameter to perform tone mapping processing on the first color component in the low-light region, and using at least one second mapping parameter to perform tone mapping processing on the first color component in the high-light region, to obtain a fourth intermediate image.

[0144] In the embodiments of the present application, the first color component in the low-light region in the third intermediate image can be subjected to tone mapping processing by using a first mapping parameter.

[0145] Taking XYZ color space as the second color space, the first color component can be Y component, and the Y component in the low-light region can be adjusted by using the following formula:

[0146] (7)

[0147] wherein c1 is a parameter for controlling the mapping of the low-light region, which can be set as a constant with a value less than 1, and is used to maintain the details of the low-light region. That is, in the low-light region, the small luminance values in the third intermediate image in SDR format should be enlarged to reveal more details, and this part of the mapping is directly performed by c1.

[0148] In the embodiments of the present application, the first color component in the high-light region in the third intermediate image can also be subjected to tone mapping processing by using at least one second mapping parameter to obtain a fourth intermediate image.

[0149] wherein the at least one second mapping parameter can be used to determine a luminance adjustment factor according to the ratio of the neutral gray luminance and the first mapping parameter, and to determine the at least one second mapping parameter according to the luminance adjustment factor, the first mapping parameter and the neutral gray luminance.

[0150] Taking XYZ color space as the second color space, the first color component can be Y component, and the Y component in the high-light region can be adjusted by using the following formula:

[0151] (8)

[0152] wherein ; .

[0153] wherein c2 determines the steepness of the logarithmic mapping curve in the formula, and a smaller c2 represents a steeper curve; c3 controls the reference luminance of the logarithmic mapping curve, and determines the degree of "compression" of the high-light part; a larger c3 raises the overall luminance value, and the high-light part of the image appears more natural, but some details can be lost; on the contrary, a smaller c3 enhances the contrast of the high-light part, and more details are displayed; c4 controls the starting point of the logarithmic mapping, and determines when the logarithmic mapping starts; a smaller c4 causes the logarithmic mapping to start to affect the high-light part earlier, and more luminance ranges are mapped to the high-light region (or the highlight region), which can result in a reduction of dark details.

[0154] Wherein, c1 and c3 in the mapping parameter can be adjusted manually, for example, the value range <1, which can better maintain the dynamic range and ensure the smooth transition of the curve.

[0155] In any one of the embodiments of the present application, the first mapping parameter can be used to perform tone mapping processing on the first color component in the low-light area, and at least one second mapping parameter can be used to perform tone mapping processing on the first color component in the high-light area to obtain a mapping image, so that in the present application, the second color component other than the first color component in the mapping image can be corrected according to the first color component in the mapping image to obtain a fourth intermediate image.

[0156] Exemplarily, taking the XYZ color space as the second color space, the first color component can be the Y component, the above formulas (7) and (8) can be used to perform tone mapping processing to obtain a mapping image, and the following formula can be used to correct the second color component (i.e., the X component and the Z component) other than the first color component in the mapping image to obtain a fourth intermediate image:

[0157] ; (9)

[0158] In summary, the luminance information of the fourth intermediate image can be expanded, and the details of the fourth intermediate image can be maintained, and the visual effect of the fourth intermediate image can be improved.

[0159] Step S506 converts the fourth intermediate image from the second color space to the first color space to obtain a second shooting image in the HDR format.

[0160] In the embodiments of the present application, the inverse conversion matrix can be used to convert the fourth intermediate image from the second color space to the first color space to obtain a second shooting image in the HDR format.

[0161] Exemplarily, taking the RGB color space as the first color space and the XYZ color space as the second color space, the above formulas (7), (8), and (9) can be converted to the RGB color space by using the following formula (10): 、 、

[0162] ; (10)

[0163] Wherein, is the inverse conversion matrix, which is obtained by inverting the conversion matrix in the above formula (5).

[0164] ​​The image processing method provided in the embodiments of the present application can effectively expand the luminance information of the image and maintain the image details.

[0165] As a possible implementation manner, Figure 6 A flowchart of another image processing method provided by the exemplary embodiments of the present application is shown in FIG. 6.

[0166] It should be noted that the image processing method can be executed alone, or in combination with any of the embodiments or possible implementation manners of the embodiments, or in combination with any of the technical solutions in the related art, and the embodiments of the present application do not limit this.

[0167] As Figure 6 shown, on the basis of any of the embodiments of the present application, the second captured image can be obtained by the following steps S601 to S608:

[0168] In step S601, in response to a capturing operation, a first captured image in SDR format is obtained, and the first captured image is converted to a linear domain to obtain a first intermediate image.

[0169] In step S602, crosstalk elimination processing is performed on a plurality of color channels of the first intermediate image according to a crosstalk matrix to obtain a second intermediate image; wherein the crosstalk matrix is determined according to the color coupling degree between the plurality of color channels.

[0170] In step S603, based on the color gamut of the first captured image, the second intermediate image is converted from a first color space to a second color space to obtain a third intermediate image.

[0171] In step S604, a low-light region and a high-light region are determined from the third intermediate image; wherein the first color component corresponding to the luminance in the low-light region is less than or equal to a luminance threshold, and the first color component in the high-light region is greater than the luminance threshold.

[0172] In step S605, the first color component in the low-light region is subjected to tone mapping processing using a first mapping parameter, and the first color component in the high-light region is subjected to tone mapping processing using at least one second mapping parameter, to obtain a fourth intermediate image.

[0173] It should be noted that the explanation and description of steps S601 to S605 can be referred to the related description in any of the embodiments of the present application, which will not be repeated here.

[0174] In step S606, the fourth intermediate image is converted from the second color space to the first color space to obtain a fifth intermediate image.

[0175] In the embodiments of the present application, the fourth intermediate image can be converted from the second color space to the first color space by using an inverse conversion matrix to obtain a fifth intermediate image in the first color space.

[0176] For example, taking the first color space as the RGB color space and the second color space as the XYZ color space, the above formula (7), (8) and (9) can be converted to the RGB color space by using the above formula (10) to obtain the fifth intermediate image. 、 、

[0177] In step S607, the inverse crosstalk matrix is used to perform inverse crosstalk elimination processing on the plurality of color channels of the fifth intermediate image to obtain a sixth intermediate image. The inverse crosstalk matrix is obtained by inverting the crosstalk matrix.

[0178] For example, taking the first color space as the RGB color space, the fifth intermediate image is denoted as , and the sixth intermediate image is denoted as , then the following formula is obtained:

[0179]

[0180] .

[0181] It should be noted that in order to maintain the color consistency and naturalness of the image, the transformation of the chrominance needs to be coordinated with the change of the luminance (Luma) information. When the color performance in the highlight area of the image is low, the chrominance information can be expanded in one step to better display the highlight information and improve the color consistency of the image. Therefore, in any one of the embodiments of the present application, the fifth intermediate image can be color expanded, and the color expanded image can be subjected to inverse crosstalk elimination processing according to the inverse crosstalk matrix to obtain the sixth intermediate image. That is, the sixth intermediate image can be obtained by using the following steps A to E:

[0182] In step A, the fifth intermediate image is converted from the first color space to a third color space to obtain a first converted image.

[0183] The third color space is a color expanded color space. For example, the third color space is the YCbCr color space.

[0184] In the present application, the fifth intermediate image can be converted from the first color space to the third color space based on the conversion relationship between the first color space and the third color space to obtain the first converted image.

[0185] ​​​Step B: determining a chroma adjustment factor according to the ratio of the first color component in the third intermediate image and the fourth intermediate image.

[0186] Exemplarily, taking the first color component as Y component as an example, the chroma adjustment factor (or called chroma expansion factor) can be calculated by using the following formula :

[0187] ; (12)

[0188] wherein, the setting of 1.095 is an adjustment experience value, which can be set according to actual application requirements.

[0189] Step C: performing a smooth adjustment on the chroma of the first conversion image according to the chroma adjustment factor, to obtain an adjusted image.

[0190] Exemplarily, taking the third color space as YCbCr color space as an example, the Cb and Cr components can be adjusted according to the chroma adjustment factor .

[0191] ; (13)

[0192] As a possible implementation manner, the expansion intensity of the HDR content can also be increased by using the following adaptive calculation process according to actual application requirements:

[0193] ; (14)

[0194] ; (15)

[0195] wherein, , the value range of R can be between [0, 2.03], which can be set as a small fluctuation, for example, set as 0.15.

[0196] In summary, the chroma of the highlight area can be expanded more strongly, while the chroma expansion of the low-light area is smaller. Among them, is a smooth function, which ensures that the chroma adjustment factor is adjusted smoothly between the highlight and dark areas, avoiding abrupt chroma changes. controlling the smoothness of the transition, a larger will make the transition smoother, reduce drastic changes and improve image quality.

[0197] Step D: converting the adjusted image from the third color space to the first color space to obtain a second conversion image.

[0198] In the embodiments of the present application, the adjusted image can be converted from the third color space to the first color space based on the conversion relationship between the third color space and the first color space, to obtain a second converted image.

[0199] Step E: performing inverse crosstalk cancellation processing on the plurality of color channels of the second converted image according to the inverse crosstalk matrix, to obtain a sixth intermediate image.

[0200] Exemplarily, the above formula (11) can be used to perform inverse crosstalk cancellation processing on the plurality of color channels of the second converted image, to obtain the sixth intermediate image. .

[0201] In summary, in order to maintain the color consistency and naturalness of the image, the transformation of the chrominance needs to be coordinated with the change of the luminance (Luma) information. When the color performance in the highlight area of the image is low, one step of expansion is performed on the chrominance information, which can better exhibit the highlight information and improve the color consistency of the image.

[0202] Step S608: performing a luminance mapping operation on the sixth intermediate image according to the luminance range supported by the display, to obtain a second photographed image in the HDR format.

[0203] Exemplarily, taking the first color space as the RGB color space as an example, the OOTF (Opto-Optical Transfer Function) operation can be performed on the sixth intermediate image to complete the luminance mapping operation. For example, the following formula can be used to perform the luminance mapping operation on the sixth intermediate image:

[0204] First, the set empirical index (which is a constant greater than 1) is used to adjust :

[0205] ; (16)

[0206] Then, the standard OOTF conversion is performed on to map to the luminance supported by the display:

[0207] ; (17)

[0208] Wherein, OETF refers to Opto-Electronic Transfer Function; EOTF refers to Electro-Optical Transfer Function; BT.709 refers to ITU-R BT.709 (High Definition Television Standard); OETF_BT709 is used to convert the brightness and color of the image to adapt to a specific display standard; BT.1886 refers to ITU-R BT.1886 (Display Gamma Standard); EOTF_BT1886 is used to convert the electrical signal into the corresponding brightness value, and further adjust the brightness performance of the image; scale is a scaling factor; is a coefficient calculated according to the brightness range supported by the display, for example, when the brightness of the display reaches the maximum brightness 10000 nit supported by the Perceptual Quantization (PQ) curve, The value of is 31.2379. If the brightness range supported by the display or other parameters changes, the value of can be scaled accordingly to ensure that the mapped image has good display effect on different displays. The value of is 31.2379. If the brightness range supported by the display or other parameters changes, the value of can be scaled accordingly to ensure that the mapped image has good display effect on different displays.

[0209] The image processing method of the embodiment of the application can effectively retain the color information of the highlight area (or the highlight area) through the double processing mode of the inverse crosstalk matrix and the color mapping, so that in the image rendering stage, the color of the highlight part can be ensured not to be weakened too much when the brightness changes, and a more natural and accurate highlight effect is presented.

[0210] In any one of the embodiments of the application, compared with the first scheme in the related art, the application uses the additional image to perform color mapping on the multiple color channels in the first photographed image, compared with the way of directly mapping the brightness, the corresponding adaptation of the color when the brightness dynamic range changes is increased, and the color information of the high dynamic range image is more rich. Compared with the second scheme in the related art, the application uses the first photographed image in SDR format as the main image, and embeds the additional image after the main image. For devices that do not support decoding the encoding format, the display can still be performed according to the main image in SDR format, and the compatibility is good. Even if the additional image is discarded, the display result is still normal.

[0211] That is, the technical solutions provided in the present application are at least used to solve the following problems: solving the problem of the influence of brightness changes on color perception under single-channel brightness mapping; solving the compatibility problem in image display and streaming transmission, ensuring that the image can still be correctly displayed in the case that the decoder does not support additional image decoding; solving the problem of inaccurate color representation in the highlight area in the inverse tone mapping process; solving the problem of fast adaptation of additional images during image debugging in SDR format.

[0212] As an example, by generating an additional image of three color channels and embedding the additional image into the main image by encoding, the brightness and color adjustment information required for storing the main image on an HDR display is stored. The implementation principle can be as shown in Figure 7 , mainly including the following steps:

[0213] 1. Input an SDR format RGB image, which is denoted as the first photographed image (i.e., the main image) in the present application.

[0214] 2. Convert the main image to the linear domain (linear space).

[0215] Suppose the input SDR format RGB image signal is , which is a Gamma-encoded color signal with a value range of [0, 1], then it can be converted to the linear domain using the following formula:

[0216] ;

[0217] Where cctf_decoding is a Gamma curve dependent on the display device, and the result of using the sRGB standard is usually used for conversion:

[0218] ;

[0219] 3. In the linear space, the interference between the R, G, and B color channels of the main image is eliminated by the crosstalk matrix to ensure the independence of each color channel and to preserve the highlight information.

[0220] Where the purpose of crosstalk elimination is to process the image itself in the linear domain, and at the same time, to eliminate the cross-influence between different color channels in the subsequent conversion process, and at the same time, by mixing a certain amount of other color channels into each color channel, the saturation of the highlight area is reduced while the color tone is unchanged, avoiding the loss of color information in the highlight part after expansion in the image processing process.

[0221] In the present application, it is assumed that the crosstalk between multiple color channels is symmetrical, and the degree of color coupling between each color channel is controlled by a parameter , therefore, the crosstalk matrix can be represented as: ;

[0222] 4. Convert the master image from RGB color space to XYZ color space as the reference for tone mapping.

[0223] The conversion process of inverse tone mapping can be defined on XYZ color space. Taking the master image with sRGB gamut as an example, the master image can be converted from RGB color space to XYZ color space by using the following formula:

[0224] ;

[0225] wherein x sdr and y sdr The chroma components are represented as follows:

[0226] ;

[0227] 5. Apply inverse tone mapping to convert the master image in XYZ color space back to HDR range, expand the brightness information of the image, and maintain details.

[0228] wherein the purpose of inverse tone mapping is to map the low-brightness area in the SDR image to the HDR space through certain conversion, so that the highlight part in the image is preserved, and the dynamic range of the image is enhanced. According to the difference of input brightness, the inverse tone mapping function in the present application processes the low-brightness area and the high-brightness area in the image through two segments.

[0229] wherein the inverse tone mapping includes: when the brightness value in the SDR image is less than or equal to the brightness threshold , using linear mapping to convert: ; when the brightness value in the SDR image is greater than the brightness threshold , using logarithmic mapping to realize color mapping of the highlight area: .

[0230] As a possible implementation, the following formula can also be used to update , x and y :

[0231] ;

[0232] 6. Convert the image from XYZ color space to RGB color space to obtain the initial image in HDR format, which is denoted as the fifth intermediate image in the present application.

[0233] Exemplarily, the following formula can be used to convert , and Convert the initial image in HDR format from RGB color space to YCbCr color space for color expansion, and then convert it from YCbCr color space to RGB color space.

[0234] ;

[0235] 7. (Optional) Convert the initial image in HDR format from RGB color space to YCbCr color space for color expansion, and then convert it from YCbCr color space to RGB color space.

[0236] In order to maintain the color consistency and naturalness of the image, the transformation of chrominance needs to be coordinated with the change of luminance (Luma) information. When the color representation in the highlight area of the image is low, the following method can be used to expand the chrominance information, so as to better display the highlight information. The chrominance expansion can be performed on the Ycbcr color space, and then converted back to the RGB color space after expansion:

[0237] First, the chrominance adjustment factor (or chrominance expansion factor) can be calculated:

[0238] ;

[0239] Then, the Cb and Cr components can be adjusted according to the chrominance adjustment factor

[0240] ;

[0241] As a possible implementation, the expansion strength of the HDR content can also be increased by the following adaptive calculation process according to the actual application requirements:

[0242] ; wherein, ;

[0243] wherein, , the value of R can be in the range of [0, 2.03], can be set to a small fluctuation, for example, set to 0.15.

[0244] The above operation can ensure that the chrominance in the highlight area is more strongly expanded, while the chrominance expansion in the low-light area is smaller. Wherein, is a smoothing function, which ensures that the chrominance adjustment factor adjusts smoothly between the highlight and dark areas, avoiding abrupt chrominance changes. controls the smoothness of the transition. A larger will make the transition smoother, reducing drastic changes and improving image quality.

[0245] ​8. Via inverse crosstalk matrix processing, the independence of RGB channels is recovered, ensuring accurate color reconstruction, and through OOTF conversion and Gamma correction, the final image in HDR format is obtained, which is recorded as the second shooting image in this application.

[0246] Since there is an inverse crosstalk operation mainly aiming at the saturation of highlights at the beginning of the processing flow, when finally processing, the inverse crosstalk operation should be considered to be counteracted, that is, the crosstalk matrix in the foregoing is inverted:

[0247]

[0248] Then, according to the inverse crosstalk matrix, the inverse crosstalk elimination processing is performed on

[0249] The above processing operation is sufficient to map a 100 nit related content to 203 nit, and the following one-step OOTF operation is performed to complete the mapping operation to the actual peak brightness of the screen, in the following manner:

[0250] wherein, is an empirical value set.

[0251] Then, after one-step standard OOTF conversion, it is mapped to the brightness supported by the display:

[0252]

[0253] 9. According to the first shooting image and the second shooting image, an additional image is calculated. The additional image stores the color change ratio information of the R, G, and B color channels in the second shooting image in HDR format and the first shooting image in SDR format. As a possible implementation manner, in order to further save storage space, the size of the additional image can be 1 / 4 or 1 / 16 of the size of the main image, and when interacting with the main image, it can be realized through bilinear interpolation, and tests can fully meet the application requirements.

[0254] The purpose of this step is to make the gain values of the three color channels of the image compatible with the HDR format, and these gain values are encoded in the form of an additional image. The implementation manner is as follows: first, the pixel value gain of each color channel in the SDR format and the HDR format is calculated, and then the gain value is converted into a format suitable for storage by applying logarithmic coding. In order to ensure that it can be presented within the 8-bit standard image display range, the encoded gain values will be appropriately limited (Clamp). Finally, on a display device supporting the HDR format, the SDR image can be displayed in a high brightness and correct manner, improving the dynamic range and detail presentation of ordinary images.

[0255] ​​​​​Wherein, the calculation method of the gain value of each pixel point in the multiple color channels is:

[0256] ;

[0257] Then, the gain value of each color channel is converted into a code in a log domain, and the minimum luminance and the maximum luminance supported by a display are read, and according to the above information, the color mapping relationship (i.e. color proportion coefficient) of each pixel point in the multiple color channels is calculated.

[0258] As a possible implementation, the color mapping relationship (i.e. color proportion coefficient) of each pixel point in the multiple color channels can also be limited (clamped) in value, i.e. limited in a suitable value interval, and scaled to the standard range of an 8-bit image.

[0259] 10. The additional graph code is added to the main graph MPF information.

[0260] Exemplarily, the additional graph can be embedded into the back of the main graph in SDR format by using the JPG MPF coding technology. In this way, on the software and display device supporting decoding function, the HDR image can be correctly displayed according to the coded image, and the dynamic range of the HDR image can be fully displayed, while on the device not supporting decoding of the additional graph, the main graph in SDR format can be correctly displayed.

[0261] In summary, the scheme provided by the present application has at least the following advantages:

[0262] (1) The additional graph contains the color mapping relationship of three channels: on the basis of correct mapping of luminance, the additional graph realizes more rich color gradation through color mapping of three channels. This method can effectively improve the color performance of the image, avoid the influence of luminance adjustment on color, and make the HDR image more realistic.

[0263] (2) Color preservation in high light area: through the double processing method of crosstalk matrix and color mapping, the additional graph can effectively preserve the color information in the high light area. This method ensures that the color of the high light part is not weakened too much when the luminance changes, so as to present a more natural and accurate high light effect.

[0264] (3) The gain value of three channels as additional graph information, this design makes the local dynamic range adjustment can be carried out according to the color information, increases the scalability of the algorithm.

[0265] (4) The additional picture can be adaptively adjusted according to the image content in the SDR format. This means that the user can optimize the adjustment for the SDR image, while the HDR image will quickly change synchronously, thereby narrowing the visual gap between the two and providing a consistent viewing experience.

[0266] (5) Storage space saving and compatibility guarantee: In image encoding, the main picture is an SDR image, and the additional picture is an HDR gain picture. The size of the additional picture can be compressed to 1 / 4 or even 1 / 16 of the size of the main picture, greatly saving storage space, while ensuring that the image can be normally displayed on devices that do not support HDR. This way takes into account storage efficiency and compatibility, ensuring that the image can be adaptively presented on different platforms and devices.

[0267] In order to realize the above-mentioned embodiments, the embodiments of the present application further propose an image processing device.

[0268] Figure 8 A structural schematic diagram of an image processing device provided for the exemplary embodiments of the present application.

[0269] As Figure 8 shown, the image processing device 800 can include an acquisition module 810, a decoding module 820, and a rendering module 830.

[0270] The acquisition module 810 is configured to acquire an encoded image; the file structure of the encoded image includes a first captured image in a standard dynamic range (SDR) format and an additional picture, and the additional picture is used to indicate a color mapping relationship between the SDR format and a high dynamic range (HDR) format of the first captured image.

[0271] The decoding module 820 is configured to decode the encoded image to obtain the first captured image and the additional picture.

[0272] The rendering module 830 is configured to perform image rendering according to the additional picture and the first captured image to display a second captured image in an HDR format.

[0273] In an implementation manner of the embodiments of the present application, the additional picture is embedded after the first captured image, and the rendering module 830 is further configured to, in response to not having a decoding function for the encoded image, perform image rendering according to the encoded image to display the first captured image in an SDR format.

[0274] In an implementation manner of the embodiments of the present application, the additional picture is obtained by using the following modules:

[0275] determining a pixel value gain of each pixel point according to a pixel value of each pixel point in the first captured image and a pixel value of a corresponding pixel point in the second captured image; wherein the pixel value gain comprises gain values of a plurality of color channels, and the gain values are used to indicate a color mapping relationship of the corresponding color channels;

[0276] encoding the pixel value gain of each pixel point according to the lowest luminance and the highest luminance supported by the display, to obtain an additional image.

[0277] In an implementation form of the embodiment of the application, the encoding module is configured to: encode the pixel value gain of each pixel point according to the lowest luminance and the highest luminance supported by the display, to obtain an initial additional image; wherein each pixel point in the initial additional image is used to indicate a color mapping relationship of corresponding pixel points in the first captured image and the second captured image in the plurality of color channels; and perform down-sampling processing on the initial additional image to obtain the additional image.

[0278] In an implementation form of the embodiment of the application, the second captured image is obtained by using the following modules:

[0279] The conversion module is configured to obtain the first captured image in response to a capturing operation, and convert the first captured image to a linear domain to obtain a first intermediate image.

[0280] The elimination module is configured to perform crosstalk elimination processing on the plurality of color channels of the first intermediate image according to a crosstalk matrix to obtain a second intermediate image; wherein the crosstalk matrix is determined according to a color coupling degree between the plurality of color channels.

[0281] The mapping module is configured to perform inverse tone mapping processing on the second intermediate image to obtain the second captured image.

[0282] In an implementation form of the embodiment of the application, the mapping module is configured to: convert the second intermediate image from a first color space to a second color space based on a color gamut of the first captured image to obtain a third intermediate image; determine a low-light area and a high-light area from the third intermediate image; wherein a first color component corresponding to luminance in the low-light area is less than or equal to a luminance threshold, and a first color component in the high-light area is greater than the luminance threshold; perform tone mapping processing on the first color component in the low-light area by using a first mapping parameter, and perform tone mapping processing on the first color component in the high-light area by using at least one second mapping parameter, to obtain a fourth intermediate image; and convert the fourth intermediate image from the second color space to the first color space to obtain the second captured image.

[0283] In an implementation form of the embodiment of the application, the luminance threshold is determined according to a neutral gray luminance of the display; the mapping module is further configured to: determine a luminance adjustment factor according to a ratio of the neutral gray luminance and the first mapping parameter; and determine the at least one second mapping parameter according to the luminance adjustment factor, the first mapping parameter and the neutral gray luminance.

[0284] In an implementation form of the embodiment of the application, the mapping module is configured to: perform tone mapping processing on the first color component in the low-light area using the first mapping parameter, and perform tone mapping processing on the first color component in the high-light area using the at least one second mapping parameter to obtain a mapping image; and perform correction on a second color component other than the first color component in the mapping image according to the first color component in the mapping image to obtain a fourth intermediate image.

[0285] In an implementation form of the embodiment of the application, the mapping module is configured to: convert the fourth intermediate image from the second color space to the first color space to obtain a fifth intermediate image; perform inverse crosstalk cancellation processing on a plurality of color channels of the fifth intermediate image according to an inverse crosstalk matrix to obtain a sixth intermediate image; wherein the inverse crosstalk matrix is obtained by inverting the crosstalk matrix; and perform a luminance mapping operation on the sixth intermediate image according to a luminance range supported by the display to obtain the second captured image.

[0286] In an implementation form of the embodiment of the application, the mapping module is configured to: convert the fifth intermediate image from the first color space to a third color space to obtain a first converted image; determine a chroma adjustment factor according to a ratio of the first color component in the third intermediate image and the fourth intermediate image; perform smooth adjustment on chroma of the first converted image according to the chroma adjustment factor to obtain an adjusted image; convert the adjusted image from the third color space to the first color space to obtain a second converted image; and perform inverse crosstalk cancellation processing on a plurality of color channels of the second converted image according to the inverse crosstalk matrix to obtain the sixth intermediate image.

[0287] It should be noted that the foregoing explanation and description of the image processing method embodiment also apply to the image processing apparatus of the embodiment, which will not be described here.

[0288] In the image processing apparatus of the embodiment of the present application, the additional graph is used to indicate the color mapping relationship (such as the mapping relationship of luminance, hue, and saturation) required for converting from the SDR format to the HDR format, the first captured image in the SDR format is rendered by using the color mapping relationship provided by the additional graph, and the second captured image in the HDR format is generated. This can not only ensure that the luminance of the image is accurately adjusted in the process of converting from the SDR format to the HDR format, avoid the overexposure or underexposure phenomenon caused by improper luminance adjustment, and retain the details of the image, but also enable the rendered second captured image in the HDR format to exhibit more rich color gradation, thereby enhancing the realism of the second captured image and improving the visual experience of the user.

[0289] To achieve the above-mentioned embodiments, the present application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image processing method according to any one of the preceding embodiments.

[0290] Figure 9 A structural schematic diagram of an electronic device provided for the exemplary embodiments of the present application. For example, the electronic device 900 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0291] Reference Figure 9 The electronic device 900 can include one or more of the following components: a processing component 902, a memory 904, a power supply component 906, a multimedia component 908, an audio component 910, an input / output (I / O) interface 912, a sensor component 914, and a communication component 916.

[0292] The processing component 902 usually controls overall operations of the electronic device 900, such as operations associated with displaying, making phone calls, data communications, camera operations, and recording operations. The processing component 902 can include one or more processors 920 to execute instructions to complete all or part of steps of the methods described above. In addition, the processing component 902 can include one or more modules to facilitate the interaction between the processing component 902 and other components. For example, the processing component 902 can include a multimedia module to facilitate the interaction between the multimedia component 908 and the processing component 902.

[0293] Memory 904 is configured to store various types of data to support the operation of electronic device 900. Examples of this data include instructions for any application or method operating on electronic device 900, contact data, phonebook data, messages, pictures, videos, etc. Memory 904 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0294] Power component 906 provides power to various components of electronic device 900. Power component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 900.

[0295] Multimedia component 908 includes a screen that provides an output interface between the electronic device 900 and the user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen may be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 908 includes a front-facing camera and / or a rear-facing camera. When the electronic device 900 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0296] The audio component 910 is configured to output and / or input audio signals. For example, the audio component 910 includes a microphone (MIC) that is configured to receive an external audio signal when the electronic device 900 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 904 or transmitted via the communication component 916. In some embodiments, the audio component 910 also includes a speaker for outputting audio signals.

[0297] The I / O interface 912 provides an interface between the processing component 902 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0298] The sensor component 914 includes one or more sensors for providing various state assessments for the electronic device 900. For example, the sensor component 914 can detect an open / closed position of the electronic device 900, relative positioning of components, such as a display and a keypad of the electronic device 900, a change in position of the electronic device 900 or a component of the electronic device 900, the presence or absence of user contact with the electronic device 900, the orientation or acceleration / deceleration of the electronic device 900, and a temperature change of the electronic device 900. The sensor component 914 can include a proximity sensor configured to detect the presence of a nearby object without any physical touch. The sensor component 914 can also include a light sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, utilized in an imaging application. In some embodiments, the sensor component 914 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0299] The communication component 916 is configured to facilitate wired or wireless communication between the electronic device 900 and other devices. The electronic device 900 can access a wireless network based on a communication standard, such as WiFi, 4G, or 5G, or a combination thereof. In an example embodiment, the communication component 916 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 916 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0300] In example embodiments, the electronic device 900 can be implemented by one or more Application-Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field-Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements for performing the above-described methods.

[0301] In example embodiments, a non-transitory computer-readable storage medium including instructions, such as the memory 904 including instructions, is also provided, which can be executed by the processor 920 of the electronic device 900 to complete the above-described methods. For example, the non-transitory computer-readable storage medium can be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0302] To achieve the above-mentioned embodiments, the present application further provides a chip, wherein the chip comprises an interface circuit and a processing circuit coupled with each other, the interface circuit is configured to input or output a signal, and the processing circuit is configured to perform the image processing method provided in any one of the above-mentioned embodiments.

[0303] Figure 10 is a structural schematic diagram of a chip provided in an exemplary embodiment of the present application. Referring to Figure 10 the structural schematic diagram of the chip 1000 shown in FIG. 1, but the present application is not limited thereto.

[0304] The chip 1000 comprises a processing circuit 1001, which is configured to perform any one of the above-mentioned image processing methods.

[0305] In some embodiments, the chip 1000 further comprises one or more interface circuits 1002. As a possible implementation, the interface circuit 1002 is connected with a memory 1003, and the interface circuit 1002 can be configured to receive a signal from the memory 1003 or other devices, and the interface circuit 1002 can be configured to send a signal to the memory 1003 or other devices. For example, the interface circuit 1002 can read an instruction stored in the memory 1003 and send the instruction to the processing circuit 1001.

[0306] In some embodiments, the interface circuit 1002 performs at least one of the communication steps such as sending and / or receiving in the above-mentioned method, and the processing circuit 1001 performs other steps.

[0307] In some embodiments, the terms such as interface circuit, interface, transceiver pin, and transceiver can be replaced with each other.

[0308] In some embodiments, the chip 1000 further comprises one or more memories 1003 for storing instructions. As a possible implementation, all or part of the memory 1003 can be outside the chip 1000.

[0309] To achieve the above-mentioned embodiments, the present application further provides a non-transitory computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the image processing method described in any one of the above-mentioned method embodiments.

[0310] To achieve the above-mentioned embodiments, the present application further provides a computer program product, which stores a computer program, and the computer program is executed by a processor to implement the image processing method described in any one of the above-mentioned method embodiments.

[0311] In the description of the application, reference to "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that a particular feature, structure, material, or characteristic being described is included in at least one embodiment or example of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment or example. Furthermore, the described specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples. Moreover, the usage of the terms "first", "second" or "third" does not limit the quantity or order of the specific features, structures, materials or characteristics, but rather the term "first", "second" or "third" can be used to distinguish different features, structures, materials or characteristics, which can be combined in any suitable manner. Furthermore, the singular forms "a", "an" and "the" include plural references unless the context clearly dictates otherwise.

[0312] Furthermore, the terms "first", "second", or the like, merely denote different instances of a similar feature, structure, material or characteristic, without necessarily implying any relative importance or any particular order. Thus, a feature defined with "first" or "second" can implicitly or explicitly include at least one of the features. The meaning of "a", "an" and "the" includes plural references unless the context clearly dictates otherwise.

[0313] Any process or method descriptions or blocks in flow charts or otherwise described herein represent embodiments which can be managed as one or more modules, segments, or portions of code which include one or more steps for implementing specific logic functions or steps, and the terms in the description are used for causing or carrying out or upgrading of an action between other hardware under their control. The description of processes and methods of operations should be considered as merely illustrative of the principles of the application.

[0314] The logic and / or steps represented in flow diagrams or otherwise described herein, for example, can be considered as a sequence of instructions to implement logic functions, and can be realized in any computer-readable medium for use by an instruction execution system, apparatus, or device, such as a computer-based system, processor- based system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this specification, a "computer-readable medium" can be any means that can contain, store, communicate, propagate or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a machine-readable storage device (e.g., magnetic, optical or other storage device), a machine-readable storage substrate, a machine-readable signal, or any combination thereof. Other, specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electrical) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber (optical), and a portable compact disc read-only memory (CDROM). Note that the computer-readable medium can even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example via the optical scanner of a device or device, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and stored in a computer memory.

[0315] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. As such, in some embodiments, the steps or methods can be implemented in a combination of hardware and software. If implemented in hardware, as in another embodiment, any of the above techniques can be implemented with or without the use of a combination of one or more of the following technologies, which are well known in the art: discrete logic circuitry having logic gates for implementing logic functions upon an application of data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0316] Those of skill in the art could readily implement the above described example methods with or without the aid of programs in suitable computer readable media for implementing the embodiments described herein, and programs to carry out these methods can be stored in any computer readable medium, which can be carried in any computer program product. The programs, when executed, can direct an associated processor to provide the functionality described herein.

[0317] In addition, each functional unit in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module. When the integrated module is realized in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0318] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. An image processing method, characterized in that, include: Acquire an encoded image; wherein the file structure of the encoded image includes a first captured image in standard dynamic range (SDR) format and an additional image, the additional image being used to indicate the color mapping relationship between the SDR format and the high dynamic range (HDR) format of the first captured image; Decoding the encoded image yields the first captured image and the additional image; Image rendering is performed based on the additional image and the first captured image to display the second captured image in HDR format; The additional diagram is obtained through the following steps: Based on the pixel values ​​of each pixel in the first captured image and the corresponding pixel values ​​in the second captured image, the pixel value gain of each pixel is determined; wherein, the pixel value gain includes gain values ​​of multiple color channels, and the gain values ​​are used to indicate the color mapping relationship of the corresponding color channels; The gain values ​​of each pixel in multiple color channels are converted into log domain codes, and the color mapping relationship of each pixel in multiple color channels is determined based on the log domain codes of each pixel in multiple color channels, the minimum brightness and the maximum brightness supported by the display. An initial additional image is generated based on the color mapping relationship of each pixel in multiple color channels; wherein, each pixel in the initial additional image is used to indicate the color mapping relationship of corresponding pixels in the first captured image and the second captured image in multiple color channels. The initial additional image is downsampled to obtain the additional image.

2. The method according to claim 1, characterized in that, The additional image is embedded after the first captured image, and the method further includes: In response to the lack of decoding capability for the encoded image, image rendering is performed based on the encoded image to display the first captured image in SDR format.

3. The method according to claim 1, characterized in that, The second captured image was obtained using the following steps: In response to the shooting operation, the first captured image is acquired, and the first captured image is converted to the linear domain to obtain a first intermediate image; Crosstalk cancellation processing is performed on multiple color channels of the first intermediate image according to the crosstalk matrix to obtain the second intermediate image; wherein, the crosstalk matrix is ​​determined according to the degree of color coupling between the multiple color channels; The second intermediate image is subjected to inverse tone mapping to obtain the second captured image.

4. The method according to claim 3, characterized in that, The step of performing inverse tone mapping on the second intermediate image to obtain the second captured image includes: Based on the color gamut of the first captured image, the second intermediate image is converted from the first color space to the second color space to obtain the third intermediate image; Low-brightness regions and high-brightness regions are determined from the third intermediate image; wherein, the first color component corresponding to brightness in the low-brightness region is less than or equal to a brightness threshold, and the first color component in the high-brightness region is greater than the brightness threshold; A first mapping parameter is used to perform tone mapping processing on the first color component in the low-brightness region, and at least one second mapping parameter is used to perform tone mapping processing on the first color component in the brightness region to obtain a fourth intermediate image. The fourth intermediate image is converted from the second color space to the first color space to obtain the second captured image.

5. The method according to claim 4, characterized in that, The brightness threshold is determined based on the neutral gray brightness of the display; The at least one second mapping parameter is determined using the following steps: The brightness adjustment factor is determined based on the ratio of the neutral gray brightness to the first mapping parameter; At least one second mapping parameter is determined based on the brightness adjustment factor, the first mapping parameter, and the neutral gray brightness.

6. The method according to claim 4, characterized in that, The step of performing tone mapping processing on the first color component in the low-brightness region using a first mapping parameter, and performing tone mapping processing on the first color component in the high-brightness region using at least one second mapping parameter, to obtain a fourth intermediate image, includes: A first mapping parameter is used to perform tone mapping processing on the first color component in the low-brightness region, and at least one second mapping parameter is used to perform tone mapping processing on the first color component in the high-brightness region to obtain a mapped image; Based on the first color component in the mapped image, the second color component in the mapped image, excluding the first color component, is corrected to obtain the fourth intermediate image.

7. The method according to claim 4, characterized in that, The step of converting the fourth intermediate image from the second color space to the first color space to obtain the second captured image includes: The fourth intermediate image is converted from the second color space to the first color space to obtain the fifth intermediate image; Based on the inverse crosstalk matrix, inverse crosstalk cancellation processing is performed on multiple color channels of the fifth intermediate image to obtain the sixth intermediate image; wherein, the inverse crosstalk matrix is ​​obtained by inverting the crosstalk matrix; Based on the brightness range supported by the display, a brightness mapping operation is performed on the sixth intermediate image to obtain the second captured image.

8. The method according to claim 7, characterized in that, The step of performing inverse crosstalk cancellation processing on multiple color channels of the fifth intermediate image according to the inverse crosstalk matrix to obtain the sixth intermediate image includes: The fifth intermediate image is converted from the first color space to the third color space to obtain the first converted image; The chromaticity adjustment factor is determined based on the ratio of the first color component in the third intermediate image to that in the fourth intermediate image; Based on the chroma adjustment factor, the first converted image is subjected to chroma smoothing adjustment to obtain an adjusted image; The adjusted image is converted from the third color space to the first color space to obtain the second converted image; Based on the inverse crosstalk matrix, inverse crosstalk cancellation processing is performed on multiple color channels of the second converted image to obtain the sixth intermediate image.

9. An image processing apparatus, characterized in that, include: An acquisition module is used to acquire an encoded image; wherein, the file structure of the encoded image includes a first captured image in standard dynamic range (SDR) format and an additional image, the additional image being used to indicate the color mapping relationship between the SDR format and the high dynamic range (HDR) format of the first captured image; A decoding module is used to decode the encoded image to obtain the first captured image and the additional image; A rendering module is used to render an image based on the additional image and the first captured image to display the second captured image in HDR format; The additional diagram is obtained using the following module: The determining module is used to determine the pixel value gain of each pixel based on the pixel value of each pixel in the first captured image and the pixel value of the corresponding pixel in the second captured image; wherein the pixel value gain includes the gain values ​​of multiple color channels, and the gain values ​​are used to indicate the color mapping relationship of the corresponding color channels; An encoding module is used to convert the gain values ​​of each pixel in multiple color channels into log-domain encodings, and determine the color mapping relationship of each pixel in multiple color channels based on the log-domain encodings of each pixel in multiple color channels, the minimum brightness supported by the display, and the maximum brightness supported by the display; generate an initial additional image based on the color mapping relationship of each pixel in multiple color channels; wherein each pixel in the initial additional image is used to indicate the color mapping relationship of corresponding pixels in the first captured image and the second captured image in multiple color channels; and perform downsampling processing on the initial additional image to obtain the additional image.

10. The apparatus according to claim 9, characterized in that, The additional image is embedded after the first captured image, and the rendering module is further configured to: In response to the lack of decoding capability for the encoded image, image rendering is performed based on the encoded image to display the first captured image in SDR format.

11. The apparatus according to claim 9, characterized in that, The second captured image was obtained using the following module: A conversion module is used to, in response to a shooting operation, acquire the first captured image and convert the first captured image to the linear domain to obtain a first intermediate image; The crosstalk elimination module is used to perform crosstalk elimination processing on multiple color channels of the first intermediate image according to the crosstalk matrix to obtain a second intermediate image; wherein, the crosstalk matrix is ​​determined according to the degree of color coupling between the multiple color channels; The mapping module is used to perform inverse tone mapping on the second intermediate image to obtain the second captured image.

12. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the steps of the method as described in any one of claims 1 to 8.

13. A non-transitory computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method according to any one of claims 1 to 8.

14. A chip, characterized in that, The chip includes an interface circuit and a processing circuit that are coupled to each other. The interface circuit is used to input or output signals, and the processing circuit is used to implement the method of any one of claims 1 to 8.

15. A computer program product, characterized in that, It includes a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

Images