Chromatic adaptation method and electronic device

Abstract

Disclosed in embodiments of the present application are a chromatic adaptation method and an electronic device. The electronic device has a capture end and a display end. At the capture end, the electronic device can acquire a first raw image under a first ambient color temperature, generate a first image file on the basis of the first raw image and store the first image file, wherein the first image file comprises a first image and first metadata, and the first metadata comprises the first ambient color temperature, a desired white point color temperature of the first image displayed under the first ambient color temperature, and a white point color temperature of the first image. At the display end, the electronic device can generate, on the basis of a second ambient color temperature, the first image that has undergone chromatic adaptation adjustment, and display same, wherein the white point color temperature of the first image that has undergone chromatic adaptation adjustment is determined on the basis of the first metadata. In this way, the electronic device can reproduce, as accurately as possible, colors perceived by human eyes in environments of different color temperatures, thereby providing more realistic capture and view experience for users.

Description

A method and electronic device for color adaptation

[0001] This application claims priority to Chinese Patent Application No. 202510052984.6, filed on January 10, 2025, with the invention entitled "A method and electronic device for color adaptation", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of terminal technology, and in particular to a method and electronic device for color adaptation. Background Technology

[0003] To provide users with a realistic color experience when viewing images, electronic devices need to offer a color adaptation method to adjust the final color temperature of the displayed image, simulating how the human eye adapts to and perceives colors at different color temperatures. One solution is for electronic devices to directly adjust the screen's color temperature based on the ambient color temperature. This reduces color perception differences caused by environmental changes when viewing images, helping users obtain a more realistic color experience and thus improving the user experience.

[0004] However, on the one hand, while the aforementioned solution can alleviate the problem of unrealistic user perception when viewing images under different color temperature environments to some extent, it is still insufficient to meet users' needs in the field of photography. On the other hand, the degree of color adaptation of the human eye varies for different shooting content. In the aforementioned solution, the electronic device's adjustment of the screen display color temperature cannot achieve different effects for different shooting content. Therefore, the electronic device cannot accurately simulate the differences in color perception of the human eye for different shooting objects, and thus cannot provide users with a realistic shooting experience for all shooting objects. Summary of the Invention

[0005] This application provides a color adaptation method and an electronic device. The electronic device has a shooting end and a display end. At the shooting end, the electronic device can capture a first original image under a first ambient color temperature, and generate and save a first image file based on the first original image. The first image file includes a first image and first metadata, which includes the first ambient color temperature, the expected white point color temperature of the first image when displayed under the first ambient color temperature, and the white point color temperature of the first image in the first image file. At the display end, the electronic device can generate and display a color-adapted first image based on a second ambient color temperature. The white point color temperature of the color-adapted first image is determined based on the aforementioned first metadata. Therefore, the electronic device can maximize the similarity between the color displayed in the color-adapted image under the second ambient color temperature and the color perceived by the human eye after color adaptation under that ambient color temperature, thereby providing the user with a more realistic shooting and viewing experience.

[0006] In a first aspect, this application provides a color adaptation method applied to a first electronic device, the first electronic device including a camera, the method including: acquiring a first original image under a first ambient color temperature; generating and saving a first image file based on the first original image, the first image file including a first image and first metadata, the first image being obtained by processing the first original image with standard white balance, the first metadata including the first ambient color temperature, the expected white point color temperature when the first image is displayed under the first ambient color temperature, and the white point color temperature of the first image in the first image file.

[0007] By implementing the method provided in the first aspect, an electronic device with a camera can capture an original image and generate an image file based on that original image. This image file contains the ambient color temperature, the actual white point color temperature (i.e., the expected white point color temperature when the first image is displayed under the first ambient color temperature), and the standard white point color temperature (i.e., the white point color temperature of the first image in the first image file). By including the ambient color temperature, the actual white point color temperature, and the standard white point color temperature in the image file, other electronic devices can perform precise color adaptation adjustments when displaying the image file. This allows the adjusted image to closely approximate the image perceived by the user's eye after color adaptation in the viewing environment, thereby providing the user with a more realistic shooting and viewing experience.

[0008] In conjunction with the first aspect, in some embodiments, generating and saving a first image file based on a first original image specifically includes: inputting the first original image into a human visual model and outputting the expected white point color temperature of the first image when displayed under a first ambient color temperature and the white point color temperature of the first image in the first image file.

[0009] By implementing the method provided in the above embodiments, the real-world white point color temperature and the standard playback white point color temperature contained in the image file can be obtained based on the HVM model. The HVM model has been trained with a large amount of data, which can ensure that the image after color adaptation adjustment based on the real-world white point color temperature and the standard playback white point color temperature is closer to the perception of the human eye.

[0010] In conjunction with the first aspect, in some embodiments, the method further includes: generating and displaying a first image after color adaptation adjustment based on a second ambient color temperature; the white point color temperature of the first image after color adaptation adjustment is different from the white point color temperature of the first image in the first image file.

[0011] By implementing the method provided in the above embodiments, the electronic device with a shooting end also has a display end, and the user can directly see the color-adapted image on the electronic device. The white point color temperature of the color-adapted image is different from the white point color temperature of the image in the stored image file.

[0012] In conjunction with the first aspect, in some embodiments, when the second ambient color temperature is higher than the first ambient color temperature, the color temperature of the first image after color adaptation adjustment is higher than the color temperature of the first image in the first image file.

[0013] By implementing the method provided in the above embodiments, the color adaptation method adjusts the color temperature of the image in the same way as the changing trend of the ambient color temperature, thereby making the color-adapted image closer to the image perceived by the human eye through color adaptation.

[0014] In conjunction with the first aspect, in some embodiments, the method further includes: acquiring a second original image at a third ambient color temperature, the second original image having the same image content as the first original image; generating and saving a second image file based on the second original image, the second image file including a second image and first metadata, the second image being obtained by standard white balance processing of the second original image, the second metadata including the third ambient color temperature, the expected white point color temperature of the second image when displayed at the third ambient color temperature, and the white point color temperature of the second image in the second image file; generating and displaying a color-adapted second image based on a fourth ambient color temperature; if the deviation between the expected white point color temperature of the second image when displayed at the third ambient color temperature and the white point color temperature of the second image in the second image file is greater than the deviation between the expected white point color temperature of the first image when displayed at the first ambient color temperature and the white point color temperature of the first image in the first image file, then the deviation between the white point color temperature of the color-adapted second image and the white point color temperature of the second image in the second image file is greater than the deviation between the white point color temperature of the color-adapted first image and the white point color temperature of the first image in the first image file.

[0015] In conjunction with the first aspect, in some embodiments, the method further includes: acquiring a third original image at a first ambient color temperature, the third original image having different image content from the first original image; generating and saving a third image file based on the third original image, the third image file including a third image and third metadata, the third image being obtained by standard white balance processing of the third original image, the third metadata including the first ambient color temperature, the expected white point color temperature of the third image when displayed at the first ambient color temperature, and the white point color temperature of the third image in the third image file; generating and displaying a color-adapted third image based on a second ambient color temperature; the white point color temperature of the color-adapted third image being different from the white point color temperature of the first image in the first image file.

[0016] By implementing the method provided in the above embodiments, the first and third images captured and saved at the same color temperature can undergo different color adaptation adjustments based on the differences in image content. That is, the electronic device can perform different color adaptation adjustments on the image based on different display content, thereby accurately simulating the differences in color perception of the human eye for different subjects, providing users with a realistic shooting experience for all subjects.

[0017] In conjunction with the first aspect, in some embodiments, before generating and displaying the first image after color adaptation adjustment based on the second ambient color temperature, the method further includes: adjusting the basic color adaptation mapping curve based on the first ambient color temperature and the expected white point color temperature when the first image is displayed under the first ambient color temperature to obtain a first color adaptation mapping curve, wherein the white point color temperature corresponding to the first ambient color temperature on the basic color adaptation mapping curve is different from the expected white point color temperature when the first image is displayed under the first ambient color temperature, and the white point color temperature corresponding to the first ambient color temperature on the first color adaptation mapping curve is the expected white point color temperature when the first image is displayed under the first ambient color temperature; and determining the white point color temperature for displaying the first image under the second ambient color temperature based on the first color adaptation mapping curve.

[0018] By implementing the method provided in the above embodiments, the electronic device can adjust the basic color adaptation mapping curve based on the ambient color temperature (i.e., the first ambient color temperature) and the actual white point color temperature (i.e., the expected white point color temperature when the first image is displayed under the first ambient color temperature) to obtain a first color adaptation mapping curve for the first image. This allows the electronic device to more accurately determine the white point color temperature of the color-adapted image based on this first color adaptation mapping curve, resulting in more precise color adaptation adjustments. The adjusted result will be closer to the image perceived by the human eye after color adaptation. Furthermore, when reviewing an image where the ambient color temperature equals the shooting ambient color temperature, the electronic device can completely restore the image to the color temperature perceived by the human eye after color adaptation under the shooting environment.

[0019] In conjunction with the first aspect, in some embodiments, the ambient color temperature includes a first adjusted color temperature and a second adjusted color temperature. The white point color temperature corresponding to the first adjusted color temperature on the basic color adaptation mapping curve is the same as the white point color temperature corresponding to the first adjusted color temperature on the first color adaptation mapping curve. The white point color temperature corresponding to the second adjusted color temperature on the basic color adaptation mapping curve is the same as the white point color temperature corresponding to the second adjusted color temperature on the first color adaptation mapping curve. The first ambient color temperature is located between the first adjusted color temperature and the second adjusted color temperature. Within the range of the first adjusted color temperature to the second adjusted color temperature, the white point color temperature corresponding to each ambient color temperature on the first color adaptation mapping curve is obtained by interpolating the expected white point color temperature when the first image is displayed at the first ambient color temperature with the white point color temperature corresponding to the first adjusted color temperature on the first color adaptation mapping curve, or by interpolating the expected white point color temperature when the first image is displayed at the first ambient color temperature with the white point color temperature corresponding to the second adjusted color temperature on the first color adaptation mapping curve.

[0020] By implementing the method provided in the above embodiments, the basic color adaptation mapping curve is adjusted using interpolation before and after the ambient color temperature is captured, resulting in the first color adaptation mapping curve. Therefore, when the electronic device subsequently determines the white point color temperature of the color-adapted image based on this first color adaptation mapping curve for the first image, it can be more accurate, leading to more precise color adaptation adjustments. The adjusted result will be closer to the image perceived by the human eye after color adaptation.

[0021] In conjunction with the first aspect, in some embodiments, there are one or more basic color adaptation mapping curves, and the multiple basic color adaptation mapping curves are basic color adaptation mapping curves corresponding to different ambient brightness. If the electronic device contains multiple basic color adaptation mapping curves, before adjusting the basic color adaptation mapping curve based on the first ambient color temperature and the desired white point color temperature when the first image is displayed at the first ambient color temperature, the method further includes: selecting one from the multiple basic color adaptation mapping curves as the basic color adaptation mapping curve for determining the white point color temperature of the first image after color adaptation adjustment.

[0022] The method provided in the above embodiments allows for the existence of multiple base color adaptation mapping curves used to adjust and obtain the first color adaptation mapping curve. This results in different effects on images after color adaptation adjustment when viewing certain ambient color temperatures.

[0023] In conjunction with the first aspect, in some embodiments, the method further includes: detecting the brightness of the environment in which the first image is displayed; selecting one from a plurality of basic color adaptation mapping curves as a basic color adaptation mapping curve for determining the white point color temperature of the first image after color adaptation adjustment, specifically including: selecting the basic color adaptation mapping curve corresponding to the brightness of the environment in which the first image is displayed from a plurality of basic color adaptation mapping curves as a basic color adaptation mapping curve for determining the white point color temperature of the first image after color adaptation adjustment, based on the brightness of the environment in which the first image is displayed.

[0024] By implementing the method provided in the above embodiments, electronic devices can select a basic color adaptation mapping curve based on ambient brightness, thereby adapting to different shooting needs.

[0025] In conjunction with the first aspect, in some embodiments, the method further includes: determining a color adaptation matrix based on the white point color temperature of the first image when it is displayed at a first ambient color temperature and the white point color temperature of the first image in the first image file; the color adaptation matrix is ​​calculated from a color space transformation matrix and a diagonal matrix; the values ​​on the diagonal of the diagonal matrix are used to indicate the scaling degree of the human eye's response to long wavelengths, medium wavelengths, and short wavelengths in the screen data; the color space transformation matrix includes a transformation matrix between the RGB color space and the XYZ color space, and a transformation matrix between the XYZ color space and the LMS color space; displaying the first image at a second ambient color temperature specifically includes: converting the first image in the first image file from the RGB color space to the XYZ color space;

[0026] The first image is converted from the XYZ color space to the LMS color space; the first image in the LMS color space is color transformed using a color adaptation matrix; the first image after color transformation is converted from the LMS color space to the XYZ color space; the first image after color transformation is converted from the XYZ color space to the RGB color space; and the first image after color transformation is displayed in the RGB color space.

[0027] In conjunction with the first aspect, in some embodiments, the first image and the second image are simultaneously displayed on different layers of the background image. The residual between the color adaptation matrix of the first image and the color adaptation matrix of the background image is used to perform color transformation on the layer containing the first image. After color transformation, the white point color temperature displayed on the layer containing the first image is closer to the white point color temperature of the first image perceived by the human eye after color adaptation under the second ambient color temperature. The residual between the color adaptation matrix of the second image and the color adaptation matrix of the background image is used to perform color transformation on the layer containing the second image. After color transformation, the white point color temperature displayed on the layer containing the second image is closer to the white point color temperature of the second image perceived by the human eye after color adaptation under the second ambient color temperature. The color adaptation matrix of the background image is used to perform color transformation on the fused image after layer fusion.

[0028] By implementing the method provided in the above embodiments, the electronic device can adjust the color of different layers and the merged image separately when displaying multiple images, that is, the color transformation mentioned above, so as to ensure that the background image can be displayed with normal color temperature after the above color adaptation processing.

[0029] In conjunction with the first aspect, in some embodiments, the method further includes: sending a first image file to a second electronic device, the second electronic device including a display screen, the second electronic device being used to generate and display a color-adapted first image based on a second ambient color temperature; the white point color temperature of the color-adapted first image is different from the white point color temperature of the first image in the first image file.

[0030] If the method provided in the above embodiments is implemented, the first electronic device does not have a display end. In this case, the first electronic device needs to send the saved first image file to the second electronic device with a display screen for display.

[0031] In conjunction with the first aspect, in some embodiments, when the second ambient color temperature is higher than the first ambient color temperature, the color temperature of the first image after color adaptation adjustment is higher than the color temperature of the first image in the first image file.

[0032] In conjunction with the first aspect, in some embodiments, the method further includes: acquiring a third original image at a first ambient color temperature, the third original image having different image content from the first original image; generating and saving a third image file based on the third original image, the third image file including a third image and third metadata, the third image being obtained by standard white balance processing of the third original image, the third metadata including the first ambient color temperature, the expected white point color temperature of the third image when displayed at the first ambient color temperature, and the white point color temperature of the third image in the third image file; generating and displaying a color-adapted third image based on a second ambient color temperature; the white point color temperature of the color-adapted third image being different from the white point color temperature of the first image in the first image file.

[0033] Secondly, this application provides a color adaptation method applied to a second electronic device, the second electronic device including a display screen, the method comprising: receiving a first image file from a first electronic device, the first image file including a first image and first metadata, the first image being obtained by standard white balance processing of a first original image, the first metadata including a first ambient color temperature, the expected white point color temperature of the first image when displayed at the first ambient color temperature, and the white point color temperature of the first image in the first image file; generating and displaying a color-adapted first image based on a second ambient color temperature; the white point color temperature of the color-adapted first image being different from the white point color temperature of the first image in the first image file.

[0034] By implementing the method provided in the second aspect, the second electronic device can, after receiving the first image file, perform color adaptation adjustment on the first image based on the first metadata in the first image file, so that the adjusted image can be close to the image perceived by the human eye after color adaptation in the viewing environment, thereby providing the user with a more realistic shooting and viewing experience.

[0035] In conjunction with the second aspect, in some embodiments, the method further includes: receiving a third image file from a first electronic device, the third image file being generated and saved from a third original image acquired by the first electronic device at a first ambient color temperature, the third image file including a third image and third metadata, the third image being obtained by standard white balance processing of the third original image, the third metadata including the first ambient color temperature, the expected white point color temperature of the third image when displayed at the first ambient color temperature, and the white point color temperature of the third image in the third image file; generating and displaying a color-adapted third image based on the second ambient color temperature; the white point color temperature of the color-adapted third image being different from the white point color temperature of the first image in the first image file.

[0036] By implementing the method provided in the above embodiments, the first and third images captured and saved at the same color temperature can undergo different color adaptation adjustments based on the differences in image content. That is, the electronic device can perform different color adaptation adjustments on the image based on different display content, thereby accurately simulating the differences in color perception of the human eye for different subjects, providing users with a realistic shooting experience for all subjects.

[0037] In conjunction with the second aspect, in some embodiments, before generating and displaying the first image after color adaptation adjustment based on the second ambient color temperature, the method further includes: adjusting the basic color adaptation mapping curve based on the first ambient color temperature and the expected white point color temperature when the first image is displayed under the first ambient color temperature to obtain a first color adaptation mapping curve, wherein the white point color temperature corresponding to the first ambient color temperature on the basic color adaptation mapping curve is different from the expected white point color temperature when the first image is displayed under the first ambient color temperature, and the white point color temperature corresponding to the first ambient color temperature on the first color adaptation mapping curve is the expected white point color temperature when the first image is displayed under the first ambient color temperature; and determining the white point color temperature for displaying the first image under the second ambient color temperature based on the first color adaptation mapping curve.

[0038] By implementing the method provided in the above embodiments, the electronic device can adjust the basic color adaptation mapping curve based on the ambient color temperature (i.e., the first ambient color temperature) and the actual white point color temperature (i.e., the expected white point color temperature when the first image is displayed under the first ambient color temperature) to obtain a first color adaptation mapping curve for the first image. This allows the electronic device to more accurately determine the white point color temperature of the color-adapted image based on this first color adaptation mapping curve, resulting in more precise color adaptation adjustments. The adjusted result will be closer to the image perceived by the human eye after color adaptation. Furthermore, when reviewing an image where the ambient color temperature equals the shooting ambient color temperature, the electronic device can completely restore the image to the color temperature perceived by the human eye after color adaptation under the shooting environment.

[0039] In conjunction with the second aspect, in some embodiments, the ambient color temperature includes a first adjusted color temperature and a second adjusted color temperature. The white point color temperature corresponding to the first adjusted color temperature on the basic color adaptation mapping curve is the same as the white point color temperature corresponding to the first adjusted color temperature on the first color adaptation mapping curve. The white point color temperature corresponding to the second adjusted color temperature on the basic color adaptation mapping curve is the same as the white point color temperature corresponding to the second adjusted color temperature on the first color adaptation mapping curve. The first ambient color temperature is located between the first adjusted color temperature and the second adjusted color temperature. Within the range of the first adjusted color temperature to the second adjusted color temperature, the white point color temperature corresponding to each ambient color temperature on the first color adaptation mapping curve is obtained by interpolating the expected white point color temperature when the first image is displayed at the first ambient color temperature with the white point color temperature corresponding to the first adjusted color temperature on the first color adaptation mapping curve, or by interpolating the expected white point color temperature when the first image is displayed at the first ambient color temperature with the white point color temperature corresponding to the second adjusted color temperature on the first color adaptation mapping curve.

[0040] By implementing the method provided in the above embodiments, the basic color adaptation mapping curve is adjusted using interpolation before and after the ambient color temperature is captured, resulting in the first color adaptation mapping curve. Therefore, when the electronic device subsequently determines the white point color temperature of the color-adapted image based on this first color adaptation mapping curve for the first image, it can be more accurate, leading to more precise color adaptation adjustments. The adjusted result will be closer to the image perceived by the human eye after color adaptation.

[0041] In conjunction with the second aspect, in some embodiments, there are one or more basic color adaptation mapping curves, and the multiple basic color adaptation mapping curves are basic color adaptation mapping curves corresponding to different ambient brightness. If the electronic device contains multiple basic color adaptation mapping curves, before adjusting the basic color adaptation mapping curve based on the first ambient color temperature and the desired white point color temperature when the first image is displayed at the first ambient color temperature, the method further includes: selecting one from the multiple basic color adaptation mapping curves as the basic color adaptation mapping curve for determining the white point color temperature of the first image after color adaptation adjustment.

[0042] The method provided in the above embodiments allows for the existence of multiple base color adaptation mapping curves used to adjust and obtain the first color adaptation mapping curve. This results in different effects on images after color adaptation adjustment when viewing certain ambient color temperatures.

[0043] In conjunction with the second aspect, in some embodiments, the method further includes: detecting the brightness of the environment in which the first image is displayed using a brightness sensor; selecting one of a plurality of basic color adaptation mapping curves as a basic color adaptation mapping curve for determining the white point color temperature of the first image after color adaptation adjustment, specifically including: selecting one of a plurality of basic color adaptation mapping curves as a basic color adaptation mapping curve for determining the white point color temperature of the first image after color adaptation adjustment based on the brightness of the environment in which the first image is displayed.

[0044] By implementing the method provided in the above embodiments, electronic devices can select a basic color adaptation mapping curve based on ambient brightness, thereby adapting to different shooting needs.

[0045] In conjunction with the second aspect, in some embodiments, the method further includes: determining a color adaptation matrix based on the white point color temperature of the first image displayed at a first ambient color temperature and the white point color temperature of the first image in the first image file; the color adaptation matrix is ​​calculated from a color space transformation matrix and a diagonal matrix; the values ​​on the diagonal of the diagonal matrix are used to indicate the scaling degree of the human eye's response to long wavelengths, medium wavelengths, and short wavelengths in the screen data; the color space transformation matrix includes a transformation matrix between the RGB color space and the XYZ color space, and a transformation matrix between the XYZ color space and the LMS color space; displaying the first image at a second ambient color temperature specifically includes: converting the first image in the first image file from the RGB color space to the XYZ color space; converting the first image from the XYZ color space to the LMS color space; performing a color transformation on the first image in the LMS color space using the color adaptation matrix; converting the color-transformed first image from the LMS color space to the XYZ color space; converting the color-transformed first image from the XYZ color space to the RGB color space; and displaying the color-transformed first image in the RGB color space.

[0046] In conjunction with the second aspect, in some embodiments, the first image and the second image are simultaneously displayed on different layers of the background image. The residual between the color adaptation matrix of the first image and the color adaptation matrix of the background image is used to perform color transformation on the layer containing the first image. After color transformation, the white point color temperature displayed on the layer containing the first image is closer to the white point color temperature of the first image perceived by the human eye after color adaptation under the second ambient color temperature. The residual between the color adaptation matrix of the second image and the color adaptation matrix of the background image is used to perform color transformation on the layer containing the second image. After color transformation, the white point color temperature displayed on the layer containing the second image is closer to the white point color temperature of the second image perceived by the human eye after color adaptation under the second ambient color temperature. The color adaptation matrix of the background image is used to perform color transformation on the fused image after layer fusion.

[0047] By implementing the method provided in the above embodiments, the electronic device can adjust the color of different layers and the merged image separately when displaying multiple images, that is, the color transformation mentioned above, so as to ensure that the background image can be displayed with normal color temperature after the above color adaptation processing.

[0048] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the method described in the first aspect and any possible implementation thereof.

[0049] Fourthly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the method described in the second aspect and any possible implementation thereof.

[0050] Fifthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect and any possible implementation thereof, or in the second aspect and any possible implementation thereof.

[0051] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect and any possible implementation thereof, or in the second aspect and any possible implementation thereof.

[0052] Understandably, the electronic devices provided in the third aspect, the fourth aspect, the fifth aspect, and the sixth aspect are all used to execute the methods provided in this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here. Attached Figure Description

[0053] Figure 1 is a schematic diagram comparing the actual image color temperature displayed under a low color warm light environment and the image color temperature perceived by the human eye after color adaptation, according to an embodiment of this application.

[0054] Figure 2 is a schematic diagram comparing the actual image color temperature displayed under the D50 standard environment and the image color temperature perceived by the human eye after color adaptation, according to an embodiment of this application.

[0055] Figure 3 is a schematic diagram of a user's perception of the image color temperature of the same image under different ambient color temperatures, provided by an embodiment of this application;

[0056] Figure 4 is a schematic diagram of another user's perception of the image color temperature of the same image under different ambient color temperatures, provided by an embodiment of this application;

[0057] Figure 5 is a schematic diagram of a color temperature adjustment mapping relationship provided in an embodiment of this application;

[0058] Figure 6 is a simplified color adaptation processing flow diagram provided in an embodiment of this application;

[0059] Figure 7 is a schematic diagram of a detailed color adaptation process provided in an embodiment of this application;

[0060] Figure 8 is a diagram showing the correspondence between different adaptive white points and image color temperatures provided in an embodiment of this application;

[0061] Figure 9 is a schematic diagram comparing the color temperature difference between the real white point on-site and the real white point on standard playback under the same color temperature range for different image contents provided in an embodiment of this application.

[0062] Figure 10 is a schematic diagram illustrating a real human factor experiment provided in an embodiment of this application;

[0063] Figure 11 is a graph of a basic color adaptation mapping relationship provided in an embodiment of this application;

[0064] Figure 12 is a schematic diagram of a basic color adaptation mapping curve based on white point color temperature adjustment provided in an embodiment of this application;

[0065] Figure 13 is a schematic diagram of a color adaptation conversion process provided in an embodiment of this application;

[0066] Figure 14 is a schematic diagram of an exemplary display end color adaptation process provided in an embodiment of this application;

[0067] Figure 15 is a schematic diagram of an image display area containing multiple images provided in an embodiment of this application;

[0068] Figure 16 is a processing flow for color adaptation when displaying multiple images according to an embodiment of this application;

[0069] Figure 17 is a schematic diagram illustrating the effect of a color adaptation method provided in an embodiment of this application;

[0070] Figure 18 is a schematic diagram of a shooting color adaptation implementation method provided in an embodiment of this application;

[0071] Figure 19 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0072] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be a limitation of this application.

[0073] In the field of photography, the color temperature of the shooting environment can affect the user's perception of image colors. Figure 1 is a schematic diagram comparing the actual displayed image color temperature and the image color temperature perceived by the human eye after color adaptation under a low color warm light environment, according to an embodiment of this application.

[0074] As shown in Figure 1, a user can take a picture of a subject (the person in Figure 1) in a low color warm light environment (such as a yellow light environment), resulting in image A. Assume that in this low color warm light environment, the actual color temperature of image A is 3800K, while the color temperature perceived by the user is 5400K. Comparing the actual color temperature of image A in Figure 1 with the color temperature perceived by the human eye after color adaptation, it can be seen that in the low color warm light environment, the color temperature of image A perceived by the user's eye is higher. In other words, when viewing image A in this low color warm light environment, the user will perceive that image A is not as yellow as it actually appears.

[0075] This is due to the human eye's ability to adapt to color. This ability refers to the human visual system's capacity to automatically adjust the relative sensitivity of the three types of cone cells in the retina when observation conditions change, in order to maintain a consistent color perception of the same physical target. Therefore, in a low-color, warm light environment as shown in Figure 1, after the human eye adapts to the environment, the sensitivity of the cone cells to yellow decreases, leading to a reduced perception of yellow within the user's field of vision. Consequently, the user perceives image A as not yellow, meaning the color temperature of image A perceived by the user's eye is too high.

[0076] However, when a user revisits image A, they may have moved beyond the aforementioned low color warm light environment. For example, the user could revisit image A in the D50 standard environment shown in Figure 2. Similarly, if the user shares image A with another user, that user can also view image A in a non-shooting environment (e.g., the aforementioned D50 standard environment). The environment in which image A is revisited is also called the revisit environment. Taking the aforementioned D50 standard environment as an example, the human eye does not need to adapt to the environment's color; that is, the sensitivity of the user's cone cells does not decrease in the D50 standard environment. Therefore, as shown in Figure 2, the color temperature perceived by the user's eye when revising image A is the actual color temperature of image A displayed in the D50 standard revisit environment. Furthermore, regardless of whether it is displayed in a low color warm light environment or the D50 standard revisit environment, the actual display color temperature of image A depends only on the displayed image itself; that is, regardless of the environment, the actual display color temperature of image A is 3800K. Therefore, under the aforementioned D50 standard environment, the color temperature of image A perceived by the user's human eye should also be 3800K.

[0077] Understandably, the human eye undergoes color adaptation in the low color temperature warm light environment shown in Figure 1. For the user, the color temperature of image A perceived by the human eye in this environment appears relatively realistic. However, comparing the color temperature perceived by the human eye in the D50 standard viewing environment shown in Figure 2 with that in the low color temperature warm light environment shown in Figure 1, it can be seen that when viewing image A in the D50 standard environment shown in Figure 2, the human eye perceives image A as yellowish. Therefore, for the same image (image A in this case), the user will feel that image A is not realistic when viewing it in the D50 standard environment. Here, the user's perception of image realism means that the image can successfully reproduce objects, scenes, or people in the real world visually, thus minimizing the difference between the image perceived by the user and what they actually see. In the scenario of color temperature changes in the shooting environment, the user's perception of image realism specifically refers to the color temperature of the image displayed in the viewing environment matching the color temperature perceived by the human eye after color adaptation under the ambient color temperature of the viewing environment; that is, the user obtains a realistic color experience. Therefore, for users, the true image they perceive refers to the image whose color temperature has been adjusted based on the environment in which it is viewed.

[0078] In order to provide users with a realistic color experience when viewing images, electronic devices need to provide a color adaptation method to adjust the final color of the image to simulate the human eye's adaptation and perception of color under different color temperatures.

[0079] One solution to the above problem is that electronic devices can directly adjust the screen display color temperature based on the ambient color temperature, thereby reducing the color perception differences caused by environmental changes when viewing images, helping users obtain a relatively realistic color experience, and thus improving the user experience.

[0080] Figure 3 is a schematic diagram of a user's perception of the color temperature of the same image under different ambient color temperatures, provided by an embodiment of this application.

[0081] As shown in Figure 3, assuming the color temperature of the shooting environment is 2800K and the color temperature of the viewing environment is 5800K, it can be seen from Figure 3 that compared to the shooting environment of 2800K, when viewing the captured image in the viewing environment of 5800K, the user perceives a lower color temperature, and the user will also perceive the image as having a yellowish tint when viewing it.

[0082] Understandably, in a 2800K shooting environment, the image directly captured by the camera reflects the actual color temperature of the environment. Compared to the colors perceived by the user in a 2800K environment, the captured image will appear yellowish. If the electronic device stores the image directly captured by the camera, the difference between the image's color temperature and the user's perceived color temperature in the shooting environment will cause the user to perceive the image as unrealistic. Therefore, electronic devices have an built-in automatic white balance (AWB) algorithm. During shooting, after the camera captures the image, the electronic device can use the AWB algorithm to adjust the image's color temperature, ensuring that white objects appear white in the processed image. This simulates the human eye's color temperature perception under sunlight, thus correcting the impact of the ambient color temperature on the image's color temperature. After processing by the AWB algorithm, the image's color temperature stored in the electronic device can be 6240K. Understandably, when viewing the image in a 2800K environment, the human eye, after color adaptation, can perceive the image's color temperature as 5400K.

[0083] Subsequently, if the user switches the environment in which they view the image from the shooting environment to the playback environment, the change in ambient color temperature will cause the human eye to undergo color adaptation again, resulting in a change in the color temperature of the image perceived by the user. As shown in Figure 3, when the color temperature of the environment in which the user views the image changes from 2800K to 5800K, for the same image, after the human eye undergoes color adaptation again, the color temperature of the image perceived by the user can be 4000K. It is understandable that the reason for this change in image color temperature is that the change in ambient light enhances the visual contrast of warm tones in the image, making the warm tones in the image more obvious to the human eye. That is, when viewing the image in the playback environment, the user will perceive the image as yellowish. It is understandable that in the above scheme, the inconsistency between the change in ambient color temperature and the change in the color temperature of the image perceived by the user leads to an unrealistic image perception.

[0084] Figure 4 is a schematic diagram illustrating another aspect of user perception of the image color temperature of the same image under different ambient color temperatures, as provided in an embodiment of this application. The electronic device shown in Figure 4 has the aforementioned function of directly adjusting the screen display color temperature based on the ambient color temperature.

[0085] As shown in Figure 4, if an electronic device can directly adjust the screen display color temperature based on the ambient color temperature, then in a 2800K shooting environment, the electronic device can adjust the screen display color temperature to match the color temperature of the shooting environment, making the screen display warmer, thus adjusting the image color temperature perceived by the user from 5400K to 4000K. Similarly, in a 5800K playback environment, the electronic device can adjust the screen display color temperature to match the color temperature of the playback environment, making the screen display cooler, thus adjusting the image color temperature perceived by the user from 4000K to 5400K. Specifically, the electronic device can detect the ambient color temperature when the user views the image, and then adjust the screen display color temperature based on the color temperature adjustment mapping relationship shown in Figure 5. This color temperature adjustment mapping relationship is used to demonstrate the correspondence between the ambient color temperature and the screen display color temperature, and this mapping relationship can be predetermined and set within the electronic device by the developers.

[0086] As can be understood, the solution shown in Figure 4, which directly adjusts the screen display color temperature based on the ambient color temperature, allows the color temperature of the image displayed in the viewing environment to match the color temperature of the image perceived by the human eye after color adaptation under the ambient color temperature when viewing the image in different environments. This makes the image look more realistic and improves the user experience.

[0087] While the above solution can adjust the screen's color temperature to ensure a more realistic image color temperature for users viewing images in different color temperature environments, it still has the following drawbacks. Firstly, the color temperature adjustment mapping relationship itself lacks precision. That is, although the electronic device can alleviate the problem of unrealistic image perception under different color temperature environments to some extent, it is still insufficient to meet the needs of users in the field of photography. Secondly, for different shooting content, such as white tablecloths and people shot in a 2800K environment, the degree of color adaptation by the human eye varies. The electronic device cannot determine the required screen display color temperature adjustment for different subjects under the same ambient color temperature using a fixed color temperature adjustment mapping relationship. In other words, in the above solution, different subjects will display the same color temperature on the electronic device when viewing images under the same ambient color temperature. Therefore, the electronic device cannot accurately simulate the differences in color perception of different subjects by the human eye, and thus cannot provide users with a realistic shooting experience for all subjects.

[0088] In view of this, embodiments of this application provide a method for color adaptation. Figure 6 is a simplified schematic diagram of a color adaptation process provided by an embodiment of this application.

[0089] As shown in Figure 6, the color adaptation process involves two parts: the image capture path and the display path. In the image capture path, the electronic device processes the input HVM image based on the human vision model (HVM), outputting an image file with the corresponding white point color temperature. In the display path, the electronic device performs linked color adaptation processing on the original image based on the ambient color temperature, the human vision model, and the white point color temperature, and displays the image after linked color adaptation. Thus, the electronic device ensures that when the color temperature of the viewing environment is the same as the shooting environment, the displayed image color matches the color perceived by the user in the shooting environment; when the color temperature of the viewing environment is different from the shooting environment, the displayed image color simulates the color perceived by the user's human eye after color adaptation to the greatest extent. That is, regardless of the color temperature of the viewing environment, the electronic device maintains the authenticity of the image color, thereby providing the user with the most realistic shooting and display experience. The shooting environment color temperature is also called the first ambient color temperature, and the viewing environment color temperature is also called the second ambient color temperature.

[0090] The above method can be applied to electronic devices such as mobile phones and tablets that have both camera and display capabilities. These devices simultaneously possess both camera and display functions. The camera module includes, but is not limited to, a camera module, an image signal processor (ISP), and the aforementioned HVM algorithm. The display device includes, but is not limited to, a display screen and a display controller.

[0091] The following section further elaborates on the processing flow of the color adaptation method described above, based on Figure 7. Specifically, the processing flow of the above method can be divided into two parts: the camera end and the display end, which will be described separately.

[0092] Photo terminal

[0093] At the camera end, the electronic device needs to capture an image under the ambient color temperature and generate and save the image as an image file containing metadata. The captured RAW image is also called the first raw image, and the saved image file is also called the first image file. This first image file includes the first image and the aforementioned metadata (also called first metadata). The first image is obtained by processing the first raw image using standard white balance. Specific steps include S101 to S104.

[0094] S101. Electronic devices can capture raw images (i.e., RAW images) via a camera.

[0095] The RAW image mentioned above refers to the raw data of the light source signal captured by the camera's image sensor and converted into a digital signal. It records the raw information of each pixel captured by the camera, including color, brightness, etc., and also records some shooting parameters, such as ISO, shutter speed, aperture value, white balance, etc. As shown in Figure 7, the electronic device can capture the above RAW image in a shooting environment of 3800K.

[0096] S102. The electronic device inputs the RAW image into the HVM model and outputs the white point color temperature corresponding to the RAW image.

[0097] The white point color temperature corresponding to the aforementioned RAW image can include both the actual white point color temperature in the shooting environment and the standard playback white point color temperature. Specifically, the actual white point color temperature in the shooting environment refers to the white point color temperature determined by the electronic device after adjusting the image based on a simulated human eye color adaptation process in the shooting environment; the standard playback white point color temperature refers to the white point color temperature of the standard playback real image. The standard playback real image refers to the image stored in the electronic device. It can be understood that an image taken using an electronic device that does not provide the color adaptation method in the embodiments of this application and ultimately stored in the album is the standard playback real image. It can be understood that the actual white point color temperature in the shooting environment is also called the expected white point color temperature when the first image is displayed under the first ambient color temperature, and the standard playback white point color temperature is also called the white point color temperature of the first image in the first image file. Therefore, compared to the white point color temperature of the first image in the first image file, the expected white point color temperature of the first image when displayed under the first ambient color temperature is closer to the white point color temperature of the first image perceived by the human eye after color adaptation under the first ambient color temperature.

[0098] Understandably, electronic devices can adjust the image color temperature by adjusting the adaptive white point color temperature. The adaptive white point color temperature refers to the color temperature of points or areas perceived as white in an image under specific lighting conditions within the CIE color space. Figure 8 is a diagram showing the correspondence between different adaptive white point color temperatures and image color temperature according to an embodiment of this application. The following explanation, using Figure 8, will more intuitively illustrate the use of the white point color temperature in image color temperature adjustment.

[0099] As is understandable, the four scenarios in Figure 8 actually present four different expected display effects. Electronic devices can adjust the image color temperature by adjusting the adaptive white point color temperature of the image, thereby presenting different display effects.

[0100] As shown in Figure 8, taking a low color temperature image shooting scenario as an example, that is, assuming the color temperature of the current shooting environment is 3800K. As shown in (1) of Figure 8, if the displayed image does not undergo any color adaptation adjustment, that is, the color temperature of the light source in the shooting environment is close to the color temperature of the light source in the image, for example, the color temperature of the image is also 3800K, the color temperature of the ambient light source observed by the user is almost completely consistent with the color temperature of the same light source displayed on the screen. For electronic devices, electronic devices do not perform color adaptation adjustment for the adaptive white point color temperature of the image. Since 6500K color temperature is widely regarded as the ideal white light perceived by the human eye, in this case, the adaptive white point color temperature of the image can be 6500K.

[0101] In another extreme case, as shown in Figure 8(4), the electronic device can use a gray card for white balance calibration and perform full color adaptation adjustment on the image based on the shooting environment. The electronic device can identify the color of the light source in the shooting environment as the ideal white light perceived by the human eye. Continuing with the above example, the electronic device can set the adaptive white point color temperature of the image to 3800K, so that the light source color temperature of the adjusted image is 6500K.

[0102] In real-world scenarios, users' perception of the color temperature of an image's light source typically falls between the extreme real-world conditions and the grayscale conditions described above. This can be further divided into real-world conditions and standard playback conditions.

[0103] As shown in Figure 8(2), in a shooting environment with a color temperature of 3800K, after the user's color adaptation to the environment, the visual perception of yellow will decrease because the color temperature perceived by the user's eyes after color adaptation at the shooting location will be higher than the actual color temperature of the environment. Electronic devices can appropriately increase the color temperature to restore the color temperature perception of the actual situation at the scene. For example, if the electronic device uses 4800K as the adaptation white point color temperature to adjust the image, the adjusted image color temperature can be 5400K. At this time, the color of the adjusted image is closer to the color temperature perceived by the human eye after color adaptation at the shooting location, thereby restoring the actual situation at the scene of the image.

[0104] As shown in Figure 8(3), when the user is no longer in a shooting environment with a color temperature of 3800K, for example, when the user views the image in other standard environments, the human eye no longer adapts to the environment and its sensitivity to yellow is normal. At this time, the electronic device can further increase the color temperature of the image to restore the human eye's perception of the color temperature of the actual situation, thereby restoring the standard playback reality of the image. For example, the electronic device adjusts the image with 4200K as the adaptation white point color temperature, and the adjusted image color temperature can be 6240K. It can be understood that the adjusted image under the standard playback reality can be regarded as the above-mentioned standard playback reality image.

[0105] Based on the above, it can be reasonably inferred that after obtaining the actual white point color temperature and the standard white point color temperature, the electronic device can identify the color deviation of the image under specific lighting conditions in the shooting environment (as shown in Figure 7, a color temperature of 3800K), thus providing a basis for correction in the subsequent restoration of the image's true colors. Specifically, the electronic device can adjust the image's color temperature by adjusting the image's adaptive white point color temperature, thereby maintaining the authenticity of the image's colors. It can be understood that during the adjustment of the image's adaptive white point color temperature, the electronic device can use the standard white point color temperature as the starting point for adjustment and the actual white point color temperature in the shooting environment as the ending point. Therefore, the electronic device can accurately restore the colors of the standard viewing image to the true colors in the shooting environment, that is, restore the colors perceived by the human eye after color adaptation in the shooting environment.

[0106] Electronic devices can obtain the white point color temperature corresponding to each RAW image through the HVM model described above, including the actual white point color temperature in the field and the standard playback white point color temperature. The HVM model described above can be a mature model that has been pre-trained based on real-world human factors experiments.

[0107] Understandably, the human visual system has a relatively fixed color memory for objects such as skin color, blue sky, and green plants. These factors can also affect the human eye's color perception of an image, thus affecting the user's perception of whether the image is realistic. Furthermore, at the same color temperature, the perceived white point color temperature of the image after color adaptation varies considerably for different subjects.

[0108] Figure 9 is a schematic diagram comparing the color temperature difference between the actual white point and the standard playback white point of different image content within the same color temperature range, according to an embodiment of this application. In Figure 9(a), the image content is a white tabletop; in Figure 9(b), the image content is a tiled floor; in Figure 9(c), the image content is red flowers on a white cloth; and in Figure 9(d), the image content is a white paper bag on a light-colored table and a light-colored wall. As shown in Figure 9, within the color temperature range around 2700K, the difference between the color temperature of the actual white point and the standard playback white point varies significantly for different subjects. In other words, restoring the image color to the actual white point based on the standard playback white point has a certain specificity for the subject.

[0109] Therefore, electronic devices cannot directly obtain the actual white point color temperature based on the aforementioned RAW image and ambient color temperature. Thus, in this embodiment, researchers can construct the aforementioned HVM model through extensive training with real human factors experiments, and then use the HVM model to output the white point color temperature corresponding to the RAW image.

[0110] For example, the above-mentioned real human factor experiment can be shown in Figure 10. Researchers can set the shooting environment color temperature to 7000K in Figure 10(a) and the standard playback environment color temperature to 6500K in Figure 10(b). Then, based on the human eye perception of the experimenters, the adaptive white point color temperature of the image is adjusted to achieve the following debugging goal: In the above shooting environment and standard playback environment, the adjusted image color can simulate the color perceived by the human eye after color adaptation in the corresponding environment.

[0111] Not limited to RAW images, data from all cameras and their internal sensors, such as multispectral data, can also be used as input to the HVM model along with the RAW images. This application does not impose any special restrictions on the specific data used to assist the output of the HVM model other than RAW images.

[0112] S103. The electronic device generates a standard review of the real image based on the RAW image.

[0113] After capturing a RAW image, through a series of image processing steps, including but not limited to AWB algorithm, color correction, exposure adjustment, etc., the electronic device can generate the above-mentioned standard image for viewing the real image.

[0114] Understandably, in step S102 above, the HVM can also output other parameters (such as sensitivity) required in the image processing steps. Optionally, after capturing the RAW image, the electronic device can adjust the RAW image based on the other parameters required in the image processing steps output by the HVM to obtain the standard viewable real image. For example, the HVM outputs the standard viewable white point color temperature, and based on the standard viewable white point color temperature output by the HVM, standard white balance processing is performed on the RAW image to generate the first image.

[0115] Understandably, the above-mentioned standard for viewing real images refers to the images that users store on their devices after taking the photos, as well as the images that users transmit to other users when sharing them.

[0116] Understandably, since the aforementioned standard playback image can be transmitted by the user to other users for viewing in different environments, taking a RAW image captured by an electronic device in a 3800K shooting environment as an example, the color temperature of the aforementioned standard playback image will be higher than the actual image color temperature in the shooting environment. In other words, if the color temperature of the saved and transmitted image is the same as the image color temperature perceived by the user after color adaptation in the shooting environment, then for other users viewing the image in other environments, the image will appear too yellow, resulting in a significant difference in visual experience compared to the user's playback environment (such as the D65 standard environment).

[0117] S104. The electronic device writes the white point color temperature and the shooting environment color temperature into the above standard, reviews the real image, obtains the updated image, and outputs it.

[0118] The aforementioned actual white point color temperature, standard playback white point color temperature, and shooting environment color temperature can be written as metadata into the standard playback real image to obtain an updated image, which is then output to the display. This updated image can also be referred to as Image B. It is understood that outputting the updated image to the display actually means that the electronic device saves the updated image to a first memory, allowing the display to access that first memory to obtain the updated image (i.e., Image B). It should be noted that the term "updated image" is only used to distinguish between the images before and after writing the white point color temperature; it does not imply that the camera first outputs an image without a white point color temperature in its header file, and then updates it using the method provided in this application embodiment. In other words, if the electronic device uses the method provided in this application embodiment, the header file of all images output by the camera should contain the aforementioned white point color temperature.

[0119] Optionally, the electronic device can write the aforementioned white point color temperature (including the standard playback real white point color temperature and the actual scene white point color temperature) and the aforementioned shooting environment color temperature into the aforementioned standard playback real photo file via the image header file (such as an ICC file) to obtain an updated image. It should be noted that for the camera end that has not performed the aforementioned writing operation, the header file of the standard playback real image output by the electronic device will not contain the aforementioned standard playback real white point color temperature and the aforementioned actual scene white point color temperature.

[0120] Understandably, throughout the entire image processing flow at the camera end, the content and tone of the image ultimately used by the electronic device for storage and transmission (i.e., the aforementioned standard playback real image) remain unchanged. This is because the aforementioned standard playback real image is obtained by processing the RAW image captured by the camera, and the color adaptation method provided in this application embodiment does not improve how the electronic device specifically processes the RAW image to obtain the aforementioned standard playback real image.

[0121] Display end

[0122] On the display end, the electronic device can generate and display a color-adapted image (such as the first image) based on the ambient color temperature. The white point color temperature of the color-adapted first image differs from the white point color temperature of the first image in the first image file. When the ambient color temperature is higher than the shooting ambient color temperature, the color temperature of the color-adapted first image can be higher than the color temperature of the first image in the first image file. Specifically, the steps performed by the electronic device on the display end include steps S105 to S107.

[0123] S105. Electronic equipment detection and review of environmental information.

[0124] The aforementioned viewing environment information includes the color temperature of the viewing environment. Specifically, the electronic device can detect the color temperature of the viewing environment using a color temperature sensor. This color temperature sensor can be embedded in the display screen of the electronic device, such as a front-facing color temperature sensor. Optionally, the color temperature sensor can be an RGB sensor that includes a color temperature detection algorithm. This application is not limited to the specific type of color temperature sensor described above.

[0125] In some embodiments, the color temperature of the viewing environment described above is not limited to the color temperature of the viewing environment; the viewing environment information may also include the brightness of the viewing environment. The viewing environment information may also include more data, not limited to the above indicators.

[0126] S106. The electronic device can adjust the basic color adaptation mapping curve based on the updated image to obtain the first color adaptation mapping curve.

[0127] Understandably, a color adaptation mapping curve can be used to illustrate the correspondence between the ambient color temperature and the target adaptation white point. For example, in the color adaptation mapping curve, the first ambient color temperature corresponds to the first target adaptation white point. Therefore, when an electronic device adjusts the white point of an image to the aforementioned first target adaptation white point, the color temperature of the adjusted image is consistent with the color temperature perceived by the human eye after color adaptation under that first ambient color temperature. Thus, it can be seen that through the aforementioned color adaptation mapping curve, an electronic device can determine the target adaptation white point of an image based on the ambient color temperature, and then adjust the image based on this target adaptation white point to obtain a more realistic display effect.

[0128] The aforementioned basic color adaptation mapping curve is a general type of color adaptation mapping curve. In some embodiments, a basic color adaptation mapping curve may be pre-set in the display of an electronic device. Optionally, the aforementioned basic color adaptation mapping curve may be obtained based on the aforementioned display real human factors experiment, and the aforementioned basic color adaptation mapping curve obtained based on the display real human factors experiment is also called a human factors color adaptation mapping curve. For example, Figure 11 is a graph of a basic color adaptation mapping relationship provided by an embodiment of this application.

[0129] In some embodiments, the display of the electronic device may have multiple basic color adaptation mapping curves preset, from which the electronic device can select one for adjustment to obtain the aforementioned first color adaptation mapping curve. Optionally, if the viewing environment information obtained by the electronic device in step S105 includes the brightness of the viewing environment, the electronic device can determine which basic color adaptation mapping curve to adjust based on the brightness of the viewing environment. For example, if the electronic device has two preset basic color adaptation mapping curves, curve a and curve b, the electronic device can use curve a to perform subsequent steps when the brightness of the viewing environment is less than a first brightness threshold, and use curve b to perform subsequent steps when the brightness of the viewing environment is greater than or equal to the first brightness threshold. It is understood that since the human eye's perception of color changes with the ambient brightness, the applicable basic color adaptation mapping curves may be different for different viewing environment brightness levels. If the electronic device can select a suitable basic color adaptation mapping curve for adjustment based on the brightness of the viewing environment, the electronic device can obtain a more accurate target white point, thereby enabling more accurate color adaptation adjustment and obtaining a more realistic color experience.

[0130] The aforementioned adjustment of the base color adaptation mapping curve based on the updated image specifically refers to the electronic device adjusting the base color adaptation mapping curve based on the shooting environment color temperature and white point color temperature (standard playback real white point color temperature and on-site real white point color temperature) parsed from the header file of the updated image.

[0131] After performing the above steps, the display of the electronic device can first receive the updated image (i.e., image B) output by the camera. Specifically, the display of the electronic device can read the updated image from the first memory. After reading image B, the electronic device can parse the white point color temperature and the shooting environment color temperature from the header file of image B, including the standard playback real white point color temperature and the actual white point color temperature on site.

[0132] Subsequently, the display of the electronic device can adjust the basic color adaptation mapping curve based on the aforementioned white point color temperature to obtain the color adaptation mapping curve of a specific image (here, image B). The color adaptation mapping curve of image B is also called the first color adaptation mapping curve. That is, the white point color temperature corresponding to the aforementioned shooting environment color temperature on the basic color adaptation mapping curve is different from the aforementioned actual white point color temperature at the scene, while the white point color temperature corresponding to the aforementioned shooting environment color temperature on the first color adaptation mapping curve is the aforementioned actual white point color temperature at the scene.

[0133] Figure 12 is a schematic diagram of a basic color adaptation mapping curve based on white point color temperature adjustment provided in an embodiment of this application.

[0134] As shown in Figure 12, the solid line represents the basic color adaptation mapping curve, while the dashed line represents the first color adaptation mapping curve. Specifically, the electronic device can determine the actual location based on the white point color temperature, and then adjust the basic color adaptation mapping curve based on the actual location.

[0135] The aforementioned "realistic point" refers to the target adaptive white point color temperature corresponding to the color temperature of the shooting environment for image B. In other words, when the ambient color temperature during review equals the shooting environment color temperature, the electronic device can adjust the image's adaptive white point to the aforementioned realistic point. This allows the electronic device to accurately restore the image's colors to the true colors of the shooting environment, ensuring that the colors perceived by the user when reviewing the image at that color temperature are consistent with the colors perceived by the human eye after color adaptation in the shooting environment. Specifically, the electronic device can use the aforementioned shooting environment color temperature as the x-axis corresponding to the aforementioned realistic point, and the aforementioned realistic white point color temperature as the y-axis corresponding to the aforementioned realistic point. The aforementioned shooting environment color temperature can be obtained by the electronic device from the header file of image B. It is understandable that the realistic point corresponding to different images B may be different.

[0136] After determining the aforementioned real-world location, the electronic device can adjust the aforementioned basic color adaptation mapping curve based on this real-world location to obtain the aforementioned first color adaptation mapping curve. In some embodiments, the electronic device can adjust the aforementioned basic color adaptation mapping curve using interpolation. Optionally, the electronic device can adjust the aforementioned basic color adaptation mapping curve using linear interpolation. For example, the electronic device can determine a first adjusted color temperature and a second adjusted color temperature before and after the ambient color temperature (e.g., 3800K as shown in Figure 12) corresponding to the real-world location on the aforementioned basic color adaptation mapping curve. The white point color temperature (also called the first data point) corresponding to the first adjusted color temperature on the aforementioned basic color adaptation mapping curve is the same as the white point color temperature corresponding to the first adjusted color temperature on the aforementioned first color adaptation mapping curve, and the white point color temperature (also called the second data point) corresponding to the second adjusted color temperature on the aforementioned basic color adaptation mapping curve is the same as the white point color temperature corresponding to the second adjusted color temperature on the aforementioned first color adaptation mapping curve. The first ambient color temperature is located between the first adjusted color temperature and the second adjusted color temperature. Understandably, the deviation between the first adjusted color temperature and the ambient color temperature of the actual point in the scene is called the first color temperature deviation, and the deviation between the second adjusted color temperature and the ambient color temperature of the actual point in the scene is called the second color temperature deviation. Optionally, the first color temperature deviation and the second color temperature deviation can be the same. Within the range of the first adjusted color temperature to the second adjusted color temperature, the white point color temperature corresponding to each ambient color temperature on the first color adaptation mapping curve can be obtained by interpolating the expected white point color temperature of the first image displayed at the first ambient color temperature with the white point color temperature corresponding to the first adjusted color temperature on the first color adaptation mapping curve, or the expected white point color temperature of the first image displayed at the first ambient color temperature with the white point color temperature corresponding to the second adjusted color temperature on the first color adaptation mapping curve. Not limited to the above linear interpolation implementation method, this application embodiment does not impose special restrictions on the specific interpolation method used by the electronic device.

[0137] Understandably, the real-world points determined by the above method must lie on the aforementioned first color adaptation mapping curve. Furthermore, because the first color adaptation mapping curve specifically considers the real-world points of image B, it has higher precision than the basic color adaptation mapping curve, thus more accurately simulating the color adaptation of the human eye under specific lighting conditions. This targeted adjustment not only improves the color realism of the image but also optimizes the user's visual experience.

[0138] S107. The electronic device can adjust the image display based on viewing environment information and the first color adaptation mapping curve.

[0139] After obtaining the first color adaptation mapping curve, the electronic device can use this curve to determine the target adaptation white point of the image corresponding to the color temperature of the current user's viewing environment. Then, based on this target adaptation white point, the image undergoes color adaptation conversion, and the color-adapted image is displayed. The color-adapted image is also called image C.

[0140] Figure 13 is a schematic diagram of a color adaptation conversion process provided in an embodiment of this application.

[0141] In Figure 13, XYZ represents the CIE XYZ color space, LMS represents the LMS color space, and P3 represents the DCI-P3 or Display P3 color space. In Figure 13, l_rgb can represent standard real-world image data processed by the AWB algorithm, l_xyz can represent the value in the CIE XYZ color space obtained after the l_RAW conversion, and l_lms can represent the value in the LMS color space obtained after the l_xyz conversion. The diagonal matrix gainMat can be considered a transformation matrix used to transform l_lms to l_lms'. It can be understood that the values ​​on the diagonal of the above diagonal matrix are used to indicate the scaling degree of the human eye's response to long wavelengths, medium wavelengths, and short wavelengths in the screen data. The above l_xys' can be used to represent the value in the CIE XYZ color space obtained after the above l_lms' conversion, and the above l_p3 can be used to represent the RGB value in the DCI-P3 or Display P3 color space obtained after the above l_xys' conversion.

[0142] As shown in Figure 13, the color adaptation conversion process at the display end of an electronic device can be understood as the electronic device sequentially converting the standard playback real image to the CIE XYZ color space, then the LMS color space, adjusting it in the LMS color space, and finally converting it through the XYZ color space to obtain the final RGB image in the DCI-P3 or Display P3 color space that can be displayed on the electronic device's display end. Optionally, the electronic device can use the color conversion function CAM2STD to convert the aforementioned standard playback real image from the RGB color space to the CIE XYZ color space. Optionally, the electronic device can use the color conversion function CAT02 to convert the image from the CIE XYZ color space to the LMS color space. Correspondingly, the electronic device can use the inverse process of the above color conversion function invCAT02 to convert the image from the LMS color space to the CIE XYZ color space. Optionally, the electronic device can use the color conversion function XYZ2P3 to convert the image from the CIE XYZ color space to an RGB image in the DCI-P3 or Display P3 color space. The above color conversion functions can be regarded as transformation matrices used to perform color space transformation. Therefore, the above color adaptation conversion process can be expressed by the following formula: XYZ2P3×invCAT02×gainMat×CAT02×CAM2STD×l_rgb=l_p3

[0143] As is understandable, in the above formulas, color conversion functions such as CAM2STD, CAT02, invCAT02, and XYZ2P3 are all general-purpose color conversion functions. Therefore, the core parameter for performing the above color adaptation conversion is the diagonal matrix gainMat, which directly determines the specific range of changes in the image in the color space and the final effect of the change. That is, gainMat scales the color data through the values ​​on its diagonal, thereby achieving adaptive color adjustment.

[0144] As shown in Figure 13, the diagonal matrix gainMat is calculated based on srcXYZ and dstXYZ. srcXYZ represents the standard playback true white point color temperature in the XYZ color space, and dstXYZ represents the target adaptive white point value in the XYZ color space corresponding to the current playback ambient color temperature. The target adaptive white point corresponding to the current playback ambient color temperature is determined based on the first color adaptation mapping curve. That is, dstXYZ is determined based on the playback ambient color temperature and the first color adaptation mapping curve in the playback environment information. LMS0 represents the value of dstXYZ converted from the XYZ color space to the LMS color space, and LMS1 represents the value of srcXYZ converted from the XYZ color space to the LMS color space. The diagonal matrix gainMat can be calculated from LMS0 and LMS1. Optionally, gainMat can be obtained by dividing LMS0 by LMS1.

[0145] Figure 14 is a schematic diagram of an exemplary display end color adaptation process provided in an embodiment of this application.

[0146] Continuing with the example above, assuming the ambient color temperature during shooting is 3800K, and the ambient color temperature during playback is also 3800K, the electronic device, after acquiring image B, can parse the standard playback true white point color temperature (4200K) and the actual ambient white point color temperature (4800K) from the header file of image B. First, the electronic device can convert these white point color temperatures into CIE XYZ color space values ​​in (x, y, z) format. The standard playback true white point color temperature in CIE XYZ color space is srcXYZ = (XwA, YwA, ZwA). The actual ambient white point color temperature in CIE XYZ color space is dstXYZ = (XwB, YwB, ZwB). Subsequently, the electronic device can convert the above srcXYZ to LMS-c1 = (LA, MA, SA), and the above dstXYZ to LMS-c2 = (LB, MB, SB), and then calculate the above diagonal matrix gainMat based on the following formula:

[0147] Correspondingly, in Figure 14, LMS-d2=(L out M out S out LMS-d1 = (L in M in S in It is calculated using the following formula:

[0148] Therefore, as shown in Figure 14, before color adaptation conversion, the screen data of the electronic device displaying the image can be denoted as RGB1. It is understandable that RGB1 can be obtained by color space conversion of the image B. In Figure 14, the portion highlighted by the dashed line can be considered a large transformation matrix, called the color adaptation matrix. This color adaptation matrix can be composed of a color space conversion matrix and a diagonal matrix gainMat. The color space conversion matrix can include conversion matrices from RGB color space to XYZ color space, XYZ color space to LMS color space, LMS color space to XYZ color space, and XYZ color space to RGB color space.

[0149] As shown in Figure 14, the aforementioned color adaptation matrix is ​​used to transform each pixel on the screen from RGB1 to the final displayed RGB2. Specifically, the electronic device can convert the RGB1 to XYZ-d1 based on the color adaptation matrix, and then to LMS-d1. The calculated diagonal matrix gainMat is then used to adjust the LMS-d1 to obtain LMS-d2. Subsequently, the electronic device can convert the LMS-d2 back to XYZ-d2 in the XYZ color space, and then obtain the corresponding RGB2 based on XYZ-d2.

[0150] As is understandable, Figure 14 illustrates a special case where the color temperature of the current viewing environment is the same as the color temperature of the shooting environment (both are 3800K). Therefore, in Figure 14, dstXYZ can be determined based on the actual white point color temperature in the scene. In fact, the above dstXYZ is determined based on the target adaptive white point, and in the above special case, the target adaptive white point is exactly equal to the actual white point color temperature in the scene. Therefore, in reality, before executing the color adaptation processing flow shown in Figure 14, the electronic device needs to determine the target adaptive white point based on the viewing environment color temperature detected in step S105 and the first color adaptation mapping curve determined in step S106. Only then can the electronic device determine the above dstXYZ based on the target adaptive white point, and then determine the above diagonal matrix and the above color adaptation matrix.

[0151] Therefore, in some embodiments, after step S105, the electronic device can first determine whether the color temperature of the current viewing environment is the same as the color temperature of the shooting environment based on the detected viewing environment information. If the color temperature of the viewing environment is the same as the color temperature of the shooting environment, the electronic device can skip calculating the first color adaptation mapping curve mentioned above, and instead, after determining that the two environment color temperatures are the same, directly use the white point color temperature parsed from the header file of image B to perform the color adaptation processing flow shown in Figure 14.

[0152] In some embodiments, when the electronic device does not need to determine the basic color adaptation mapping curve that needs to be adjusted based on the above-mentioned review environment information, for example, when the electronic device contains only one basic color adaptation mapping curve, the execution order of steps S105-S106 is not particularly limited.

[0153] When displaying images using the above methods, there is a certain correlation between the color temperature of the color-adapted image and the white point color temperature output by HVM.

[0154] Suppose that an electronic device acquires a second original image at a third ambient color temperature. This second original image has the same image content as the first original image mentioned above. A second image file is generated and saved based on the second original image. The second image file includes a second image and first metadata. The second image is obtained by processing the second original image with standard white balance. The second metadata includes the third ambient color temperature, the expected white point color temperature of the second image when displayed at the third ambient color temperature, and the white point color temperature of the second image in the second image file. Subsequently, the electronic device can generate and display the second image after color adaptation adjustment based on a fourth ambient color temperature.

[0155] Understandably, if the deviation between the expected white point color temperature of the second image displayed at the third ambient color temperature and the white point color temperature of the second image in the second image file is greater than the deviation between the expected white point color temperature of the first image displayed at the first ambient color temperature and the white point color temperature of the first image in the first image file, then the deviation between the white point color temperature of the second image after color adaptation adjustment and the white point color temperature of the second image in the second image file is greater than the deviation between the white point color temperature of the first image after color adaptation adjustment and the white point color temperature of the first image in the first image file.

[0156] Based on the above method, the electronic device can perform color adaptation adjustment on an image at the display end, thereby ensuring that the electronic device can maintain the authenticity of the image colors regardless of the user's viewing environment at any color temperature, thus providing the user with the most realistic shooting and display experience. Not limited to displaying a single image at the display end, in some embodiments, the electronic device can display two images at the display end. For example, as shown in Figure 15, the electronic device can simultaneously display Image 1 and Image 2 on a white background. The white background can be considered Image 0, also called the background image. It is understood that in the case shown in Figure 15, Image 1, Image 2, and Image 0 can be displayed on different layers.

[0157] Optionally, in the above situation, the electronic device can calculate the color adaptation matrix corresponding to the image in each of the above layers, and then perform color adaptation transformation on each layer based on the color adaptation matrix. Subsequently, the electronic device can merge the multiple layers that have undergone color adaptation processing to display the final merged image.

[0158] In some embodiments, the color adaptation method described above can also display the final fused image based on the color adaptation processing flow for multi-image display shown in Figure 16, following the process shown in Figure 14. The color adaptation processing flow described above will be illustrated below using the case shown in Figure 16, where the electronic device needs to display two images simultaneously.

[0159] When the display of an electronic device detects that the image display area contains multiple images, for example, after a user imports multiple small images into a large image using image processing software, the electronic device can execute the processing flow shown in Figure 16.

[0160] Specifically, the electronic device can first perform preprocessing on the image using a display hardware module. This module compares the color adaptation matrix corresponding to each image with a standard color adaptation matrix, calculating the difference between each image and the standard, also known as the residual. This residual can be considered an image feature, representing the deviation between the image and the standard. Optionally, the display hardware module may include a graphics processing unit (GPU). In Figure 16, gamma refers to gamma correction. After preprocessing, the electronic device can receive the residual information calculated by the display hardware module and perform gamma correction on the images in each layer based on these features, making the images more consistent with human visual perception. Subsequently, the electronic device can perform post-processing on the image using gamut mapping (GMP). This post-processing includes fusing the layers and transforming the fused image (i.e., the fused image) using the standard color adaptation matrix, ensuring accurate color display for each image when displaying multiple images.

[0161] For example, taking Figure 16 as an example, the color adaptation matrix corresponding to image 1 can be denoted as CATMAT1, the color adaptation matrix corresponding to image 2 can be denoted as CATMAT2, and the above-mentioned standard color adaptation matrix can be denoted as CATMAT0. It can be understood that the above-mentioned standard color adaptation matrix is ​​the color adaptation matrix of image 0.

[0162] In the above method, the electronic device can determine CATMat1 and CATMat2 based on the process shown in Figure 14. It is understood that the key to determining CATMat1 and / or CATMat2 lies in determining the diagonal matrix in the process shown in Figure 14, and the key to determining the diagonal matrix lies in finding the accurate dstXYZ, that is, the key is that the electronic device can find the accurate target adaptive white point under the current viewing environment color temperature through the first color adaptation mapping curve. Specific steps are described above and will not be elaborated here. It is understood that since the user's perception of the color temperature of image 0 is also affected by the viewing environment color temperature, although CATMat0 is a standard color adaptation matrix, it is not a fixed color adaptation matrix within the electronic device; instead, the standard color adaptation matrix (i.e., CATMat0) should be a color adaptation matrix that changes with the viewing environment color temperature, just like CATMat1 and CATMat2. Similar to CATMat1 and CATMat2 mentioned above, determining CATMat0 hinges on the electronic device's ability to accurately determine the target white point of image 0 under the current ambient color temperature. Optionally, the electronic device can determine the accurate target white point of image 0 under the current ambient color temperature based on the base color adaptation mapping curve, thereby determining CATMat0.

[0163] Specifically, after calculating CATMat0 to CATMat2, the electronic device can call the display hardware module (such as a GPU) to perform residual calculations on image 1 and image 0, and image 2 and image 0, respectively. Subsequently, the electronic device can perform gamma correction on the preprocessed images 1 and 2 to adjust the brightness and contrast of the images, making the corrected images conform to the non-linear perception characteristics of light by the human eye. Afterward, the electronic device can fuse the layers containing images 0 to 2, and use CATMat0 to perform another color adaptation change on the fused image. In some embodiments, the electronic device can use GMP technology to implement the above-mentioned processing of performing another color adaptation change on the fused image using CATMat0. Optionally, the above GMP can be implemented using a 3D color lookup table (LUT). Not limited to the above method, other technologies capable of implementing the above post-processing can also be used. This application embodiment does not impose special limitations on how the electronic device specifically implements the above post-processing steps.

[0164] Understandably, in the above process, the preprocessing steps are used to save the characteristics in the color adaptation matrices of each image, and the postprocessing steps are used to supplement the commonalities in the color adaptation matrices of each image. Thus, after the preprocessing and postprocessing steps, the electronic device can accurately display the colors of multiple images displayed simultaneously. Furthermore, in actual production and application, the processing flow shown in Figure 16 is more convenient to implement in hardware, which is beneficial for the applicability of the method.

[0165] It is understood that Figure 16 above is only used as an example to illustrate the color adaptation method described above, assuming that the electronic device needs to display two images simultaneously. The method is not limited to displaying two images; the electronic device can also perform the combined image capture and display color adaptation method provided in this application embodiment while displaying more images simultaneously. This application embodiment does not impose any special limitations on this.

[0166] As described above, the color adaptation method provided in this application is a combined shooting and display color adaptation method. As shown in Figure 17, using the above color adaptation method, an image captured in an environment with a color temperature of 3721K can be perceived by the user to have a color temperature of 4000K. At this time, the image color temperature after color adaptation processing using the color temperature adjustment mapping relationship in the prior art is 3750K, the image color temperature after color adaptation processing based on the human color adaptation mapping curve is 4100K, and the image color temperature obtained after processing based on the aforementioned first color adaptation mapping curve is 3980K. Therefore, by performing color adaptation processing on the image based on the first color adaptation mapping curve generated in the color adaptation method provided in this application, the electronic device can maximize the accuracy of the adjusted displayed color to the color perceived by the user's eye after color adaptation in the ambient color temperature of the image, thereby providing the user with a more realistic shooting and viewing experience.

[0167] It is understandable that after implementing the color adaptation method provided in this application embodiment, under the same viewing ambient color temperature, the electronic device performs different color adaptation processing on different image contents captured under the same shooting ambient color temperature. Specifically, the electronic device can capture a third image under the first ambient color temperature, and the image content of the third image is different from that of the first image. Subsequently, the electronic device can save the third image as a third image file and display the third image under the second ambient color temperature. The third image file includes the third image and second metadata, which includes the first ambient color temperature, the expected white point color temperature when the third image is displayed under the first ambient color temperature, and the white point color temperature of the third image in the third image file.

[0168] At this time, the white point color temperature of the first image and the third image is different when they are displayed, that is, from the user's point of view, the color temperature of the first image and the third image is different.

[0169] Understandably, in the above scheme, the electronic device's ability to perform color adaptation at the display end is based on the fact that the electronic device writes the white point color temperature into the ICC header file at the camera end, and this ICC header file is embedded in the image file or packaged and stored and transmitted with the image file. Therefore, if the electronic device cannot parse the white point color temperature from the image file at the display end, the electronic device cannot execute the above scheme.

[0170] Understandably, the aforementioned ICC file is an editable file. Developers can remove the white point color temperature from the ICC file through editing. This means that the electronic device cannot parse the white point color temperature from the image file at the display end. Consequently, for the first and third images, after the electronic device removes the white point color temperature through editing, the white point color temperature is the same when the first and third images are displayed under the same ambient color temperature. In other words, the user will perceive that the first and third images have the same image color temperature when displayed under the same ambient color temperature.

[0171] Therefore, in some embodiments, if the electronic device fails to parse the white point color temperature from the aforementioned image file at the display end, as shown in Figure 18, the electronic device can directly perform the aforementioned color adaptation process based on the aforementioned basic color adaptation mapping curve. The case where the image file does not contain the white point color temperature can also be referred to as a shot-and-display independent method. In some implementations, a basic color adaptation mapping curve can be preset in the electronic device.

[0172] In some implementations, the electronic device can pre-set multiple different base color adaptation mapping curves. Optionally, these multiple base color adaptation mapping curves can be differentiated according to the scene. For example, the electronic device can identify the content of the photograph based on the photograph file. If the photograph is detected to be a portrait image, the electronic device can select base color adaptation mapping curve A to perform the color adaptation process. It is understood that compared to a general base color adaptation mapping curve, the image obtained by the electronic device using base color adaptation mapping curve A has a cooler overall color temperature, thus making the image displayed after color adaptation more in line with the user's preference for portrait photography. Optionally, these multiple base color adaptation mapping curves can also be differentiated according to the brightness of the shooting environment. Specifically, the electronic device can detect the brightness of the environment displaying the first image through a brightness sensor, and then select one from multiple base color adaptation mapping curves as the base color adaptation mapping curve for determining the white point color temperature of the first image when displayed under the ambient color temperature. Here, the multiple base color adaptation mapping curves are base color adaptation mapping curves corresponding to different ambient brightness. For example, when the ambient brightness is less than a first illumination threshold, the electronic device can use the basic color adaptation mapping curve B to perform the aforementioned color adaptation process. It is understandable that when the ambient brightness is less than the first illumination threshold (i.e., under low-light conditions), the human eye's ability to perceive color decreases. Therefore, the electronic device does not need to adjust the color adaptation of the image too drastically. This is because in low-light environments, the human eye focuses more on visual clarity and comfort than on accurate color reproduction. Therefore, color adaptation adjustment may not be a priority, and the electronic device can focus more on adjusting brightness and contrast to ensure image clarity. Compared to the commonly used basic color adaptation mapping curve, the overall color temperature adjustment of the image obtained by the electronic device using the aforementioned basic color adaptation mapping curve B is smaller, thereby reducing the computational load on the electronic device.

[0173] It is understood that in the above description, the electronic device has both a camera and a display. However, in some embodiments, the electronic device may only have the aforementioned camera or display, and may only execute the method corresponding to the camera or display in the above-described combined shooting and display color adaptation method. For example, for some laptops without cameras, as long as the photo file received by the laptop contains the aforementioned white point color temperature, the laptop can still perform color adaptation display based on the method of the display. In other words, when the electronic device at the camera end can write the aforementioned white point color temperature into the image file, the electronic device at the display end can read the white point color temperature in the image file and perform the above-described color adaptation display. The above-described combined shooting and display color adaptation method does not require the camera and display to be located on the same electronic device. For example, user A's electronic device can capture a photo and add the white point color temperature to the header of the photo file. Subsequently, user A can share the photo file with user B. User B's electronic device can parse the aforementioned white point color temperature after receiving the photo and perform color adaptation display based on the aforementioned white point color temperature. Here, the electronic device with the camera is also called the first electronic device, and the electronic device with the display is also called the second electronic device.

[0174] If the first electronic device only includes a camera, then after saving the first image as a first image file, the first electronic device also needs to send the first image file to the second electronic device. Similarly, if the second electronic device only includes a display screen, then the process of the second electronic device performing the color adaptation method is as follows: receiving the first image file from the first electronic device and displaying the first image at a second ambient color temperature. For specific method details, please refer to the color adaptation method for electronic devices that simultaneously have a camera and a display screen; it will not be elaborated here.

[0175] Figure 19 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Here, the electronic device has both a camera and a display.

[0176] As shown in Figure 19, the electronic device may include components such as a processor 211, a memory 212, a wireless communication processing module 213, a power switch 214, a display screen 215, an audio module 216, a speaker 217, and a camera 218. The components in the electronic device are connected to each other via a bus and communicate based on the bus.

[0177] Processor 211 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0178] Memory 212 is coupled to processor 211 and is used to store various software programs and / or multiple sets of instructions. Memory 212 can be used to store computer executable program code, which includes instructions. Processor 211 executes various functional applications and data processing of the electronic device by running the instructions stored in memory 212. Memory may also be provided in processor 211 for storing instructions and data.

[0179] The memory 212 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM). The RAM can be directly read and written by the processor 211. The RAM can be used to store executable programs (e.g., machine instructions) of the operating system or other running programs, as well as user and application data. The NVM can also store executable programs and user and application data. Executable programs, i.e., user data, stored in the NVM can be pre-loaded into the RAM for direct reading and writing by the processor 211.

[0180] The executable program code and data (including but not limited to various voice data) used to implement the method of combined color adaptation for photography and display provided in the embodiments of this application can be stored in non-volatile memory. In the process of implementing the above-described method of combined color adaptation for photography and display, the electronic device can load the executable program code and user data from the non-volatile memory into the random access memory, thereby maximally simulating the color perceived by the user's human eye after color adaptation, and thus providing the user with the most realistic shooting and display experience.

[0181] The wireless communication processing module 213 can provide wireless communication solutions including WLAN, such as Wi-Fi, Bluetooth communication, ZigBee communication, NFC communication, infrared communication, and UWB communication.

[0182] The power switch 214 can be used to control the power supply to electronic devices, thereby supplying power to the processor 211, memory 212, wireless communication processing module 213, display screen 215, audio module 216, speaker 217, camera 218, etc.

[0183] Display screen 215 can be used to display images captured by camera 218. Display screen 215 includes a display panel. A touch sensor can be installed in display screen 215. The touch sensor is used to detect touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Furthermore, the electronic device can provide visual output related to the touch operation through display screen 215. The electronic device can implement display functions through GPU, display screen 215, touch sensor, and application processor, etc. In this embodiment, the electronic device can use the display functions provided by GPU, display screen 215, and application processor, etc., to display a user interface of a program that allows viewing images, and then display an image after combined color adaptation processing of capture and display.

[0184] Audio module 216 can be used to convert digital audio signals into analog audio signals for output, and can also be used to convert analog audio input into digital audio signals. Speaker 217 can be used to convert the audio signals transmitted by audio module 216 into sound signals. Electronic devices can implement audio playback functions through audio module 216, speaker 217, etc. In some embodiments, audio module 216 may also include a microphone for converting sound signals into electrical signals.

[0185] Camera 218 may include a lens, a photosensor, and a flexible printed circuit board (FPCB). The FPCB is responsible for connecting other components of camera 218 to processor 211, such as transmitting raw data output from the photosensor to processor 211. During image capture, the shutter of camera 218 is opened, allowing light to enter and illuminate the photosensor. The photosensor converts the light signal into an electrical signal, which is then further converted into a digital signal via an analog-to-digital converter (ADC) for transmission to the ISP for processing. The ISP can perform the following processing on the photosensor's output data: automatic exposure control (AEC), automatic gain control (AGC), automatic white balance (AWB), color correction, dead pixel removal, etc. The ISP can also be integrated within camera 218. In this embodiment, camera 218 can be used to acquire RAW images and perform AWB processing on the RAW images. The aforementioned RAW image can also be used as input to the HVM large model, thereby enabling the electronic device to obtain the white point color temperature corresponding to the RAW image. The camera 218 can write the aforementioned white point color temperature and the shooting environment color temperature into the header file of the image.

[0186] It is understood that in some embodiments, if the electronic device only has the aforementioned shooting end, then the structure of these electronic devices can be reduced by removing the aforementioned display screen 215 based on Figure 18; if the electronic device only has the aforementioned display end, then the structure of these electronic devices can be reduced by removing the aforementioned camera 218 based on Figure 18. The remaining hardware structure of these electronic devices can be referred to the relevant description in Figure 18 above, and will not be repeated here.

[0187] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. Components may be implemented in hardware, software, or a combination of software and hardware.

[0188] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of a computer program from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer. Embodiments of this application also provide a computer program product, including a computer program that, when run on a processor, implements the steps in the various method embodiments described above.

[0189] The above detailed embodiments further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A color adaptation method, characterized in that, Applied to a first electronic device, the first electronic device including a camera, the method includes: The first original image was acquired under the first ambient color temperature. A first image file is generated and saved based on the first original image. The first image file includes a first image and first metadata. The first image is obtained by processing the first original image with standard white balance. The first metadata includes the first ambient color temperature, the expected white point color temperature when the first image is displayed under the first ambient color temperature, and the white point color temperature of the first image in the first image file.

2. The method according to claim 1, characterized in that, The step of generating and saving a first image file based on the first original image specifically includes: The first original image is input into the human visual model, and the expected white point color temperature and the white point color temperature of the first image in the first image file are output when the first image is displayed under the first ambient color temperature.

3. The method according to claim 1 or 2, characterized in that, The method further includes: The first image after color adaptation adjustment is generated and displayed based on the second ambient color temperature; the white point color temperature of the first image after color adaptation adjustment is different from the white point color temperature of the first image in the first image file.

4. The method according to claim 3, characterized in that, When the second ambient color temperature is higher than the first ambient color temperature, the color temperature of the first image after color adaptation adjustment is higher than the color temperature of the first image in the first image file.

5. The method according to claim 3 or 4, characterized in that, The method further includes: A second original image is acquired under a third ambient color temperature, and the image content of the second original image is the same as that of the first original image. A second image file is generated and saved based on the second original image. The second image file includes a second image and first metadata. The second image is obtained by processing the second original image with standard white balance. The second metadata includes the third ambient color temperature, the expected white point color temperature when the second image is displayed under the third ambient color temperature, and the white point color temperature of the second image in the second image file. The second image, after color adaptation adjustment, is generated and displayed based on the fourth ambient color temperature; If the deviation between the expected white point color temperature of the second image when displayed at the third ambient color temperature and the white point color temperature of the second image in the second image file is greater than the deviation between the expected white point color temperature of the first image when displayed at the first ambient color temperature and the white point color temperature of the first image in the first image file, then the deviation between the white point color temperature of the second image after color adaptation adjustment and the white point color temperature of the second image in the second image file is greater than the deviation between the white point color temperature of the first image after color adaptation adjustment and the white point color temperature of the first image in the first image file.

6. The method according to any one of claims 2-5, characterized in that, The method further includes: A third original image is acquired under the first ambient color temperature, and the content of the third original image is different from that of the first original image. A third image file is generated and saved based on the third original image. The third image file includes a third image and third metadata. The third image is obtained by processing the third original image with standard white balance. The third metadata includes the first ambient color temperature, the expected white point color temperature when the third image is displayed under the first ambient color temperature, and the white point color temperature of the third image in the third image file. The third image, after color adaptation adjustment, is generated and displayed based on the second ambient color temperature; the white point color temperature of the third image after color adaptation adjustment is different from the white point color temperature of the first image in the first image file.

7. The method according to any one of claims 2-6, characterized in that, Before generating and displaying the first image after color adaptation adjustment based on the second ambient color temperature, the method further includes: Based on the first ambient color temperature and the desired white point color temperature of the first image when displayed under the first ambient color temperature, the basic color adaptation mapping curve is adjusted to obtain the first color adaptation mapping curve. The white point color temperature corresponding to the first ambient color temperature on the basic color adaptation mapping curve is different from the desired white point color temperature of the first image when displayed under the first ambient color temperature. The white point color temperature corresponding to the first ambient color temperature on the first color adaptation mapping curve is the desired white point color temperature of the first image when displayed under the first ambient color temperature. The white point color temperature for displaying the first image under the second ambient color temperature is determined based on the first color adaptation mapping curve.

8. The method according to claim 7, characterized in that, The ambient color temperature includes a first adjustable color temperature and a second adjustable color temperature. The white point color temperature corresponding to the first adjustable color temperature on the basic color adaptation mapping curve is the same as the white point color temperature corresponding to the first adjustable color temperature on the first color adaptation mapping curve. The white point color temperature corresponding to the second adjustable color temperature on the basic color adaptation mapping curve is the same as the white point color temperature corresponding to the second adjustable color temperature on the first color adaptation mapping curve. The first ambient color temperature is located between the first adjustable color temperature and the second adjustable color temperature. Within the range of the first adjusted color temperature to the second adjusted color temperature, the white point color temperature corresponding to each ambient color temperature on the first color adaptation mapping curve is obtained by interpolating the expected white point color temperature of the first image when displayed at the first ambient color temperature with the white point color temperature corresponding to the first adjusted color temperature on the first color adaptation mapping curve, or the expected white point color temperature of the first image when displayed at the first ambient color temperature with the white point color temperature corresponding to the second adjusted color temperature on the first color adaptation mapping curve.

9. The method according to claim 7 or 8, characterized in that, The basic color adaptation mapping curve is one or more, and the multiple basic color adaptation mapping curves are basic color adaptation mapping curves corresponding to different ambient brightness; if the electronic device contains multiple basic color adaptation mapping curves, before adjusting the basic color adaptation mapping curve based on the first ambient color temperature and the desired white point color temperature when the first image is displayed under the first ambient color temperature, the method further includes: Select one of the multiple basic color adaptation mapping curves as the basic color adaptation mapping curve for determining the white point color temperature of the first image after color adaptation adjustment.

10. The method according to claim 9, characterized in that, The method further includes: The brightness of the environment in which the first image is displayed is detected; The step of selecting one of multiple basic color adaptation mapping curves as the basic color adaptation mapping curve for determining the white point color temperature of the first image after color adaptation adjustment specifically includes: Based on the brightness of the environment in which the first image is displayed, a base color adaptation mapping curve corresponding to the brightness of the environment in which the first image is displayed is selected from the plurality of base color adaptation mapping curves as the base color adaptation mapping curve for determining the white point color temperature of the first image after color adaptation adjustment.

11. The method according to any one of claims 7-10, characterized in that, The method further includes: A color adaptation matrix is ​​determined based on the white point color temperature of the first image when it is displayed under the first ambient color temperature and the white point color temperature of the first image in the first image file. The color adaptation matrix is ​​calculated by a color space transformation matrix and a diagonal matrix. The values ​​on the diagonal of the diagonal matrix are used to indicate the scaling degree of the human eye's response to long wavelengths, medium wavelengths, and short wavelengths in the screen data. The color space transformation matrix includes a transformation matrix between the RGB color space and the XYZ color space, and a transformation matrix between the XYZ color space and the LMS color space. Displaying the first image at the second ambient color temperature specifically includes: Convert the first image in the first image file from the RGB color space to the XYZ color space; The first image is converted from the XYZ color space to the LMS color space; The color adaptation matrix is ​​used to perform color transformation on the first image in the LMS color space; The first image after color transformation is converted from the LMS color space to the XYZ color space; The first image after color transformation is converted from the XYZ color space to the RGB color space; The first image after color transformation is displayed in the RGB color space.

12. The method according to claim 11, characterized in that, The first image and the second image are simultaneously displayed on different layers of the background image. The residual between the color adaptation matrix of the first image and the color adaptation matrix of the background image is used to perform color transformation on the layer where the first image is located. After the color transformation, the white point color temperature displayed on the layer where the first image is located is closer to the white point color temperature of the first image perceived by the human eye after color adaptation under the second ambient color temperature. The residual between the color adaptation matrix of the second image and the color adaptation matrix of the background image is used to perform color transformation on the layer where the second image is located. After the color transformation, the white point color temperature displayed on the layer where the second image is located is closer to the white point color temperature of the second image perceived by the human eye after color adaptation under the second ambient color temperature. The color adaptation matrix of the background image is used to perform color transformation on the merged image after layer fusion.

13. The method according to claim 1, characterized in that, The method further includes: The first image file is sent to a second electronic device, the second electronic device including a display screen, the second electronic device being used to generate and display the first image after color adaptation adjustment based on a second ambient color temperature; the white point color temperature of the first image after color adaptation adjustment is different from the white point color temperature of the first image in the first image file.

14. The method according to claim 13, characterized in that, When the second ambient color temperature is higher than the first ambient color temperature, the color temperature of the first image after color adaptation adjustment is higher than the color temperature of the first image in the first image file.

15. The method according to claim 13 or 14, characterized in that, The method further includes: A third original image is acquired under the first ambient color temperature, and the content of the third original image is different from that of the first original image. A third image file is generated and saved based on the third original image. The third image file includes a third image and third metadata. The third image is obtained by processing the third original image with standard white balance. The third metadata includes the first ambient color temperature, the expected white point color temperature when the third image is displayed under the first ambient color temperature, and the white point color temperature of the third image in the third image file. The third image, after color adaptation adjustment, is generated and displayed based on the second ambient color temperature; the white point color temperature of the third image after color adaptation adjustment is different from the white point color temperature of the first image in the first image file.

16. A color adaptation method, characterized in that, Applied to a second electronic device, the second electronic device including a display screen, the method includes: Receive a first image file from a first electronic device. The first image file includes a first image and first metadata. The first image is obtained by processing the first original image with standard white balance. The first metadata includes the first ambient color temperature, the expected white point color temperature when the first image is displayed under the first ambient color temperature, and the white point color temperature of the first image in the first image file. The first image after color adaptation adjustment is generated and displayed based on the second ambient color temperature; the white point color temperature of the first image after color adaptation adjustment is different from the white point color temperature of the first image in the first image file.

17. The method according to claim 16, characterized in that, The method further includes: The first electronic device receives a third image file, which is generated and saved from a third original image acquired by the first electronic device under the first ambient color temperature. The third image file includes a third image and third metadata. The third image is obtained by processing the third original image with standard white balance. The third metadata includes the first ambient color temperature, the expected white point color temperature when the third image is displayed under the first ambient color temperature, and the white point color temperature of the third image in the third image file. The third image, after color adaptation adjustment, is generated and displayed based on the second ambient color temperature; the white point color temperature of the third image after color adaptation adjustment is different from the white point color temperature of the first image in the first image file.

18. The method according to any one of claims 16-17, characterized in that, Before generating and displaying the first image after color adaptation adjustment based on the second ambient color temperature, the method further includes: Based on the first ambient color temperature and the desired white point color temperature of the first image when displayed under the first ambient color temperature, the basic color adaptation mapping curve is adjusted to obtain the first color adaptation mapping curve. The white point color temperature corresponding to the first ambient color temperature on the basic color adaptation mapping curve is different from the desired white point color temperature of the first image when displayed under the first ambient color temperature. The white point color temperature corresponding to the first ambient color temperature on the first color adaptation mapping curve is the desired white point color temperature of the first image when displayed under the first ambient color temperature. The white point color temperature for displaying the first image under the second ambient color temperature is determined based on the first color adaptation mapping curve.

19. The method according to claim 18, characterized in that, The ambient color temperature includes a first adjustable color temperature and a second adjustable color temperature. The white point color temperature corresponding to the first adjustable color temperature on the basic color adaptation mapping curve is the same as the white point color temperature corresponding to the first adjustable color temperature on the first color adaptation mapping curve. The white point color temperature corresponding to the second adjustable color temperature on the basic color adaptation mapping curve is the same as the white point color temperature corresponding to the second adjustable color temperature on the first color adaptation mapping curve. The first ambient color temperature is located between the first adjustable color temperature and the second adjustable color temperature. Within the range of the first adjusted color temperature to the second adjusted color temperature, the white point color temperature corresponding to each ambient color temperature on the first color adaptation mapping curve is obtained by interpolating the expected white point color temperature of the first image when displayed at the first ambient color temperature with the white point color temperature corresponding to the first adjusted color temperature on the first color adaptation mapping curve, or the expected white point color temperature of the first image when displayed at the first ambient color temperature with the white point color temperature corresponding to the second adjusted color temperature on the first color adaptation mapping curve.

20. The method according to claim 18 or 19, characterized in that, The basic color adaptation mapping curve is one or more, and the multiple basic color adaptation mapping curves are basic color adaptation mapping curves corresponding to different ambient brightness; if the electronic device contains multiple basic color adaptation mapping curves, before adjusting the basic color adaptation mapping curve based on the first ambient color temperature and the desired white point color temperature when the first image is displayed under the first ambient color temperature, the method further includes: Select one of the multiple basic color adaptation mapping curves as the basic color adaptation mapping curve for determining the white point color temperature of the first image after color adaptation adjustment.

21. The method according to claim 20, characterized in that, The method further includes: The brightness of the environment in which the first image is displayed is detected by a brightness sensor; The step of selecting one of multiple basic color adaptation mapping curves as the basic color adaptation mapping curve for determining the white point color temperature of the first image after color adaptation adjustment specifically includes: Based on the brightness of the environment in which the first image is displayed, one of the multiple basic color adaptation mapping curves is selected as the basic color adaptation mapping curve for determining the white point color temperature of the first image after color adaptation adjustment.

22. The method according to any one of claims 18-21, characterized in that, The method further includes: A color adaptation matrix is ​​determined based on the white point color temperature of the first image when it is displayed under the first ambient color temperature and the white point color temperature of the first image in the first image file. The color adaptation matrix is ​​calculated by a color space transformation matrix and a diagonal matrix. The values ​​on the diagonal of the diagonal matrix are used to indicate the scaling degree of the human eye's response to long wavelengths, medium wavelengths, and short wavelengths in the screen data. The color space transformation matrix includes a transformation matrix between the RGB color space and the XYZ color space, and a transformation matrix between the XYZ color space and the LMS color space. Displaying the first image at the second ambient color temperature specifically includes: Convert the first image in the first image file from a RAW format image to the RGB color space; Convert the first image from the RGB color space to the XYZ color space; The first image is converted from the XYZ color space to the LMS color space; The color adaptation matrix is ​​used to perform color transformation on the first image in the LMS color space; The first image after color transformation is converted from the LMS color space to the XYZ color space; The first image after color transformation is converted from the XYZ color space to the RGB color space; The first image after color transformation is displayed in the RGB color space.

23. The method according to claim 22, characterized in that, The first image and the second image are simultaneously displayed on different layers of the background image. The residual between the color adaptation matrix of the first image and the color adaptation matrix of the background image is used to perform color transformation on the layer where the first image is located. After the color transformation, the white point color temperature displayed on the layer where the first image is located is closer to the white point color temperature of the first image perceived by the human eye after color adaptation under the second ambient color temperature. The residual between the color adaptation matrix of the second image and the color adaptation matrix of the background image is used to perform color transformation on the layer where the second image is located. After the color transformation, the white point color temperature displayed on the layer where the second image is located is closer to the white point color temperature of the second image perceived by the human eye after color adaptation under the second ambient color temperature. The color adaptation matrix of the background image is used to perform color transformation on the merged image after layer fusion.

24. An electronic device, characterized in that, The method includes one or more processors, one or more memories, and a camera; wherein the camera is used to acquire a first image at a first ambient color temperature, the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when executed by the one or more processors, cause the method as described in any one of claims 1-15 to be performed.

25. An electronic device, characterized in that, The device includes one or more processors, one or more memories, and a display screen; wherein the display screen is used to display a first image at a second ambient color temperature, the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when executed by the one or more processors, cause the method as described in any one of claims 16-23 to be performed.

26. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run on an electronic device, it causes the method as described in any one of claims 1-15 or 16-23 to be performed.

27. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-15 or 16-23.

Images