An image processing method based on color temperature rendering, an electronic device and a related medium

Abstract

Embodiments of the present application provide an image processing method based on color temperature rendering, an electronic device and related media. The method comprises: generating i first color temperature rendering images of a to-be-processed image based on the to-be-processed image, each of the i first color temperature rendering images corresponding to a different color temperature value, i≥3, i being a positive integer; determining a fusion weight corresponding to each of the i first color temperature rendering images; and generating a first fused image based on the i first color temperature rendering images and the fusion weight corresponding to each of the first color temperature rendering images. By implementing the method, the color temperature of an image collected in a mixed light source scene can be adjusted, which helps to make the image color harmonious and closer to the observation effect of the human eye, presents an image with balanced color to the user, and improves the user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computers, and particularly relates to an image processing method based on color temperature rendering, an electronic device and related media. BACKGROUND

[0002] With the enrichment of the camera function of electronic devices and the improvement of the camera effect, the frequency of users using electronic devices to take pictures or videos is increasing, and the scenes of users taking pictures are also becoming diversified. In order to meet the shooting needs of users, the current electronic devices have good waterproof ability and night shooting ability, etc., but there is little research on the problem of color temperature imbalance of images collected in complex scenes (such as mixed light source scenes, that is, there are multiple light sources with different color temperatures in the shooting scene)..

[0003] The prior art can estimate the color temperature according to the black level and the brightness value in the image collection device, and correct the color temperature error, so as to obtain an accurate color temperature. Although the prior art can restore the image corresponding to the color temperature of the shooting scene more accurately, for the image obtained in the shooting scene where cold color temperature light sources and warm color temperature light sources exist at the same time, the image obtained by the prior art still has the problem of color temperature imbalance (such as the case that a white object under the irradiation of a cold color temperature light source appears blue), which has a color temperature difference with the image observed by the human eye, not only affecting the beauty of the image, but also affecting the shooting experience of the user. SUMMARY

[0004] In a first aspect, the present application provides an image processing method based on color temperature rendering, an electronic device and related media, which can include:

[0005] Based on the to-be-processed image, i first color temperature rendering images of the to-be-processed image are generated, each first color temperature rendering image in the i first color temperature rendering images corresponds to a different color temperature value, i≥3, i is a positive integer;

[0006] Determine the fusion weight corresponding to each first color temperature rendering image in the i first color temperature rendering images;

[0007] Based on the i first color temperature rendering images and the fusion weight corresponding to each first color temperature rendering image, a first fusion image is generated.

[0008] The method provided by the first aspect can be implemented. The electronic device can render the to-be-processed image into a plurality of rendering images with different color temperatures, determine the fusion weight of each rendering image corresponding to a color temperature value, and finally generate a first fused image based on the rendering images with different color temperatures and the fusion weight corresponding to each rendering image. It can be seen that the embodiments of the present application can fuse the rendering images with different color temperature values based on different fusion ratios, which helps to make the image presented to the user in a color temperature balanced state, avoids the phenomenon that part of the objects in the image is blue, and provides a good shooting experience for the user. In the high-low color temperature mixed light source scene, the prior art can only restore the color temperature of part of the objects in the shooting scene, without considering the human eye adaptation adjustment of the camera in the high-low color temperature mixed light source scene, thereby causing the color temperature deviation between the image obtained by the electronic device and the image observed by the human eye of the user, making the image not meet the shooting expectation of the user, and affecting the shooting experience of the user. However, the method of the embodiments of the present application can render the obtained image according to a plurality of different color temperatures, and fuse the rendering images rendered according to different color temperatures according to a certain ratio, so as to obtain a color temperature balanced fused image. The fused image is not only more beautiful than the prior art, but also more in line with the observation habit of the human eye. It can be considered that the method of the embodiments of the present application improves the shooting experience of the user compared with the prior art.

[0009] In some embodiments, the method provided by the first aspect can be implemented. Based on the to-be-processed image, i first color temperature rendering images of the to-be-processed image can be generated, which can include:

[0010] The to-be-processed image is input into the first neural network model, and i first color temperature rendering images of the to-be-processed image are output.

[0011] The method provided by the above embodiments can be implemented. The electronic device can generate i first color temperature rendering images corresponding to the to-be-processed image through the first neural network, and then fuse the i first color temperature rendering images according to a certain ratio, which helps to make the fused image closer to the scene observed by the human eye of the user, and improve the shooting experience of the user.

[0012] In some embodiments, the method provided by the first aspect can be implemented. The i first color temperature rendering images corresponding to each first color temperature rendering image can be determined, which can include:

[0013] The i first color temperature rendering images are input into the second neural network model, and the fusion weight corresponding to each first color temperature rendering image is output.

[0014] The method provided by the above embodiments can be implemented. The electronic device can determine the fusion weight corresponding to each color temperature rendering image through the second neural network, which helps to determine a more suitable color temperature rendering image fusion scheme, so as to ensure that the fused image is closer to the scene observed by the human eye of the user, and improve the shooting experience of the user.

[0015] In some embodiments of the method provided in the first aspect, the i different color temperature values can include n different standard color temperature values and m different offset color temperature values.

[0016] The standard color temperature values are different color temperature values on the Planck curve, and 3≤n≤i, where n is a positive integer.

[0017] The offset color temperature values are different color temperature values outside the Planck curve, and n+m=i, where m≥0 and m is an integer.

[0018] In the method provided in the above embodiments, the method provided in the embodiments of the present application not only selects the standard color temperature values as the color temperature rendering target values, but also adds m offset color temperature values as the color temperature rendering target values, which helps to cover the color temperature values that users can contact in daily life, so that the color temperature adjustment effect is better. Specifically, the standard color temperature values are different color temperature values on the Planck curve, and the offset color temperature values are different color temperature values outside the Planck curve. By adding the offset color temperature values, the second neural network model can interpolate more color temperature values and their corresponding fusion ratios, which helps to improve the color temperature adjustment effect and provide better shooting experience for users.

[0019] In some embodiments of the method provided in the first aspect, before generating i first color temperature rendering images of the to-be-processed image based on the to-be-processed image, the method can further include:

[0020] In response to an operation instruction of a user, the to-be-processed image is determined. The operation instruction can include a shooting instruction and a repair instruction.

[0021] In the method provided in the above embodiments, the electronic device can adjust the color temperature of the image in different scenes. For example, when the user shoots a photo, the electronic device can immediately perform color temperature adjustment processing on the image obtained by the camera; after the user saves the image from another way, the electronic device can perform the color temperature adjustment function in response to the repair instruction of the user, which helps to meet the diversified image repair needs of the user and improve the user experience.

[0022] In some embodiments of the method provided in the first aspect, when the operation instruction is a shooting instruction, the method can further include:

[0023] In response to the shooting instruction of the user, a RAW image is obtained.

[0024] The RAW image is subjected to local white balance processing to generate the to-be-processed image.

[0025] When the method provided in the above embodiment is implemented, after the electronic device acquires a RAW image in response to the user's shooting instruction, it will perform local white balance processing on the RAW image and generate an image to be processed, so that the subsequent color temperature adjustment step can be performed. After the RAW image is locally white balanced, it helps to adjust the color of the image to a degree close to the real color of the object, correct the color temperature to a certain extent, and preliminarily correct the color cast caused by ambient light to the image. Compared with the method provided in the first aspect, which is a rough adjustment, the method provided in the first aspect is a fine adjustment. The coarse adjustment of this embodiment helps to improve the color temperature adjustment effect of the method provided in the first aspect, so that the fused image is closer to the user's observation effect, and improves the user's shooting experience.

[0026] In some embodiments of the method provided in the first aspect, when the operation instruction is a repair instruction, the method may further include:

[0027] In response to a repair instruction from a user, an image corresponding to the repair instruction is determined as an image to be processed.

[0028] In some embodiments of the method provided in the first aspect, the second neural network model is a model trained using i second color temperature rendered images of the first training image as training samples and using a first standard output image corresponding to the first training image as an output label;

[0029] The i second color temperature rendered images of the first training image are output by the first neural network model based on the first training image.

[0030] By implementing the method provided in the above embodiment, a second fused image corresponding to the first training image can be generated based on the i second color temperature rendered images corresponding to the first training image, and a second neural network model can be trained based on the error matrix between the second fused image and the first standard output image, which helps the second neural network model to iterate more appropriate fusion weights for different color temperature values, thereby obtaining a better color temperature adjustment effect, making the color temperature of the fused image more balanced and more in line with the user's observation effect. Among them, the number of first training images is several, and a large number of training images helps the second neural network model to iterate more reasonable fusion weights, thereby obtaining a better color temperature adjustment effect. Furthermore, the first standard output image can be obtained from the first training image PS, or it can be calculated from the standard white balance map corresponding to the first training image.

[0031] In some embodiments of the method provided in the first aspect, the first neural network model is a model obtained by training using the second training image as a training sample and a second standard output image corresponding to the second training image as an output label;

[0032] The second standard output image includes a rendering image corresponding to the second training image and i different color temperature values.

[0033] The second training image and the second standard output image corresponding to any one of the i different color temperature values serve as a pair of training data.

[0034] Implementing the method provided in the above embodiments, a single second training image can correspond to i pairs of training data, which helps the first neural network model to have the ability to render a single image (such as a to-be-processed image) into multiple (such as i) rendering images corresponding to different color temperature values, and helps subsequent fusion of various rendering images at a certain ratio to make the fused image closer to the observation effect of the user, thereby providing a good shooting experience for the user.

[0035] In some embodiments, the number of the first neural network models is 1 or i.

[0036] The method can further include:

[0037] In the case where the number of the first neural network models is 1, the to-be-processed image is input into the first neural network model, and i first color temperature rendering images corresponding to different color temperature values are generated.

[0038] In the case where the number of the first neural network models is i, the to-be-processed image is respectively input into each of the i first neural network models based on that each first neural network model corresponds to a rendering color temperature value, so as to generate a first color temperature rendering image corresponding to the rendering color temperature value through each first neural network model.

[0039] Implementing the method provided in the above embodiments, the number of the first neural network models in the embodiments of the present application can be 1 or i (i.e., the same as the number of the color temperature rendering target values), which can respectively realize rendering of multiple rendering images corresponding to different color temperature values for a single to-be-processed image. Specifically, in the case where the number of the first neural network models is 1, the to-be-processed image is input into the first neural network model, and i first color temperature rendering images corresponding to different color temperature values are generated; in the case where the number of the first neural network models is i, the to-be-processed image is respectively input into each of the i first neural network models based on that each first neural network model corresponds to a rendering color temperature value, so as to generate a first color temperature rendering image corresponding to the rendering color temperature value through each first neural network model, which helps subsequent generation of a fused image with more balanced color temperature and improves the shooting experience of the user, regardless of whether the number of the first neural network models is 1 or i.

[0040] In a second aspect, the embodiments of the present application provide an electronic device, which can include an image processing module.

[0041] The image processing module can be configured to generate i first color temperature rendering images of the to-be-processed image based on the to-be-processed image, each of the i first color temperature rendering images corresponding to a different color temperature value, i ≥ 3, i being a positive integer.

[0042] The image processing module can be further configured to determine a fusion weight corresponding to each of the i first color temperature rendering images.

[0043] The image processing module can be further configured to generate a first fusion image based on the i first color temperature rendering images and the fusion weight corresponding to each of the first color temperature rendering images.

[0044] In some embodiments of the method provided in the second aspect, the electronic device can further include:

[0045] The image processing module can be further configured to input the to-be-processed image into the first neural network model to output the i first color temperature rendering images of the to-be-processed image.

[0046] In some embodiments of the method provided in the second aspect, the electronic device can further include:

[0047] The image processing module can be further configured to input the i first color temperature rendering images into the second neural network model to output the fusion weight corresponding to each of the first color temperature rendering images.

[0048] In some embodiments of the method provided in the second aspect, the i different color temperature values can include n different standard color temperature values and m different offset color temperature values.

[0049] The standard color temperature values are mutually different color temperature values located on the Planck curve, 3 ≤ n ≤ i, n being a positive integer.

[0050] The offset color temperature values are mutually different color temperature values located outside the Planck curve, n + m = i, m ≥ 0, m being an integer.

[0051] In some embodiments of the method provided in the second aspect, the electronic device can further include a control module.

[0052] The control module can be configured to determine the to-be-processed image in response to an operation instruction of a user, the operation instruction including a shooting instruction and a repair instruction.

[0053] In some embodiments of the method provided in the second aspect, the electronic device can further include an image acquisition module.

[0054] The image acquisition module can be configured to acquire a RAW image in response to a shooting instruction of a user.

[0055] The image processing module can also be configured to perform local white balance processing on the RAW image to generate the to-be-processed image.

[0056] In some embodiments of the method provided in the second aspect, the electronic device can further include:

[0057] The control module can also be configured to determine, in response to a repair instruction of a user, an image corresponding to the repair instruction as the to-be-processed image.

[0058] In some embodiments of the method provided in the second aspect, the second neural network model is a model trained by taking i second color temperature rendering images of a first training image as training samples and taking a first standard output image corresponding to the first training image as an output label.

[0059] The i second color temperature rendering images of the first training image are output by the first neural network model based on the first training image.

[0060] In some embodiments of the method provided in the second aspect, the first neural network model is a model trained by taking a second training image as a training sample and taking a second standard output image corresponding to the second training image as an output label.

[0061] The second standard output image includes rendering images of the second training image corresponding to i different color temperature values.

[0062] The second training image and the second standard output image corresponding to any one of the i different color temperature values serve as a pair of training data.

[0063] In some embodiments of the method provided in the second aspect, the number of the first neural network models is 1 or i.

[0064] The electronic device can further include:

[0065] The image processing module can also be configured to, when the number of the first neural network models is 1, input the to-be-processed image into the first neural network model and generate i first color temperature rendering images corresponding to different color temperature values.

[0066] The image processing module can also be configured to, when the number of the first neural network models is i, input the to-be-processed image into each of the i first neural network models based on that each first neural network model corresponds to a rendering color temperature value, so as to generate a first color temperature rendering image corresponding to the rendering color temperature value through each first neural network model.

[0067] In a third aspect, an electronic device is provided, which can include one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, and are configured to store computer program codes; the computer program codes can include computer instructions, which, when executed by the one or more processors, cause the execution of the method according to the first aspect and any possible implementation manner of the first aspect.

[0068] In a fourth aspect, a chip is provided, which can include a logic circuit and an interface; the interface is configured to input and / or output code instructions, and the logic circuit is configured to execute the code instructions, so that the method according to the first aspect or any possible implementation manner of the first aspect is executed.

[0069] In a fifth aspect, a computer readable storage medium is provided, which can include instructions; when the instructions are executed on a target terminal, the method according to the first aspect and any possible implementation manner of the first aspect is executed.

[0070] In a sixth aspect, a computer program product is provided, which includes instructions; when the computer program product is executed on an electronic device, the electronic device executes the method according to the first aspect and any possible implementation manner of the first aspect.

[0071] It can be understood that the device for adjusting screen color temperature according to the second aspect, the electronic device according to the third aspect, the chip according to the fourth aspect, the computer readable storage medium according to the fifth aspect, and the computer program product according to the sixth aspect are all related to the method for adjusting screen color temperature according to the first aspect, and can be used to execute the method provided in the present application. Therefore, the beneficial effects achieved thereby can refer to the beneficial effects of the method for adjusting screen color temperature according to the first aspect, which will not be described herein again. BRIEF DESCRIPTION OF DRAWINGS

[0072] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0073] Figure 1 is a flowchart of a color temperature compensation method in a white balance algorithm provided by the prior art;

[0074] Figure 2 is a flowchart of an image processing method based on color temperature rendering provided by the embodiments of the present application;

[0075] Figure 3 is a distribution diagram of a color temperature white point provided by an embodiment of the present application;

[0076] Figure 4 is a chromaticity diagram provided by an embodiment of the present application;

[0077] Figures 5a-5b is a rendering diagram of a first neural network model provided by an embodiment of the present application;

[0078] Figures 6a-6d is an image processing scene diagram based on color temperature rendering provided by an embodiment of the present application;

[0079] Figure 7 is a flow diagram of training a second neural network model provided by an embodiment of the present application;

[0080] Figure 8 is a composition diagram of an electronic device 100 provided by an embodiment of the present application;

[0081] Figure 9 is a hardware structure diagram of an electronic device 100 provided by an embodiment of the present application;

[0082] Figure 10 is a software structure block diagram provided by an embodiment of the present application. DETAILED DESCRIPTION

[0083] The technical solutions in the embodiments of the present application will be described clearly and exhaustively below with reference to the drawings. The terms used in the following embodiments of the present application are only for the purpose of describing the specific embodiments and are not intended to be limiting to the present application. As used in the specification and the appended claims of the present application, the singular forms "a," "an," and "the" are intended to include plural forms as well, unless the context clearly indicates otherwise. It should also be understood that, in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone, and in addition, "multiple" in the description of the embodiments of the present application means two or more than two.

[0084] Hereinafter, the terms "first", "second", "third", "fourth", "fifth", "sixth", "seventh" and "eighth" are used only for descriptive purposes and should not be construed as implying or suggesting relative importance or an implied direction indicating the number of indicated technical features. Therefore, the features defined with "first", "second", "third", "fourth", "fifth", "sixth", "seventh" and "eighth" can explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0085] The term "user interface (UI)" in the following embodiments of the present application is a medium interface for interaction and information exchange between an application program or an operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is source code written in a specific computer language such as Java, extensible markup language (XML), etc., and the interface source code is parsed, rendered and finally presented as content recognizable by the user on the electronic device. The commonly used form of user interface is graphic user interface (GUI), which refers to a user interface displayed in a graphical manner related to computer operation. It can be a visual interface element such as text, icon, button, menu, tab, text box, dialog box, status bar, navigation bar, Widget, etc. displayed in the display screen of the electronic device.

[0086] For the sake of clear and concise description of the following embodiments, first, a brief introduction of the related art is given:

[0087] (1) RAW image

[0088] The RAW image is the original data of the digital signal converted by the image sensor from the light source signal captured by the image sensor, and is the data format of the output image of the sensor. The sensor can be referred to as a light-sensitive element, which is a device for converting an optical image into an electronic signal, such as a charge coupled device image sensor (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. In the present application, the image sensor can be provided in the image acquisition module. In the image technology field, the RAW format is an unprocessed and uncompressed format, i.e. the original image encoding data (digital negative), and the common suffixes of the RAW format include.ARW,.SRF,.SR2,.crw,.cr2,.cr3, etc.

[0089] (2) Planck curve, color temperature curve

[0090] The Planckian curve, also known as the blackbody radiation spectrum, is a graph that describes the spectrum of light emitted by a black object (or a dark-colored object). In this curve, the brightness (or color warmth) increases with increasing energy until infinity, and then gradually decreases. The color temperature curve is a graph that describes the color temperature of light, indicating the energy level of different colored light. In this curve, red light has the highest energy, followed by orange light, then gradually decreasing, and violet light has the lowest energy. Both the Planckian curve and the color temperature curve are important curves in physics related to the color and energy conversion of light. The unit of color temperature is "K" (Kelvin).

[0091] (3) CIE 1931 XYZ Color Space

[0092] The CIE 1931 XYZ color space, also known as the CIE 1931 color space, is the first color space defined mathematically, established by the International Commission on Illumination (CIE) in 1931. In this color space, each color can be represented by three numerical values corresponding to the X, Y, and Z channels. This representation method can accurately describe any color and facilitate color conversion and calculation. The establishment of the CIE 1931 standard has important significance in the fields of lighting, display, and photography. In lighting engineering, people can choose appropriate light sources and lamps based on the CIE 1931 standard to ensure that the obtained light meets the human eye's perception of color. In display technology, the CIE 1931 standard can help people design and calibrate display devices to ensure accurate color display. In the field of photography, the CIE 1931 standard can help photographers perform color correction and post-processing to obtain more realistic and accurate colors.

[0093] In summary, the CIE 1931 standard is an important standard for color perception, which establishes a system for describing and representing colors by studying and experimenting on the human eye's perception of different wavelengths of light. This standard has important application value in the fields of lighting, display, and photography, and has important significance for improving the accuracy and authenticity of color performance.

[0094] (4) Auto White Balance

[0095] Auto White Balance (AWB) refers to the process of restoring the white color in images taken under different color temperature environment light to the true white color (usually the white color observed by the human eye under natural daylight environment light) through certain algorithms. Since the sensor cannot change its light sensing characteristics like the human eye according to the changes in environment light, additional modules are needed to simulate human perception characteristics to restore the white color under different environment light to the true white color. AWB algorithms are mainly divided into local white balance and global white balance.

[0096] Please refer to Figure 1 , Figure 1 a flowchart of a color temperature compensation method in a white balance algorithm provided by the prior art.

[0097] As Figure 1 shown, the method can include:

[0098] S111: Obtain a target black level value corresponding to a current shooting environment, and determine a first color temperature error generated by a preset white balance algorithm under the target black level value.

[0099] S112: Read a target brightness value corresponding to the current shooting environment, and determine a second color temperature error generated by the preset white balance algorithm under the target brightness value.

[0100] S113: For an estimated color temperature generated by the preset white balance algorithm under the current shooting environment, correct the estimated color temperature based on the first color temperature error and the second color temperature error.

[0101] The corrected color temperature is used to determine the current intensity of the flash.

[0102] As can be seen, the prior art can estimate the color temperature according to the black level value and the brightness value in the image acquisition device, and correct the color temperature error, so as to obtain an accurate color temperature. This technical solution can provide an accurate basis for subsequent determination of the current intensity of the double-color temperature flash, so that an image with a higher degree of color restoration can be shot. However, in a high-low color temperature mixed light source scene, the prior art can only restore the color temperature shown by part of the objects in the shooting scene, without considering the human eye adaptation adjustment of the camera to different color temperature regions in the high-low color temperature mixed light source scene, thereby causing a color temperature deviation between the image obtained by the electronic device and the image observed by the user's eyes, so that the image does not meet the user's shooting expectation, affecting the user's shooting experience.

[0103] Unlike the prior art, the method of the present application embodiment can render the obtained image according to a plurality of different color temperatures, and fuse the rendered images rendered according to different color temperatures according to a certain proportion, so as to obtain a color temperature balanced fusion image. The fusion image is not only more beautiful than the prior art, but also more in line with the observation habits of the human eye. It can be considered that the method of the present application embodiment improves the user's shooting experience compared with the prior art.

[0104] Please refer to Figure 2 , Figure 2 a flowchart of an image processing method based on color temperature rendering provided by the present application embodiment.

[0105] As Figure 2 shown, the method can include:

[0106] S201: Based on an image to be processed, generate i first color temperature rendering images of the image to be processed.

[0107] Specifically, the electronic device 100 may input the image to be processed into the first neural network model, and output i first color temperature rendered images of the image to be processed.

[0108] The shooting scene corresponding to the image to be processed may be a single light source scene or a mixed light source scene, where there are at least two light sources. Furthermore, i≥3, where i is a positive integer.

[0109] Specifically, the i different color temperature values ​​may include n different standard color temperature values ​​and m different offset color temperature values; the standard color temperature value is a color temperature value that is located on the Planck curve and is different from each other, 3≤n≤i, and n is a positive integer; the offset color temperature value is a color temperature value that is located outside the Planck curve and is different from each other, n+m=i, m≥0, and m is an integer.

[0110] For example, see Figure 3 , Figure 3 This is a schematic diagram of the distribution of color temperature white points provided in an embodiment of the present application.

[0111] like Figure 3 As shown, the first curve 31 can be considered as the Planck curve in the RG-BG coordinate system. If n=3 and m=3, then the first white point 32, the second white point 33 and the third white point 34 can be selected on the first curve 31. The color temperature values ​​corresponding to these three white points decrease in sequence. For example, the first white point 32 can be regarded as a white point D75 of cold white light (the corresponding color temperature is 7500K), the second white point 33 can be regarded as a common D50 white point (the corresponding color temperature is 5000K), and the third white point 34 can be regarded as a warmer white point H (the corresponding color temperature value is 2300K). Since the color temperature values ​​of the light sources that users are exposed to in daily life are not necessarily exactly the same as the color temperature values ​​of the white points on the Planck curve, it is necessary to add some color temperature white points outside the Planck curve (such as Figure 3 The fourth white point 35, the fifth white point 36, the sixth white point 37, and the seventh white point 38 in the image are used to increase the DUV distribution. DUV is the offset of the newly added color temperature white point relative to the normal direction of the color temperature curve.

[0112] Specifically, see Figure 4 , Figure 4 A chromaticity diagram is provided in the embodiment of the present application. Figure 4As shown, the second curve 41 can be considered as a Planck curve in the CIE 1931 chromaticity space, and the eighth white point 42 (which can be corresponded to the position of the first white point 32 in the CIE 1931 chromaticity space), the ninth white point 43 (which can be corresponded to the position of the second white point 33 in the CIE 1931 chromaticity space) and the tenth white point 44 (which can be corresponded to the position of the third white point 34 in the CIE 1931 chromaticity space) exist on the second curve 41, and the eleventh white point 45 (which can be corresponded to the position of the fourth white point 35 in the CIE 1931 chromaticity space), the twelfth white point 46 (which can be corresponded to the position of the fifth white point 36 in the CIE 1931 chromaticity space), the thirteenth white point 47 (which can be corresponded to the position of the sixth white point 37 in the CIE 1931 chromaticity space), and the fourteenth white point 48 (which can be corresponded to the position of the seventh white point 38 in the CIE 1931 chromaticity space) can be newly added outside the second curve 41. The method of the present application determines the coordinates of the eleventh white point 45, the twelfth white point 46, the thirteenth white point 47 and the fourteenth white point 48 based on the offset of each color temperature white point in the CIE 1931 chromaticity space compared to the normal direction of the second curve 41, and further determines the color temperature values corresponding to the eleventh white point 45, the twelfth white point 46, the thirteenth white point 47 and the fourteenth white point 48.

[0113] Further, the method of the present application helps the subsequent second neural network model to interpolate more different color temperature values and their corresponding fusion ratios by rendering the to-be-processed image into a rendered image with the same color temperature value as the color temperature values corresponding to these color temperature white points, helps to improve the effect of color temperature adjustment, and provides better shooting experience for users.

[0114] Among them, in order to guarantee the color temperature adjustment effect of the present application, the skilled person needs to set at least 3 different standard color temperature values (corresponding to lower color temperature value, medium color temperature value and higher color temperature value respectively), which helps to cover the color temperature values corresponding to various light sources that users may encounter in daily life. It should be noted that the skilled person can set m different offset color temperature values (i.e. the color temperature values corresponding to the newly added color temperature white points) according to actual needs, which is not limited in the present application.

[0115] In some possible embodiments, the number of the first neural network model can be 1 or i; the method can further include:

[0116] In the case where the number of the first neural network model is 1, the to-be-processed image is input into the first neural network model, and i first color temperature rendered images corresponding to different color temperature values are generated;

[0117] In a case where the number of the first neural network models is i, based on each first neural network model corresponding to a rendering color temperature value, the image to be processed is respectively input into each of the i first neural network models, so as to generate a first color temperature rendering image corresponding to the rendering color temperature value through each first neural network model.

[0118] Exemplarily, please refer to Figures 5a-5b , Figures 5a-5b A rendering schematic diagram of a first neural network model provided by an embodiment of the present application.

[0119] As shown in Figure 5a , in a case where the number of the first neural network models is 1, after the image to be processed is input into the first neural network model, the first neural network model generates i first color temperature rendering images corresponding to different color temperature values.

[0120] As shown in Figure 5b , in a case where the number of the first neural network models is i, each first neural network model corresponds to a rendering color temperature value, for example, the low first neural network model, the middle first neural network model and the high first neural network model in Figure 5b , it can be considered that i = 3, n = 3 and m = 0 at this time. At this time, the image to be processed needs to be respectively input into the low first neural network model, the middle first neural network model and the high first neural network model, and rendering images corresponding to the rendering color temperature values are respectively generated by each first neural network model (for example, a low color temperature rendering image corresponding to the low first neural network model, a middle color temperature rendering image corresponding to the middle first neural network model, and a high color temperature rendering image corresponding to the high first neural network model).

[0121] In some possible implementation manners, before i first color temperature rendering images of the image to be processed are generated based on the image to be processed, the method can further include:

[0122] In response to an operation instruction of a user, the image to be processed is determined, and the operation instruction can include a shooting instruction and a repair instruction.

[0123] Further, in a case where the operation instruction is the shooting instruction, the method can further include:

[0124] In response to the shooting instruction of the user, a RAW image is acquired.

[0125] The RAW image is subjected to local white balance processing to generate the image to be processed.

[0126] In response to a user's shooting instruction, the electronic device acquires a RAW image, performs local white balance processing on the RAW image, and generates a to-be-processed image, so that the subsequent color temperature adjustment step can be performed. After the local white balance processing is performed on the RAW image, the color of the image is adjusted to an extent close to the true color of the object, the color temperature is corrected to a certain extent, the color cast caused by the ambient light to the image is preliminarily corrected, and the method provided in the first aspect is a rough adjustment, while the method provided in the first aspect is a fine adjustment. Through the coarse adjustment of the present embodiment, the color temperature adjustment effect of the method provided in the first aspect is improved, the fused image is closer to the observation effect of the user, and the shooting experience of the user is improved.

[0127] In some possible implementation manners, the RAW image acquired by the electronic device in response to the user's shooting instruction is an image acquired in a mixed light source scene. If only global white balance processing is performed on the RAW image, the color temperature distribution of the image is likely to be unbalanced. For example, if the user shoots indoors at noon (there is a warm light source indoors, and the color temperature value of the sunlight outdoors is higher than that of the warm light source indoors), the white wall and the glass on the window side in the shot image are likely to appear blue. If the user shoots a white light-emitting display screen indoors (there is a warm light source indoors), the display screen in the shot image is also likely to appear blue. The global white balance cannot effectively eliminate the color temperature difference in the shot image, so that the image is not beautiful enough, and the shooting experience of the user is affected. However, the method provided in the present embodiment can perform color temperature adjustment processing on the shot image after the global white balance operation. By fusing the rendering images of different color temperature values corresponding to the shot image in a certain proportion, a color temperature balanced fused image can be obtained, the fused image is closer to the observation effect of the user, and the shooting experience of the user is improved.

[0128] Further, in the case where the operation instruction is a repair instruction, the method can further include:

[0129] In response to the user's repair instruction, the image corresponding to the repair instruction is determined as the to-be-processed image.

[0130] S202: Determine a fusion weight corresponding to each of the i first color temperature rendering images.

[0131] Specifically, the electronic device 100 can input the to-be-processed image into the first neural network model, and output i first color temperature rendering images of the to-be-processed image.

[0132] S203: Generate a first fused image based on the i first color temperature rendering images and the fusion weight corresponding to each first color temperature rendering image.

[0133] Please refer toFigures 6a-6d , Figures 6a-6d A schematic diagram of an image processing scenario based on color temperature rendering provided in an embodiment of the present application.

[0134] When the user's operation instruction is a shooting instruction, you can refer to Figures 6a-6b The corresponding scene diagram. Figure 6a As shown, assume that there is a mixed light source scene with a first light source 61 (set as a warm color temperature light source), a second light source 62 (set as a cold color temperature light source) and a third light source 63 (set as a cold color temperature light source). In this mixed light source scene, the electronic device 100 responds to the user's shooting instruction, obtains a RAW image, and performs local white balance processing on the RAW image to generate an image to be processed. Since the local white balance processing cannot adjust the color temperature value of the entire RAW image to a balanced state, there will be a situation where the local color temperature is low or high in the image to be processed. For example, the first area 64 of the image to be processed will show an image effect with a low color temperature under the influence of the first light source 61, the second area 65 will show an image effect with a high color temperature under the influence of the second light source 62, and the third area 66 will show an image effect with a high color temperature under the influence of the third light source 63. Therefore, it is necessary to continue to perform the color temperature adjustment operation on the image to be processed. Through the embodiment method of the present application, a fused image can be generated by fusing rendered images of different color temperature values ​​corresponding to the image to be processed in a certain proportion. Furthermore, as Figure 6b As shown, the electronic device 100 can also present the fused image on the display screen of the electronic device 100 in response to the user's viewing instructions, which helps to achieve the purpose of balancing the image color temperature value, so that the image finally presented to the user is closer to the effect observed by the user's human eyes.

[0135] Furthermore, when the user's operation instruction is a repair instruction, you can refer to Figures 6c-6d The corresponding scene diagram. Figure 6c As shown, the user can select the image to be repaired in the gallery application of the electronic device 100 and select the "Repair" function. The electronic device 100 can then determine the image to be repaired as the above-mentioned image to be processed, perform a color temperature adjustment operation on it, and generate a repaired image. Furthermore, the electronic device 100 can replace the image to be repaired in the gallery with the repaired image, and Figure 6d is presented to the user.

[0136] It can be seen that the method of the embodiment of the present application can perform fusion operations on rendered images of different color temperature values ​​based on different fusion ratios, which helps to make the image presented to the user in a color temperature balanced state, avoid the phenomenon that some objects in the image appear blue or yellow, and provide users with a good shooting experience.

[0137] In some possible implementation manners, the method provided in the embodiments of the present application can further include: training the first neural network model and the second neural network model before inputting the image to be processed into the first neural network model.

[0138] In some possible implementation manners, the method provided in the embodiments of the present application can further include: training the first neural network model and the second neural network model before inputting the image to be processed into the first neural network model. Figure 7 Figure 7 In some possible implementation manners, the method provided in the embodiments of the present application can further include: training the first neural network model and the second neural network model before inputting the image to be processed into the first neural network model.

[0139] In some possible implementation manners, the method provided in the embodiments of the present application can further include: training the first neural network model and the second neural network model before inputting the image to be processed into the first neural network model. Figure 7 In some possible implementation manners, the method provided in the embodiments of the present application can further include: training the first neural network model and the second neural network model before inputting the image to be processed into the first neural network model.

[0140] In some possible implementation manners, the method provided in the embodiments of the present application can further include: training the first neural network model and the second neural network model before inputting the image to be processed into the first neural network model.

[0141] In some possible implementation manners, the method provided in the embodiments of the present application can further include: training the first neural network model and the second neural network model before inputting the image to be processed into the first neural network model.

[0142] In some possible implementation manners, the method provided in the embodiments of the present application can further include: training the first neural network model and the second neural network model before inputting the image to be processed into the first neural network model.

[0143] In some possible implementation manners, the method provided in the embodiments of the present application can further include: training the first neural network model and the second neural network model before inputting the image to be processed into the first neural network model.

[0144] In some possible implementation manners, the method provided in the embodiments of the present application can further include: training the first neural network model and the second neural network model before inputting the image to be processed into the first neural network model.

[0145] ​determining, based on the second training image and the second standard output image, the i pairs of training data, the second standard output image can include rendering images corresponding to i different color temperature values of the second training image;

[0146] training the first neural network model based on the i pairs of training data.

[0147] determining, based on the second training image and the second standard output image, the i pairs of training data, can include:

[0148] taking the second training image and the second standard output image corresponding to any one of the i different color temperature values as a pair of data.

[0149] For example, if there are three standard output images (low color temperature output image, medium color temperature output image and high color temperature output image) for the second training image, then three pairs of training data can be determined, i.e., “second training image-low color temperature output image”, “second training image-medium color temperature output image” and “second training image-high color temperature output image”, which are used to train the first neural network model to generate three output images (low color temperature output image, medium color temperature output image and high color temperature output image) for a single input image, which helps the first neural network model to generate multiple rendering images corresponding to different color temperature values based on a single to-be-processed image.

[0150] It should be noted that there are several first training images and second training images, and the embodiments of the present application only illustrate the main training steps of training the first neural network model and the second neural network model, so only a single training data example is used for illustration, and the above example should not be construed as limiting the present application.

[0151] The following will be described in detail Figures 8-10 The electronic device 100 related to the embodiments of the present application is introduced.

[0152] Please refer to Figure 8 , Figure 8 The composition of the electronic device 100 provided by the embodiments of the present application is shown in the following schematic diagram.

[0153] As Figure 8 shown, the electronic device 100 can include an image processing module 810.

[0154] The image processing module 810 can be configured to generate i first color temperature rendering images of a to-be-processed image based on the to-be-processed image, each of the i first color temperature rendering images corresponding to a different color temperature value, i≥3, i being a positive integer.

[0155] The image processing module 810 can also be configured to determine a fusion weight corresponding to each of the i first color temperature rendering images.

[0156] The image processing module 810 can be further configured to generate a first fusion image based on the i first color temperature rendered images and the fusion weights corresponding to the respective first color temperature rendered images.

[0157] In some possible implementation manners, the electronic device can further include:

[0158] The image processing module 810 can be further configured to input the to-be-processed image into the first neural network model, and output i first color temperature rendered images of the to-be-processed image.

[0159] In some possible implementation manners, the electronic device can further include:

[0160] The image processing module 810 can be further configured to input the i first color temperature rendered images into the second neural network model, and output the fusion weights corresponding to the respective first color temperature rendered images.

[0161] In some possible implementation manners, the i different color temperature values can include n different standard color temperature values and m different offset color temperature values.

[0162] The standard color temperature values are different color temperature values on the Planck curve, 3≤n≤i, and n is a positive integer.

[0163] The offset color temperature values are different color temperature values outside the Planck curve, n+m=i, and m≥0, where m is an integer.

[0164] In some possible implementation manners, the electronic device can further include a control module 820.

[0165] The control module 820 can be configured to determine the to-be-processed image in response to an operation instruction of a user, and the operation instruction can include a shooting instruction and a repair instruction.

[0166] In some possible implementation manners, the electronic device can further include an image acquisition module 830.

[0167] The image acquisition module 830 can be configured to acquire a RAW image in response to a shooting instruction of a user.

[0168] The image processing module 810 can be further configured to perform local white balance processing on the RAW image to generate the to-be-processed image.

[0169] In some possible implementation manners, the electronic device can further include:

[0170] The control module 820 can be further configured to determine an image corresponding to a repair instruction of a user as the to-be-processed image in response to the repair instruction.

[0171] In some possible implementation manners, the second neural network model is a model trained by taking the i second color temperature rendering images of the first training image as training samples and taking the first standard output image corresponding to the first training image as an output label.

[0172] The i second color temperature rendering images of the first training image are output by the first neural network model based on the first training image.

[0173] In some possible implementation manners, the first neural network model is a model trained by taking the second training image as a training sample and taking the second standard output image corresponding to the second training image as an output label.

[0174] The second standard output image includes rendering images corresponding to i different color temperature values of the second training image.

[0175] The second training image and the second standard output image corresponding to any one of the i different color temperature values serve as a pair of training data.

[0176] In some possible implementation manners, the number of the first neural network models is 1 or i.

[0177] The electronic device can further include:

[0178] The image processing module 810 can be further configured to, when the number of the first neural network models is 1, input the to-be-processed image into the first neural network model, and generate i first color temperature rendering images corresponding to different color temperature values.

[0179] The image processing module 810 can be further configured to, when the number of the first neural network models is i, input the to-be-processed image into each of the i first neural network models based on that each first neural network model corresponds to one rendering color temperature value, so as to generate a first color temperature rendering image corresponding to the rendering color temperature value through each first neural network model.

[0180] Further, please refer to Figure 9 , Figure 9 FIG. 1 shows a hardware structure schematic diagram of an electronic device 100 provided by an embodiment of the present application.

[0181] The electronic device 100 can include a processor 101, a memory 102, a wireless communication module 103, a mobile communication module 104, an antenna 103A, an antenna 104A, a power switch 105, a sensor module 106, a focusing motor 107, a camera 108, a display screen 109, and the like. Among them, the sensor module 106 can include a gyroscope sensor 106A, an acceleration sensor 106B, an ambient light sensor 106C, an image sensor 106D, a distance sensor 106E, and the like. Among them, the wireless communication module 103 can include a WLAN communication module, a Bluetooth communication module, and the like. The above-mentioned multiple parts can transmit data through a bus.

[0182] The processor 101 can include one or more processing units, for example: the processor 101 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0183] The memory 102 can be used to store computer executable program codes, and the executable program codes can include instructions. The processor 101 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the memory 102. The memory 102 can include a program storage area and a data storage area. In specific implementation, the memory 102 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more disk storage devices, flash memory devices or other non-volatile solid-state storage devices.

[0184] The wireless communication function of the electronic device 100 can be realized through the antenna 103A, the antenna 104A, the mobile communication module 104, the wireless communication module 103, the modem processor and the baseband processor, etc.

[0185] The antenna 103A and the antenna 104A can be used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antenna.

[0186] The mobile communication module 104 can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied to the electronic device 100.

[0187] The wireless communication module 103 can provide a solution for wireless communication including wireless local area networks (WLAN), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc. applied to the electronic device 100.

[0188] The gyroscope sensor 106A can be used to determine the motion posture of the electronic device 100.

[0189] The acceleration sensor 106B can detect the magnitude of acceleration of the electronic device 100 in each direction (generally three axes).

[0190] The electronic device 100 can implement a photographing function through an ISP, a camera 108, a video codec, a GPU, a display screen 109, and an application processor, etc.

[0191] The electronic device 100 can implement a display function through a GPU, a display screen 109, and an application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 109 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 101 can include one or more GPUs that execute program instructions to generate or change display information.

[0192] The display screen 109 is used to display images, videos, etc. The display screen 109 includes a display panel. In some embodiments, the electronic device 100 can include 1 or N display screens 109, N being a positive integer greater than 1.

[0193] The structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0194] In the embodiments of the present application:

[0195] Any one or more of the wireless communication module 103, the mobile communication module 104, the sensor module 106, the focusing motor 107, the camera 108, and the like can be used to detect the current environmental features of the electronic device 100.

[0196] The display screen 109 is configured to display the user interface provided by the electronic device 100 as described above, for example, a photographing interface or a photo album interface. The user interface displayed by the display screen 109 can refer to the UI embodiments described above.

[0197] The operations performed by the components of the electronic device 100 can refer to the descriptions of the method embodiments described above.

[0198] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The software structure of the electronic device 100 is exemplarily described in the embodiments of the present application by taking a layered architecture of a mobile operating system as an example.

[0199] It can be understood that the software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture, which is not limited in the present application. For ease of understanding, exemplarily, please refer to Figure 10 , Figure 10 is a software structure block diagram provided by the embodiments of the present application. As shown in the layered architecture, the software is divided into several layers, each layer has a clear role and division of labor, and the layers can communicate with each other through a software interface. Figure 10

[0200] In some embodiments, the system of the electronic device can be divided into five layers from top to bottom, namely, an application layer, an application framework layer, a system runtime library layer, a hardware abstraction layer (HAL), and a kernel layer. The above-mentioned layers are described as follows:

[0201] The application layer can include a series of application packages. Exemplarily, the application packages of the application layer can include camera, gallery, calendar, call, map, navigation, browser, video, music, and short message applications.

[0202] The application framework layer can provide application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer can include some pre-defined functions.

[0203] ​Exemplarily, the application framework layer can include an activity manager, a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, an accelerated graphics port (AGP), and the like. Among them:

[0204] The activity manager can be used to manage the life cycle of each application and the general navigation back function.

[0205] The window manager can be used to manage the window program. Exemplarily, the window manager can obtain the display screen size of the electronic device 100, lock the screen, intercept the screen, and determine whether there is a status bar, and the like.

[0206] The content provider can be used to store and obtain data, and make the data accessible to the application, so that different applications can access or share data. Exemplarily, the above-mentioned data can include videos, images, audios, dialed and received calls, browsing history and bookmarks, and phone books, and the like.

[0207] The view system includes visual controls, such as a control for displaying text, a control for displaying pictures, and the like. The view system can be used to build an application. A display interface can be composed of one or more views. For example, a display interface including a short message notification icon can include a view for displaying text and a view for displaying pictures.

[0208] The telephony manager is used to provide the communication function of the electronic device 100, such as the management of the call state (including connecting a call, hanging up a call, and the like).

[0209] The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and the like.

[0210] The notification manager enables the application to display notification information in the status bar, which can be used to convey a type of message that can automatically disappear after a short stay without user interaction. Exemplarily, the notification manager can be used to inform the completion of downloading, message reminders, and the like. The notification manager can also be a notification in the form of a chart or a scroll bar text appearing in the top status bar of the system, such as a notification of an application running in the background, and can also be a notification in the form of a dialogue window appearing on the screen. For example, prompting text information in the status bar, issuing a prompt sound, the electronic device vibrating, the indicator light flashing, and the like.

[0211] The AGP in the application framework layer can be used to improve the rendering performance of the graphics card, such as providing more cache capacity to the graphics card to achieve faster image processing speed.

[0212] The system runtime layer can include a system library and an Android runtime. Among them:

[0213] The Android runtime includes a core library and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system. Among them, the core library contains two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java file of the application layer and the application framework layer into a binary file. The virtual machine is used to perform object lifecycle management, stack management, thread management, security and exception management, and garbage collection functions.

[0214] The system library can be understood as the support of the application framework, which is an important link connecting the application framework layer and the kernel layer. The system layer can include multiple functional modules, for example, it can include a surface manager, media libraries, a three-dimensional graphics processing library (such as OpenGL ES), a two-dimensional graphics engine (such as SGL), etc. Among them:

[0215] The surface manager can be used to manage the display subsystem, such as managing the interaction between display and access operations when the electronic device 100 executes multiple application programs. The surface manager can also be used to provide two-dimensional and three-dimensional layer fusion for multiple application programs.

[0216] The media library can support multiple commonly used audio, video format playback and recording, and static image files, etc. The media library can support multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0217] The three-dimensional graphics processing library is used to realize three-dimensional graphics drawing, image rendering, synthesis, and layer processing, etc.

[0218] The two-dimensional graphics engine can be understood as a two-dimensional drawing engine.

[0219] The hardware abstraction layer provides a standard interface, such as a HAL interface definition language (HIDL) interface or an Android interface definition language (AIDL) interface.

[0220] The kernel layer can be understood as an abstraction layer between hardware and software. The kernel layer can include system services such as security, memory management, process management, power management, network protocol management, and driver management. Illustratively, the kernel layer can include drivers, such as display drivers, camera drivers, audio drivers, and sensor drivers. Alternatively, the kernel layer can also be referred to as an Android kernel or a kernel program. Illustratively, the hardware layer of the electronic device can include a touch panel (TP), a liquid crystal display (LCD), and the like.

[0221] In some embodiments, the kernel layer can be understood as a kernel state, and the other four layers (i.e., the application layer, the application framework layer, the system runtime library layer, and the HAL) can be understood as a user state.

[0222] It can be understood that various applications that can exist in the application layer have video recording and / or playing functions. Illustratively, a camera has a video recording function, and applications such as a gallery, navigation, and a browser can all have video playing functions. Further, based on different lengths, resolutions, and shooting parameters of a video, the recording and / or playing of the video can be affected. The shooting parameters can be an aperture value, a shutter speed, a sensitivity, an exposure, a focal length, and a depth of field.

[0223] Illustratively, related parameters of video recording and / or playing supported by various applications can be stored in a sensor binary file (sensor bin). The sensor bin file can be stored in an external storage, such as a disk or a hard disk. When a user uses a specific application to record and / or play a video, the application can read related parameters of video recording and / or playing to the memory, so that the user can normally use the application and ensure the user experience.

[0224] It should be understood that each step in the above method embodiments can be completed by integrated logic circuits of hardware in a processor or instructions in the form of software. The method steps disclosed in conjunction with the embodiments of the present application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0225] The present application also provides an electronic device, which can include a memory and a processor. The memory can be used to store a computer program, and the processor can be used to call the computer program in the memory, so that the electronic device executes the method executed by the electronic device in any one of the above embodiments.

[0226] The application also provides a chip system, which comprises at least one processor for implementing the functions performed by the electronic device in any of the above embodiments.

[0227] In some possible designs, the chip system further comprises a memory for storing program instructions and data, and the memory is located in or outside the processor.

[0228] The chip system can be composed of a chip or can comprise a chip and other discrete devices.

[0229] Optionally, the processor in the chip system can be one or more. The processor can be implemented by hardware or software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit or the like. When implemented by software, the processor can be a general-purpose processor, which is implemented by reading software codes stored in the memory.

[0230] Optionally, the memory in the chip system can also be one or more. The memory can be integrated with the processor or can be arranged separately from the processor, and the embodiments of the application do not make any limitation in this aspect. For example, the memory can be a non-transient processor such as a read-only memory (ROM), which can be integrated on the same chip as the processor or can be arranged on different chips respectively, and the embodiments of the application do not make any limitation on the type of the memory and the arrangement manner of the memory and the processor.

[0231] For example, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD) or other integrated chip.

[0232] The application also provides a computer program product, which comprises a computer program (also referred to as code or instruction), which, when executed, causes a computer to perform the method performed by the electronic device in any of the above embodiments.

[0233] The application also provides a computer readable storage medium storing a computer program (also referred to as code or instructions). When the computer program is run, it causes a computer to perform the method performed by the electronic device in any of the above embodiments.

[0234] In the above embodiments, the term "when" can be interpreted as meaning "if" or "after" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "on determining" or "if detecting (a stated condition or event)" can be interpreted as meaning "if determining" or "in response to determining" or "on detecting (a stated condition or event)" or "in response to detecting (a stated condition or event)" depending on the context.

[0235] The embodiments of the application can be combined in any manner to achieve different technical effects.

[0236] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as DVD), or semiconductor media (such as solid state disk (SSD)) and the like.

[0237] Those skilled in the art can understand that all or part of the processes in the above-mentioned method embodiments can be implemented by a computer program instructing relevant hardware to complete, and the program can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned method embodiments. The aforementioned storage medium includes ROM, random access memory (RAM), magnetic disk or optical disk, and various storage media that can store program codes.

[0238] In summary, the above only describes the embodiments of the technical scheme of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of the present application shall be included in the protection scope of the present application.

Claims

1. An image processing method based on color temperature rendering, characterized in that, The method comprises: Based on the image to be processed, generate i first color temperature rendering images of the image to be processed, each of the i first color temperature rendering images corresponds to a different color temperature value, wherein the i different color temperature values include n different standard color temperature values and m different offset color temperature values, the standard color temperature values are different color temperature values located on the Planck curve, the offset color temperature values are different color temperature values located outside the Planck curve, i≥3, 3≤n≤i, n+m=i, m≥0, i, n, m are positive integers; Determine the fusion weight corresponding to each of the i first color temperature rendering images; Based on the i first color temperature rendering images and the fusion weight corresponding to each first color temperature rendering image, generate a first fusion image; The method comprises: Input the image to be processed into a first neural network model, and output i first color temperature rendering images of the image to be processed; The method comprises: Input the i first color temperature rendering images into a second neural network model, and output the fusion weight corresponding to each first color temperature rendering image; Before the method comprises: In response to a user's shooting instruction, a RAW image is obtained; The RAW image is subjected to local white balance processing to generate the image to be processed.

2. The method of claim 1, wherein, Before the method comprises: In response to a user's operation instruction, the image to be processed is determined, and the operation instruction includes the shooting instruction and a repair instruction.

3. The method of claim 2, wherein, In the case where the operation instruction is the repair instruction, the method further comprises: In response to the user's repair instruction, the image corresponding to the repair instruction is determined as the image to be processed.

4. The method of claim 3, wherein, The second neural network model is a model obtained by training a first training image i second color temperature rendering images as training samples and a first standard output image corresponding to the first training image as an output label; The i second color temperature rendering images of the first training image are output by the first neural network model based on the first training image.

5. The method according to any one of claims 1-4, characterized in that, The first neural network model is a model obtained by training a second training image as a training sample and a second standard output image corresponding to the second training image as an output label; The second standard output image includes rendering images corresponding to i different color temperature values of the second training image; The second training image and the second standard output image corresponding to any one of the i different color temperature values are a pair of training data.

6. The method of claim 5, wherein, The number of the first neural network model is 1 or i; The method further comprises: In a case where the number of the first neural network models is 1, inputting the to-be-processed image into the first neural network model, and generating i first color temperature rendering images corresponding to different color temperature values; In a case where the number of the first neural network models is i, inputting the to-be-processed image into each of the i first neural network models based on that each first neural network model corresponds to a rendering color temperature value, to generate a first color temperature rendering image corresponding to the rendering color temperature value through each first neural network model.

7. An electronic device, comprising: The electronic device comprises an image processing module and an image acquisition module. The image processing module is configured to input a to-be-processed image into a first neural network model, and generate i first color temperature rendering images corresponding to different color temperature values, the to-be-processed image corresponding to a mixed light source scene, and at least two light sources existing in the mixed light source scene, wherein i different color temperature values include n different standard color temperature values and m different offset color temperature values, the standard color temperature values are different color temperature values on the Planck curve, the offset color temperature values are different color temperature values outside the Planck curve, i≥3, 3≤n≤i, n+m=i, m≥0, i, n, and m are positive integers. The image processing module is further configured to input the i first color temperature rendering images corresponding to different color temperature values into a second neural network model, and determine a fusion weight corresponding to each first color temperature rendering image. The image processing module is further configured to generate a first fusion image based on each first color temperature rendering image and the fusion weight corresponding to each first color temperature rendering image. The image processing module is further configured to input a to-be-processed image into a first neural network model, and output i first color temperature rendering images of the to-be-processed image. The image processing module is further configured to input the i first color temperature rendering images into a second neural network model, and output a fusion weight corresponding to each first color temperature rendering image. The image acquisition module is configured to acquire a RAW image in response to a shooting instruction of a user. The image processing module is further configured to perform local white balance processing on the RAW image, and generate the to-be-processed image.

8. An electronic device, comprising: The computer readable storage medium stores a computer program, the computer program comprising program instructions, and the program instructions are executed by the processor to cause the method in any one of claims 1-6 to be performed.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, the computer program comprising program instructions, and the program instructions are executed by the processor to cause the method in any one of claims 1-6 to be performed.

Images