Image processing method based on color temperature rendering, electronic equipment and related medium

Abstract

The embodiment of the invention provides an image processing method based on color temperature rendering, electronic equipment and a related medium, and the method comprises the steps: generating i first color temperature rendering images of a to-be-processed image based on the to-be-processed image, each first color temperature rendering image in the i first color temperature rendering images corresponding to different color temperature values, i > = 3, and i is a positive integer; determining a fusion weight corresponding to each first color temperature rendering image in the i first color temperature rendering images; and generating a first fused image based on the i first color temperature rendering images and the fusion weight corresponding to each first color temperature rendering image. By implementing the method provided by the invention, the color temperature adjustment processing can be performed on the image collected in the mixed light source scene, so that the color of the image is harmonious, the color of the image is closer to the observation effect of human eyes, the image with balanced color is presented for a user, and the use experience of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to an image processing method based on color temperature rendering, an electronic device and related media. Background Art

[0002] With the enrichment of electronic device camera functions and the improvement of camera effects, users use electronic devices to shoot images or videos more and more frequently, and the scenes in which users shoot have also become more diverse. In order to meet users' shooting needs, current electronic devices have good waterproof capabilities and night shooting capabilities, but there is little research on the problem of color temperature imbalance in images collected in complex scenes (such as mixed light source scenes, that is, there are multiple light sources with different color temperatures in the shooting scene).

[0003] The existing technology can estimate the color temperature based on the black level and brightness value in the image acquisition device, and correct the color temperature error to obtain an accurate color temperature. Although the existing technology can restore the image of the corresponding color temperature of the shooting scene more accurately, for the image obtained in the shooting scene where there are both cold color temperature light sources and warm color temperature light sources, the image obtained by the existing technology will still have an unbalanced color temperature value (such as a white object under the cold color temperature light source appears bluish), and there is a color temperature difference with the image observed by the user's human eye, which not only affects the beauty of the image, but also affects the user's shooting experience. Summary of the invention

[0004] In a first aspect, the present application provides an image processing method based on color temperature rendering, an electronic device, and related media. The method may include:

[0005] Based on the image to be processed, i first color temperature rendering images of the image to be processed are generated, each of the i first color temperature rendering images corresponds to a different color temperature value, i≥3, and i is a positive integer;

[0006] Determine a fusion weight corresponding to each first color temperature rendering image in i first color temperature rendering images;

[0007] A first fused image is generated based on the i first color temperature rendered images and the fusion weights corresponding to each of the first color temperature rendered images.

[0008] By implementing the method provided in the first aspect, the electronic device can render the image to be processed into a plurality of rendered images with different color temperatures, and determine the fusion weights of the rendered images corresponding to each color temperature value, and finally generate a first fused image based on the rendered images of different color temperatures and their corresponding fusion weights. It can be seen that the embodiment of the present application can perform fusion operations on rendered images with different color temperature values ​​based on different fusion ratios, which helps to make the image presented to the user in a state of color temperature balance, avoid the phenomenon that some objects in the image are blue, and provide the user with a good shooting experience. In the scene of mixed light sources of high and low color temperatures, the prior art can only restore the color temperature exhibited by some objects in the shooting scene, and does not take into account the adaptation and adjustment of the camera to the human eye in different color temperature areas in the scene of mixed light sources of high and low color temperatures, resulting in a color temperature deviation between the image obtained by the electronic device and the image observed by the user's human eye, so that the image does not meet the user's shooting expectations and affects the user's shooting experience. However, the method of the embodiment of the present application can render the acquired image according to a variety of different color temperatures, and fuse the rendered images after being rendered at different color temperatures in a certain proportion, so as to obtain a fused image with balanced color temperature. This fused image is not only more beautiful than the prior art, but also more in line with the observation habits of the human eye. It can be considered that the method of the embodiment of the present application improves the user's shooting experience compared with the prior art.

[0009] In some embodiments of the method provided in the first aspect, generating i first color temperature rendered images of the image to be processed based on the image to be processed may include:

[0010] The image to be processed is input into the first neural network model, and i first color temperature rendered images of the image to be processed are output.

[0011] By implementing the method provided in the above embodiment, the electronic device can generate i first color temperature rendering images corresponding to the image to be processed through the first neural network, and then can fuse these i first color temperature rendering images in a certain proportion, which helps to make the fused image closer to the scene observed by the user's human eyes and improve the user's shooting experience.

[0012] In some embodiments of the method provided in the first aspect, determining the fusion weight corresponding to each of the i first color temperature rendered images may include:

[0013] The i first color temperature rendering images are input into the second neural network model, and the fusion weights corresponding to the first color temperature rendering images are output.

[0014] By implementing the method provided in the above embodiment, the electronic device can determine the fusion weights corresponding to each color temperature rendered image through the second neural network, which helps to determine a more appropriate color temperature rendered image fusion scheme, thereby ensuring that the fused image is closer to the scene observed by the user's human eye and improving the user's shooting experience.

[0015] In some embodiments of the method provided in the first aspect, the i different color temperature values ​​may include n different standard color temperature values ​​and m different offset color temperature values;

[0016] The standard color temperature value is a different color temperature value located on the Planck curve, 3≤n≤i, n is a positive integer;

[0017] The offset color temperature value is a color temperature value that is outside the Planck curve and is different from each other, n+m=i, m≥0, and m is an integer.

[0018] By implementing the method provided in the above embodiment, the method in the embodiment of the present application will not only select the standard color temperature value as the color temperature rendering target value, but also add m offset color temperature values ​​as the color temperature rendering target value, which helps to cover the color temperature values ​​that users can come into contact with in daily life, so that the effect of color temperature adjustment is better. Specifically, the standard color temperature value is a color temperature value that is located on the Planck curve and is different from each other, and the offset color temperature value is a color temperature value that is located outside the Planck curve and is different from each other. By adding an offset color temperature value, the embodiment of the present application can allow the second neural network model to interpolate more color temperature values ​​and their corresponding fusion ratios, which helps to improve the effect of color temperature adjustment and provide users with a better shooting experience.

[0019] In some embodiments of the method provided in the first aspect, before generating i first color temperature rendered images of the image to be processed based on the image to be processed, the method may further include:

[0020] In response to an operation instruction of a user, an image to be processed is determined, and the operation instruction may include a shooting instruction and a repair instruction.

[0021] By implementing the method provided in the above embodiment, the electronic device can adjust the color temperature of the image in different scenarios. For example, when the user takes a photo, the electronic device can immediately adjust the color temperature of the image obtained by the camera; after the user saves the image from another way, the electronic device can respond to the user's repair instruction to perform the color temperature adjustment function, which helps to meet the user's diversified image repair needs and improve the user's experience.

[0022] In some embodiments of the method provided in the first aspect, when the operation instruction is a shooting instruction, the method may further include:

[0023] Responding to a shooting instruction from a user, acquiring a RAW image;

[0024] Perform local white balance processing on the RAW image to generate an image to be processed.

[0025] By implementing the method provided in the above embodiment, after the electronic device acquires the RAW image in response to the user's shooting instruction, it will perform local white balance processing on the RAW image and generate an image to be processed, so that the subsequent color temperature adjustment step can be performed. After the RAW image is locally white balanced, it helps to adjust the color of the image to a degree close to the real color of the object, correct the color temperature to a certain extent, and preliminarily correct the color cast caused by ambient light to the image. Compared with the rough adjustment method provided in the first aspect, the method provided in the first aspect is a fine adjustment. The coarse adjustment of this embodiment helps to improve the color temperature adjustment effect of the method provided in the first aspect, so that the fused image is closer to the user's observation effect, and the user's shooting experience is improved.

[0026] In some embodiments of the method provided in the first aspect, when the operation instruction is a repair instruction, the method may further include:

[0027] In response to a repair instruction from a user, an image corresponding to the repair instruction is determined as an image to be processed.

[0028] In some embodiments of the method provided in the first aspect, the second neural network model is a model obtained by training using i second color temperature rendered images of the first training image as training samples and using the first standard output image corresponding to the first training image as an output label;

[0029] The i second color temperature rendered images of the first training image are output by the first neural network model based on the first training image.

[0030] By implementing the method provided in the above embodiment, a second fused image corresponding to the first training image can be generated based on i second color temperature rendering images corresponding to the first training image, and the second neural network model can be trained based on the error matrix between the second fused image and the first standard output image, which helps the second neural network model to iterate more appropriate fusion weights for different color temperature values, thereby obtaining a better color temperature adjustment effect, making the color temperature of the fused image more balanced and more in line with the user's observation effect. Among them, the number of first training images is several, and a large number of training images helps the second neural network model to iterate more reasonable fusion weights, thereby obtaining a better color temperature adjustment effect. Furthermore, the first standard output image can be obtained from the first training image PS, or it can be calculated from the standard white balance map corresponding to the first training image.

[0031] In some embodiments of the method provided in the first aspect, the first neural network model is a model obtained by training using the second training image as a training sample and the second standard output image corresponding to the second training image as an output label;

[0032] The second standard output image includes rendered images of i different color temperature values ​​corresponding to the second training image;

[0033] The second training image and the second standard output image corresponding to any one of the i different color temperature values ​​are used as a pair of training data.

[0034] By implementing the method provided in the above embodiment, a single second training image can correspond to i training data pairs, which helps the first neural network model to have the ability to render a single image (such as the image to be processed) into multiple (such as i) rendered images corresponding to different color temperature values, and helps to subsequently fuse various rendered images in a certain proportion, so that the fused image is closer to the user's observation effect, providing the user with a good shooting experience.

[0035] Implementing the method provided by the first aspect, in some embodiments, the number of the first neural network models is 1 or i;

[0036] The method may further include:

[0037] When the number of the first neural network model is 1, the image to be processed is input into the first neural network model, and i first color temperature rendering images corresponding to different color temperature values ​​are generated;

[0038] When the number of first neural network models is i, based on the fact that each first neural network model corresponds to a rendering color temperature value, the image to be processed is respectively input into each of the i first neural network models to generate a first color temperature rendering image corresponding to the rendering color temperature value through each first neural network model.

[0039] By implementing the method provided in the above-mentioned embodiment, the number of first neural network models in the embodiment of the present application can be 1 or i (i.e., the same as the number of color temperature rendering target values), and both can realize rendering of multiple rendered images corresponding to different color temperature values ​​for a single image to be processed. Specifically, when the number of first neural network models is 1, the image to be processed is input into the first neural network model, and i first color temperature rendered images corresponding to different color temperature values ​​are generated; when the number of first neural network models is i, based on each first neural network model corresponding to a rendering color temperature value, the image to be processed is input into each of the i first neural network models, so as to generate a first color temperature rendered image corresponding to the rendering color temperature value through each first neural network model. Regardless of whether the number of first neural network models is 1 or i, it is helpful to subsequently generate a more color temperature balanced fusion image, thereby improving the user's shooting experience.

[0040] In a second aspect, an embodiment of the present application provides an electronic device, which may include: an image processing module;

[0041] The image processing module can be used to generate i first color temperature rendering images of the image to be processed based on the image to be processed, each of the i first color temperature rendering images corresponds to a different color temperature value, i≥3, and i is a positive integer;

[0042] The image processing module may also be used to determine a fusion weight corresponding to each of the i first color temperature rendered images;

[0043] The image processing module may also be configured to generate a first fused image based on the i first color temperature rendered images and the fusion weights corresponding to the respective first color temperature rendered images.

[0044] To implement the method provided in the second aspect, in some embodiments, the electronic device may further include:

[0045] The image processing module can also be used to input the image to be processed into the first neural network model and output i first color temperature rendered images of the image to be processed.

[0046] To implement the method provided in the second aspect, in some embodiments, the electronic device may further include:

[0047] The image processing module can also be used to input i first color temperature rendering images into the second neural network model, and output the fusion weights corresponding to each first color temperature rendering image.

[0048] In some embodiments of the method provided in the second aspect, the i different color temperature values ​​may include n different standard color temperature values ​​and m different offset color temperature values;

[0049] The standard color temperature value is a different color temperature value located on the Planck curve, 3≤n≤i, n is a positive integer;

[0050] The offset color temperature value is a color temperature value that is outside the Planck curve and is different from each other, n+m=i, m≥0, and m is an integer.

[0051] To implement the method provided in the second aspect, in some embodiments, the electronic device may further include: a control module;

[0052] The control module can be used to determine the image to be processed in response to the user's operation instruction, and the operation instruction can include a shooting instruction and a repair instruction.

[0053] To implement the method provided in the second aspect, in some embodiments, the electronic device may further include: an image acquisition module;

[0054] The image acquisition module can be used to obtain a RAW image in response to a user's shooting instruction;

[0055] The image processing module can also be used to perform local white balance processing on the RAW image to generate an image to be processed.

[0056] To implement the method provided in the second aspect, in some embodiments, the electronic device may further include:

[0057] The control module may also be configured to respond to a repair instruction from a user and determine the image corresponding to the repair instruction as the image to be processed.

[0058] In some embodiments of the method provided in the second aspect, the second neural network model is a model obtained by training using i second color temperature rendered images of the first training image as training samples and using the first standard output image corresponding to the first training image as an output label;

[0059] The i second color temperature rendered images of the first training image are output by the first neural network model based on the first training image.

[0060] In some embodiments of the method provided in the second aspect, the first neural network model is a model obtained by training with the second training image as a training sample and the second standard output image corresponding to the second training image as an output label;

[0061] The second standard output image includes rendered images of i different color temperature values ​​corresponding to the second training image;

[0062] The second training image and the second standard output image corresponding to any one of the i different color temperature values ​​are used as a pair of training data.

[0063] Implementing the method provided by the second aspect, in some embodiments, the number of the first neural network models is 1 or i;

[0064] The electronic device may also include:

[0065] The image processing module may also be used to input the image to be processed into the first neural network model when the number of the first neural network models is 1, and generate i first color temperature rendering images corresponding to different color temperature values;

[0066] The image processing module can also be used to input the image to be processed into each of the i first neural network models respectively when the number of first neural network models is i, based on the fact that each first neural network model corresponds to a rendering color temperature value, so as to generate a first color temperature rendering image with corresponding rendering color temperature value through each first neural network model.

[0067] In a third aspect, an embodiment of the present application provides an electronic device, which may include one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, and the computer program code may include computer instructions, and when the one or more processors execute the computer instructions, the method described in the first aspect and any possible implementation method of the first aspect is executed.

[0068] In a fourth aspect, an embodiment of the present application provides a chip, which may include a logic circuit and an interface, wherein the logic circuit and the interface are coupled; the interface is used to input and / or output code instructions, and the logic circuit is used to execute the code instructions so that the method in the first aspect or any possible implementation of the first aspect is executed.

[0069] In a fifth aspect, the present application provides a computer-readable storage medium that may include instructions, which, when executed on a target terminal, execute the method described in the first aspect and any possible implementation of the first aspect.

[0070] In a sixth aspect, the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect.

[0071] It can be understood that the device for adjusting screen color temperature provided in the second aspect, the electronic device provided in the third aspect, the chip provided in the fourth aspect, the computer-readable storage medium provided in the fifth aspect, and the computer program product provided in the sixth aspect are all related to the method for adjusting screen color temperature provided in the first aspect, and can be used to execute the method provided in the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the method for memory management in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0073] Figure 1 It is a flowchart of a color temperature compensation method in a white balance algorithm provided by the prior art;

[0074] Figure 2 It is a flowchart of an image processing method based on color temperature rendering provided by an embodiment of the present application;

[0075] Figure 3 is a schematic diagram of the distribution of a color temperature white point provided in an embodiment of the present application;

[0076] Figure 4 is a chromaticity diagram provided in an embodiment of the present application;

[0077] Figure 5a-Figure 5b A rendering schematic diagram of a first neural network model provided in an embodiment of the present application;

[0078] Figure 6a-6d A schematic diagram of an image processing scenario based on color temperature rendering provided in an embodiment of the present application;

[0079] Figure 7 A schematic diagram of a process for training a second neural network model provided in an embodiment of the present application;

[0080] Figure 8 A schematic diagram of the composition of an electronic device 100 provided in an embodiment of the present application;

[0081] Fig. 9 A schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of the present application;

[0082] Fig.10 A software structure block diagram provided for an embodiment of the present application. DETAILED DESCRIPTION

[0083] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear contrary indication in the context. It should also be understood that in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a kind of association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B, can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" refers to two or more than two.

[0084] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0085] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device, and finally presented as content that can be recognized by the user. The commonly used form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0086] In order to make the description of the following embodiments clear and concise, a brief introduction to the related technology is first given:

[0087] (1) RAW image

[0088] RAW image is the raw data that the image sensor converts the captured light source signal into a digital signal, and is the data format of the image output by the image sensor. A sensor can be called a photosensitive element, which is a device that converts an optical image into an electronic signal, such as a charge coupled device (CCD) image sensor or a complementary metal-oxide-semiconductor (CMOS) phototransistor. In the present application, the image sensor can be arranged in an image acquisition module. In the field of image technology, the RAW format is an unprocessed, uncompressed format, that is, the original image encoding data (digital negative). Common suffixes of the RAW format include .ARW, .SRF, .SR2, .crw, .cr2, .cr3, etc.

[0089] (2) Planck curve, color temperature curve

[0090] The Planck curve, also known as the blackbody radiation spectrum, is a spectrum line that describes the light emitted by black (or dark colors). In this curve, the brightness (or color warming) increases with energy until it reaches infinity, and then gradually decreases. The color temperature curve is a curve that describes the color temperature of light and is used to indicate the energy levels of different colors of light. In this curve, red light has the highest energy, followed by orange light, and then gradually decreases, with purple light having the lowest energy. Both the Planck curve and the color temperature curve are important curves in physics related to the color and energy conversion of light. The unit of color temperature is "K" (Kelvin).

[0091] (3) CIE 1931XYZ color space

[0092] The CIE 1931XYZ color space (also called the CIE 1931 color space) is the first color space to be defined mathematically. It was created by the International Commission on Illumination (CIE) in 1931. In this color space, each color can be represented by three numerical values, corresponding to the three channels of X, Y, and Z. This representation method can accurately describe any color and can easily perform color conversion and calculation. The formulation of the CIE 1931 standard is of great significance to fields such as lighting, display, and photography. In lighting engineering, people can choose appropriate light sources and lamps according to the CEE1931 standard to ensure that the light obtained conforms to the human eye's perception of color. In display technology, the CIE1931 standard can help people design and calibrate display devices to ensure that the displayed colors are accurate. In the field of photography, the CIE1931 standard can help photographers perform color correction and post-processing to obtain more realistic and accurate colors.

[0093] In short, the CIE 1931 standard is an important set of standards for color perception. It establishes a system for describing and expressing colors by studying and experimenting on the human eye's perception of light of different wavelengths. This standard has important application value in the fields of lighting, display, photography, etc., and is of great significance for improving the accuracy and authenticity of color expression.

[0094] (4) Automatic white balance

[0095] Auto White Balance (AWB) refers to the use of an algorithm to restore the white imaged under ambient light of different color temperatures to true white (usually the white observed by people under natural daylight). Since the sensor cannot change its photosensitive characteristics according to changes in ambient light like the human eye, an additional module is needed to simulate human perception characteristics and restore the white image under different ambient light to true white. The AWB algorithm is mainly divided into two types: local white balance and global white balance.

[0096] See also Figure 1 , Figure 1 The present invention provides a flowchart of a color temperature compensation method in a white balance algorithm provided in the prior art.

[0097] like Figure 1 As shown, the method includes:

[0098] S111: Acquire a target black level value corresponding to the current shooting environment, and determine a first color temperature error generated by a preset white balance algorithm under the target black level value.

[0099] S112: Reading a target brightness value corresponding to the current shooting environment, and determining a second color temperature error generated by a preset white balance algorithm under the target brightness value.

[0100] S113: For the estimated color temperature generated by the preset white balance algorithm under the current shooting environment, correct the estimated color temperature based on the first color temperature error and the second color temperature error.

[0101] The corrected color temperature is used to determine the current intensity of the flash lamp.

[0102] It can be seen that the prior art can estimate the color temperature based on the black level value and brightness value in the image acquisition device, and correct the color temperature error to obtain an accurate color temperature. This technical solution can provide an accurate basis for the subsequent determination of the current intensity of the dual-color temperature flash, so that images with a higher degree of color restoration can be captured. However, in the high and low color temperature mixed light source scene, the prior art can only restore the color temperature of some objects in the shooting scene, without taking into account the camera's adaptation and adjustment to the human eye in different color temperature areas in the high and low color temperature mixed light source scene, resulting in a color temperature deviation between the image obtained by the electronic device and the image observed by the user's human eye, making the image not meet the user's shooting expectations and affecting the user's shooting experience.

[0103] Different from the prior art, the method of the embodiment of the present application can render the acquired image according to a variety of different color temperatures, and fuse the rendered images after being rendered at different color temperatures in a certain proportion, so as to obtain a fused image with balanced color temperature. The fused image is not only more beautiful than the prior art, but also more in line with the observation habits of the human eye. It can be considered that the method of the embodiment of the present application improves the user's shooting experience compared with the prior art.

[0104] See also Figure 2 , Figure 2 A flowchart of an image processing method based on color temperature rendering provided in an embodiment of the present application.

[0105] like Figure 2 As shown, the method may include:

[0106] S201: Based on an image to be processed, generating i first color temperature rendering images of the image to be processed.

[0107] Specifically, the electronic device 100 can input the image to be processed into the first neural network model, and output i first color temperature rendered images of the image to be processed.

[0108] The shooting scene corresponding to the image to be processed may be a single light source scene or a mixed light source scene, where there are at least two light sources. Furthermore, i≥3, i is a positive integer.

[0109] Specifically, the i different color temperature values ​​may include n different standard color temperature values ​​and m different offset color temperature values; the standard color temperature value is a color temperature value that is located on the Planck curve and is different from each other, 3≤n≤i, n is a positive integer; the offset color temperature value is a color temperature value that is located outside the Planck curve and is different from each other, n+m=i, m≥0, m is an integer.

[0110] For example, see Figure 3 , Figure 3 This is a schematic diagram of the distribution of color temperature white points provided in an embodiment of the present application.

[0111] like Figure 3 As shown, the first curve 31 can be considered as the Planck curve in the RG-BG coordinate system. If n=3, m=3, then the first white point 32, the second white point 33 and the third white point 34 can be selected on the first curve 31, and the color temperature values ​​corresponding to these three white points decrease in sequence. For example, the first white point 32 can be regarded as a white point D75 of cold white light (the corresponding color temperature is 7500K), the second white point 33 can be regarded as a common D50 white point (the corresponding color temperature is 5000K), and the third white point 34 can be regarded as a warmer white point H (the corresponding color temperature value is 2300K). Since the color temperature values ​​of the light sources that users are exposed to in daily life are not necessarily exactly the same as the color temperature values ​​of the white points on the Planck curve, it is necessary to add some color temperature white points outside the Planck curve (such as Figure 3 The fourth white point 35, the fifth white point 36, the sixth white point 37 and the seventh white point 38 in the color temperature curve are used to increase the DUV distribution. Wherein, DUV is the offset value of the newly added color temperature white point compared to the normal direction of the color temperature curve.

[0112] Specifically, see Figure 4 , Figure 4 A chromaticity diagram provided in an embodiment of the present application. Figure 4As shown, the second curve 41 can be considered as the Planck curve in the CIE 1931 chromaticity space, and there are an eighth white point 42 (which can be corresponded to the position of the first white point 32 in the CIE 1931 chromaticity space), a ninth white point 43 (which can be corresponded to the position of the second white point 33 in the CIE 1931 chromaticity space) and a tenth white point 44 (which can be corresponded to the position of the third white point 34 in the CIE 1931 chromaticity space) on the second curve 41, and an eleventh white point 45 (which can be corresponded to the position of the fourth white point 35 in the CIE 1931 chromaticity space), a twelfth white point 46 (which can be corresponded to the position of the fifth white point 36 in the CIE 1931 chromaticity space), a thirteenth white point 47 (which can be corresponded to the position of the sixth white point 37 in the CIE 1931 chromaticity space), and a fourteenth white point 48 (which can be corresponded to the position of the seventh white point 38 in the CIE 1931 chromaticity space) can be added outside the second curve 41. 1931 chromaticity space) The embodiment method of the present application determines the coordinates of the eleventh white point 45, the twelfth white point 46, the thirteenth white point 47 and the fourteenth white point 48 based on the offset of each color temperature white point in the CIE 1931 chromaticity space compared to the normal direction of the second curve 41, and then determines the color temperature values ​​corresponding to the eleventh white point 45, the twelfth white point 46, the thirteenth white point 47 and the fourteenth white point 48.

[0113] Furthermore, the method of the embodiment of the present application, by rendering the image to be processed into a rendered image with the same color temperature value as that corresponding to these color temperature white points, helps the subsequent second neural network model to interpolate more different color temperature values ​​and their corresponding fusion ratios, helps to improve the effect of color temperature adjustment, and provides users with a better shooting experience.

[0114] Among them, in order to ensure the color temperature adjustment effect of the embodiment of the present application, the technician needs to set at least 3 different standard color temperature values ​​(corresponding to lower color temperature values, medium color temperature values ​​and higher color temperature values, respectively), which helps to cover the color temperature values ​​corresponding to various light sources that users may encounter in daily life. It should be noted that the technician can set m different offset color temperature values ​​(i.e., the color temperature values ​​corresponding to the newly added color temperature white point) according to actual needs, and this application does not limit this.

[0115] In some other possible embodiments, the number of the first neural network models may be 1 or i; the method may further include:

[0116] When the number of the first neural network model is 1, the image to be processed is input into the first neural network model, and i first color temperature rendering images corresponding to different color temperature values ​​are generated;

[0117] When the number of first neural network models is i, based on the fact that each first neural network model corresponds to a rendering color temperature value, the image to be processed is respectively input into each of the i first neural network models to generate a first color temperature rendering image corresponding to the rendering color temperature value through each first neural network model.

[0118] For example, see Figure 5a-Figure 5b , Figure 5a-Figure 5b A rendering schematic diagram of a first neural network model provided in an embodiment of the present application.

[0119] like Figure 5a As shown, when the number of first neural network models is 1, after the image to be processed is input into the first neural network model, the first neural network model will generate i first color temperature rendering images corresponding to different color temperature values.

[0120] like Figure 5b As shown, when the number of first neural network models is i, each first neural network model corresponds to a rendering color temperature value, such as Figure 5b The low first neural network model, the medium first neural network model and the high first neural network model in the example, it can be considered that i=3, n=3, and m=0 at this time. At this time, the image to be processed needs to be input into the low first neural network model, the medium first neural network model and the high first neural network model respectively, and each first neural network model generates a rendering image with a corresponding rendering color temperature value (such as a low color temperature rendering image corresponding to the low first neural network model, a medium color temperature rendering image corresponding to the medium first neural network model, and a high color temperature rendering image corresponding to the high first neural network model).

[0121] In some possible implementations, before generating i first color temperature rendering images of the image to be processed based on the image to be processed, the method may further include:

[0122] In response to an operation instruction of a user, an image to be processed is determined, and the operation instruction may include a shooting instruction and a repair instruction.

[0123] Furthermore, when the operation instruction is a shooting instruction, the method may further include:

[0124] Responding to a shooting instruction from a user, acquiring a RAW image;

[0125] Perform local white balance processing on the RAW image to generate an image to be processed.

[0126] Among them, after the electronic device acquires the RAW image in response to the user's shooting instruction, it will perform local white balance processing on the RAW image and generate an image to be processed, so that the subsequent color temperature adjustment step can be performed. After the RAW image is locally white balanced, it helps to adjust the color of the image to a degree close to the real color of the object, correct the color temperature to a certain extent, and preliminarily correct the color cast caused by ambient light to the image. Compared with the rough adjustment method provided in the first aspect, the method provided in the first aspect is a fine adjustment. The coarse adjustment of this embodiment helps to improve the color temperature adjustment effect of the method provided in the first aspect, so that the fused image is closer to the user's observation effect, and improves the user's shooting experience.

[0127] In some other possible implementations, it is assumed that the electronic device acquires the RAW image in response to the user's shooting instruction as an image acquired in a mixed light source scene. If only the RAW image is processed globally with white balance, it is easy to cause the problem of uneven color temperature distribution of the image. For example, if the user shoots indoors at noon (there is a warm light source indoors, and the outdoor sunlight has a higher color temperature value than the warm light source indoors), it may cause the white wall and glass by the window in the captured image to appear bluish in tone; if the user uses a camera to shoot a white luminous display screen indoors (there is a warm light source indoors), it may also cause the display screen in the captured image to appear bluish in tone. However, global white balance cannot effectively eliminate the color temperature difference in the captured image, resulting in an image that is not beautiful enough, affecting the user's shooting experience. However, the method of the embodiment of the present application can perform color temperature adjustment on the captured image after the global white balance operation. By fusing the rendered images of different color temperature values ​​corresponding to the captured image in a certain proportion, a fused image with balanced color temperature can be obtained, so that the fused image is closer to the user's observation effect and improves the user's shooting experience.

[0128] Furthermore, in the case where the operation instruction is a repair instruction, the method may further include:

[0129] In response to a repair instruction from a user, an image corresponding to the repair instruction is determined as an image to be processed.

[0130] S202: Determine a fusion weight corresponding to each of the i first color temperature rendering images.

[0131] Specifically, the electronic device 100 can input the image to be processed into the first neural network model, and output i first color temperature rendered images of the image to be processed.

[0132] S203: Generate a first fused image based on the i first color temperature rendering images and the fusion weights corresponding to each first color temperature rendering image.

[0133] See also Figure 6a-6d , Figure 6a-6d A schematic diagram of an image processing scenario based on color temperature rendering provided in an embodiment of the present application.

[0134] When the user's operation instruction is a shooting instruction, you can refer to Figure 6a-6b The corresponding scene diagram. Figure 6a As shown, assume that there is a mixed light source scene with a first light source 61 (set as a warm color temperature light source), a second light source 62 (set as a cold color temperature light source) and a third light source 63 (set as a cold color temperature light source). In this mixed light source scene, the electronic device 100 responds to the user's shooting instruction, obtains a RAW image, and performs local white balance processing on the RAW image to generate an image to be processed. Since the local white balance processing cannot adjust the color temperature value of the entire RAW image to a balanced state, the local color temperature may be low or high in the image to be processed. For example, the first area 64 of the image to be processed will present an image effect with a low color temperature under the influence of the first light source 61, the second area 65 will present an image effect with a high color temperature under the influence of the second light source 62, and the third area 66 will present an image effect with a high color temperature under the influence of the third light source 63. Therefore, it is necessary to continue to perform the color temperature adjustment operation on the image to be processed. Through the method of the embodiment of the present application, a fused image can be generated by fusing rendered images of different color temperature values ​​corresponding to the image to be processed in a certain proportion. Furthermore, as Figure 6b As shown, the electronic device 100 can also present the fused image on the display screen of the electronic device 100 in response to the user's viewing instructions, which helps to achieve the purpose of balancing the image color temperature value, so that the image finally presented to the user is closer to the effect observed by the user's human eyes.

[0135] Furthermore, when the user's operation instruction is a repair instruction, you can refer to Figure 6c-6d The corresponding scene diagram. Figure 6c As shown, the user can select the image to be repaired in the gallery application of the electronic device 100 and select the "repair" function, then the electronic device 100 can determine the image to be repaired as the above-mentioned image to be processed, and perform a color temperature adjustment operation on it, and generate a repaired image. Further, the electronic device 100 can replace the image to be repaired in the gallery with the repaired image, and Figure 6d is presented to the user.

[0136] It can be seen that the method of the embodiment of the present application can perform fusion operations on rendered images of different color temperature values ​​based on different fusion ratios, which helps to make the image presented to the user in a color temperature balanced state, avoid the phenomenon that some objects in the image are blue or yellow, and provide users with a good shooting experience.

[0137] In some other possible implementations, the method of the embodiment of the present application needs to establish and train a first neural network model and a second neural network model before inputting the image to be processed into the first neural network model.

[0138] Among them, see Figure 7 , Figure 7 A schematic diagram of a process for training a second neural network model provided in an embodiment of the present application.

[0139] like Figure 7 As shown, the process of training the second neural network model may include:

[0140] First, the first training image is input into the first neural network model, and the first neural network model generates i second color temperature rendering images corresponding to different color temperature values. Then, the i second color temperature rendering images corresponding to different color temperature values ​​are input into the second neural network model to determine the fusion weights corresponding to each second color temperature rendering image. Among them, the technician can set an initial weight, and the second neural network model can automatically iterate the initial weight according to the training data until it iterates out the appropriate fusion weight corresponding to each color temperature value.

[0141] Further, based on each second color temperature rendering image and the fusion weights corresponding to each second color temperature rendering image, a second fused image is generated, and based on the second fused image and the first standard output image corresponding to the first training image, an error matrix is ​​generated. The error matrix represents the color temperature difference between the corresponding pixels of the second fused image and the first standard output image.

[0142] Optionally, the first standard output image can be obtained from the first training image PS, or calculated from a standard white balance map corresponding to the first training image. For example, when the first training image is an image taken in a single light source scene, a first standard output image corresponding to the first training image can be produced by a simple PS operation; when the first training image is an image taken in a mixed light source scene, the PS operation cannot produce an accurate first standard output image, and the first standard output image can be generated based on the standard white balance map corresponding to the first training image, which is not elaborated in this application.

[0143] Furthermore, based on the second fused image and the error matrix, the second neural network model is trained so that the second neural network model updates the fusion weights corresponding to i different color temperature values.

[0144] In some other possible implementations, before inputting the first training image into the first neural network model and generating i second color temperature rendering images corresponding to different color temperature values, the method may further include:

[0145] Based on the second training image and the second standard output image, determining i pairs of training data pairs, where the second standard output image may include rendered images of the second training image corresponding to i different color temperature values;

[0146] The first neural network model is trained with i pairs of training data.

[0147] Wherein, determining i pairs of training data pairs based on the second training image and the second standard output image may include:

[0148] The second training image and the second standard output image corresponding to any one of the i different color temperature values ​​are taken as a data pair.

[0149] Exemplarily, assuming that the second training image has three standard output images (respectively, a low color temperature output image, a medium color temperature output image, and a high color temperature output image), three pairs of training data pairs, namely, "second training image-low color temperature output image", "second training image-medium color temperature output image" and "second training image-high color temperature output image", can be determined, which are used to train the first neural network model to generate three output images (respectively, a low color temperature output image, a medium color temperature output image, and a high color temperature output image) for a single input image, which helps the first neural network model to subsequently generate multiple rendered images corresponding to different color temperature values ​​based on a single image to be processed.

[0150] It should be noted that there are several first training images and second training images mentioned above. The embodiment of the present application is only to explain in detail the main training steps of training the first neural network model and the second neural network model, so only an example of a single training data is used for illustration. The above example should not constitute a limitation to the present application.

[0151] Combine the following Figure 8-Figure 10 An electronic device 100 according to an embodiment of the present application is introduced.

[0152] See also Figure 8 , Figure 8 A schematic diagram of the composition of an electronic device 100 provided in an embodiment of the present application.

[0153] like Figure 8 As shown, the electronic device 100 may include: an image processing module 810;

[0154] The image processing module 810 may be used to generate i first color temperature rendering images of the image to be processed based on the image to be processed, each of the i first color temperature rendering images corresponds to a different color temperature value, i≥3, and i is a positive integer;

[0155] The image processing module 810 may also be used to determine a fusion weight corresponding to each of the i first color temperature rendering images;

[0156] The image processing module 810 may also be configured to generate a first fused image based on i first color temperature rendered images and fusion weights corresponding to each first color temperature rendered image.

[0157] In some possible implementations, the electronic device may further include:

[0158] The image processing module 810 can also be used to input the image to be processed into the first neural network model and output i first color temperature rendered images of the image to be processed.

[0159] In some other possible implementations, the electronic device may further include:

[0160] The image processing module 810 may also be used to input i first color temperature rendering images into the second neural network model, and output fusion weights corresponding to each first color temperature rendering image.

[0161] In some other possible implementations, the i different color temperature values ​​may include n different standard color temperature values ​​and m different offset color temperature values;

[0162] The standard color temperature value is a different color temperature value located on the Planck curve, 3≤n≤i, n is a positive integer;

[0163] The offset color temperature value is a color temperature value that is outside the Planck curve and is different from each other, n+m=i, m≥0, and m is an integer.

[0164] In some other possible implementations, the electronic device may further include: a control module 820;

[0165] The control module 820 may be used to determine the image to be processed in response to the user's operation instruction, and the operation instruction may include a shooting instruction and a repair instruction.

[0166] In some other possible implementations, the electronic device may further include: an image acquisition module 830;

[0167] The image acquisition module 830 may be used to acquire a RAW image in response to a user's shooting instruction;

[0168] The image processing module 810 may also be used to perform local white balance processing on the RAW image to generate an image to be processed.

[0169] In some other possible implementations, the electronic device may further include:

[0170] The control module 820 may also be configured to respond to a repair instruction from a user and determine an image corresponding to the repair instruction as an image to be processed.

[0171] In some other possible implementations, the second neural network model is a model obtained by training using i second color temperature rendered images of the first training image as training samples and using the first standard output image corresponding to the first training image as an output label;

[0172] The i second color temperature rendered images of the first training image are output by the first neural network model based on the first training image.

[0173] In some other possible implementations, the first neural network model is a model obtained by training with the second training image as a training sample and the second standard output image corresponding to the second training image as an output label;

[0174] The second standard output image includes rendered images of i different color temperature values ​​corresponding to the second training image;

[0175] The second training image and the second standard output image corresponding to any one of the i different color temperature values ​​are used as a pair of training data.

[0176] In some other possible implementations, the number of the first neural network models is 1 or i;

[0177] The electronic device may also include:

[0178] The image processing module 810 may also be used to input the image to be processed into the first neural network model when the number of the first neural network models is 1, and generate i first color temperature rendering images corresponding to different color temperature values;

[0179] The image processing module 810 can also be used to input the image to be processed into each of the i first neural network models respectively, based on the fact that each first neural network model corresponds to a rendering color temperature value, when the number of first neural network models is i, so as to generate a first color temperature rendering image corresponding to the rendering color temperature value through each first neural network model.

[0180] For further information, see Fig. 9 , Fig. 9 A schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of the present application is shown.

[0181] The electronic device 100 may include a processor 101, a memory 102, a wireless communication module 103, a mobile communication module 104, an antenna 103A, an antenna 104A, a power switch 105, a sensor module 106, a focus motor 107, a camera 108, a display screen 109, etc. Among them, the sensor module 106 may include a gyroscope sensor 106A, an acceleration sensor 106B, an ambient light sensor 106C, an image sensor 106D, a distance sensor 106E, etc. Among them, the wireless communication module 103 may include a WLAN communication module, a Bluetooth communication module, etc. The above multiple parts can transmit data through a bus.

[0182] The processor 101 may include one or more processing units, for example, the processor 101 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0183] The memory 102 may be used to store computer executable program codes, and the executable program codes may include instructions. The processor 101 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the memory 102. The memory 102 may include a program storage area and a data storage area. In a specific implementation, the memory 102 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices.

[0184] The wireless communication function of the electronic device 100 can be implemented through the antenna 103A, the antenna 104A, the mobile communication module 104, the wireless communication module 103, the modem processor and the baseband processor.

[0185] Antenna 103A and antenna 104A can be used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas.

[0186] The mobile communication module 104 can provide wireless communication solutions including 2G / 3G / 4G / 5G etc. applied on the electronic device 100 .

[0187] The wireless communication module 103 can provide wireless communication solutions including wireless local area networks (WLAN), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. for application in the electronic device 100.

[0188] The gyro sensor 106A may be used to determine the motion posture of the electronic device 100 .

[0189] The acceleration sensor 106B can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes).

[0190] The electronic device 100 can realize the shooting function through the ISP, the camera 108, the video codec, the GPU, the display screen 109 and the application processor.

[0191] The electronic device 100 can realize the display function through a GPU, a display screen 109, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 109 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 101 may include one or more GPUs, which execute program instructions to generate or change display information.

[0192] The display screen 109 is used to display images, videos, etc. The display screen 109 includes a display panel. In some embodiments, the electronic device 100 may include 1 or N display screens 109, where N is a positive integer greater than 1.

[0193] The structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0194] In the embodiments of the present application:

[0195] Any one or more of the wireless communication module 103 , the mobile communication module 104 , the sensor module 106 , the focus motor 107 , the camera 108 , etc. may be used to detect the current environmental characteristics of the electronic device 100 .

[0196] The display screen 109 is used to display the user interface provided by the electronic device 100, such as a photo taking interface or an album interface. The user interface displayed by the display screen 109 can refer to the UI embodiment described above.

[0197] For details of the operations performed by the various components in the electronic device 100, please refer to the relevant description of the above method embodiments.

[0198] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The present application embodiment takes a mobile operating system with a layered architecture as an example to exemplify the software structure of the electronic device 100.

[0199] It is understandable that the software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture, and this application does not limit this. Fig.10 , Fig.10 This is a software structure diagram provided by the embodiment of the present application. Fig.10 The layered architecture shown divides the software into several layers. Each layer has a clear role and division of labor, and the layers can communicate through software interfaces.

[0200] In some embodiments, the system of the electronic device can be divided into five layers, from top to bottom, namely, the application layer, the application framework layer, the system runtime layer, the hardware abstraction layer (HAL) and the kernel layer. The description of each of the above layers is as follows:

[0201] The application layer may include a series of application packages. For example, the application packages of the application layer may include applications such as camera, gallery, calendar, call, map, navigation, browser, video, music, and short message.

[0202] The application framework layer can provide an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer can include some predefined functions.

[0203] Exemplarily, the application framework layer may include an activity manager, a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and an accelerated graphics port (AGP). Among them:

[0204] The activity manager can be used to manage the individual application lifecycles as well as general navigation back functionality.

[0205] The window manager can be used to manage window programs. For example, the window manager can obtain the display screen size of the electronic device 100, lock the screen, capture the screen, and determine whether there is a status bar.

[0206] Content providers can be used to store and retrieve data and make the data accessible to applications so that different applications can access or share data. For example, the data may include videos, images, audio, calls made and received, browsing history and bookmarks, and phone books.

[0207] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0208] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering a call, hanging up a call, etc.).

[0209] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, etc.

[0210] The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager can be used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the top status bar of the system in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0211] AGP in the application framework layer can be used to improve the rendering performance of the graphics card, such as providing the graphics card with more cache capacity to achieve faster image processing speed.

[0212] The system runtime layer may include system libraries and Android runtime.

[0213] The Android runtime includes a core library and a virtual machine. The Android runtime is responsible for scheduling and management of the Android system. The core library consists of two parts: one is the function that the Java language needs to call, and the other is the Android core library. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0214] The system library can be understood as the support of the application framework, and is an important link between the application framework layer and the kernel layer. The system layer can include multiple functional modules, such as surface managers, media libraries, 3D graphics processing libraries (such as OpenGL ES), 2D graphics engines (such as SGL), etc. Among them:

[0215] The surface manager can be used to manage the display subsystem, for example, when the electronic device 100 executes multiple applications, it is responsible for managing the interaction between display and access operations. The surface manager can also be used to provide fusion of two-dimensional and three-dimensional layers for multiple applications.

[0216] The media library can support playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0217] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing.

[0218] A two-dimensional graphics engine can be understood as a drawing engine for two-dimensional drawing.

[0219] The hardware abstraction layer provides a standard interface, such as the HAL interface definition language (HIDL) interface or the Android interface definition language (AIDL) interface.

[0220] The kernel layer can be understood as an abstract layer between hardware and software. The kernel layer may include system services such as security, memory management, process management, power management, network protocol management, and driver management. Exemplarily, the kernel layer may include drivers, such as display drivers, camera drivers, audio drivers, and sensor drivers. Optionally, the kernel layer may also be called an Android kernel or kernel program. Exemplarily, the hardware layer of an electronic device may include a touch panel (TP), a liquid crystal display (LCD), and the like.

[0221] In some embodiments, the kernel layer can be understood as kernel state, and the other four layers (ie, application layer, application framework layer, system runtime layer, and HAL) can be understood as user state.

[0222] It is understandable that multiple applications that may exist in the application layer have video recording and / or playback functions. For example, if a camera has a video recording function, applications such as gallery, navigation, and browser may have a video playback function. Furthermore, based on the length, resolution, and shooting parameters of the video, the recording and / or playback of the video may be affected. The shooting parameters may include aperture value, shutter speed, sensitivity, exposure, focal length, depth of field, and the like.

[0223] For example, the video recording and / or playback related parameters supported by various applications can be stored in a sensor binary file (sensor bin), and the sensor bin file can be stored in an external memory, such as a disk or a hard disk. When a user uses a specific application to record and / or play a video, the application can read the video recording and / or playback related parameters into the memory so that the user can use the application normally and ensure the user's experience.

[0224] It should be understood that each step in the above method embodiment can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0225] The present application also provides an electronic device, which may include: a memory and a processor. The memory may be used to store a computer program; the processor may be used to call the computer program in the memory so that the electronic device executes the method executed by the electronic device side in any of the above embodiments.

[0226] The present application also provides a chip system, which includes at least one processor for implementing the functions involved in the electronic device side in any of the above embodiments.

[0227] In some possible designs, the chip system also includes a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

[0228] The chip system may be composed of the chip, or may include the chip and other discrete devices.

[0229] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0230] Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

[0231] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0232] The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instruction), which, when executed, enables a computer to execute the method executed by the electronic device side in any of the above embodiments.

[0233] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes the method executed by the electronic device side in any of the above embodiments.

[0234] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0235] The various implementation modes of the present application can be combined arbitrarily to achieve different technical effects.

[0236] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)), etc.

[0237] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0238] In short, the above description is only an embodiment of the technical solution of this application, and is not intended to limit the protection scope of this application. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of this application shall be included in the protection scope of this application.

Claims

1. An image processing method based on color temperature rendering, characterized in that: The method comprises: Based on the image to be processed, i first color temperature rendering images of the image to be processed are generated, each of the i first color temperature rendering images corresponds to a different color temperature value, i≥3, and i is a positive integer; Determine a fusion weight corresponding to each first color temperature rendering image in the i first color temperature rendering images; A first fused image is generated based on the i first color temperature rendered images and the fusion weights corresponding to each first color temperature rendered image.

2. The method according to claim 1, characterized in that The step of generating i first color temperature rendering images of the image to be processed based on the image to be processed includes: The image to be processed is input into the first neural network model, and i first color temperature rendered images of the image to be processed are output.

3. The method according to claim 2, characterized in that The determining a fusion weight corresponding to each of the i first color temperature rendered images includes: The i first color temperature rendering images are input into a second neural network model, and the fusion weights corresponding to the i first color temperature rendering images are output.

4. The method according to claim 3, characterized in that The i different color temperature values ​​include n different standard color temperature values ​​and m different offset color temperature values; The standard color temperature values ​​are different color temperature values ​​located on the Planck curve, 3≤n≤i, n is a positive integer; The offset color temperature value is a color temperature value that is outside the Planck curve and is different from each other, n+m=i, m≥0, and m is an integer.

5. The method according to any one of claims 1 to 4, characterized in that: Before generating i first color temperature rendering images of the image to be processed based on the image to be processed, the method further includes: In response to an operation instruction of a user, the image to be processed is determined, wherein the operation instruction includes a shooting instruction and a repair instruction.

6. The method according to claim 5, characterized in that In the case where the operation instruction is the shooting instruction, the method further includes: In response to the shooting instruction of the user, acquiring a RAW image; Performing local white balance processing on the RAW image to generate the image to be processed.

7. The method according to claim 6, characterized in that In the case where the operation instruction is the repair instruction, the method further includes: In response to the user's repair instruction, an image corresponding to the repair instruction is determined as an image to be processed.

8. The method according to claim 7, characterized in that The second neural network model is a model obtained by training using i second color temperature rendered images of the first training image as training samples and using the first standard output image corresponding to the first training image as an output label; The i second color temperature rendered images of the first training image are output by the first neural network model based on the first training image.

9. The method according to any one of claim 8, characterized in that: The first neural network model is a model obtained by training with the second training image as a training sample and the second standard output image corresponding to the second training image as an output label; The second standard output image includes rendered images of i different color temperature values ​​corresponding to the second training image; The second training image and the second standard output image corresponding to any one of the i different color temperature values ​​serve as a pair of training data.

10. The method according to claim 9, characterized in that The number of the first neural network models is 1 or i; The method further comprises: When the number of the first neural network models is 1, inputting the image to be processed into the first neural network model, and generating i first color temperature rendering images corresponding to different color temperature values; When the number of the first neural network models is i, based on the fact that each first neural network model corresponds to a rendering color temperature value, the image to be processed is respectively input into each of the i first neural network models to generate a first color temperature rendering image corresponding to the rendering color temperature value through each first neural network model.

11. An electronic device, characterized in that: The electronic device comprises: an image processing module; The image processing module is used to input the image to be processed into the first neural network model, and generate i first color temperature rendering images corresponding to different color temperature values, the shooting scene corresponding to the image to be processed is a mixed light source scene, there are at least two light sources in the mixed light source scene, i≥3, i is a positive integer; The image processing module is further used to input the i first color temperature rendering images corresponding to different color temperature values ​​into the second neural network model to determine the fusion weight corresponding to each of the first color temperature rendering images; The image processing module is further configured to generate a first fused image based on each of the first color temperature rendered images and a fusion weight corresponding to each of the first color temperature rendered images.

12. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions so that the method according to any one of claims 1 to 10 is executed.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the method according to any one of claims 1 to 10 is executed.

Images