Image processing method and device and storage medium

Abstract

The invention discloses an image processing method and device and a storage medium, and belongs to the technical field of shooting. The method is applied to the electronic equipment comprising a plurality of cameras, the plurality of cameras comprise a reference camera, and the method comprises the following steps: under the condition that a first camera is called for shooting, determining a first white point of a current frame shot by the first camera and a first environment color temperature of a shooting environment corresponding to the current frame; according to an environment color temperature difference between the first environment color temperature and a second environment color temperature of a shooting environment corresponding to a memory frame of a reference camera, carrying out fusion processing on white point information of the first white point and white point information of a second white point of the memory frame to obtain white point information of a third white point; and performing white balance correction on the current frame according to the white point information of the third white point. Therefore, the color difference of the images shot by other cameras and the reference camera in the same scene and the visual difference caused by the color difference can be reduced, and the color consistency of multi-camera shooting in the same scene is improved.

Description

Technical Field

[0001] This application relates to the field of shooting technologies, and in particular, to an image processing method, device, and storage medium. Background Art

[0002] With the improvement of users' shooting requirements, the camera modules of electronic devices such as mobile phones have developed from single cameras to multi-cameras, that is, an electronic device is equipped with multiple cameras, such as a main camera, a telephoto camera, and an ultra-wide-angle camera, etc. Users can select any camera for shooting according to their shooting needs. For example, the main camera is selected by default for shooting, the telephoto camera is selected for shooting distant objects, and the ultra-wide-angle camera is selected for shooting large-scale buildings or landscapes.

[0003] However, since the sensors of each camera may respond differently to colors, there may be a certain deviation in the colors of the images captured by different cameras in the same scene among these multiple cameras, which affects the user's shooting experience. For example, in the same scene, the user first selects the main camera for shooting and then switches to the telephoto camera for shooting. However, there may be a large difference in the colors of the images captured before and after the switch. For example, the image captured by the main camera is yellowish, while the image captured by the telephoto camera is bluish, resulting in inconsistent colors of the images captured by the user for the same scene and affecting the user's shooting experience. Summary of the Invention

[0004] This application provides an image processing method, device, and storage medium, which can reduce the color difference in multi-camera shooting in the same scene and improve the user's shooting experience. The technical solutions are as follows:

[0005] In a first aspect, an image processing method is provided, which is applied to an electronic device. The electronic device includes multiple cameras, and the multiple cameras include a reference camera. The method includes:

[0006] When calling the first camera for shooting, determine the first white point of the current frame captured by the first camera and the first ambient color temperature of the shooting environment corresponding to the current frame. The first camera is any one of the multiple cameras other than the reference camera; according to the ambient color temperature difference between the first ambient color temperature and the second ambient color temperature of the shooting environment corresponding to the memory frame of the reference camera, perform fusion processing on the white point information of the first white point and the white point information of the second white point of the memory frame to obtain the white point information of the third white point; according to the white point information of the third white point, perform white balance correction on the current frame.

[0007] In the embodiments of the present application, a certain camera among multiple cameras configured in an electronic device can be set as a reference camera in advance, and the white point information of the second white point of the memory frame (i.e., the last frame captured) of the reference camera and the second ambient color temperature of the shooting environment corresponding to the memory frame can be stored, so as to perform color correction on the images captured by other cameras based on the relevant information of the memory frame of the reference camera. For example, when the electronic device calls any camera other than the reference camera for shooting, the white point information of the first white point of the current frame captured by the other camera and the first ambient color temperature of the shooting environment corresponding to the current frame can be obtained. Then, according to the difference between the second ambient color temperature of the shooting environment corresponding to the memory frame of the reference camera and the first ambient color temperature, the white point information of the first white point of the current frame and the white point information of the second white point of the memory frame are fused, and the current frame is corrected for white balance according to the fused white point information.

[0008] In this way, when the ambient color temperature difference between the other camera and the reference camera is small, it can be determined that the shooting scene difference is small, and it is likely to be shooting in the same scene. By combining the white point information of the memory frame of the reference camera and the white point information of the current frame, the current frame is color-corrected by performing white balance correction on the current frame, so as to reduce the color difference between the current frame and the memory frame of the reference camera, thereby reducing the color difference between the images captured by the other camera and the reference camera in the same scene and the resulting visual difference, improving the color consistency of multi-camera shooting in the same scene, and further improving the user's visual experience and shooting experience.

[0009] As an example, the fusion weight can be determined according to the ambient color temperature difference; according to the fusion weight, the white point information of the first white point and the white point information of the second white point are fused to obtain the white point information of the third white point. Among them, the smaller the ambient color temperature difference, the greater the fusion weight.

[0010] Among them, the ambient color temperature difference between the first ambient color temperature and the second ambient color temperature can represent the environmental difference between the current shooting environment and the shooting environment of the memory frame. The ambient color temperature difference is inversely proportional to the fusion weight, and the smaller the ambient color temperature difference, the greater the fusion weight.

[0011] Thus, when the ambient color temperature difference is small, it indicates that the difference between the current shooting environment and the shooting environment of the memory frame is small. In this case, by fusing the first white point of the memory frame to a greater extent according to a larger fusion weight, a greater degree of color correction can be performed on the current frame, so that the color of the corrected image of the current frame is closer to the color of the image of the memory frame to a greater extent; when the ambient color temperature difference is large, it indicates that the difference between the current shooting environment and the shooting environment of the memory frame is large. In the case of a large difference in the shooting environment, it is normal for the current frame and the memory frame to have a color difference. In this case, a smaller degree of color correction can be performed on the current frame by fusing the first white point of the memory frame to a smaller extent according to a smaller fusion weight.

[0012] As an example, before fusing the white point information of the first white point and the white point information of the second white point according to the fusion weight, the second white point can be mapped to the first camera to obtain a mapped white point. Then, according to the fusion weight, the white point information of the first white point and the white point information of the mapped white point are fused to obtain the white point information of the third white point.

[0013] By first mapping the second white point of the memory frame of the reference camera to the first camera to obtain a mapped white point, and then fusing the white point information of the mapped white point with the white point information of the first white point of the current frame, the accuracy of multi-camera color correction can be further improved.

[0014] As an example, the operation of mapping the second white point to the first camera to obtain a mapped white point may include: obtaining the calibration data of the reference camera and the first camera, where the calibration data includes the white point information and color temperature under multiple standard light sources; determining at least one standard light source from multiple standard light sources according to the color temperature difference from the estimated color temperature of the memory frame; determining the white point mapping matrix between the reference camera and the first camera according to the white point information of the reference camera under at least one standard light source and the white point information of the first camera under at least one standard light source; and mapping the second white point to the first camera according to the white point mapping matrix to obtain a mapped white point.

[0015] As an example, before determining the fusion weight according to the ambient color temperature difference, the first estimated color temperature of the current frame can be obtained first. Then, according to the ambient color temperature difference, the first sub-weight is determined; according to the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature of the memory frame, the second sub-weight is determined; and the fusion weight is determined according to the first sub-weight and the second sub-weight. Among them, the smaller the ambient color temperature difference, the larger the first sub-weight. The larger the estimated color temperature difference, the larger the second sub-weight.

[0016] Among them, the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature can indicate the shooting effect difference between the first camera and the reference camera. The estimated color temperature difference is proportional to the second sub-weight. The larger the estimated color temperature difference, the larger the second sub-weight.

[0017] In this way, when the estimated color temperature difference is small, it means that the shooting effect difference between the first camera and the reference camera is small. In this case, a small degree of color correction can be performed on the current frame according to the small second sub-weight. When the estimated color temperature difference is large, it means that the shooting effect difference between the first camera and the reference camera is large. In this case, a large degree of color correction is performed on the current frame according to the large second sub-weight to improve the shooting effect difference.

[0018] By combining the ambient color temperature difference and the estimated color temperature difference to determine the fusion weight, the accuracy of image correction can be further improved.

[0019] As an example, before determining the fusion weight according to the first sub-weight and the second sub-weight, it may further include: determining a third sub-weight according to the ambient brightness information of the shooting environment corresponding to the memory frame. Then, the fusion weight is determined according to the first sub-weight, the second sub-weight, and the third sub-weight. Among them, the larger the brightness indicated by the ambient brightness information, the larger the third sub-weight.

[0020] Among them, the ambient brightness information of the shooting environment corresponding to the memory frame is used to indicate the ambient brightness of the shooting environment corresponding to the memory frame, such as brightness parameters such as LV. Generally, the smaller the ambient brightness, the more complex the shooting environment and the worse the shooting effect. The larger the ambient brightness, the better the shooting effect. The brightness indicated by the ambient brightness information is proportional to the third sub-weight. The larger the brightness indicated by the ambient brightness information, the larger the third sub-weight.

[0021] In this way, when the brightness indicated by the ambient brightness information is smaller, the image effect of the memory frame of the reference camera is worse. In this case, a small degree of color correction can be performed on the current frame according to the small third sub-weight. When the brightness indicated by the ambient brightness information is larger, it means that the image effect of the memory frame of the reference camera is better. In this case, a large degree of color correction can be performed on the current frame according to the large third sub-weight. Adopting this strategy can ensure the accuracy of color correction.

[0022] By combining the ambient color temperature difference, the estimated color temperature difference, and the ambient brightness information corresponding to the memory frame to determine the fusion weight, the accuracy of image correction can be further improved.

[0023] As an example, the operation of determining the fusion weight according to the first sub-weight, the second sub-weight, and the third sub-weight may include: determining the product among the first sub-weight, the second sub-weight, and the third sub-weight as the fusion weight. For example, the fusion weight, the white point information of the first white point, and the white point information of the mapped white point satisfy the following formula: the white point information of the third white point = the white point information of the first white point * (1 - W) + the white point information of the mapped white point * W; where W is the fusion weight.

[0024] As an example, the operation of performing white balance correction on the current frame according to the white point information of the third white point may include: determining the white balance gain according to the white point information of the third white point; and correcting the current frame according to the white balance gain.

[0025] As an example, before determining the white balance gain according to the white point information of the third white point, it may further include: performing a fusion process on the first estimated color temperature of the current frame and the second estimated color temperature of the memory frame corresponding to the shooting environment to obtain a third estimated color temperature; after performing white balance correction on the current frame according to the third white point information, it may further perform color correction on the current frame after white balance correction according to the third estimated color temperature. In this way, the accuracy of color correction can be improved, thereby further improving the color consistency of multi-camera images.

[0026] In a second aspect, there is provided an image processing method applied to an electronic device. The electronic device includes a first camera, and the first camera supports multiple magnification factors. The multiple magnification factors include a reference magnification factor, and the output image mode corresponding to the reference magnification factor is a first output image mode. The method includes: when the first camera shoots at a first magnification factor, determining a first white point of the current frame shot by the first camera at the first magnification factor, and a first environmental color temperature of the shooting environment corresponding to the current frame. The first magnification factor is a magnification factor other than the reference magnification factor among the multiple magnification factors and the output image mode corresponding to it is different from the first output image mode; performing a fusion process on the white point information of the first white point and the white point information of the second white point of the memory frame according to the environmental color temperature difference between the first environmental color temperature and the second environmental color temperature of the shooting environment corresponding to the memory frame of the first camera, to obtain the white point information of the third white point. The memory frame refers to the last frame shot by the first camera at the reference magnification factor; performing white balance correction on the current frame according to the white point information of the third white point.

[0027] That is, for a certain camera configured in an electronic device that can support multiple magnification ratios, a certain magnification ratio supported by the camera can be set as a reference magnification ratio in advance, and the second white point of the memory frame of the reference magnification ratio (i.e., the last frame captured by the camera at the reference magnification ratio) and the second ambient color temperature of the shooting environment corresponding to the memory frame can be stored, so as to perform color correction on the images captured by the camera at other magnification ratios based on the relevant information of the memory frame. For example, when the electronic device calls any other camera other than the reference camera for shooting, the first white point of the current frame captured by the other camera and the first ambient color temperature of the shooting environment corresponding to the current frame can be obtained. Then, according to the difference between the second ambient color temperature of the shooting environment corresponding to the memory frame of the reference camera and the first ambient color temperature, the white point information of the first white point of the current frame and the white point information of the second white point of the memory frame are fused, and the current frame is corrected for white balance according to the fused white point information.

[0028] In this way, when the ambient color temperature of the same camera at other magnification ratios is close to that at the reference magnification ratio, it can be determined that the shooting scene difference is small, and it is likely to be shooting in the same scene. By combining the white point information of the memory frame of the reference magnification ratio and the white point information of the current frame, color correction is performed on the current frame by correcting the white balance of the current frame, so as to reduce the color difference between the current frame and the memory frame at the reference magnification ratio, thereby reducing the color difference between the images captured by the same camera using other magnification ratios and the reference magnification ratio in the same scene and the resulting visual difference, improving the color consistency of the images captured by the same camera using different magnification ratios, and further improving the user's visual experience and shooting experience.

[0029] As an example, the fusion weight can be determined according to the ambient color temperature difference, and then, according to the fusion weight, the white point information of the first white point and the white point information of the second white point are fused to obtain the white point information of the third white point. Among them, the smaller the ambient color temperature difference, the greater the fusion weight.

[0030] Among them, the ambient color temperature difference between the first ambient color temperature and the second ambient color temperature can represent the environmental difference between the current shooting environment and the shooting environment of the memory frame. The ambient color temperature difference is inversely proportional to the fusion weight, and the smaller the ambient color temperature difference, the greater the fusion weight.

[0031] Thus, when the environmental color temperature difference is small, it indicates that the current shooting environment has a small difference from the shooting environment of the memory frame. In this case, by performing a greater degree of fusion on the first white point of the memory frame according to a larger fusion weight, a greater degree of color correction can be performed on the current frame, so that the color of the corrected image of the current frame is closer to the color of the image of the memory frame to a greater extent; when the environmental color temperature difference is large, it indicates that the current shooting environment has a large difference from the shooting environment of the memory frame. In the case of a large difference in the shooting environment, it is normal for the current frame and the memory frame to have a color difference. In this case, a smaller degree of fusion can be performed on the first white point of the memory frame according to a smaller fusion weight to perform a smaller degree of color correction on the current frame.

[0032] As an example, before determining the fusion weight according to the environmental color temperature difference, the first estimated color temperature of the current frame can also be obtained. Then, according to the environmental color temperature difference, the first sub-weight is determined; according to the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature of the memory frame, the second sub-weight is determined; according to the first sub-weight and the second sub-weight, the fusion weight is determined.

[0033] Among them, the greater the environmental color temperature difference, the smaller the first sub-weight; the greater the estimated color temperature difference, the greater the second sub-weight.

[0034] Among them, the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature can indicate the shooting effect difference between the first camera and the reference camera. The estimated color temperature difference is proportional to the second sub-weight, and the greater the estimated color temperature difference, the greater the second sub-weight.

[0035] Thus, when the estimated color temperature difference is small, it indicates that the shooting effect difference between the first camera and the reference camera is small. In this case, a smaller degree of color correction can be performed on the current frame according to the smaller second sub-weight. When the estimated color temperature difference is small, it indicates that the shooting effect difference between the first camera and the reference camera is large. In this case, a greater degree of color correction is performed on the current frame according to the larger second sub-weight to improve the shooting effect difference.

[0036] By combining the environmental color temperature difference and the estimated color temperature difference to determine the fusion weight, the accuracy of image correction can be further improved.

[0037] As an example, before determining the fusion weight according to the first sub-weight and the second sub-weight, the third sub-weight can also be determined according to the environmental brightness information of the shooting environment corresponding to the memory frame. Then, according to the first sub-weight, the second sub-weight and the third sub-weight, the fusion weight is determined. Among them, the greater the brightness indicated by the environmental brightness information, the greater the third sub-weight.

[0038] Among them, the ambient brightness information of the shooting environment corresponding to the memory frame is used to indicate the ambient brightness of the shooting environment corresponding to the memory frame, such as brightness parameters like LV. Generally, the smaller the ambient brightness, the more complex the shooting environment and the worse the shooting effect. The larger the ambient brightness, the better the shooting effect. The brightness indicated by the ambient brightness information is directly proportional to the third sub-weight. The larger the brightness indicated by the ambient brightness information, the larger the third sub-weight.

[0039] In this way, when the brightness indicated by the ambient brightness information is smaller, the image effect of the memory frame of the reference camera is worse. In this case, a smaller degree of color correction can be performed on the current frame according to the smaller third sub-weight. When the brightness indicated by the ambient brightness information is larger, it means that the image effect of the memory frame of the reference camera is better. In this case, a larger degree of color correction can be performed on the current frame according to the larger third sub-weight. Adopting this strategy can ensure the accuracy of color correction.

[0040] By combining the ambient color temperature difference, the estimated color temperature difference, and the ambient brightness information corresponding to the memory frame to determine the fusion weight, the accuracy of image correction can be further improved.

[0041] As an example, the operation of determining the fusion weight according to the first sub-weight, the second sub-weight, and the third sub-weight includes: determining the product among the first sub-weight, the second sub-weight, and the third sub-weight as the fusion weight. For example, the fusion weight, the white point information of the first white point, and the white point information of the second white point satisfy the following formula: the white point information of the third white point = the white point information of the first white point * (1 - W) + the white point information of the second white point * W; where W is the fusion weight.

[0042] As an example, the operation of performing white balance correction on the current frame according to the white point information of the third white point includes: determining the white balance gain according to the white point information of the third white point; and performing correction on the current frame according to the white balance gain.

[0043] As an example, before performing white balance correction on the current frame according to the white point information of the third white point, the first estimated color temperature of the current frame and the second estimated color temperature of the memory frame can also be fused to obtain a third estimated color temperature; after performing white balance correction on the current frame according to the third white point information, color correction can also be performed on the current frame after white balance correction according to the third estimated color temperature. In this way, the accuracy of color correction can be improved, thereby further improving the color consistency of multi-camera images.

[0044] In a third aspect, an image processing device is provided. The image processing device has the function of implementing the behavior of the image processing method in the first aspect above. The image processing device includes at least one module, and the at least one module is used to implement the image processing method provided in the first aspect or the second aspect above.

[0045] In a fourth aspect, an image processing apparatus is provided. The structure of the image processing apparatus includes a processor and a memory. The memory is used to store a program that supports the image processing apparatus to execute the image processing method provided in the first aspect or the second aspect, and to store data involved in implementing the image processing method described in the first aspect or the second aspect. The processor is configured to execute the program stored in the memory. The image processing apparatus may further include a communication bus, and the communication bus is used to establish a connection between the processor and the memory.

[0046] In a fifth aspect, a computer-readable storage medium is provided. Instructions are stored in the computer-readable storage medium, and when it runs on a computer, the computer is caused to execute the image processing method described in the first aspect or the second aspect.

[0047] In a sixth aspect, a computer program product including instructions is provided. When it runs on a computer, the computer is caused to execute the image processing method described in the first aspect or the second aspect.

[0048] The technical effects obtained in the above third aspect, fourth aspect, fifth aspect and sixth aspect are similar to the technical effects obtained by the corresponding technical means in the above first aspect or second aspect, and will not be elaborated here. Description of the Drawings

[0049] Figure 1 is a schematic diagram of the spatial distribution of the cameras of an electronic device provided in an embodiment of the present application;

[0050] Figure 2 is a schematic diagram of the color comparison of the images captured by different cameras in the same scene provided in an embodiment of the present application;

[0051] Figure 3 is a schematic diagram of the comparison of the output images of the same camera at 1X and 2X magnification provided in an embodiment of the present application;

[0052] Figure 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0053] Figure 5 is a block diagram of the software system of an electronic device provided in an embodiment of the present application;

[0054] Figure 6 is a logical schematic diagram of an algorithm library provided in an embodiment of the present application;

[0055] Figure 7 is a schematic flowchart of a multi-camera color correction algorithm provided in an embodiment of the present application;

[0056] Figure 8 It is a schematic diagram for mapping the white point of the memory frame of a reference camera to a first camera provided by an embodiment of the present application;

[0057] Figure 9 It is a schematic diagram of a multi-camera color correction scenario provided by an embodiment of the present application;

[0058] Figure 10 It is a schematic diagram for comparing the effects before and after color correction of an image frame provided by an embodiment of the present application;

[0059] Figure 11 It is a schematic flowchart of a multi-magnification color correction algorithm provided by an embodiment of the present application;

[0060] Figure 12 It is a flowchart of a multi-camera color correction method provided by an embodiment of the present application;

[0061] Figure 13 It is a flowchart of a multi-magnification color correction method provided by an embodiment of the present application. Detailed implementation manners

[0062] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the implementation manners of the present application in detail with reference to the accompanying drawings.

[0063] It should be understood that the "multiple" mentioned in the present application refers to two or more. In the description of the present application, unless otherwise specified, " / " means "or", for example, A / B can represent A or B; the "and / or" herein is only a description of the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, for the convenience of clearly describing the technical solutions of the present application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the terms "first" and "second" do not limit the quantity and execution order, and the terms "first" and "second" do not necessarily limit to be different.

[0064] For the convenience of understanding, the names involved in the embodiments of the present application will be explained first.

[0065] White point: That is, the point on a white object. White refers to the visual response formed by the light reflected into the human eye due to the same proportion of blue, green, and red light and a certain brightness. A white object refers to an object that a human eye considers white when seen in any scene, such as a white wall, a gray tabletop, white paper, etc. However, it should be noted that these white objects generally have colors when photographed by a camera (i.e., the corresponding red (R), green (G), and blue (B) values are not equal), and their color values are determined by the light source. Therefore, the white objects in the scene (as perceived by the human eye) are generally not white (image values) in the image. Like white objects, white points actually also have colors.

[0066] White point information: White point information is used to indicate the color information of the corresponding white point. For example, the white point information can be the white point coordinates of the corresponding white point. The white point coordinates refer to the coordinates in a coordinate system with R / G as the abscissa and B / G as the ordinate. That is, the white point coordinates are (R / G, B / G).

[0067] Color temperature: Color temperature is a measure of the light color of a light source, with the unit of kelvin (K). Color temperature is defined according to an absolute blackbody. When the radiation of the light source in the visible region is exactly the same as that of the absolute blackbody, the temperature of the blackbody at this time is called the color temperature of the light source. The higher the color temperature, the bluer the white light tone; the lower the color temperature, the yellower the white light tone.

[0068] White balance (WB): Since the sensor in the camera cannot change its photosensitive characteristics according to the change of ambient light like the human eye, under light sources with different color temperatures, the response of white in the camera's sensor will be bluish or reddish. White balance is to restore the white color after imaging under ambient light with different color temperatures to the real white (usually the white observed by the human eye under natural daylight ambient light) through the white balance algorithm.

[0069] The white balance algorithm can make the white color truly presented by adjusting the intensities of the three color channels of R, G, and B. For example, for the image to be processed, the white points in the image can be first counted, and then the white balance gain can be determined according to the white point information of the white points, and the values of the three color channels of R, G, and B in the image can be corrected according to the white balance gain. In addition, the color temperature of the image can also be estimated through the white balance algorithm, and the estimated color temperature of the image can be output, such as estimating the color temperature according to the white point information.

[0070] Magnification ratio: The magnification ratio is used to indicate the ratio between the image formed on the photosensitive element of the camera and the original object. The higher the magnification ratio, the stronger the close-up ability of the camera. For example, the magnification ratio can be 0.5X, 1X, 2X, 5X, etc. 1X is the normal photo-taking size mode. 0.5X means increasing the shooting distance, and the width and height of the photographed object are respectively reduced to 0.5 times the original. 2X means shortening the shooting distance, and the width and height of the photographed object are respectively enlarged to 2 times the original. 5X means shortening the shooting distance, and the width and height of the photographed object are respectively enlarged to 5 times the original.

[0071] Output mode: The sensor in the camera can have different sensor configurations, that is, different configuration parameters can be set, such as resolution, frame rate, output mode (also called sensor output mode), etc., to perform shooting under different configuration parameters. The output mode refers to the way of reading the original raw image output by the sensor to obtain the actual output image, such as binding output mode, quadra output mode, or remosaic output mode, etc.

[0072] In the binning output mode, the charges sensed by adjacent pixels (of the same color) are added together and read out in the mode of one pixel. For example, four adjacent pixels are combined into one pixel for reading, so the resolution of the actual output image will be 1 / 4 of the resolution of the original image. The remosaic output mode can convert the pixel arrangement of the original image into a Bayer structure, and the quadra output mode does not convert the pixel arrangement of the original image. The resolution of the actual output images of the remosaic output mode and the quadra output mode is the same as that of the original image.

[0073] Generally, in order to ensure the resolution of the output image, different zoom ratios can correspond to different output modes, that is, under different zoom ratios, the sensor can use different output modes for output. For example, 1X corresponds to the binning output mode, 2X corresponds to the quadra output mode or the remosaic output mode, etc.

[0074] The image processing method provided by the embodiments of this application is applicable to any electronic device with a shooting function, such as terminals like mobile phones, tablet computers, cameras, and smart wearable devices. The embodiments of this application do not make any limitations in this regard. For example, the image processing method provided by the embodiments of this application can include a multi-camera image color correction method for correcting the color differences of the images captured by different cameras. This multi-camera image color correction method is applicable to electronic devices equipped with multiple cameras and is mainly applied to the single-camera shooting scenario of the electronic device, that is, the scenario where one of the multiple cameras is independently called for shooting.

[0075] As an example, different cameras among multiple cameras configured in an electronic device have different shooting capabilities. For example, the electronic device is configured with, but not limited to, a wide-angle camera, a telephoto camera (such as a periscope telephoto camera), an ultra-wide-angle camera, and a depth camera, etc.

[0076] As an example, multiple cameras configured in an electronic device include cameras arranged on different sides, or cameras arranged on the same side. That is to say, the embodiments of the present application can be applied to scenarios where any camera on different sides (such as a front camera or a rear camera) is called for shooting, or can also be applied to scenarios where any camera on the same side (such as a rear camera) is called for shooting.

[0077] As an example, when multiple cameras are configured on the same side of an electronic device, any camera on the same side can be called for shooting. For example, when multiple rear cameras are configured on the back of the electronic device, any rear camera can be called for shooting. Or, when multiple front cameras are configured on the front side of the electronic device, any front camera can be called for shooting separately.

[0078] Generally, an electronic device is configured with a main camera and at least one auxiliary camera. For example, please refer to Figure 1 , the spatial position distribution of multiple cameras can be as shown in Figure (a) of Figure 1 , or the spatial position distribution of multiple cameras can also be as shown in Figure (b) of Figure 1 . The multiple cameras are respectively camera 00, camera 01, camera 02, and camera 03. Exemplarily, camera 00 is the main camera, and the others are auxiliary cameras.

[0079] After the electronic device starts the camera application, it usually defaults to shooting through the main camera, and then can automatically switch according to the shooting requirements or switch to a certain auxiliary camera according to the user's switching operation for shooting. For example, please refer to Figure 1 , shooting is performed through camera 00 by default, and then switched to camera 01, camera 02, or camera 03 according to the user's switching operation for shooting.

[0080] As described in the background art, when an electronic device is equipped with multiple cameras, if the sensors of each camera have different responses to colors, there may be a certain deviation in the colors of the images captured by different cameras among these multiple cameras in the same scene. The visual difference caused by the color difference in multi-camera shooting in the same scene will affect the user's shooting experience. For example, when the user switches cameras to shoot in the same scene, there may be a deviation in the colors of the images captured before and after the camera switch. For the user, the user will see that the colors of the same object captured before and after are inconsistent, resulting in a poor visual experience for the user. Moreover, this situation will also cause certain doubts to the user, and may make the user suspect whether their shooting operation is incorrect, affecting the user's shooting experience.

[0081] Please refer to Figure 2 , Figure 2 which is a schematic diagram of color comparison of images captured by different cameras in the same scene provided by an embodiment of the present application. In the same scene, the user can first use the main camera to shoot, and then switch to the telephoto camera to shoot. Figure 2 Figure (a) in Figure 2 is the image captured by the main camera, and figure (b) in

[0082] is the image captured by the telephoto camera. By comparing the image captured by the main camera with the image captured by the telephoto camera, it can be seen that the field of view (FOV) of the image captured by the telephoto camera is smaller than that of the image captured by the main camera. Moreover, there is a large deviation in the colors of these two images, and the colors of the same object (such as a street lamp, etc.) in the shooting scene are inconsistent in these two images, resulting in a poor visual experience for the user.

[0083] In this way, when the ambient color temperature difference between other cameras and the reference camera is small, it can be determined that the shooting scene difference is small, and it is possible that the shooting is carried out in the same scene. By combining the white point information of the memory frame of the reference camera and the white point information of the current frame, the current frame is color-corrected by performing white balance correction on the current frame, so as to reduce the color difference between the current frame and the memory frame of the reference camera, thereby reducing the color difference between the images captured by other cameras and the reference camera in the same scene and the resulting visual difference, improving the color consistency of multi-camera shooting in the same scene, and further improving the user's visual experience and shooting experience.

[0084] It should be noted that the specific algorithm of this multi-camera color correction method will be described in detail in the following Figure 7 embodiment, and the embodiments of the present application will not be elaborated here.

[0085] Furthermore, considering the different responses of sensors in different cameras, before fusing the white point information of the current frame and the memory frame, the white point information of the second white point of the memory frame can be mapped to the current camera first, and then the white point information of the first white point of the current frame and the white point information of the mapped white point can be fused. In this way, the accuracy of color correction can be further improved.

[0086] After introducing the above multi-camera color correction method and related application scenarios and inventive concepts, next, another image processing method (i.e., multi-magnification color correction method) provided by the embodiments of the present application and related application scenarios and inventive concepts will be introduced.

[0087] For the same camera, the same camera usually supports multiple magnification ratios to meet different shooting needs of users. However, when the same camera supports multiple magnification ratios, since the output modes corresponding to different magnification ratios may be different, the colors of the images captured at different magnification ratios in the same scene may be different, and the resulting visual difference will affect the user's shooting experience.

[0088] For example, assume that a certain camera supports at least two magnification ratios of 1X and 2X, and the resolution of the sensor in the camera is 50M. In the same scene, when the camera shoots at a 1X magnification ratio, the sensor outputs an image in the binning output mode, and the resolution of the output image is 12.5M, and the FOV is the full FOV. When the camera shoots at a 2X magnification ratio, in order to ensure the resolution of the output image, the sensor outputs an image in the quadra output mode or the remosaic output mode, and the resolution of the output image is also 12.5M, and the FOV is 1 / 4 of the 1X output image. Please refer to Figure 3 ,Figure 3 It is a comparison schematic diagram of the output images of the same camera provided by the embodiments of the present application at 1X and 2X magnification ratios. Figure 3 In Figure (a) in [reference], it is the output image at 1X magnification ratio, and the FOV is the full FOV; Figure 3 In Figure (b) in [reference], it is the output image at 2X magnification ratio, and the FOV is 1 / 4 of the 1X output image. For the output image of the camera, it is usually necessary to perform white balance correction on it. However, for the white balance algorithm, due to the large difference in the statistical information of the output images in different image output modes, it is very difficult to calculate relatively similar white balance results for the output images in different image output modes, which will result in a large color difference between the images taken at 1X magnification ratio and 2X magnification ratio in the same scene. Thus, when the user switches the magnification ratio to take pictures in the same scene, there may be a deviation in the color of the pictures taken before and after the magnification ratio switch. For the user, the user will see that the colors of the same object taken before and after are inconsistent, resulting in a poor visual experience for the user. Moreover, this situation will also bring certain doubts to the user, and may make the user suspect that there is an error in their shooting operation, affecting the user's shooting experience.

[0089] Regarding the color difference of the images taken by the same camera at different magnification ratios in the same scene, the embodiments of the present application also provide another image processing method, that is, the multi-magnification color correction method, which is used to correct the color difference of the images taken by the same camera at different magnification ratios. In this method, for a certain camera configured in the electronic device that can support multiple magnification ratios, a certain magnification ratio supported by the camera can be pre-set as the reference magnification ratio, and the second white point of the memory frame of the reference magnification ratio (that is, the last frame taken by the camera at the reference magnification ratio) and the second ambient color temperature of the shooting environment corresponding to the memory frame can be stored, so as to perform color correction on the images taken by the camera at other magnification ratios based on the relevant information of the memory frame. For example, when the electronic device calls any other camera other than the reference camera to take pictures, the first white point of the current frame taken by the other camera and the first ambient color temperature of the shooting environment corresponding to the current frame can be obtained. Then, according to the difference between the second ambient color temperature of the shooting environment corresponding to the memory frame of the reference camera and the first ambient color temperature, the white point information of the first white point of the current frame and the white point information of the second white point of the memory frame are fused, and the current frame is white balance corrected according to the fused white point information.

[0090] In this way, when the ambient color temperature of the same camera at other magnification ratios is close to that at the reference magnification ratio, it can be determined that the shooting scene difference is small, and it is likely to be shooting in the same scene. By combining the white point information of the memory frame at the reference magnification ratio and the white point information of the current frame, color correction is performed on the current frame by performing white balance correction on the current frame, so as to reduce the color difference between the current frame and the memory frame at the reference magnification ratio, thereby reducing the color difference between the images captured by the same camera at other magnification ratios and at the reference magnification ratio in the same scene and the resulting visual difference, improving the color consistency of the images captured by the same camera at different magnification ratios, and further improving the user's visual experience and shooting experience.

[0091] It should be noted that the specific algorithm of this multi-camera color correction method will be described in detail in the following Figure 9 embodiment, and will not be elaborated here in the embodiments of this application.

[0092] It should also be noted that the multi-magnification color correction method provided in the embodiments of this application is mainly applicable to electronic devices equipped with cameras supporting multiple magnification ratios. Therefore, different from the above multi-camera image color correction method, this method can be applied to electronic devices equipped with multiple cameras or electronic devices equipped with only one camera.

[0093] For the convenience of description, the following will take an example of an electronic device equipped with multiple cameras for introduction.

[0094] Figure 4 is a schematic structural diagram of an electronic device provided in the embodiments of this application. Refer to Figure 4, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0095] It can be understood that the structure schematically shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0096] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modulation and demodulation processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0097] Among them, the controller may be the nerve center and command center of the electronic device 100. The controller may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching instructions and executing instructions.

[0098] A memory can also be set in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly called from this memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0099] The wireless communication function of the electronic device 100 can be implemented by the antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modulation and demodulation processor, and baseband processor, etc.

[0100] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, the antenna 1 can be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0101] The mobile communication module 150 can provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 and at least some of the modules of the processor 110 can be disposed in the same device.

[0102] The wireless communication module 160 may provide solutions for wireless communications applied to the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSSs), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive the signals to be sent from the processor 110, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through the antenna 2 for radiation.

[0103] The electronic device 100 realizes the display function through the GPU, the display screen 194, the application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to execute mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change the display information.

[0104] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel may adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is an integer greater than 1.

[0105] The electronic device 100 may realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, the application processor, etc.

[0106] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and light passes through the lens and is transmitted to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also optimize the noise, brightness, and skin color of the image through algorithms. The ISP can also optimize parameters such as the exposure and color temperature of the shooting scene. In some embodiments, the ISP can be provided in the camera 193.

[0107] The camera 193 is used to capture still images or videos. An object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal and then transmits the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in standard RGB, YUV, etc. formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is an integer greater than 1.

[0108] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0109] The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0110] The NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission pattern between human brain neurons, it can quickly process the input information and can also continuously learn on its own. Through the NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as: image recognition, face recognition, speech recognition, text understanding, etc.

[0111] The external memory interface 120 can be used to connect to an external memory card, such as a Micro SD card, to implement the storage capacity expansion of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. For example, files such as music and videos are saved in the external memory card.

[0112] The internal memory 121 can be used to store computer-executable program codes, and the computer-executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, the image playback function, etc.). The data storage area can store the data created during the use of the electronic device 100 (such as audio data, phone book, etc.). In addition, the internal memory 121 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0113] The electronic device 100 can implement audio functions, such as music playback, recording, etc., through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor.

[0114] The pressure sensor 180A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can include at least two parallel plates with conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 can also calculate the position of the touch according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations with the same touch position but different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the pressure threshold acts on the short message application icon, the instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.

[0115] The touch sensor 180K, also known as the "touch panel". The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also known as the "touch control screen". The touch sensor 180K is used to detect touch operations acting thereon or nearby. The touch sensor 180K can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In some other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a different position from the display screen 194.

[0116] Next, the software system of the electronic device 100 will be described.

[0117] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of the present application, the Android system with a layered architecture is taken as an example to exemplarily describe the software system of the electronic device 100.

[0118] Figure 5 It is a block diagram of a software system of an electronic device 100 provided by an embodiment of the present application. Refer to Figure 5 , the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, as Figure 5 shown, the system architecture of the electronic device 100 includes an application layer 510, an application framework layer 520, a hardware abstraction layer (HAL) 530, and a driver layer 540.

[0119] It can be understood that Figure 5 only as an example, the layers divided in the electronic device 100 are not limited to Figure 5 the layers shown. For example, between the application framework layer and the HAL layer, there may also be an Android runtime and a libraries layer, etc.

[0120] The application layer 510 may include a series of application program packages. As Figure 5 shown, the application program packages may include a camera, a gallery, and other application programs. Other application programs include but are not limited to: calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, and other application programs.

[0121] The application framework layer 520 provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer 520 includes some predefined functions. For example, the application framework layer 520 may include a camera access interface. The camera access interface may include camera management and a camera device. Among them, the camera management can be used to provide an access interface for managing the camera; the camera device can be used to provide an interface for accessing the camera.

[0122] In addition, the application framework layer 520 may also include a content provider, a resource manager, a notification manager, a window manager, a view system, a phone manager, etc. Similarly, the camera application can also call the content provider, the resource manager, the notification manager, the window manager, the view system, etc. according to actual business requirements. The embodiments of the present application do not impose any restrictions on this.

[0123] The hardware abstraction layer 530 is used to abstract the hardware. For example, it can encapsulate the driver programs in the driver layer and provide an interface for the application framework layer to call, shielding the implementation details of the underlying hardware. For example, the hardware abstraction layer 530 may include a camera hardware abstraction layer (Camera HAL) and other hardware device abstraction layers. The camera hardware abstraction layer can be connected to the algorithm library to call the algorithms in the algorithm library.

[0124] For example, please refer to Figure 6, the algorithm library may include a color correction module, which may include a multi-camera color correction module and / or a multi-magnification color correction module. The multi-camera color correction module integrates the multi-camera color correction algorithm provided by the embodiments of the present application, and is used to perform color correction on the image frame captured by the current camera using the multi-camera color correction algorithm. The multi-magnification color correction module integrates the multi-magnification color correction algorithm provided by the embodiments of the present application, and is used to perform color correction on the image frame captured by the current camera using the multi-magnification color correction algorithm. In addition, the algorithm library may further include an auto white balance (AWB) module and other image processing modules. The AWB module integrates the AWB algorithm, and can calculate the AWB white point and AWB color temperature of the image frame captured by the current camera using the AWB algorithm, and send the AWB white point and AWB color temperature to the color correction module. The color correction module processes them using an appropriate color correction algorithm according to the AWB white point and AWB color temperature to obtain a fused white point and a fused color temperature, and then returns the fused white point and fused color temperature to the AWB module. The AWB module can perform white balance correction on the current image frame according to the fused white point to obtain an AWB image frame. In addition, the AWB module can also continue to send the AWB image frame and the fused color temperature to other image processing modules, so that the other image processing modules can perform image processing on the AWB image frame according to the fused color temperature, such as performing color correction or lens shadow correction, etc.

[0125] The driver layer 540 is used to provide drivers for different hardware devices. For example, the driver layer 540 may include a camera device driver, a digital signal processor driver, a graphics processor driver, etc.

[0126] In addition, the hardware layer 550 includes hardware modules that can be driven, such as a camera device. For example, the camera device includes multiple cameras, such as camera 1, camera 2,..., camera n. In addition, the camera device may further include a multi-spectral sensor, a depth sensor (time of flight, TOF), etc., which are not limited in the embodiments of the present application.

[0127] In the present application, by calling the hardware abstraction layer interface in the hardware abstraction layer 530, the connection between the application layer 510 and the application framework layer 520 above the hardware abstraction layer 530 and the driver layer 550 and the hardware layer 550 below can be realized, and camera data transmission and function control can be realized.

[0128] Next, in combination with the capture and photographing scenario, the working processes of the software and hardware of the electronic device 100 will be exemplarily described.

[0129] The camera application in the application layer 510 can be displayed on the screen of the electronic device 100 in the form of an icon. When the icon of the camera application is clicked by the user to be triggered, the electronic device 100 starts running the camera application. When the camera application is running on the electronic device 100, the camera application calls the corresponding interface of the camera application in the application framework layer 520, and then starts the camera device driver by calling the hardware abstraction layer 530, turns on any camera 193 on the electronic device 100, and takes a picture through the camera 193 to obtain a captured image. Exemplarily, the camera hardware abstraction layer 530 can issue an instruction to call a certain camera to the camera device driver to obtain a captured image through the called camera.

[0130] Taking the reference camera as the main camera as an example, after the main camera is called, for example, when the main camera captures the last frame of the image (memory frame), the multispectral sensor can be called to obtain the ambient color temperature when the memory frame is captured, and at the same time, the image signal processor is called to perform white balance processing on the memory frame captured by the main camera, and obtain the white point of the memory frame calculated by the white balance algorithm during the white balance processing, and then store the white point information and ambient color temperature of the memory frame. After that, when any camera other than the main camera in the hardware layer is called, that is, when the main camera is switched to another camera, during the process of capturing the current frame by the current camera, the multispectral sensor can be called to obtain the ambient color temperature when the current frame is captured, and at the same time, the image signal processor is called to perform white balance processing on the current frame captured by this camera, and obtain the white point of the current frame calculated by the white balance algorithm during the white balance processing. Then, obtain the white point information and ambient color temperature of the memory frame, and perform fusion processing on the white point information of the memory frame and the white point information of the current frame according to the ambient color temperature difference between the ambient color temperature of the memory frame and the ambient color temperature of the current frame, and feedback the fused white point information to the image signal processor, so that the image signal processor performs white balance correction and other processing on the current frame according to the fused white point information to obtain a target image. Then, the image signal processor returns the target image to the camera hardware abstraction layer through the camera device driver, and the camera hardware abstraction layer further processes the target image, and sends the processed image back to the camera application through the camera access interface for display and storage.

[0131] The execution subject of the image processing method provided by the embodiments of the present application can be the above-mentioned electronic device, or a functional module and / or functional entity in the electronic device that can implement the image processing method, and the solution of the present application can be implemented in a hardware and / or software manner, which can be specifically determined according to actual usage requirements, and the embodiments of the present application do not make limitations.

[0132] Next, taking the electronic device configured with multiple cameras as an example, the multi-camera color correction algorithm involved in the embodiments of the present application will be described in detail.

[0133] In the embodiments of the present application, a certain camera among multiple cameras configured in an electronic device can be set as a reference camera in advance, so as to correct the colors of the images captured by other cameras based on the image parameters of the reference camera, making the colors of the images captured by other cameras approach the colors of the images captured by the reference camera, thereby reducing the color differences in multi-camera shooting in the same scene and improving the color consistency of multi-camera shooting. Among them, the reference camera can be any one of the multiple cameras. In one example, the camera that is defaultly launched by the camera application can be set as the reference camera. For example, usually, the camera that is defaultly launched by the camera application is usually the main camera, so the main camera can be set as the reference camera.

[0134] Figure 7 It is a schematic flowchart of a multi-camera color correction algorithm provided by the embodiments of the present application. As Figure 7 shown, the method includes the following steps:

[0135] Step A1: Obtain and store the white point information of the second white point of the memory frame of the reference camera, and the second ambient color temperature of the shooting environment corresponding to the memory frame. The memory frame refers to the last frame captured by the reference camera.

[0136] In the embodiments of the present application, when it is detected that the camera application switches from the reference camera to other cameras for shooting, the ambient color temperature (i.e., the second ambient color temperature) of the shooting environment corresponding to the last frame (i.e., the memory frame) captured by the reference camera, and the white point information of the white point of the memory frame (i.e., the white point information of the second white point) can be obtained, and the white point information of the second white point of the memory frame and the second ambient color temperature are stored. Exemplarily, the memory frame can be the last frame previewed by the reference camera.

[0137] Among them, the camera switching event of switching from the reference camera to other cameras for shooting can be triggered by the user's camera switching operation, or can be automatically triggered by the camera application according to the shooting requirements. The embodiments of the present application do not make limitations in this regard.

[0138] Among them, the second white point can be the white point obtained by performing white point estimation on the memory frame using a white balance algorithm. For example, the second white point is the AWB white point of the memory frame, that is, the white point represented by the white point information output after the AWB module in the image signal processor performs white point estimation on the memory frame. The white point information of the second white point is used to indicate the second white point, and can be the color information of the second white point. For example, the white point coordinates of the second white point (R / G, B / G). The white point coordinates refer to the coordinates in the coordinate system with R / G as the abscissa and B / G as the ordinate. The second ambient color temperature is used to indicate the color temperature of the corresponding shooting environment, and can be the correlated color temperature (CCT) of the shooting environment. Exemplarily, the second ambient color temperature is the color temperature detected by a multispectral sensor in the corresponding shooting environment.

[0139] Furthermore, other information corresponding to the memory frame can also be obtained and stored. For example, one or more of the following information can also be obtained: the second estimated color temperature of the memory frame, and the brightness information of the shooting environment corresponding to the memory frame.

[0140] Among them, the second estimated color temperature is the color temperature obtained by performing color temperature estimation on the memory frame. For example, it is the color temperature obtained by performing color temperature estimation based on the second white point. Exemplarily, the second estimated color temperature can be the color temperature obtained by performing color temperature estimation on the memory frame using the white balance algorithm according to the second white point. For example, the second estimated color temperature is the AWB color temperature, that is, the color temperature output after the AWB module in the image signal processor performs color temperature estimation on the memory frame.

[0141] Among them, the brightness information is used to indicate the ambient brightness of the corresponding shooting environment. For example, the brightness information can be the light value (LV), and of course, it can also be other parameters used to measure the ambient brightness. This brightness information can be the brightness information obtained by performing brightness statistics on the memory frame. For example, this brightness information is the brightness information output after the auto exposure (AE) module in the image signal processor performs brightness statistics on the memory frame.

[0142] Step A2: After the first camera other than the reference camera is called for shooting, obtain the white point information of the first white point of the current frame captured by the first camera, and the first ambient color temperature of the shooting environment corresponding to the current frame.

[0143] Among them, the first camera is the current camera, which can be any one of the multiple cameras except the reference camera. For example, the first camera is the current camera after camera switching. Exemplarily, the call scenarios of the first camera can include any one of the following scenarios: directly switching from the reference camera to the first camera for shooting; or, first switching from the reference camera to other cameras, and then switching from other cameras to the first camera for shooting, etc. Exemplarily, the current frame can be the current frame of the real-time preview of the first camera.

[0144] Among them, the first white point can be the white point obtained by performing white point estimation on the current frame using a white balance algorithm. For example, the first white point is the AWB white point, that is, the white point represented by the white point information output by the AWB module in the image signal processor after performing white point estimation on the memory frame. The white point information of the first white point is used to indicate the first white point, which can be the color information of the first white point, such as the white point coordinates (R / G, B / G) of the first white point. The first ambient color temperature is used to indicate the color temperature of the corresponding shooting environment, which can be the correlated color temperature (CCT) of the shooting environment. Exemplarily, the first ambient color temperature is the color temperature detected by a multispectral sensor in the corresponding shooting environment.

[0145] Furthermore, other information corresponding to the current frame captured by the first camera can also be obtained. For example, the first estimated color temperature of the current frame is obtained. The first estimated color temperature is the color temperature obtained by performing color temperature estimation on the current frame, such as the color temperature obtained by performing color temperature estimation based on the first white point. Exemplarily, the first estimated color temperature can be the color temperature obtained by performing color temperature estimation on the current frame using a white balance algorithm. For example, the first estimated color temperature is the AWB color temperature, that is, the color temperature output by the AWB module in the image processor after performing color temperature estimation on the current frame.

[0146] Step A3: Map the second white point of the memory frame of the reference camera to the first camera to obtain a mapped white point.

[0147] By first mapping the second white point of the memory frame of the reference camera to the first camera to obtain a mapped white point, and then performing fusion processing on the white point information of the mapped white point and the white point information of the first white point of the current frame, the accuracy of multi-camera color correction can be further improved.

[0148] As an example, the operation of mapping the second white point of the memory frame of the reference camera to the first camera can include the following steps:

[0149] 1) Obtain the calibration data of the reference camera and the first camera. The calibration data of each camera includes the white point information and color temperature of each camera under multiple standard light sources.

[0150] In the embodiments of the present application, for multiple cameras configured in an electronic device, the white point information and color temperature of each camera under multiple standard light sources can be pre-calibrated, and the calibration data of each camera can be stored.

[0151] Among them, multiple standard light sources can be specified as needed. A standard light source is a light source specified by a standard, such as the light sources specified by the International Commission on Illumination for unified color detection. The color temperatures of these multiple standard light sources are different. For example, these multiple standard light sources can be at least two of the standard light sources such as D75, D65, D50, CWF, TL84, U30, A, H, etc. Of course, other standard light sources can also be included, and the embodiments of the present application do not limit this. Among them, the color temperatures of the standard light sources D75, D65, D50, CWF, TL84, U30, A, H decrease in sequence, which are 7500K, 6500K, 5000K, 4150K, 4100K, 3000K, 2856K, and 2300K respectively.

[0152] As an example, when calibrating the white point information and color temperature of a certain camera under multiple standard light sources, the camera can be used to take pictures under each standard light source respectively, and the white point information of the white point of the captured image can be used as the white point information of the camera under the corresponding standard light source. The color temperature of the camera under each standard light source can be the estimated color temperature of the image captured by the camera under each standard light source, or the color temperature detected by an illuminometer under the corresponding standard light source, or the color temperature of the corresponding standard light source. The embodiments of the present application do not limit this. For example, assuming that the color temperatures of each standard light source are used as the color temperatures of the camera under each standard light source, the color temperatures of the camera under the standard light sources D75, D65, D50, CWF, TL84, U30, A, H are 7500K, 6500K, 5000K, 4150K, 4100K, 3000K, 2856K, and 2300K respectively.

[0153] 2) Determine at least one standard light source from multiple standard light sources according to the color temperature difference from the second estimated color temperature of the memory frame of the reference camera.

[0154] That is to say, determine at least one standard light source from multiple standard light sources whose corresponding color temperature is closest to the second estimated color temperature. The number of this at least one standard light source can be preset, such as preset to 1, 2, or 3, etc. The embodiments of the present application do not limit this.

[0155] For example, assuming that the number of this at least one standard light source is 2, then the 2 standard light sources ranked first can be determined from these multiple standard light sources in ascending order of the color temperature difference between the corresponding color temperature and the second estimated color temperature. That is to say, determine the 2 standard light sources from multiple standard light sources whose color temperatures are closest to the second estimated color temperature.

[0156] 3) Determine a white point mapping matrix between the reference camera and the first camera according to the white point information of the reference camera under at least one standard light source and the white point information of the first camera under at least one standard light source.

[0157] As an example, the logarithm of the white point information of the reference camera under at least one standard light source can be taken, and the logarithm of the white point information of the first camera under at least one standard light source can be taken. According to the logarithmic results, the white point mapping matrix between the reference camera and the first camera is determined. Among them, taking the logarithm can be taking the common logarithm (i.e., log) or taking the natural logarithm (i.e., ln), and the embodiments of the present application do not limit this.

[0158] For example, assuming that the number of at least one standard light source is 2, and the 2 standard light sources are denoted as L1 and L2, and the white point information is the white point coordinates (R / G, B / G), the white point mapping matrix between the reference camera and the first camera can be determined by the following formula (1):

[0159]

[0160] Among them, T is the white point mapping matrix. W dst is the logarithmic result of the white point coordinates of the reference camera under L1 and L2, that is Among them, is the logarithmic result of the white point coordinates (R / G, B / G) of the reference camera under L1; is the logarithmic result of the white point coordinates (R / G, B / G) of the reference camera under L2.

[0161] Among them, W src is the logarithmic result of the white point coordinates of the first camera under L1 and L2, that is Among them, is the logarithmic result of the white point coordinates (R / G, B / G) of the first camera under L1; is the logarithmic result of the white point coordinates (R / G, B / G) of the first camera under L2.

[0162] 4) Map the second white point to the first camera according to the white point mapping matrix to obtain a mapped white point.

[0163] For example, according to the white point mapping matrix, the second white point can be mapped to the first camera by the following formula (2) to obtain the logarithm of the white point coordinates of the mapped white point:

[0164]

[0165] Among them, W syncis the logarithm result of the white point coordinates for mapping the white point, T is the white point mapping matrix, log(rg ref ) and log(bg ref ) are the logarithm results of the white point coordinates of the second white point in the memory frame of the reference camera.

[0166] After that, the white point coordinates of the mapped white point can be determined according to the logarithm result of the white point coordinates of the mapped white point. For example, performing an exponential operation on the logarithm result of the white point coordinates of the mapped white point to obtain the white point coordinates of the mapped white point, so as to convert the mapped white point from the log domain to the original domain.

[0167] As an example, please refer to Figure 8 , Figure 8 which is a schematic diagram of mapping the white point of the memory frame of the reference camera to the first camera provided by an embodiment of the present application. As shown in Figure 8 , A1 and B1 are the white points of the reference camera under L1 and L2 respectively, A2 and B2 are the white points of the first camera under L1 and L2 respectively, C1 is the first white point of the memory frame of the reference camera, and C2 is the mapped white point. According to A1, B1, A2 and B2, the white point mapping relationship between the reference camera and the first camera can be determined. According to this white point mapping relationship, C1 can be mapped to the first camera to obtain the mapped white point C2. Among them, Figure 8 the white points in are represented by the coordinates of the white points in the coordinate system with log(R / G) as the abscissa and log(B / G) as the ordinate.

[0168] Step A4: Determine the fusion weight according to the ambient color temperature difference between the first ambient color temperature and the second ambient color temperature of the shooting environment corresponding to the memory frame.

[0169] Among them, the ambient color temperature difference between the first ambient color temperature and the second ambient color temperature can represent the ambient difference between the current shooting environment and the shooting environment of the memory frame. The ambient color temperature difference is inversely proportional to the fusion weight. The smaller the ambient color temperature difference, the greater the fusion weight.

[0170] In this way, when the ambient color temperature difference is small, it means that the difference between the current shooting environment and the shooting environment of the memory frame is small. In this case, by performing a greater degree of fusion on the first white point of the memory frame according to the greater fusion weight, a greater degree of color correction can be performed on the current frame, so that the color of the corrected image of the current frame is closer to the color of the image of the memory frame to a greater extent; when the ambient color temperature difference is large, it means that the difference between the current shooting environment and the shooting environment of the memory frame is large. In the case of a large shooting environment difference, it is normal for the current frame and the memory frame to have a color difference. In this case, a smaller degree of fusion can be performed on the first white point of the memory frame according to the smaller fusion weight to perform a smaller degree of color correction on the current frame.

[0171] In one embodiment, the fusion weight can be determined based on the ambient color temperature difference and the corresponding relationship between the ambient color temperature difference and the fusion weight. In the corresponding relationship between the ambient color temperature difference and the fusion weight, the ambient color temperature difference is inversely proportional to the fusion weight, that is, the smaller the ambient color temperature difference, the larger the fusion weight.

[0172] In another embodiment, determining the fusion weight according to the ambient color temperature difference may further include the following steps:

[0173] 1) Determine the first sub-weight according to the ambient color temperature difference. Among them, the smaller the ambient color temperature difference, the larger the first sub-weight.

[0174] For example, the first sub-weight can be determined according to the current ambient color temperature difference and the corresponding relationship between the ambient color temperature difference and the sub-weight. In the corresponding relationship between the ambient color temperature difference and the sub-weight, the ambient color temperature difference is inversely proportional to the sub-weight.

[0175] For example, the corresponding relationship may include multiple ambient color temperature differences and their respective corresponding sub-weights. If the corresponding relationship includes the sub-weight corresponding to the current ambient color temperature difference, the sub-weight corresponding to the current ambient color temperature difference can be used as the first sub-weight. In addition, if the corresponding relationship does not include the sub-weight corresponding to the current ambient color temperature difference, the sub-weight corresponding to the current ambient color temperature difference can be interpolated based on the sub-weights corresponding to the multiple ambient color temperature differences in the corresponding relationship to obtain the first sub-weight.

[0176] 2) Determine the second sub-weight according to the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature of the memory frame. Among them, the larger the estimated color temperature difference, the larger the second sub-weight.

[0177] Among them, the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature can indicate the shooting effect difference between the first camera and the reference camera. The estimated color temperature difference is proportional to the second sub-weight, and the larger the estimated color temperature difference, the larger the second sub-weight.

[0178] In this way, when the estimated color temperature difference is small, it means that the shooting effect difference between the first camera and the reference camera is small. In this case, the current frame can be color-corrected to a small extent according to the smaller second sub-weight. When the estimated color temperature difference is large, it means that the shooting effect difference between the first camera and the reference camera is large. In this case, the current frame is color-corrected to a large extent according to the larger second sub-weight to improve the shooting effect difference.

[0179] As an example, the second sub-weight can be determined according to the currently estimated color temperature difference and the corresponding relationship between the estimated color temperature difference and the sub-weight. In the corresponding relationship between the estimated color temperature difference and the sub-weight, the estimated color temperature difference is proportional to the sub-weight. Exemplarily, the corresponding relationship may include multiple estimated color temperature differences and their respective corresponding sub-weights. If the corresponding relationship includes the sub-weight corresponding to the currently estimated color temperature difference, the sub-weight corresponding to the currently estimated color temperature difference can be used as the second sub-weight. Additionally, if the corresponding relationship does not include the sub-weight corresponding to the currently estimated color temperature difference, interpolation can be performed on the sub-weights corresponding to the multiple estimated color temperature differences in the corresponding relationship to obtain the second sub-weight.

[0180] 3) Determine the fusion weight according to the first sub-weight and the second sub-weight.

[0181] For example, the product of the first sub-weight and the second sub-weight can be determined as the fusion weight. It should be understood that other methods can also be used to process the first sub-weight and the second sub-weight to obtain the fusion weight, and the embodiments of the present application do not limit this.

[0182] In another embodiment, determining the fusion weight according to the ambient color temperature difference may further include the following steps:

[0183] 1) Determine the first sub-weight according to the ambient color temperature difference. Among them, the smaller the ambient color temperature difference, the larger the first sub-weight.

[0184] 2) Determine the second sub-weight according to the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature of the memory frame. Among them, the larger the estimated color temperature difference, the larger the second sub-weight.

[0185] 3) Determine the third sub-weight according to the ambient brightness information of the shooting environment corresponding to the memory frame. Among them, the greater the brightness indicated by the ambient brightness information, the larger the third sub-weight.

[0186] Among them, the ambient brightness information of the shooting environment corresponding to the memory frame is used to indicate the ambient brightness of the shooting environment corresponding to the memory frame. For example, it can be a brightness parameter such as LV. Generally, the smaller the ambient brightness, the more complex the shooting environment and the worse the shooting effect. The greater the ambient brightness, the better the shooting effect. The brightness indicated by the ambient brightness information is proportional to the third sub-weight, and the greater the brightness indicated by the ambient brightness information, the larger the third sub-weight.

[0187] In this way, when the brightness indicated by the ambient brightness information is smaller, the image effect of the reference camera's memory frame is worse. In this case, a smaller color correction can be performed on the current frame according to a smaller third sub-weight. When the brightness indicated by the ambient brightness information is larger, it means that the image effect of the reference camera's memory frame is better. In this case, a larger color correction can be performed on the current frame according to a larger third sub-weight. Adopting this strategy can ensure the accuracy of color correction.

[0188] As an example, the third sub-weight can be determined according to the target ambient brightness information of the shooting environment corresponding to the memory frame and the corresponding relationship between the ambient brightness information and the sub-weight. Among them, in the corresponding relationship between the ambient brightness information and the sub-weight, the brightness indicated by the ambient brightness information is proportional to the sub-weight. Exemplarily, the corresponding relationship may include multiple ambient brightness information and their respective corresponding sub-weights. If the corresponding relationship includes the sub-weight corresponding to the target ambient brightness information, the sub-weight corresponding to the target ambient brightness information can be used as the third sub-weight. In addition, if the corresponding relationship does not include the sub-weight corresponding to the target ambient brightness information, interpolation can be performed according to the sub-weights corresponding to the multiple ambient brightness information in the corresponding relationship to obtain the third sub-weight.

[0189] 4) Determine the fusion weight according to the first sub-weight, the second sub-weight and the third sub-weight.

[0190] For example, the product of the first sub-weight, the second sub-weight and the third sub-weight can be determined as the fusion weight. It should be understood that other methods can also be used to process the first sub-weight, the second sub-weight and the third sub-weight to obtain the fusion weight, and the embodiments of the present application do not limit this.

[0191] Step A5: Perform a fusion process on the white point information of the first white point and the white point information of the mapped white point according to the fusion weight to obtain the white point information of the third white point.

[0192] For example, according to the fusion weight, the following formula (3) can be used to perform a fusion process on the white point information of the first white point and the white point information of the mapped white point to obtain the logarithm result of the white point information of the third white point:

[0193] W act =W cur *(1 - w)+W sync *w(3)

[0194] Where, W act is the logarithm result of the white point information of the third white point, W cur is the logarithm result of the white point information of the first white point, w is the fusion weight, and W sync is the logarithm result of the white point information of the mapped white point.

[0195] After that, the white point information of the third white point can be determined according to the logarithmic result of the white point information of the third white point. For example, an exponential operation can be performed on the logarithmic result of the white point information of the third white point to obtain the white point information of the third white point, so as to convert the white point information of the third white point from the log domain to the original domain.

[0196] Step A6: Perform white balance correction on the current frame according to the white point information of the third white point.

[0197] As an example, the white balance gain can be determined according to the white point information of the third white point, and the current frame can be white balance corrected according to the white balance gain.

[0198] For example, the white point information of the third white point is the white point coordinates (R / G, B / G). The white balance gains determined according to the white point coordinates of the third white point include: R channel gain R_Gain = G / R, B channel gain B_Gain = G / B. Correspondingly, for each pixel point in the current frame after white balance correction, R' = R * R_Gai; B' = B * B_Gain; the G channel value remains unchanged. Where R and B are the original R value and B value corresponding to each pixel point in the current frame respectively.

[0199] Step A7: Perform a fusion process on the first estimated color temperature and the second estimated color temperature according to the fusion weight to obtain a third estimated color temperature.

[0200] For example, according to the fusion weight, the first estimated color temperature and the second estimated color temperature can be fused using the following formula (4) to obtain the third estimated color temperature:

[0201] CCT3 = CCT1 * (1 - w) + CCT2 * w (4)

[0202] Where CCT3 is the third estimated color temperature, CCT1 is the first estimated color temperature, w is the fusion weight, and CCT2 is the second estimated color temperature.

[0203] Step A8: Perform image processing on the current frame after white balance correction according to the third estimated color temperature.

[0204] Among them, the image processing may include one or more of color correction (color correction matrix, CCM) and lens shade correction (lens shade correction, LSC), etc.

[0205] For example, according to the third estimated color temperature, the color correction matrix of the current frame after white balance correction can be determined, and the current frame after white balance correction can be color corrected according to the color correction matrix.

[0206] Please refer to Figure 9 ,Figure 9 This is a schematic diagram of the scenario for multi-camera color correction provided by an embodiment of the present application. As Figure 9 shown, it is possible to store information of the memory frame previewed by the main camera at the default magnification (AWB color temperature, AWB white point coordinates, and LV), as well as the ambient color temperature detected by the multispectral sensor. After the main camera switches to the telephoto / ultrawide-angle camera, it is possible to obtain information of the current frame (AWB color temperature, AWB white point coordinates, and LV) of the real-time preview of the current camera, as well as the ambient color temperature detected by the multispectral sensor, and map the white point of the main camera (the white point coordinates of the memory frame) to the current camera to obtain the mapped white point. Then, based on the information differences between the current frame and the memory frame (such as the ambient color temperature difference, AWB color temperature difference, and the brightness information of the memory frame, etc.), the fusion weight can be calculated. Then, based on the fusion weight, the white point of the current frame and the mapped white point are fused to obtain the actual white point, for example, the fusion process is performed using the above formula (3). Alternatively, based on the fusion weight, the AWB color temperature of the current frame and the AWB color temperature of the memory frame are fused to obtain the actual AWB color temperature, for example, the fusion process is performed using the above formula (3).

[0207] Please refer to Figure 10 , Figure 10 This is a schematic diagram showing the comparison of the effects before and after color correction of an image frame provided by an embodiment of the present application. Assume that the camera application switches from the main camera to the telephoto camera for shooting in the same scene. Among them, Figure 10 Figure (a) in Figure 10 is the memory frame captured by the main camera (i.e., the last frame captured by the main camera); Figure 10 Figure (b) in Figure 10 is the image frame captured by the telephoto camera, that is, the image frame before color correction; Figure 10 Figure (c) in

[0208] Next, taking an electronic device configured with one or more cameras, and one of the one or more cameras supporting multiple magnification ratios as an example, the multi-magnification color correction algorithm involved in the embodiments of the present application will be described in detail.

[0209] In an embodiment of the present application, for a camera among one or more cameras configured in an electronic device that supports multiple magnification factors, a certain magnification factor among the multiple magnification factors supported by the camera can be set as a reference magnification factor. Based on the image parameters of the image captured by the camera at the reference magnification factor, the color of the image captured by the camera at other magnification factors is corrected, so that the colors of the images captured by the camera at other magnification factors all approach the color of the image captured at the reference magnification factor. Furthermore, the color difference between the images captured by the same camera at different magnification factors in the same scene is reduced, and the color consistency of the images captured at different magnification factors is improved. Among them, the reference magnification factor can be any one of the multiple magnification factors supported by the camera. In one example, the magnification factor that the camera defaults to when starting up can be set as the reference magnification factor. For example, usually, the magnification factor that the camera defaults to when starting up is usually the 1X magnification factor. Therefore, the 1X magnification factor can be set as the reference magnification factor.

[0210] Figure 11 is a schematic flowchart of a multi-magnification color correction algorithm provided by an embodiment of the present application. As Figure 11 shown, the method includes the following steps:

[0211] Step B1: Obtain and store the white point information of the second white point of the memory frame of the first camera at the reference magnification factor, and the second ambient color temperature of the shooting environment corresponding to the memory frame. The memory frame refers to the last frame captured by the first camera at the reference magnification factor.

[0212] Among them, the first camera is any one of the one or more cameras configured in the electronic device that supports multiple magnification factors. In an embodiment of the present application, when it is detected that the first camera switches from the reference magnification factor to other magnification factors, the ambient color temperature (i.e., the second ambient color temperature) of the shooting environment corresponding to the last frame (i.e., the memory frame) captured by the camera at the reference magnification factor, and the white point information of the white point of the captured memory frame (i.e., the white point information of the second white point) can be obtained, and the white point information of the second white point and the second ambient color temperature of the memory frame are stored. Exemplarily, the memory frame is the last frame previewed by the first camera at the reference magnification factor.

[0213] Among them, the magnification factor switching event from the reference magnification factor to other magnification factors is triggered by the user's magnification factor switching operation, or can also be automatically triggered by the camera application according to the shooting requirements. The embodiment of the present application does not limit this.

[0214] Furthermore, other information corresponding to the memory frame can also be obtained and stored. For example, one or more of the following information can also be obtained: the second estimated color temperature of the memory frame, the brightness information of the shooting environment corresponding to the memory frame.

[0215] It should be noted that the meanings of the above-mentioned second white point, the white point information of the second white point, the second ambient color temperature, the second estimated color temperature, and the luminance information can be referred to the relevant descriptions in the above Figure 7 embodiments, and the embodiments of the present application will not be elaborated herein.

[0216] Step B2: When the first camera shoots at a first magnification other than the reference magnification, determine the first white point of the current frame shot by the first camera at the first magnification, and the first ambient color temperature of the shooting environment corresponding to the current frame.

[0217] As an example, the current frame is the current frame in the real-time preview at the first magnification after the first camera switches to the first magnification.

[0218] Furthermore, other information corresponding to the current frame shot by the first camera at the first magnification can also be obtained, such as obtaining the first estimated color temperature of the current frame.

[0219] It should be noted that the meanings of the above-mentioned first white point, the white point information of the first white point, the first ambient color temperature, and the first estimated color temperature can be referred to the relevant descriptions in the above Figure 7 embodiments, and the embodiments of the present application will not be elaborated herein.

[0220] Step B3: Determine the fusion weight according to the ambient color temperature difference between the first ambient color temperature and the second ambient color temperature of the shooting environment corresponding to the memory frame.

[0221] Among them, the ambient color temperature difference between the first ambient color temperature and the second ambient color temperature can represent the environmental difference between the current shooting environment and the shooting environment of the memory frame. The ambient color temperature difference is inversely proportional to the fusion weight. The smaller the ambient color temperature difference, the larger the fusion weight.

[0222] In this way, when the ambient color temperature difference is small, it means that the difference between the current shooting environment and the shooting environment of the memory frame is small. In this case, by performing a greater degree of fusion on the first white point of the memory frame according to a larger fusion weight, a greater degree of color correction can be performed on the current frame, so that the color of the corrected image of the current frame is closer to the image color of the memory frame to a greater extent; when the ambient color temperature difference is large, it means that the difference between the current shooting environment and the shooting environment of the memory frame is large. In the case of a large shooting environment difference, it is normal for the current frame and the memory frame to have a color difference. In this case, a smaller degree of fusion can be performed on the first white point of the memory frame according to a smaller fusion weight to perform a smaller degree of color correction on the current frame.

[0223] In one embodiment, the fusion weight can be determined according to the environmental color temperature difference and the corresponding relationship between the environmental color temperature difference and the fusion weight. In the corresponding relationship between the environmental color temperature difference and the fusion weight, the environmental color temperature difference is inversely proportional to the fusion weight, that is, the smaller the environmental color temperature difference, the larger the fusion weight.

[0224] In another embodiment, determining the fusion weight according to the environmental color temperature difference may further include the following steps:

[0225] 1) Determine the first sub-weight according to the environmental color temperature difference. Among them, the smaller the environmental color temperature difference, the larger the first sub-weight.

[0226] 2) Determine the second sub-weight according to the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature of the memory frame. Among them, the larger the estimated color temperature difference, the larger the second sub-weight.

[0227] 3) Determine the fusion weight according to the first sub-weight and the second sub-weight.

[0228] For example, the product of the first sub-weight and the second sub-weight can be determined as the fusion weight. It should be understood that other methods can also be used to process the first sub-weight and the second sub-weight to obtain the fusion weight, and the embodiments of the present application do not limit this.

[0229] In another embodiment, determining the fusion weight according to the environmental color temperature difference may further include the following steps:

[0230] 1) Determine the first sub-weight according to the environmental color temperature difference. Among them, the smaller the environmental color temperature difference, the larger the first sub-weight.

[0231] 2) Determine the second sub-weight according to the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature of the memory frame. Among them, the larger the estimated color temperature difference, the larger the second sub-weight.

[0232] 3) Determine the third sub-weight according to the environmental brightness information of the shooting environment corresponding to the memory frame. Among them, the larger the brightness indicated by the environmental brightness information, the larger the third sub-weight.

[0233] 4) Determine the fusion weight according to the first sub-weight, the second sub-weight and the third sub-weight.

[0234] For example, the product of the first sub-weight, the second sub-weight and the third sub-weight can be determined as the fusion weight. It should be understood that other methods can also be used to process the first sub-weight, the second sub-weight and the third sub-weight to obtain the fusion weight, and the embodiments of the present application do not limit this.

[0235] Among them, the method for determining each sub-weight can refer to the relevant description in the above Figure 7 embodiment, and the embodiments of the present application will not repeat it here.

[0236] Step B4: According to the fusion weight, fuse the white point information of the first white point and the white point information of the second white point to obtain the white point information of the third white point.

[0237] For example, according to the fusion weight, the following formula (5) can be used to fuse the white point coordinates of the first white point and the white point coordinates of the second white point to obtain the logarithm result of the white point coordinates of the third white point:

[0238] W cur =W cur *(1 - w)+W3*w(5)

[0239] Where, W act is the logarithm result of the white point coordinates of the third white point, W ref is the logarithm result of the white point coordinates of the first white point, w is the fusion weight, and W sync is the logarithm result of the white point coordinates of the second white point.

[0240] After that, according to the logarithm result of the white point coordinates of the third white point, determine the white point coordinates of the third white point. For example, the exponential operation can be performed on the logarithm result of the white point coordinates of the third white point to obtain the white point coordinates of the third white point, so as to convert the white point coordinates of the third white point from the log domain to the original domain.

[0241] It should be noted that compared with the above multi-camera color correction algorithm, since the multi-magnification color correction algorithm performs color correction on the images captured by the same camera, it is not necessary to map the second white point of the memory frame, and the white point information of the first white point and the white point information of the second white point can be directly fused according to the fusion weight.

[0242] Step B5: Perform white balance correction on the current frame according to the white point information of the third white point.

[0243] For example, according to the white point information of the third white point, the white balance gain can be determined, and the current frame is white balance corrected according to the white balance gain. For example, the white point information of the third white point is the white point coordinates (R / G, B / G), and the white balance gains determined according to the white point information of the third white point include: R channel gain R_Gain = G / R, B channel gain B_Gain = G / B. Correspondingly, for each pixel point in the current frame after white balance correction, R' = R * R_Gai, B' = B * B_Gain, and the G channel value remains unchanged. Where, R and B are the original R value and B value corresponding to each pixel point in the current frame respectively.

[0244] Step B6: According to the fusion weight, fuse the first estimated color temperature and the second estimated color temperature to obtain the third estimated color temperature.

[0245] For example, according to the fusion weight, the first estimated color temperature and the second estimated color temperature can be fused using the above formula (4) to obtain the third estimated color temperature.

[0246] Step B7: Perform image processing on the current frame after white balance correction according to the third estimated color temperature.

[0247] Among them, the image processing may include one or more of color correction and lens shadow correction, etc.

[0248] For example, according to the third estimated color temperature, a color correction matrix of the current frame after white balance correction can be determined, and the current frame after white balance correction can be color-corrected according to the color correction matrix.

[0249] Next, in combination with the above Figure 5 and Figure 6 , an example of the multi-camera color correction method provided in the embodiments of the present application will be described.

[0250] Please refer to Figure 12 , Figure 12 which is a flowchart of a multi-camera color correction method provided in the embodiments of the present application. This method is applied to an electronic device. In the embodiments of the present application, the main camera started by default is used as an example of the reference camera. As Figure 12 shown, the method includes the following steps:

[0251] Step 1201: The user starts the camera application.

[0252] For example, the user can click on the application icon of the camera application to start the camera application.

[0253] Step 1202: In response to the user's start operation, the camera application starts.

[0254] Step 1203: The camera application sends a call instruction 1 to the main camera.

[0255] The call instruction 1 is used to call the main camera. After the camera application is started, it can default to call the main camera for shooting.

[0256] Step 1204: The main camera previews at the default magnification according to the call instruction 1.

[0257] For example, the default magnification can be 1X, etc.

[0258] Step 1205: The multispectral sensor detects the ambient color temperature.

[0259] During the preview of the main camera, the multispectral sensor can detect the ambient color temperature of the shooting environment.

[0260] As an example, after the camera application is launched, it can send a start instruction to the multispectral sensor so that the multispectral sensor can detect the ambient color temperature according to the start instruction.

[0261] Step 1206: The main camera sends the preview image frame to the ISP.

[0262] The preview image frame of the main camera can be first sent to the ISP for processing.

[0263] Step 1207: The ISP sends the image frame to the AWB module.

[0264] For example, the ISP can call the AWB module according to the image frame to send the image frame to the AWB module for processing.

[0265] Step 1208: The AWB module uses the AWB algorithm to calculate the AWB white point coordinates, AWB color temperature, and LV of the image frame.

[0266] For example, the AWB module can use the AWB algorithm to estimate the white point of the image frame to obtain the AWB white point coordinates, estimate the color temperature according to the AWB white point coordinates to obtain the AWB color temperature, and estimate the brightness of the image frame to obtain the brightness information LV.

[0267] It should be noted that in the embodiments of the present application, only the example of obtaining LV by estimating the brightness of the image frame through the AWB module is used for illustration. It should be understood that LV of the image frame can also be determined by other means. For example, LV can be obtained through the AE module, that is, obtaining the LV output by the AE module, or LV can be obtained through a brightness sensor. The embodiments of the present application do not limit this.

[0268] Step 1209: The user switches to the telephoto camera.

[0269] The user can, according to the need, switch the main camera to the telephoto camera for shooting. For example, the main camera can be switched to the telephoto camera by clicking on the option corresponding to the telephoto camera displayed in the preview interface frame of the camera application.

[0270] Step 1210: In response to the user's camera switching operation, the camera application sends a call instruction 2 to the telephoto camera.

[0271] Among them, the call instruction 2 is used to call the telephoto camera.

[0272] Step 1211: The camera application sends a camera switching notification to the multi-camera color correction module.

[0273] Among them, the camera switching notification can carry the identifier of the switched camera and can also carry the identifier of the pre-switched camera. The embodiments of the present application do not limit this.

[0274] Step 1212: According to the camera switching notification, the multi-camera color correction module sends a memory frame data acquisition request to the AWB module.

[0275] Among them, the memory frame refers to the last frame of the main camera preview. That is, the memory frame data acquisition request is used to request the data of the last frame of the main camera preview, such as the AWB white point coordinates, AWB color temperature, and LV of the last frame.

[0276] Step 1213: According to the memory frame data acquisition request, the AWB module sends the AWB white point coordinate 1, AWB color temperature 1, and LV1 corresponding to the memory frame to the multi-camera color correction module.

[0277] Step 1214: The multi-camera color correction module sends a color temperature data acquisition request to the multi-spectral sensor.

[0278] Among them, the color temperature data acquisition request is used to request the ambient color temperature of the shooting environment, such as obtaining the ambient color temperature 1 of the shooting environment corresponding to the memory frame.

[0279] Step 1215: According to the color temperature data acquisition request, the multi-spectral sensor sends the detected ambient color temperature 1 to the multi-camera color correction module.

[0280] Among them, the ambient color temperature 1 is used to indicate the ambient color temperature of the shooting environment corresponding to the memory frame of the main camera.

[0281] Step 1216: The multi-camera color correction module stores the AWB white point coordinate 1, AWB color temperature 1, LV1, and ambient color temperature 1 corresponding to the memory frame.

[0282] That is, the multi-camera color correction module stores the data corresponding to the memory frame, so as to perform color correction on the image frames previewed by other cameras according to the data corresponding to the memory frame later.

[0283] Step 1217: The telephoto camera performs real-time preview according to the call instruction 2.

[0284] Step 1218: The telephoto camera sends the current frame of the real-time preview to the ISP.

[0285] Step 1219: The ISP sends the current frame to the AWB module.

[0286] The ISP can call the AWB module according to the current frame to send the current frame to the AWB module for processing.

[0287] Step 1220: The AWB module calculates the AWB white point coordinate 2 and AWB color temperature 2 of the current frame using the AWB algorithm.

[0288] For example, the AWB module can use the AWB algorithm to estimate the white point of the image frame, obtain the AWB white point coordinates 2, and perform color temperature estimation based on the AWB white point coordinates 2 to obtain the AWB color temperature 2.

[0289] Step 1221: The AWB module sends the AWB white point coordinates 2 and the AWB color temperature 2 to the multi-camera color correction module.

[0290] Step 1222: The multi-spectral sensor sends the detected ambient color temperature 2 to the multi-camera color correction module.

[0291] Among them, the ambient color temperature 2 is used to indicate the ambient color temperature of the shooting environment corresponding to the current frame.

[0292] Step 1223: The multi-camera color correction module calculates the fused white point coordinates and the fused color temperature according to the memory frame and the data corresponding to the current frame by using the multi-camera color correction algorithm.

[0293] Among them, the specific implementation process of calculating the fused white point coordinates and the fused color temperature by using the multi-camera color correction algorithm can refer to the relevant description in the above Figure 7 embodiments, and the embodiments of the present application will not be elaborated herein.

[0294] Step 1224: The multi-camera color correction module sends the fused white point coordinates and the fused color temperature to the AWB module.

[0295] Step 1225: The AWB module performs white balance correction on the current frame according to the fused white point coordinates.

[0296] Step 1226: The AWB module sends the fused color temperature and the current frame after white balance correction to the image processing module.

[0297] Step 1227: The image processing module processes the current frame after white balance correction according to the fused color temperature to obtain the target image frame.

[0298] For example, the image processing module may include one or more of the image processing modules such as the CCM module and the LSC module. Exemplarily, color correction and lens shadow correction and other processes may be performed on the current frame after white balance correction according to the fused color temperature, so as to obtain the processed target image frame.

[0299] Step 1228: The image processing module sends the target image frame to the camera application.

[0300] For example, the image processing module may first return the target image frame to the ISP, and then the ISP sends the target image frame to other subsequent processing modules.

[0301] Step 1229: The camera application displays the target image frame.

[0302] For example, the camera application can display the target image frame on the preview interface.

[0303] Next, in combination with the above Figure 5 and Figure 6 , an example of the multi-magnification color correction method provided in the embodiments of the present application will be given. Figure 13 is a flowchart of a multi-magnification color correction method provided in the embodiments of the present application. This method is applied to an electronic device. In the embodiments of the present application, the default magnification after the camera is started is used as the reference magnification. As Figure 13 shown, the method includes the following steps:

[0304] Step 1301: The user starts the camera application.

[0305] For example, the user can click on the application icon of the camera application to start the camera application.

[0306] Step 1302: In response to the user's start operation, the camera application starts.

[0307] Step 1303: The camera application sends a call instruction 1 to the main camera.

[0308] The call instruction 1 is used to call the main camera. After the camera application is started, it can default to calling the main camera for shooting.

[0309] Step 1304: The main camera previews at the default 1X magnification according to the call instruction 1.

[0310] Step 1305: The multi-spectral sensor detects the ambient color temperature.

[0311] During the preview of the main camera, the multi-spectral sensor can detect the ambient color temperature of the shooting environment.

[0312] As an example, after the camera application is started, it can send a start instruction to the multi-spectral sensor so that the multi-spectral sensor detects the ambient color temperature according to this start instruction.

[0313] Step 1306: The main camera sends the previewed image frame to the ISP.

[0314] The image frame previewed by the main camera can be first sent to the ISP for processing.

[0315] Step 1307: The ISP sends this image frame to the AWB module.

[0316] For example, the ISP can call the AWB module according to the image frame to send the image frame to the AWB module for processing.

[0317] Step 1308: The AWB module uses the AWB algorithm to calculate the AWB white point coordinates, AWB color temperature, and LV of the image frame.

[0318] For example, the AWB module can use the AWB algorithm to estimate the white point of the image frame, obtain the AWB white point coordinates, estimate the color temperature based on the AWB white point coordinates to obtain the AWB color temperature, and estimate the brightness of the image frame to obtain the brightness information LV.

[0319] It should be noted that in the embodiments of the present application, only the example of obtaining LV by estimating the brightness of the image frame through the AWB module is described. It should be understood that LV of the image frame can also be determined by other means. For example, LV can be obtained through the AE module, or LV can be obtained through a brightness sensor. The embodiments of the present application do not limit this.

[0320] Step 1309: The user switches to the 2X magnification.

[0321] The user can, according to needs, switch the 1X magnification to the 2X magnification for shooting. For example, the 1X magnification can be switched to the 2X magnification by clicking on the option corresponding to the 2X magnification displayed in the preview interface frame of the camera application. Or, the 1X magnification can be switched to the 2X magnification through a zoom operation on the preview interface. The embodiments of the present application do not limit the operation of switching the magnification.

[0322] Step 1310: In response to the user's magnification switching operation, the camera application sends a magnification switching instruction to the camera, and the magnification switching instruction carries the to-be-switched 2X magnification.

[0323] Step 1311: The camera application sends a magnification switching notification to the multi-magnification color correction module.

[0324] Among them, the magnification switching notification can carry the switched magnification and can also carry the magnification before switching. The embodiments of the present application do not limit this.

[0325] Step 1313: The multi-magnification color correction module sends a memory frame data acquisition request to the AWB module according to the magnification switching notification.

[0326] Among them, the memory frame refers to the last frame previewed by the main camera at the default 1X magnification. That is, the memory frame data acquisition request is used to request the data of the last frame previewed by the main camera at the default 1X magnification, such as data such as the AWB white point coordinates, AWB color temperature, and LV of the last frame.

[0327] Step 1313: The AWB module sends the AWB white point coordinate 1, AWB color temperature 1, and LV1 corresponding to the memory frame to the multi-magnification color correction module according to the memory frame data acquisition request.

[0328] Step 1314: The multi-magnification color correction module sends a color temperature data acquisition request to the multispectral sensor.

[0329] Among them, the color temperature data acquisition request is used to request the acquisition of the ambient color temperature of the shooting environment, such as obtaining the ambient color temperature of the shooting environment corresponding to the memory frame.

[0330] Step 1315: The multispectral sensor sends the detected ambient color temperature 1 to the multi-magnification color correction module according to the color temperature data acquisition request.

[0331] Among them, the ambient color temperature 1 is used to indicate the ambient color temperature of the shooting environment corresponding to the memory frame of the main camera.

[0332] Step 1316: The multi-magnification color correction module stores the AWB white point coordinates 1, AWB color temperature 1, LV1, and ambient color temperature 1 corresponding to the memory frame.

[0333] That is to say, the multi-magnification color correction module stores the data corresponding to the memory frame, so as to perform color correction on the image frames previewed by other cameras according to the data corresponding to the memory frame in the future.

[0334] Step 1317: The main camera switches to the 2X magnification for real-time preview according to the magnification switching instruction.

[0335] Step 1318: The main camera sends the current frame of the real-time preview to the ISP.

[0336] Step 1319: The ISP sends the current frame to the AWB module.

[0337] The ISP can call the AWB module according to the current frame to send the current frame to the AWB module for processing.

[0338] Step 1320: The AWB module calculates the AWB white point coordinates 2 and AWB color temperature 2 of the current frame using the AWB algorithm.

[0339] For example, the AWB module can use the AWB algorithm to estimate the white point of the image frame to obtain the AWB white point coordinates 2, and estimate the color temperature according to the AWB white point coordinates 2 to obtain the AWB color temperature 2.

[0340] Step 1321: The AWB module sends the AWB white point coordinates 2 and AWB color temperature 2 to the multi-magnification color correction module.

[0341] Step 1322: The multispectral sensor sends the detected ambient color temperature 2 to the multi-magnification color correction module.

[0342] Among them, the ambient color temperature 2 is used to indicate the ambient color temperature of the shooting environment corresponding to the current frame.

[0343] Step 1323: The multi-magnification color correction module calculates the fused white point coordinates and the fused color temperature according to the memory frame and the data corresponding to the current frame by using the multi-magnification color correction algorithm.

[0344] Among them, the specific implementation process of calculating the fused white point coordinates and the fused color temperature by using the multi-magnification color correction algorithm can refer to the relevant descriptions in the above Figure 7 embodiment, and the embodiments of the present application will not be elaborated herein.

[0345] Step 1324: The multi-magnification color correction module sends the fused white point coordinates and the fused color temperature to the AWB module.

[0346] Step 1325: The AWB module performs white balance correction on the current frame according to the fused white point coordinates.

[0347] Step 1326: The AWB module sends the fused color temperature and the current frame after white balance correction to the image processing module.

[0348] Step 1327: The image processing module processes the current frame after white balance correction according to the fused color temperature to obtain the target image frame.

[0349] For example, the image processing module may include one or more of the image processing modules such as the CCM module and the LSC module. Exemplarily, color correction and lens shadow correction and other processes may be performed on the current frame after white balance correction according to the fused color temperature, so as to obtain the processed target image frame.

[0350] Step 1328: The image processing module sends the target image frame to the camera application.

[0351] For example, the image processing module may first return the target image frame to the ISP, and then the ISP sends the target image frame to other subsequent processing modules.

[0352] Step 1329: The camera application displays the target image frame.

[0353] For example, the camera application may display the target image frame on the preview interface.

[0354] The present application also provides a chip, which is coupled to the memory. The chip is used to read and execute the computer program or instruction stored in the memory to execute the methods in the above embodiments.

[0355] The present application also provides an electronic device, which includes a chip. The chip is used to read and execute the computer program or instruction stored in the memory, so that the methods in the embodiments are executed.

[0356] This embodiment also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions run on an electronic device, the electronic device is enabled to execute the above-related method steps to implement the method in the above embodiment.

[0357] This embodiment also provides a computer program product. The computer-readable storage medium stores program codes. When the computer program product runs on a computer, the computer is enabled to execute the above-related steps to implement the method in the above embodiment.

[0358] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, a component or a module. The device may include a processor and a memory connected to each other. The memory is used to store computer execution instructions. When the device runs, the processor may execute the computer execution instructions stored in the memory, so that the chip executes the methods in the above method embodiments.

[0359] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be elaborated here.

[0360] The embodiments of the present application do not specifically limit the specific structure of the execution subject of the method provided in the embodiments of the present application. As long as a program recording the code of the method provided in the embodiments of the present application can be run to perform video processing according to the method provided in the embodiments of the present application. For example, the execution subject of the method provided in the embodiments of the present application may be an electronic device, or a functional module in the electronic device that can call and execute the program.

[0361] In several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces. The indirect coupling or communication connection of the device or unit may be in an electrical, mechanical or other form.

[0362] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0363] In addition, in each embodiment of the present application, each functional unit may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0364] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a computer software product. This computer software product is stored in a storage medium. The computer software product includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium may include, but is not limited to: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0365] As mentioned above, the above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image processing method, characterized in that, Applied to an electronic device, the electronic device includes a plurality of cameras, and the plurality of cameras include a reference camera. The method includes: When calling a first camera for shooting, determining a first white point of a current frame captured by the first camera and a first ambient color temperature of a shooting environment corresponding to the current frame, where the first camera is any one of the plurality of cameras except the reference camera; According to an ambient color temperature difference between the first ambient color temperature and a second ambient color temperature of a shooting environment corresponding to a memory frame of the reference camera, performing a fusion process on white point information of the first white point and white point information of a second white point of the memory frame to obtain white point information of a third white point; Performing white balance correction on the current frame according to the white point information of the third white point.

2. The method according to claim 1, characterized in that, The performing a fusion process on white point information of the first white point and white point information of the second white point of the memory frame according to an ambient color temperature difference between the first ambient color temperature and a second ambient color temperature of a shooting environment corresponding to a memory frame of the reference camera to obtain white point information of a third white point includes: Determining a fusion weight according to the ambient color temperature difference; wherein, the smaller the ambient color temperature difference, the larger the fusion weight; Performing a fusion process on white point information of the first white point and white point information of the second white point according to the fusion weight to obtain white point information of the third white point.

3. The method according to claim 2, wherein Before the performing a fusion process on white point information of the first white point and white point information of the second white point according to the fusion weight, it further includes: Mapping the second white point to the first camera to obtain a mapped white point; The performing a fusion process on white point information of the first white point and white point information of the second white point according to the fusion weight to obtain white point information of the third white point includes: Performing a fusion process on white point information of the first white point and white point information of the mapped white point according to the fusion weight to obtain white point information of the third white point.

4. The method according to claim 3, wherein The mapping the second white point to the first camera to obtain a mapped white point includes: Obtaining calibration data of the reference camera and the first camera, where the calibration data includes white point information and color temperature under a plurality of standard light sources; Determining at least one standard light source from the plurality of standard light sources according to a color temperature difference from an estimated color temperature of the memory frame; Determining a white point mapping matrix between the reference camera and the first camera according to white point information of the reference camera under the at least one standard light source and white point information of the first camera under the at least one standard light source; Mapping the second white point to the first camera according to the white point mapping matrix to obtain the mapped white point.

5. The method according to claim 2, wherein Before the determining a fusion weight according to the ambient color temperature difference, it further includes: Obtaining a first estimated color temperature of the current frame; The determining a fusion weight according to the ambient color temperature difference includes: Determining a first sub - weight according to the ambient color temperature difference; wherein, the smaller the ambient color temperature difference, the larger the first sub - weight; Determine a second sub-weight according to the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature of the memory frame; wherein, the larger the estimated color temperature difference, the larger the second sub-weight; Determine the fusion weight according to the first sub-weight and the second sub-weight.

6. The method according to claim 5, wherein Before determining the fusion weight according to the first sub-weight and the second sub-weight, it further includes: Determine a third sub-weight according to the ambient brightness information of the shooting environment corresponding to the memory frame, and the greater the brightness indicated by the ambient brightness information, the larger the third sub-weight; The determining the fusion weight according to the first sub-weight and the second sub-weight includes: Determine the fusion weight according to the first sub-weight, the second sub-weight and the third sub-weight.

7. The method according to claim 6, wherein The determining the fusion weight according to the first sub-weight, the second sub-weight and the third sub-weight includes: Determine the product between the first sub-weight, the second sub-weight and the third sub-weight as the fusion weight.

8. The method according to claim 3, characterized in that, The fusion weight, the white point information of the first white point and the white point information of the mapped white point satisfy the following formula: The white point information of the third white point = the white point information of the first white point * (1 - W) + the white point information of the mapped white point * W; where W is the fusion weight.

9. The method according to any one of claims 1-8, characterized in that, The performing white balance correction on the current frame according to the white point information of the third white point includes: Determine the white balance gain according to the white point information of the third white point; Perform correction on the current frame according to the white balance gain.

10. The method according to any one of claims 1-9, characterized in that Before performing white balance correction on the current frame according to the white point information of the third white point, it further includes: Perform a fusion process on the first estimated color temperature of the current frame and the second estimated color temperature of the memory frame to obtain a third estimated color temperature; After performing white balance correction on the current frame according to the white point information of the third white point, it further includes: Perform color correction on the current frame after white balance correction according to the third estimated color temperature.

11. An image processing method, characterized in that, Applied to an electronic device, the electronic device includes a first camera, the first camera supports multiple magnification factors, the multiple magnification factors include a reference magnification factor, and the output image mode corresponding to the reference magnification factor is a first output image mode. The method includes: When the first camera shoots at a first magnification factor, determine the first white point of the current frame shot by the first camera at the first magnification factor, and the first ambient color temperature of the shooting environment corresponding to the current frame. The first magnification factor is a magnification factor other than the reference magnification factor among the multiple magnification factors and the output image mode corresponding to it is different from the first output image mode; Perform a fusion process on the white point information of the first white point and the white point information of the second white point of the memory frame according to the ambient color temperature difference between the first ambient color temperature and the second ambient color temperature of the shooting environment corresponding to the memory frame of the first camera. The memory frame refers to the last frame shot by the first camera at the reference magnification factor; Perform white balance correction on the current frame according to the white point information of the third white point.

12. The method according to claim 11, wherein The fusing the white point information of the first white point and the white point information of the second white point of the memory frame according to the environmental color temperature difference between the first environmental color temperature and the second environmental color temperature of the shooting environment corresponding to the memory frame of the first camera to obtain the white point information of the third white point includes: Determine a fusion weight according to the environmental color temperature difference; wherein, the smaller the environmental color temperature difference, the larger the fusion weight; Fuse the white point information of the first white point and the white point information of the second white point according to the fusion weight to obtain the white point information of the third white point.

13. The method according to claim 12, characterized in that, Before determining the fusion weight according to the environmental color temperature difference, further includes: Obtain the first estimated color temperature of the current frame; The determining the fusion weight according to the environmental color temperature difference includes: Determine a first sub-weight according to the environmental color temperature difference; wherein, the larger the environmental color temperature difference, the smaller the first sub-weight; Determine a second sub-weight according to the estimated color temperature difference between the first estimated color temperature of the current frame and the second estimated color temperature of the memory frame; wherein, the larger the estimated color temperature difference, the larger the second sub-weight; Determine the fusion weight according to the first sub-weight and the second sub-weight.

14. The method according to claim 13, characterized in that, Before determining the fusion weight according to the first sub-weight and the second sub-weight, further includes: Determine a third sub-weight according to the environmental brightness information of the shooting environment corresponding to the memory frame; wherein, the greater the brightness indicated by the environmental brightness information, the larger the third sub-weight; The determining the fusion weight according to the first sub-weight and the second sub-weight includes: Determine the fusion weight according to the first sub-weight, the second sub-weight and the third sub-weight.

15. The method according to claim 14, characterized in that The determining the fusion weight according to the first sub-weight, the second sub-weight and the third sub-weight includes: Determine the product of the first sub-weight, the second sub-weight and the third sub-weight as the fusion weight.

16. The method according to claim 12, wherein The fusion weight, the white point information of the first white point and the white point information of the second white point satisfy the following formula: The white point information of the third white point = the white point information of the first white point * (1 - W) + the white point information of the second white point * W; where W is the fusion weight.

17. The method according to any one of claims 11-16, characterized in that, The performing white balance correction on the current frame according to the white point information of the third white point includes: Determine a white balance gain according to the white point information of the third white point; Perform correction on the current frame according to the white balance gain.

18. The method according to any one of claims 11-17, characterized in that, Before performing white balance correction on the current frame according to the white point information of the third white point, further includes: Fuse the first estimated color temperature of the current frame and the second estimated color temperature of the memory frame to obtain a third estimated color temperature; After performing white balance correction on the current frame according to the third white point information, further includes: Perform color correction on the current frame after white balance correction according to the third estimated color temperature.

19. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the method described in any one of claims 1-10 or claims 11-18.

20. A computer-readable storage medium, characterized in that, Instructions are stored in the computer-readable storage medium. When it runs on a computer, it causes the computer to execute the method described in any one of claims 1-10 or claims 11-18.

Images