Image processing method, device and storage medium

Abstract

The application discloses an image processing method, device and storage medium, and belongs to the photographic technical field.The method is applied to an electronic device comprising a plurality of cameras, wherein the plurality of cameras comprise a reference camera, and the method comprises: in the case of calling a first camera to shoot, determining a first white point of a current frame shot by the first camera and a first ambient color temperature of a shooting environment corresponding to the current frame; performing fusion processing on white point information of the first white point and white point information of a second white point of a memory frame according to an ambient color temperature difference between the first ambient color temperature and a second ambient color temperature of a shooting environment corresponding to the memory frame of the reference camera, to obtain white point information of a third white point; and performing white balance correction on the current frame according to the white point information of the third white point.Thus, the color difference of images shot by other cameras and the reference camera in the same scene and the visual difference caused thereby can be reduced, and the color consistency of multi-camera shooting in the same scene can be improved.

Description

Technical Field

[0001] This application relates to the field of imaging technology, and in particular to an image processing method, device and storage medium. Background Technology

[0002] As users' photography needs increase, camera modules in mobile phones and other electronic devices have evolved from single-camera to multi-camera systems. This means that electronic devices are equipped with multiple cameras, such as a main camera, a telephoto camera, and an ultra-wide-angle camera. Users can choose any camera to shoot according to their needs. For example, the main camera can be used by default, the telephoto camera can be used to shoot distant objects, and the ultra-wide-angle camera can be used to shoot large areas of buildings or landscapes.

[0003] However, because the color responses of different camera sensors may vary, the colors of images captured by different cameras in the same scene may deviate, affecting the user's shooting experience. For example, if a user first uses the main camera to shoot in the same scene and then switches to the telephoto camera, the colors of the images taken before and after the switch may differ significantly. For instance, the image taken with the main camera may appear yellowish, while the image taken with the telephoto camera may appear bluish. This inconsistency in the colors of images taken by the user for the same scene negatively impacts the user's shooting experience. Summary of the Invention

[0004] This application provides an image processing method, device, and storage medium that can reduce color differences in multi-camera shots of the same scene and improve the user's shooting experience. The technical solution is as follows:

[0005] In a first aspect, an image processing method is provided, applied in an electronic device, the electronic device including multiple cameras, the multiple cameras including a reference camera, the method comprising:

[0006] When the first camera is used for shooting, the first white point of the current frame captured by the first camera and the first ambient color temperature of the shooting environment corresponding to the current frame are determined. The first camera is any one of multiple cameras except for the reference camera. Based on the ambient color temperature difference between the first ambient color temperature and the second ambient color temperature of the shooting environment corresponding to the memory frame of the reference camera, the white point information of the first white point and the white point information of the second white point of the memory frame are fused to obtain the white point information of the third white point. Based on the white point information of the third white point, white balance correction is performed on the current frame.

[0007] In this embodiment, one of the multiple cameras configured in the electronic device can be pre-set as a reference camera. The white point information of the second white point in the memory frame (i.e., the last frame captured) of the reference camera, as well as the second ambient color temperature of the shooting environment corresponding to the memory frame, can be stored. This allows for color correction of images captured by other cameras based on the relevant information from the memory frame of the reference camera. For example, when the electronic device uses any camera other than the reference camera to capture an image, the white point information of the first white point in the current frame captured by the other camera, as well as the first ambient color temperature of the shooting environment corresponding to the current frame, can be obtained. Then, based on the difference between the second and first ambient color temperatures of the shooting environment corresponding to the memory frame of the reference camera, the white point information of the first white point in the current frame and the white point information of the second white point in the memory frame are fused. White balance correction is then performed on the current frame based on the fused white point information.

[0008] In this way, when the difference between the ambient color temperature of other cameras and the ambient color temperature of the reference camera is small, it can be determined that the shooting scene is relatively similar, and it is likely that the shooting was carried out in the same scene. By combining the white point information of the memory frame of the reference camera and the white point information of the current frame, the color of the current frame is corrected by performing white balance correction, thereby reducing the color difference between the current frame and the memory frame of the reference camera. This reduces the color difference between the images captured by other cameras and the reference camera in the same scene, as well as the resulting visual difference, improving the color consistency of multi-camera shooting in the same scene, and thus improving the user's visual and shooting experience.

[0009] As an example, the fusion weight can be determined based on the ambient color temperature difference. Based on this weight, the white point information of the first white point and the second white point are fused to obtain the white point information of the third white point. The smaller the ambient color temperature difference, the greater the fusion weight.

[0010] The ambient color temperature difference between the first and second ambient color temperatures represents the environmental difference between the current shooting environment and the shooting environment of the memory frame. The ambient color temperature difference is inversely proportional to the fusion weight; the smaller the ambient color temperature difference, the greater the fusion weight.

[0011] Thus, when the ambient color temperature difference is small, it means that the difference between the current shooting environment and the shooting environment of the memory frame is small. In this case, by applying a larger fusion weight to the first white point of the memory frame to a greater extent, the color correction of the current frame can be performed to a greater extent, making the color of the corrected image of the current frame closer to the color of the image of the memory frame. When the ambient color temperature difference is large, it means that the difference between the current shooting environment and the shooting environment of the memory frame is large. When the shooting environment difference is large, it is normal for the current frame and the memory frame to have color differences. In this case, by applying a smaller fusion weight to the first white point of the memory frame to a smaller extent, the color correction of the current frame can be performed to a smaller extent.

[0012] As an example, before fusing the white point information of the first white point and the second white point according to the fusion weights, the second white point can first be mapped to the first camera to obtain the mapped white point. Then, according to the fusion weights, the white point information of the first white point and the mapped white point are fused to obtain the white point information of the third white point.

[0013] By first mapping the second white point of the memory frame of the reference camera to the first camera to obtain the mapped white point, and then fusing the white point information of the mapped white point with the white point information of the first white point in the current frame, the accuracy of multi-camera color correction can be further improved.

[0014] As an example, the operation of mapping a second white point to a first camera to obtain a mapped white point may include: acquiring calibration data of a reference camera and a first camera, the calibration data including white point information and color temperature under multiple standard light sources; determining at least one standard light source from the multiple standard light sources based on the color temperature difference between the estimated color temperature of the reference camera and the memory frame; determining a white point mapping matrix between the reference camera and the first camera based on the white point information of the reference camera under at least one standard light source and the white point information of the first camera under at least one standard light source; and mapping the second white point to the first camera based on the white point mapping matrix to obtain the mapped white point.

[0015] As an example, before determining the fusion weights based on the ambient color temperature difference, the first estimated color temperature of the current frame can be obtained first. Then, based on the ambient color temperature difference, a first sub-weight is determined; based on the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature of the memory frame, a second sub-weight is determined; and based on the first and second sub-weights, the fusion weights are determined. The smaller the ambient color temperature difference, the larger the first sub-weight. The larger the estimated color temperature difference, the larger the second sub-weight.

[0016] The estimated color temperature difference between the first and second estimated color temperatures indicates the difference in shooting effects between the first and reference cameras. The estimated color temperature difference is proportional to the second sub-weight; the larger the estimated color temperature difference, the larger the second sub-weight.

[0017] Thus, when the estimated color temperature difference is small, it indicates that the shooting effects of the first camera and the reference camera are relatively similar. In this case, a smaller degree of color correction can be applied to the current frame based on the smaller second sub-weight. Conversely, when the estimated color temperature difference is small, it indicates that the shooting effects of the first camera and the reference camera are relatively similar. In this case, a larger degree of color correction can be applied to the current frame based on the larger second sub-weight to mitigate the difference in shooting effects.

[0018] By combining the ambient color temperature difference and the estimated color temperature difference to determine the fusion weights, the accuracy of image correction can be further improved.

[0019] As an example, before determining the fusion weight based on the first and second sub-weights, the process may further include: determining a third sub-weight based on the ambient brightness information of the shooting environment corresponding to the memory frame. Then, the fusion weight is determined based on the first, second, and third sub-weights. The greater the brightness indicated by the ambient brightness information, the greater the third sub-weight.

[0020] The ambient brightness information of the shooting environment corresponding to the memory frame is used to indicate the ambient brightness of the shooting environment, such as LV (Level of Detail). Generally, the lower the ambient brightness, the more complex the shooting environment, and the worse the shooting effect. The higher the ambient brightness, the better the shooting effect. The brightness indicated by the ambient brightness information is proportional to the third sub-weight; the higher the brightness indicated by the ambient brightness information, the higher the third sub-weight.

[0021] Thus, the lower the brightness indicated by the ambient light information, the worse the image quality of the reference camera's memory frame. In this case, a smaller degree of color correction can be applied to the current frame based on a smaller third sub-weight. Conversely, the higher the brightness indicated by the ambient light information, the better the image quality of the reference camera's memory frame. In this case, a larger degree of color correction can be applied to the current frame based on a larger third sub-weight. This strategy ensures the accuracy of color correction.

[0022] By combining the ambient color temperature difference, the estimated color temperature difference, and the ambient brightness information corresponding to the memory frame to determine the fusion weights, the accuracy of image correction can be further improved.

[0023] As an example, determining the fusion weight based on the first, second, and third sub-weights can include multiplying the first, second, and third sub-weights to obtain the fusion weight. For instance, the fusion weight, the white point information of the first white point, and the white point information of the mapped white point satisfy the following formula: White point information of the third white point = White point information of the first white point. (1-W) + white point information mapped to white points W; where W is the fusion weight.

[0024] As an example, the operation of white balance correction for the current frame based on the white point information of the third white point may include: determining the white balance gain based on the white point information of the third white point; and correcting the current frame based on the white balance gain.

[0025] As an example, before determining the white balance gain based on the white point information of the third white point, the process could further include: fusing the first estimated color temperature of the current frame with the second estimated color temperature of the memory frame to obtain a third estimated color temperature; after performing white balance correction on the current frame based on the third white point information, color correction could then be performed on the white balance-corrected current frame based on the third estimated color temperature. This improves the accuracy of color correction, thereby further enhancing the color consistency of multi-camera images.

[0026] Secondly, an image processing method is provided, applied in an electronic device. The electronic device includes a first camera, which supports multiple magnifications, including a reference magnification, and the output mode corresponding to the reference magnification is a first output mode. The method includes: when the first camera takes a picture using the first magnification, determining the first white point of the current frame taken by the first camera using the first magnification, and the first ambient color temperature of the shooting environment corresponding to the current frame. The first magnification is a magnification other than the reference magnification among the multiple magnifications and whose corresponding output mode is different from the first output mode; based on the ambient color temperature difference between the first ambient color temperature and the second ambient color temperature of the shooting environment corresponding to the memory frame of the first camera, fusing the white point information of the first white point and the white point information of the second white point of the memory frame to obtain the white point information of the third white point. The memory frame refers to the last frame taken by the first camera using the reference magnification; and performing white balance correction on the current frame based on the white point information of the third white point.

[0027] In other words, for an electronic device equipped with a camera that supports multiple magnification levels, a certain magnification level supported by the camera can be pre-set as a reference magnification level. The second white point of the memory frame at the reference magnification level (i.e., the last frame captured by the camera at the reference magnification level) and the second ambient color temperature of the shooting environment corresponding to the memory frame can be stored. Based on the relevant information of the memory frame, color correction can be performed on images captured by the camera at other magnification levels. For example, when the electronic device uses any other camera besides the reference camera to take a picture, the first white point of the current frame captured by the other camera and the first ambient color temperature of the shooting environment corresponding to the current frame can be obtained. Then, based on the difference between the second and first ambient color temperatures of the shooting environment corresponding to the memory frame of the reference camera, the white point information of the first white point of the current frame and the white point information of the second white point of the memory frame are fused. White balance correction is then performed on the current frame based on the fused white point information.

[0028] In this way, when the difference between the ambient color temperature at other magnification levels and the ambient color temperature at the reference magnification level is small, it can be determined that the shooting scene is relatively similar, and the images were likely taken in the same scene. By combining the white point information of the memory frame at the reference magnification level and the white point information of the current frame, white balance correction is performed on the current frame to correct its color, thereby reducing the color difference between the current frame and the memory frame at the reference magnification level. This reduces the color difference between images taken by the same camera at other magnification levels and those taken at the reference magnification level in the same scene, as well as the resulting visual differences. This improves the color consistency of images taken by the same camera at different magnification levels, and ultimately enhances the user's visual and shooting experience.

[0029] As an example, the fusion weight can be determined based on the ambient color temperature difference. Then, based on the fusion weight, the white point information of the first white point and the white point information of the second white point are fused to obtain the white point information of the third white point. The smaller the ambient color temperature difference, the larger the fusion weight.

[0030] The ambient color temperature difference between the first and second ambient color temperatures represents the environmental difference between the current shooting environment and the shooting environment of the memory frame. The ambient color temperature difference is inversely proportional to the fusion weight; the smaller the ambient color temperature difference, the greater the fusion weight.

[0031] Thus, when the ambient color temperature difference is small, it means that the difference between the current shooting environment and the shooting environment of the memory frame is small. In this case, by applying a larger fusion weight to the first white point of the memory frame to a greater extent, the color correction of the current frame can be performed to a greater extent, making the color of the corrected image of the current frame closer to the color of the image of the memory frame. When the ambient color temperature difference is large, it means that the difference between the current shooting environment and the shooting environment of the memory frame is large. When the shooting environment difference is large, it is normal for the current frame and the memory frame to have color differences. In this case, by applying a smaller fusion weight to the first white point of the memory frame to a smaller extent, the color correction of the current frame can be performed to a smaller extent.

[0032] As an example, before determining the fusion weights based on the ambient color temperature difference, the first estimated color temperature of the current frame can be obtained. Then, the first sub-weight is determined based on the ambient color temperature difference; the second sub-weight is determined based on the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature of the memory frame; and the fusion weight is determined based on the first and second sub-weights.

[0033] Among them, the greater the ambient color temperature difference, the smaller the weight of the first sub-sub ...

[0034] The estimated color temperature difference between the first and second estimated color temperatures indicates the difference in shooting effects between the first and reference cameras. The estimated color temperature difference is proportional to the second sub-weight; the larger the estimated color temperature difference, the larger the second sub-weight.

[0035] Thus, when the estimated color temperature difference is small, it indicates that the shooting effects of the first camera and the reference camera are relatively similar. In this case, a smaller degree of color correction can be applied to the current frame based on the smaller second sub-weight. Conversely, when the estimated color temperature difference is small, it indicates that the shooting effects of the first camera and the reference camera are relatively similar. In this case, a larger degree of color correction can be applied to the current frame based on the larger second sub-weight to mitigate the difference in shooting effects.

[0036] By combining the ambient color temperature difference and the estimated color temperature difference to determine the fusion weights, the accuracy of image correction can be further improved.

[0037] As an example, before determining the fusion weight based on the first and second sub-weights, a third sub-weight can be determined based on the ambient brightness information of the shooting environment corresponding to the memory frame. Then, the fusion weight is determined based on the first, second, and third sub-weights. The greater the brightness indicated by the ambient brightness information, the greater the third sub-weight.

[0038] The ambient brightness information of the shooting environment corresponding to the memory frame is used to indicate the ambient brightness of the shooting environment, such as LV (Level of Detail). Generally, the lower the ambient brightness, the more complex the shooting environment, and the worse the shooting effect. The higher the ambient brightness, the better the shooting effect. The brightness indicated by the ambient brightness information is proportional to the third sub-weight; the higher the brightness indicated by the ambient brightness information, the higher the third sub-weight.

[0039] Thus, the lower the brightness indicated by the ambient light information, the worse the image quality of the reference camera's memory frame. In this case, a smaller degree of color correction can be applied to the current frame based on a smaller third sub-weight. Conversely, the higher the brightness indicated by the ambient light information, the better the image quality of the reference camera's memory frame. In this case, a larger degree of color correction can be applied to the current frame based on a larger third sub-weight. This strategy ensures the accuracy of color correction.

[0040] By combining the ambient color temperature difference, the estimated color temperature difference, and the ambient brightness information corresponding to the memory frame to determine the fusion weights, the accuracy of image correction can be further improved.

[0041] As an example, determining the fusion weight based on the first, second, and third sub-weights involves multiplying the first, second, and third sub-weights to obtain the fusion weight. For instance, the fusion weight, the white point information of the first white point, and the white point information of the second white point satisfy the following formula: White point information of the third white point = White point information of the first white point. (1-W) + information on the second white point W; where W is the fusion weight.

[0042] As an example, the operation of white balance correction for the current frame based on the white point information of the third white point includes: determining the white balance gain based on the white point information of the third white point; and correcting the current frame based on the white balance gain.

[0043] As an example, before performing white balance correction on the current frame based on the white point information of the third white point, the first estimated color temperature of the current frame and the second estimated color temperature of the memory frame can be fused to obtain the third estimated color temperature. After performing white balance correction on the current frame based on the third white point information, color correction can be performed on the white balance-corrected current frame based on the third estimated color temperature. In this way, the accuracy of color correction can be improved, thereby further improving the color consistency of multi-camera images.

[0044] Thirdly, an image processing apparatus is provided, which has the function of implementing the image processing method behavior described in the first aspect above. The image processing apparatus includes at least one module for implementing the image processing method provided in the first or second aspect above.

[0045] Fourthly, an image processing apparatus is provided, comprising a processor and a memory. The memory stores a program that supports the image processing apparatus in executing the image processing method provided in the first or second aspect, and stores data related to implementing the image processing method described in the first or second aspect. The processor is configured to execute the program stored in the memory. The image processing apparatus may further include a communication bus for establishing a connection between the processor and the memory.

[0046] Fifthly, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform the image processing method described in the first or second aspect above.

[0047] In a sixth aspect, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to perform the image processing method described in the first or second aspect above.

[0048] The technical effects achieved by the third, fourth, fifth and sixth aspects mentioned above are similar to the technical effects achieved by the corresponding technical means in the first or second aspects mentioned above, and will not be repeated here. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the spatial distribution of a camera in an electronic device provided in an embodiment of this application;

[0050] Figure 2 This is a color comparison diagram of images captured by different cameras in the same scene, provided in an embodiment of this application.

[0051] Figure 3 This is a comparative schematic diagram of the output images of the same camera at 1X and 2X magnification provided in the embodiments of this application;

[0052] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0053] Figure 5 This is a block diagram of a software system for an electronic device provided in an embodiment of this application;

[0054] Figure 6This is a logical schematic diagram of an algorithm library provided in an embodiment of this application;

[0055] Figure 7 This is a flowchart illustrating a multi-camera color correction algorithm provided in an embodiment of this application;

[0056] Figure 8 This is a schematic diagram illustrating how white dots from a memory frame of a reference camera are mapped to a first camera, according to an embodiment of this application.

[0057] Figure 9 This is a schematic diagram of a multi-camera color correction scenario provided in an embodiment of this application;

[0058] Figure 10 This is a schematic diagram showing the effect comparison before and after color correction of an image frame, provided in an embodiment of this application.

[0059] Figure 11 This is a flowchart illustrating a multi-magnification color correction algorithm provided in an embodiment of this application;

[0060] Figure 12 This is a flowchart of a multi-camera color correction method provided in an embodiment of this application;

[0061] Figure 13 This is a flowchart of a multi-magnification color correction method provided in an embodiment of this application. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0063] It should be understood that "multiple" as mentioned in this application refers to two or more. In the description of this application, unless otherwise stated, " / " indicates "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, to facilitate a clear description of the technical solutions of this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and that "first," "second," etc., do not necessarily imply differences.

[0064] To facilitate understanding, the names involved in the embodiments of this application will be explained first.

[0065] White dot: A dot on a white object. White refers to the visual perception of light reflected to the human eye as having equal proportions of blue, green, and red light with a certain brightness. A white object is any object that the human eye perceives as white in any scene, such as a white wall, a gray tabletop, or white paper. However, it's important to note that these white objects generally have color when photographed (i.e., their corresponding red (R), green (G), and blue (B) values ​​are not equal). Their color values ​​are determined by the light source, so what appears white to the human eye in a scene is generally not white (image value) in the image. Like white objects, white dots also actually have color.

[0066] White point information: White point information is used to indicate the color information of the corresponding white point. For example, white point information can be the white point coordinates of the corresponding white point. White point coordinates refer to the coordinates in a coordinate system with R / G as the x-axis and B / G as the y-axis, that is, the white point coordinates are (R / G, B / G).

[0067] Color temperature: Color temperature is a measure of the color of light from a light source, measured in Kelvin (K). It is defined based on a blackbody; when the radiation from a light source in the visible region is identical to that of a blackbody, the temperature of the blackbody is called the color temperature of the light source. The higher the color temperature, the more bluish the white light appears; the lower the color temperature, the more yellow the white light appears.

[0068] White balance (WB): Because the sensors in a camera cannot change their light-sensing characteristics according to changes in ambient light like the human eye, white will appear bluish or reddish in the camera's sensor under different color temperature light sources. White balance is the process of restoring the white image captured under different color temperature ambient light to true white (usually the white that the human eye perceives under natural sunlight).

[0069] White balance algorithms adjust the intensity of the R, G, and B color channels to make white appear more realistic. For example, for an image to be processed, the white points in the image can be counted first, and then the white balance gain can be determined based on the white point information. This white balance gain is then used to correct the values ​​of the R, G, and B color channels of the image. Additionally, white balance algorithms can also estimate the color temperature of the image, outputting the estimated color temperature, for example, by estimating the color temperature based on white point information.

[0070] Magnification: Magnification indicates the ratio of the image formed on the camera's sensor to the original object. A higher magnification means a stronger close-up capability. For example, magnification can be 0.5X, 1X, 2X, or 5X. 1X is the normal photo size mode. 0.5X means increasing the shooting distance, reducing the width and height of the object to 0.5 times its original size. 2X means shortening the shooting distance, magnifying the width and height of the object to twice its original size. 5X means shortening the shooting distance, magnifying the width and height of the object to five times its original size.

[0071] Image Output Mode: The sensor in a camera can have different sensor settings, meaning different configuration parameters can be set, such as resolution, frame rate, and image output mode (also called sensor output mode), to capture images under different configuration parameters. Image output mode refers to the method of reading the raw image output by the sensor to obtain the actual output image, such as binging image output mode, quadra image output mode, or remosaic image output mode.

[0072] Binning output mode combines the induced charges of adjacent pixels (of the same color) and reads them out as a single pixel. For example, it merges four adjacent pixels into one pixel for reading out, resulting in an output image resolution that is 1 / 4 of the original image resolution. Remosaic output mode converts the pixel arrangement of the original image into a Bayer structure, while quadra output mode does not convert the pixel arrangement of the original image. Both remosaic and quadra output modes produce output images with the same resolution as the original image.

[0073] Generally, to ensure the resolution of the output image, different scaling ratios correspond to different output modes. That is, the sensor can use different output modes to output images at different scaling ratios. For example, 1X corresponds to binning output mode, 2X corresponds to quadra or remosaic output mode, etc.

[0074] The image processing method provided in this application is applicable to any electronic device with a shooting function, such as a mobile phone, tablet computer, camera, or smart wearable device. This application does not limit the application to such devices. For example, the image processing method provided in this application may include a multi-camera image color correction method for correcting color differences in images captured by different cameras. This multi-camera image color correction method is applicable to electronic devices equipped with multiple cameras, and is mainly used in single-camera shooting scenarios, i.e., scenarios where only one of the multiple cameras is used for shooting.

[0075] As an example, different cameras among the multiple cameras configured in an electronic device have different shooting capabilities. For instance, the electronic device may be equipped with, but is not limited to, wide-angle cameras, telephoto cameras (such as periscope telephoto cameras), ultra-wide-angle cameras, and depth cameras.

[0076] As an example, an electronic device may be equipped with multiple cameras, including cameras located on different sides, or cameras located on the same side. That is, the embodiments of this application can be applied to scenarios where any camera on different sides (such as a front-facing camera or a rear-facing camera) is used for shooting, or to scenarios where any camera on the same side (such as a rear-facing camera) is used for shooting.

[0077] As an example, an electronic device can have multiple cameras on the same side, allowing any one of those cameras to be used for taking a picture. Similarly, an electronic device with multiple rear cameras on the back can use any one of them. Or, an electronic device with multiple front cameras on the front can use any one of them individually.

[0078] Typically, electronic devices are equipped with one main camera and at least one secondary camera. For example, please refer to... Figure 1 The spatial distribution of multiple cameras can be as follows: Figure 1 As shown in Figure (a), or, the spatial distribution of multiple cameras can also be as shown in Figure (a). Figure 1 As shown in Figure (b), the multiple cameras are camera 00, camera 01, camera 02 and camera 03. For example, camera 00 is the main camera and the others are auxiliary cameras.

[0079] When an electronic device launches its camera application, it typically defaults to using the main camera for shooting. It can then automatically switch to a secondary camera based on shooting needs or user input. For example, please refer to... Figure 1 By default, it takes pictures through camera 00, and then switches to camera 01, camera 02 or camera 03 to take pictures according to the user's switching operation.

[0080] As mentioned in the background section, when electronic devices are equipped with multiple cameras, if the sensors of each camera respond differently to color, the colors of the images captured by different cameras in the same scene may vary. This visual difference caused by color variations in multi-camera shots of the same scene will affect the user's shooting experience. For example, when a user switches cameras to shoot in the same scene, the colors of the images captured before and after the switch may differ. The user will see the same object in inconsistent colors in the previous and subsequent shots, resulting in a poor visual experience. Furthermore, this situation can cause confusion for the user, potentially leading them to doubt their shooting operation and negatively impacting the overall shooting experience.

[0081] Please refer to Figure 2 , Figure 2 This is a color comparison diagram of images captured by different cameras in the same scene, provided in an embodiment of this application. In the same scene, the user can first use the main camera to take a picture, and then switch to the telephoto camera to take another picture. Figure 2 Image (a) in the image is the one taken by the main camera. Figure 2 Figure (b) shows an image taken by the telephoto camera. Comparing the image taken by the main camera with the image taken by the telephoto camera, it can be seen that the field of view (FOV) of the image taken by the telephoto camera is smaller than that of the image taken by the main camera. Moreover, there is a large color difference between the two images. The same object in the shooting scene (such as a street lamp) appears in different colors in the two images, resulting in a poor visual experience for the user.

[0082] To address color differences in multi-camera shots within the same scene, this application provides an image processing method, namely a multi-camera color correction method. In this method, one of the multiple cameras configured in an electronic device can be pre-set as a reference camera. The white point information of the second white point in the memory frame (i.e., the last frame captured) of the reference camera, as well as the second ambient color temperature of the shooting environment corresponding to the memory frame, are stored. Based on the relevant information of the memory frame of the reference camera, color correction is performed on images captured by other cameras. For example, when the electronic device uses any other camera besides the reference camera to capture an image, the white point information of the first white point in the current frame captured by the other camera, as well as the first ambient color temperature of the shooting environment corresponding to the current frame, can be obtained. Then, based on the difference between the second and first ambient color temperatures of the shooting environment corresponding to the memory frame of the reference camera, the white point information of the first white point in the current frame and the white point information of the second white point in the memory frame are fused. White balance correction is then performed on the current frame based on the fused white point information.

[0083] In this way, when the difference between the ambient color temperature of other cameras and the ambient color temperature of the reference camera is small, it can be determined that the shooting scene is relatively similar, and it is likely that the shooting was carried out in the same scene. By combining the white point information of the memory frame of the reference camera and the white point information of the current frame, the color of the current frame is corrected by performing white balance correction, thereby reducing the color difference between the current frame and the memory frame of the reference camera. This reduces the color difference between the images captured by other cameras and the reference camera in the same scene, as well as the resulting visual difference, improving the color consistency of multi-camera shooting in the same scene, and thus improving the user's visual and shooting experience.

[0084] It should be noted that the specific algorithm of this multi-camera color correction method will be explained below. Figure 7 The embodiments are described in detail below, but the embodiments of this application will not be repeated here.

[0085] Furthermore, considering the differences in sensor responses in different cameras, before fusing the white point information of the current frame and the memory frame, the white point information of the second white point in the memory frame can be mapped to the current camera, and then the white point information of the first white point in the current frame and the mapped white point can be fused. This can further improve the accuracy of color correction.

[0086] After introducing the above-mentioned multi-camera color correction method, related application scenarios and inventive concepts, the following will introduce another image processing method (i.e., multi-magnification color correction method) and related application scenarios and inventive concepts provided by the embodiments of this application.

[0087] For a single camera, it can typically support multiple magnification levels to meet different user shooting needs. However, when the same camera supports multiple magnification levels, the output image mode may differ for each level. This can lead to color differences in images taken at different magnification levels in the same scene, thus affecting the user's shooting experience.

[0088] For example, suppose a camera supports at least 1X and 2X magnification, and its sensor resolution is 50MP. In the same scene, when the camera shoots at 1X magnification, the sensor uses binning mode to output the image, resulting in a resolution of 12.5MP and a full FOV. When the camera shoots at 2X magnification, to maintain the output image resolution, the sensor uses either quadra or remosaic mode, also producing a 12.5MP resolution image, but with a FOV that is one-quarter of the 1X output image. Please refer to [reference needed]. Figure 3 , Figure 3 This is a comparative schematic diagram of the output images of the same camera at 1X and 2X magnification provided in the embodiments of this application. Figure 3 Figure (a) shows the output image at 1X magnification, with FOV being the full FOV; Figure 3 Figure (b) shows the output image at 2X magnification, where the field of view (FOV) is one-quarter of the 1X output image. White balance correction is typically required for camera output images. However, due to significant differences in statistical information between output images from different output modes, it's difficult to calculate similar white balance results for each mode. This leads to substantial color differences between images taken at 1X and 2X magnification in the same scene. Consequently, when a user switches magnification levels in the same scene, the colors of the images before and after the switch may differ. The user will perceive inconsistent colors for the same object in the previous and subsequent shots, resulting in a poor visual experience. Furthermore, this situation can cause confusion, potentially leading the user to question their shooting technique and negatively impacting their overall experience.

[0089] To address color differences in images captured by the same camera at different magnifications in the same scene, this application provides another image processing method: a multi-magnification color correction method. This method corrects color differences in images captured by the same camera at different magnifications. In this method, for a camera configured in an electronic device that supports multiple magnifications, a certain magnification supported by the camera can be pre-set as a reference magnification. The second white point of a memory frame at the reference magnification (i.e., the last frame captured by the camera at the reference magnification) and the second ambient color temperature of the shooting environment corresponding to the memory frame are stored. Based on the relevant information of the memory frame, color correction is performed on images captured by the camera at other magnifications. For example, when the electronic device uses any other camera besides the reference camera to capture images, the first white point of the current frame captured by the other camera and the first ambient color temperature of the shooting environment corresponding to the current frame can be obtained. Then, based on the difference between the second ambient color temperature and the first ambient color temperature of the shooting environment corresponding to the memory frame of the reference camera, the white point information of the first white point in the current frame and the white point information of the second white point in the memory frame are fused, and the white balance is corrected for the current frame based on the fused white point information.

[0090] In this way, when the difference between the ambient color temperature at other magnification levels and the ambient color temperature at the reference magnification level is small, it can be determined that the shooting scene is relatively similar, and the images were likely taken in the same scene. By combining the white point information of the memory frame at the reference magnification level and the white point information of the current frame, the color of the current frame is corrected by performing white balance correction, thereby reducing the color difference between the current frame and the memory frame at the reference magnification level. This reduces the color difference between images taken by the same camera at other magnification levels and those taken at the reference magnification level in the same scene, as well as the resulting visual differences. This improves the color consistency of images taken by the same camera at different magnification levels, and thus enhances the user's visual and shooting experience.

[0091] It should be noted that the specific algorithm of this multi-camera color correction method will be explained below. Figure 9 The embodiments are described in detail below, but the embodiments of this application will not be repeated here.

[0092] It should also be noted that the multi-magnification color correction method provided in this application embodiment is mainly applicable to electronic devices equipped with cameras that support multiple magnifications. Therefore, unlike the multi-camera image color correction method described above, this method can be applied to electronic devices equipped with multiple cameras as well as electronic devices equipped with only one camera.

[0093] For ease of explanation, the following will use an electronic device with multiple cameras as an example.

[0094] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. See also... Figure 4The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0095] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0096] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0097] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0098] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0099] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0100] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0101] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0102] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0103] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0104] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is an integer greater than 1.

[0105] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0106] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's image sensor. The light signal is converted into an electrical signal, and the image sensor transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set within the camera 193.

[0107] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is an integer greater than 1.

[0108] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP performs Fourier transforms on the frequency energy.

[0109] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0110] NPU stands for Neural Network (NN) Computing Processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0111] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions, such as saving music, video, and other files on the external memory card.

[0112] Internal memory 121 can be used to store computer-executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created by electronic device 100 during use (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0113] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D and application processor.

[0114] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the touch operation intensity based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than the pressure threshold is applied to the SMS application icon, a command to view an SMS message is executed. When a touch operation with an intensity greater than or equal to the pressure threshold is applied to the SMS application icon, a command to create a new SMS message is executed.

[0115] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touch display." Touch sensor 180K detects touch operations applied to or near it. Touch sensor 180K can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0116] The software system of electronic device 100 will be described next.

[0117] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to illustrate the software system of electronic device 100.

[0118] Figure 5 This is a block diagram of a software system for an electronic device 100 provided in an embodiment of this application. See also... Figure 5 A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, such as... Figure 5 As shown, the system architecture of electronic device 100 includes an application layer 510, an application framework layer 520, a hardware abstraction layer (HAL) 530, and a driver layer 540.

[0119] Understandable. Figure 5 As an example only, the layers in electronic device 100 are not limited to... Figure 5 The layers shown, for example, between the application framework layer and the HAL layer, may also include the Android runtime and system library layers.

[0120] Application layer 510 may include a series of application packages. For example... Figure 5 As shown, the application package may include a camera, gallery, and other applications, including but not limited to: calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS applications.

[0121] The application framework layer 520 provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer 520 includes some predefined functions. For example, the application framework layer 520 may include a camera access interface. The camera access interface may include camera management and camera devices. Specifically, camera management can provide an access interface for managing the camera; camera devices can provide an interface for accessing the camera.

[0122] In addition, the application framework layer 520 may also include a content provider, a resource manager, a notification manager, a window manager, a view system, a phone manager, etc. Similarly, the camera application may also call the content provider, resource manager, notification manager, window manager, view system, etc. according to actual business needs. This application embodiment does not impose any restrictions on this.

[0123] The hardware abstraction layer 530 is used to abstract hardware, such as encapsulating drivers in the driver layer and providing an interface for the application framework layer to call them, thus shielding the implementation details of the underlying hardware. For example, the hardware abstraction layer 530 can encapsulate the camera hardware abstraction layer (Camera HAL) and other hardware device abstraction layers. The camera hardware abstraction layer can connect to an algorithm library to call algorithms from that library.

[0124] For example, please refer to Figure 6The algorithm library may include a color correction module, which may include a multi-camera color correction module and / or a multi-magnification color correction module. The multi-camera color correction module integrates the multi-camera color correction algorithm provided in this application embodiment, and is used to perform color correction on the image frame captured by the current camera using the multi-camera color correction algorithm. The multi-magnification color correction module integrates the multi-magnification color correction algorithm provided in this application embodiment, and is used to perform color correction on the image frame captured by the current camera using the multi-magnification color correction algorithm. In addition, the algorithm library may also include an automatic white balance (AWB) module and other image processing modules. The AWB module integrates the AWB algorithm, which can calculate the AWB white point and AWB color temperature of the image frame captured by the current camera, and send the AWB white point and AWB color temperature to the color correction module. The color correction module processes the AWB white point and AWB color temperature using a suitable color correction algorithm to obtain a fused white point and a fused color temperature, and then returns the fused white point and fused color temperature to the AWB module. The AWB module can perform white balance correction on the current image frame based on the fused white point to obtain an AWB image frame. In addition, the AWB module can also send the AWB image frame and the blended color temperature to other image processing modules so that the other image processing modules can perform image processing on the AWB image frame according to the blended color temperature, such as color correction or lens shadow correction.

[0125] The driver layer 540 is used to provide drivers for different hardware devices. For example, the driver layer 540 may include camera device drivers, digital signal processor drivers, and graphics processor drivers.

[0126] Additionally, the hardware layer 550 includes driveable hardware modules, such as camera devices. For example, a camera device may include multiple cameras, such as camera 1, camera 2, ..., camera n. Furthermore, the camera device may also include multispectral sensors, time-of-flight (TOF) sensors, etc., but this embodiment does not limit the scope of the application.

[0127] In this application, by calling the hardware abstraction layer interface in the hardware abstraction layer 530, the application layer 510 and application framework layer 520 above the hardware abstraction layer 530 can be connected to the driver layer 540 and hardware layer 550 below, so as to realize camera data transmission and function control.

[0128] The following example, using a scene of capturing a photograph, illustrates the workflow of the software and hardware of the electronic device 100.

[0129] The camera application in application layer 510 can be displayed as an icon on the screen of electronic device 100. When the user clicks the camera application icon to trigger it, electronic device 100 starts running the camera application. When the camera application is running on electronic device 100, it calls the corresponding interface of the camera application in application framework layer 520, and then calls hardware abstraction layer 530 to start the camera device driver, turning on any camera 193 on electronic device 100 to capture images. For example, camera hardware abstraction layer 530 can send a command to camera device driver to call a certain camera to capture images through the called camera.

[0130] Taking the reference camera as the main camera as an example, after the main camera is invoked, for instance, when it captures the last frame (memory frame), the multispectral sensor can be called to obtain the ambient color temperature at the time the memory frame was captured. Simultaneously, the image signal processor is invoked to perform white balance processing on the memory frame captured by the main camera, and the white point of the memory frame calculated using the white balance algorithm during the white balance processing is obtained. Then, the white point information and ambient color temperature of the memory frame are stored. Afterwards, when any camera other than the main camera in the hardware layer is invoked, i.e., when the main camera is switched to another camera, during the current camera's capture process, the multispectral sensor can be called to obtain the ambient color temperature at the time the current frame was captured. Simultaneously, the image signal processor is invoked to perform white balance processing on the current frame captured by that camera, and the white point of the current frame calculated using the white balance algorithm during the white balance processing is obtained. Then, the white point information and ambient color temperature of the memory frame are acquired. Based on the difference between the ambient color temperature of the memory frame and the current frame, the white point information of the memory frame and the current frame are fused. The fused white point information is then fed back to the image signal processor (ISP), which performs white balance correction and other processing on the current frame to obtain the target image. The ISP then sends the target image back to the camera hardware abstraction layer (HAL) via the camera device driver. The HAL further processes the target image and then sends the processed image back to the camera application for display and storage via the camera access interface.

[0131] The execution subject of the image processing method provided in this application embodiment can be the aforementioned electronic device, or a functional module and / or functional entity in the electronic device that can implement the image processing method. Furthermore, the solution of this application can be implemented by hardware and / or software, and the specific implementation can be determined according to actual usage requirements. This application embodiment does not impose any limitations.

[0132] Next, taking an electronic device equipped with multiple cameras as an example, the multi-camera color correction algorithm involved in the embodiments of this application will be described in detail.

[0133] In this embodiment, one of the multiple cameras configured in the electronic device can be pre-set as a reference camera. This allows for color correction of images captured by other cameras based on the image parameters of the reference camera, ensuring that the colors of images captured by other cameras approximate those of the reference camera. This reduces color differences in multi-camera shots of the same scene and improves color consistency. The reference camera can be any one of the multiple cameras. In one example, the camera that the camera application launches by default can be set as the reference camera. For instance, the camera that the camera application launches by default is usually the main camera, so the main camera can be set as the reference camera.

[0134] Figure 7 This is a flowchart illustrating a multi-camera color correction algorithm provided in an embodiment of this application, as shown below. Figure 7 As shown, the method includes the following steps:

[0135] Step A1: Obtain and store the white dot information of the second white dot of the memory frame of the reference camera, as well as the second ambient color temperature of the shooting environment corresponding to the memory frame. The memory frame refers to the last frame captured by the reference camera.

[0136] In this embodiment, when the camera application detects that it has switched from a reference camera to another camera for shooting, the ambient color temperature (i.e., the second ambient color temperature) of the shooting environment corresponding to the last frame (i.e., the memory frame) captured by the reference camera, and the white point information of the white point of the captured memory frame (i.e., the white point information of the second white point) are obtained, and the white point information of the second white point and the second ambient color temperature of the memory frame are stored. For example, the memory frame can be the last frame of the reference camera preview.

[0137] The camera switching event, which switches from the reference camera to other cameras for shooting, can be triggered by the user's camera switching operation or by the camera application automatically according to shooting needs. This application embodiment does not limit this.

[0138] The second white point can be the white point estimated by using a white balance algorithm on the memory frame. For example, the second white point is the AWB white point of the memory frame, that is, the white point represented by the white point information output by the AWB module in the image signal processor after estimating the white point of the memory frame. The white point information of the second white point is used to indicate the second white point, and can be the color information of the second white point, such as the white point coordinates (R / G, B / G). The white point coordinates refer to the coordinates in a coordinate system with R / G as the horizontal axis and B / G as the vertical axis. The second ambient color temperature is used to indicate the color temperature of the corresponding shooting environment, and can be the correlated color temperature (CCT) of the shooting environment. For example, the second ambient color temperature is the color temperature detected by a multispectral sensor in the corresponding shooting environment.

[0139] Furthermore, other information corresponding to the memory frame can also be acquired and stored, such as one or more of the following: the second estimated color temperature of the memory frame, and the brightness information of the shooting environment corresponding to the memory frame.

[0140] The second estimated color temperature is the color temperature obtained by estimating the color temperature of the memory frame, such as the color temperature estimated based on the second white point. For example, the second estimated color temperature can be obtained by estimating the color temperature of the memory frame using a white balance algorithm based on the second white point. For instance, the second estimated color temperature can be the AWB color temperature, which is the color temperature output by the AWB module in the image signal processor after estimating the color temperature of the memory frame.

[0141] The brightness information indicates the ambient brightness of the corresponding shooting environment. For example, the brightness information can be a light value (LV), or other parameters used to measure ambient brightness. This brightness information can be obtained by performing brightness statistics on the memory frames. For instance, this brightness information is the output of the auto exposure (AE) module in the image signal processor, which performs brightness statistics on the memory frames.

[0142] Step A2: After the first camera other than the reference camera is called to take a picture, obtain the white point information of the first white point in the current frame captured by the first camera, and the first ambient color temperature of the shooting environment corresponding to the current frame.

[0143] Here, the first camera refers to the current camera, which can be any one of multiple cameras other than the reference camera. For example, the first camera can be the current camera after a camera switch. For instance, the scenarios in which the first camera is invoked can include any of the following: switching directly from the reference camera to the first camera for shooting; or switching from the reference camera to another camera first, and then switching from that camera to the first camera for shooting, etc. For instance, the current frame can be the current frame of the real-time preview from the first camera.

[0144] The first white point can be the white point estimated by a white balance algorithm on the current frame. For example, the first white point is the AWB white point, which is the white point information output by the AWB module in the image signal processor after estimating the white point of the memory frame. The white point information of the first white point is used to indicate the first white point and can be the color information of the first white point, such as the white point coordinates (R / G, B / G). The first ambient color temperature is used to indicate the color temperature of the corresponding shooting environment and can be the correlated color temperature (CCT) of the shooting environment. For example, the first ambient color temperature is the color temperature detected by a multispectral sensor in the corresponding shooting environment.

[0145] Furthermore, other information corresponding to the current frame captured by the first camera can also be obtained, such as the first estimated color temperature of the current frame. The first estimated color temperature is the color temperature obtained by estimating the color temperature of the current frame, for example, it is the color temperature estimated based on the first white point. For example, the first estimated color temperature can be the color temperature obtained by estimating the color temperature of the current frame using a white balance algorithm. For example, the first estimated color temperature is the AWB color temperature, that is, the color temperature output by the AWB module in the image processor after estimating the color temperature of the current frame.

[0146] Step A3: Map the second white point of the memory frame of the reference camera to the first camera to obtain the mapped white point.

[0147] By first mapping the second white point of the memory frame of the reference camera to the first camera to obtain the mapped white point, and then fusing the white point information of the mapped white point with the white point information of the first white point in the current frame, the accuracy of multi-camera color correction can be further improved.

[0148] As an example, mapping the second white point of a memory frame from a reference camera to a first camera may include the following steps:

[0149] 1) Obtain the calibration data of the reference camera and the first camera. The calibration data of each camera includes the white point information and color temperature of each camera under multiple standard light sources.

[0150] In this embodiment of the application, for multiple cameras configured in an electronic device, the white point information and color temperature of each camera under multiple standard light sources can be pre-calibrated, and the calibration data of each camera can be stored.

[0151] Multiple standard light sources can be specified as needed. Standard light sources are light sources defined by standards, such as those specified by the International Commission on Illumination (ICI) for standardizing color detection. These multiple standard light sources have different color temperatures. For example, these multiple standard light sources can be at least two of the following standard light sources: D75, D65, D50, CWF, TL84, U30, A, and H. Other standard light sources can also be included, but this application does not limit this. The color temperatures of the standard light sources D75, D65, D50, CWF, TL84, U30, A, and H decrease sequentially: 7500K, 6500K, 5000K, 4150K, 4100K, 3000K, 2856K, and 2300K.

[0152] As an example, when calibrating the white point information and color temperature of a camera under multiple standard light sources, the camera can be used to take pictures under each standard light source, and the white point information of the captured images can be used as the white point information of the camera under the corresponding standard light source. The color temperature of the camera under each standard light source can be the estimated color temperature of the images captured by the camera under each standard light source, the color temperature detected by an illuminance meter under the corresponding standard light source, or the color temperature of the corresponding standard light source. This application does not limit this. For example, assuming that the color temperature of each standard light source is used as the color temperature of the camera under each standard light source, the color temperatures of the camera under the standard light sources D75, D65, D50, CWF, TL84, U30, A, and H are 7500K, 6500K, 5000K, 4150K, 4100K, 3000K, 2856K, and 2300K, respectively.

[0153] 2) Determine at least one standard light source from a plurality of standard light sources based on the color temperature difference between the second estimated color temperature and the memory frame of the reference camera.

[0154] That is, at least one standard light source is determined from multiple standard light sources whose corresponding color temperature is closest to the second estimated color temperature. The number of these at least one standard light source can be preset, such as 1, 2 or 3, etc., and this application embodiment does not limit this.

[0155] For example, assuming there are at least two standard light sources, the two standard light sources ranked first can be determined from these multiple standard light sources in ascending order of the color temperature difference between their corresponding color temperatures and the second estimated color temperature. That is, the two standard light sources whose color temperatures are closest to the second estimated color temperature are determined from the multiple standard light sources.

[0156] 3) Determine the white point mapping matrix between the reference camera and the first camera based on the white point information of the reference camera under at least one standard light source and the white point information of the first camera under at least one standard light source.

[0157] As an example, the logarithm of the white point information of the reference camera under at least one standard light source and the logarithm of the white point information of the first camera under at least one standard light source can be taken. Based on the logarithm results, the white point mapping matrix between the reference camera and the first camera can be determined. The logarithm can be either the common logarithm (i.e., log) or the natural logarithm (i.e., ln), and this embodiment does not limit the choice.

[0158] For example, assuming that the number of at least one standard light source is 2, and these two standard light sources are denoted as L1 and L2, and the white point information is the white point coordinates (R / G, B / G), then the white point mapping matrix between the reference camera and the first camera can be determined by the following formula (1):

[0159] (1)

[0160] Where T is the white point mapping matrix. It is the logarithmic result of the coordinates of the white point of the reference camera under L1 and L2, that is... .in, , It is the logarithmic result of the white point coordinates (R / G, B / G) of the reference camera under L1; , It is the logarithmic result of the white point coordinates (R / G, B / G) of the reference camera in L2.

[0161] in, It is the logarithmic result of the coordinates of the white point of the first camera under L1 and L2, that is... .in, , It is the logarithmic result of the white point coordinates (R / G, B / G) of the first camera under L1; , It is the logarithmic result of the white point coordinates (R / G, B / G) of the first camera under L2.

[0162] 4) Based on the white point mapping matrix, map the second white point to the first camera to obtain the mapped white point.

[0163] For example, based on the white point mapping matrix, the second white point can be mapped to the first camera using the following formula (2), and the logarithm of the white point coordinates of the mapped white point can be obtained:

[0164] (2)

[0165] in, Let T be the logarithm of the coordinates of the mapped white points, and let T be the white point mapping matrix. , It is the logarithmic result of the coordinates of the second white point in the memory frame of the reference camera.

[0166] Then, the coordinates of the mapped white point can be determined based on the logarithm of the white point coordinates. For example, by performing an exponential operation on the logarithm of the white point coordinates, the coordinates of the mapped white point can be obtained, thus transforming the mapped white point from the log domain to the original domain.

[0167] As an example, please refer to Figure 8 , Figure 8 This is a schematic diagram illustrating how white dots from a memory frame of a reference camera are mapped to a first camera, as provided in an embodiment of this application. Figure 8 As shown, A1 and B1 are the white points of the reference camera under L1 and L2, respectively; A2 and B2 are the white points of the first camera under L1 and L2, respectively; C1 is the first white point of the memory frame of the reference camera; and C2 is the mapped white point. Based on A1, B1, A2, and B2, the white point mapping relationship between the reference camera and the first camera can be determined. According to this white point mapping relationship, C1 can be mapped to the first camera, resulting in the mapped white point C2. Figure 8 The white point in the diagram is represented by its coordinates in a coordinate system with log(R / G) as the x-axis and log(B / G) as the y-axis.

[0168] Step A4: Determine the fusion weights based on the ambient color temperature difference between the first ambient color temperature and the second ambient color temperature of the shooting environment corresponding to the memory frame.

[0169] The ambient color temperature difference between the first and second ambient color temperatures represents the environmental difference between the current shooting environment and the shooting environment of the memory frame. The ambient color temperature difference is inversely proportional to the fusion weight; the smaller the ambient color temperature difference, the greater the fusion weight.

[0170] Thus, when the ambient color temperature difference is small, it means that the current shooting environment and the shooting environment of the memory frame are relatively similar. In this case, by applying a larger fusion weight to the white points of the memory frame, a greater degree of color correction can be applied to the current frame, making the corrected image color of the current frame much closer to the image color of the memory frame. When the ambient color temperature difference is large, it means that the current shooting environment and the shooting environment of the memory frame are significantly different. In this case, color differences between the current frame and the memory frame are normal. In this case, by applying a smaller fusion weight to the white points of the memory frame, a smaller degree of color correction can be applied to the current frame.

[0171] In one embodiment, the fusion weight can be determined based on the ambient color temperature difference and the correspondence between the ambient color temperature difference and the fusion weight. In the correspondence between the ambient color temperature difference and the fusion weight, the ambient color temperature difference is inversely proportional to the fusion weight; that is, the smaller the ambient color temperature difference, the larger the fusion weight.

[0172] In another embodiment, determining the blending weights based on the ambient color temperature difference may further include the following steps:

[0173] 1) Determine the first sub-weight based on the ambient color temperature difference. The smaller the ambient color temperature difference, the larger the first sub-weight.

[0174] For example, the first sub-weight can be determined based on the current ambient color temperature difference and the correspondence between the ambient color temperature difference and the sub-weights. In the correspondence between the ambient color temperature difference and the sub-weights, the ambient color temperature difference is inversely proportional to the sub-weight.

[0175] For example, the correspondence can include multiple ambient color temperature differences and their corresponding sub-weights. If the correspondence includes the sub-weight corresponding to the current ambient color temperature difference, then the sub-weight corresponding to the current ambient color temperature difference can be used as the first sub-weight. Alternatively, if the correspondence does not include the sub-weight corresponding to the current ambient color temperature difference, then the sub-weight corresponding to the current ambient color temperature difference can be interpolated based on the multiple sub-weights corresponding to ambient color temperature differences in the correspondence to obtain the first sub-weight.

[0176] 2) Determine the second sub-weight based on the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature of the memory frame. The larger the estimated color temperature difference, the larger the second sub-weight.

[0177] The estimated color temperature difference between the first and second estimated color temperatures indicates the difference in shooting effects between the first and reference cameras. The estimated color temperature difference is proportional to the second sub-weight; the larger the estimated color temperature difference, the larger the second sub-weight.

[0178] Thus, when the estimated color temperature difference is small, it indicates that the shooting effects of the first camera and the reference camera are relatively similar. In this case, a smaller degree of color correction can be applied to the current frame based on the smaller second sub-weight. Conversely, when the estimated color temperature difference is small, it indicates that the shooting effects of the first camera and the reference camera are relatively similar. In this case, a larger degree of color correction can be applied to the current frame based on the larger second sub-weight to mitigate the difference in shooting effects.

[0179] As an example, a second sub-weight can be determined based on the current estimated color temperature difference and the correspondence between the estimated color temperature difference and the sub-weights. In the correspondence between the estimated color temperature difference and the sub-weights, the estimated color temperature difference is proportional to the sub-weight. For example, this correspondence can include multiple estimated color temperature differences and their corresponding sub-weights. If the correspondence includes the sub-weight corresponding to the current estimated color temperature difference, then the sub-weight corresponding to the current estimated color temperature difference can be used as the second sub-weight. Alternatively, if the correspondence does not include the sub-weight corresponding to the current estimated color temperature difference, then the sub-weight corresponding to the current estimated color temperature difference can be interpolated based on the multiple sub-weights corresponding to estimated color temperature differences in the correspondence to obtain the second sub-weight.

[0180] 3) Determine the fusion weight based on the first sub-weight and the second sub-weight.

[0181] For example, the product of the first sub-weight and the second sub-weight can be determined as the fusion weight. It should be understood that other methods can also be used to process the first sub-weight and the second sub-weight to obtain the fusion weight, and the embodiments of this application do not limit this.

[0182] In another embodiment, determining the blending weights based on the ambient color temperature difference may further include the following steps:

[0183] 1) Determine the first sub-weight based on the ambient color temperature difference. The smaller the ambient color temperature difference, the larger the first sub-weight.

[0184] 2) Determine the second sub-weight based on the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature of the memory frame. The larger the estimated color temperature difference, the larger the second sub-weight.

[0185] 3) Determine the third sub-weight based on the ambient brightness information of the shooting environment corresponding to the memory frame. The greater the brightness indicated by the ambient brightness information, the greater the third sub-weight.

[0186] The ambient brightness information of the shooting environment corresponding to the memory frame is used to indicate the ambient brightness of the shooting environment, such as LV (Level of Detail). Generally, the lower the ambient brightness, the more complex the shooting environment, and the worse the shooting effect. The higher the ambient brightness, the better the shooting effect. The brightness indicated by the ambient brightness information is proportional to the third sub-weight; the higher the brightness indicated by the ambient brightness information, the higher the third sub-weight.

[0187] Thus, the lower the brightness indicated by the ambient light information, the worse the image quality of the reference camera's memory frame. In this case, a smaller degree of color correction can be applied to the current frame based on a smaller third sub-weight. Conversely, the higher the brightness indicated by the ambient light information, the better the image quality of the reference camera's memory frame. In this case, a larger degree of color correction can be applied to the current frame based on a larger third sub-weight. This strategy ensures the accuracy of color correction.

[0188] As an example, a third sub-weight can be determined based on the target ambient brightness information of the shooting environment corresponding to the memory frame, and the correspondence between the ambient brightness information and the sub-weights. In the correspondence between ambient brightness information and sub-weights, the brightness indicated by the ambient brightness information is proportional to the sub-weight. For example, this correspondence can include multiple ambient brightness information items and their corresponding sub-weights. If the correspondence includes the sub-weight corresponding to the target ambient brightness information, then the sub-weight corresponding to the target ambient brightness information can be used as the third sub-weight. Alternatively, if the correspondence does not include the sub-weight corresponding to the target ambient brightness information, then interpolation can be performed based on the sub-weights corresponding to multiple ambient brightness information items in the correspondence to obtain the third sub-weight.

[0189] 4) Determine the fusion weight based on the first sub-weight, the second sub-weight, and the third sub-weight.

[0190] For example, the product of the first sub-weight, the second sub-weight, and the third sub-weight can be determined as the fusion weight. It should be understood that other methods can also be used to process the first sub-weight, the second sub-weight, and the third sub-weight to obtain the fusion weight, and the embodiments of this application do not limit this.

[0191] Step A5: Based on the fusion weight, fuse the white point information of the first white point and the white point information of the mapped white point to obtain the white point information of the third white point.

[0192] For example, based on the fusion weight, the white point information of the first white point and the white point information of the mapped white point can be fused using the following formula (3) to obtain the logarithmic result of the white point information of the third white point:

[0193] (3)

[0194] in, The logarithm of the white point information for the third white point. The logarithm of the white point information for the first white point, where w is the fusion weight. This is the logarithmic result of the white point information mapped to white points.

[0195] Then, the information of the third white point can be determined based on the logarithm of the third white point's information. For example, the logarithm of the third white point's information can be used to perform an exponential operation to obtain the third white point's information, thus transforming the third white point's information from the log domain to the original domain.

[0196] Step A6: Perform white balance correction on the current frame based on the white point information of the third white point.

[0197] As an example, the white balance gain can be determined based on the white point information of the third white point, and the white balance can be corrected for the current frame based on the white balance gain.

[0198] For example, the white point information of the third white point is its coordinates (R / G, B / G). The white balance gain determined based on the coordinates of the third white point includes: R channel gain R_Gain = G / R, and B channel gain B_Gain = G / B. Correspondingly, the R' of each pixel in the current frame after white balance correction is R' = R'. R_Gai;B'=B B_Gain; The G channel value remains unchanged. Here, R and B are the original R and B values ​​for each pixel in the current frame, respectively.

[0199] Step A7: Based on the fusion weight, the first estimated color temperature and the second estimated color temperature are fused to obtain the third estimated color temperature.

[0200] For example, based on the fusion weight, the first estimated color temperature and the second estimated color temperature can be fused using the following formula (4) to obtain the third estimated color temperature:

[0201] (4)

[0202] in, For the third estimated color temperature, Let w be the first estimated color temperature and w be the blending weight. This is the second estimated color temperature.

[0203] Step A8: Based on the third estimated color temperature, perform image processing on the current frame after white balance correction.

[0204] The image processing may include one or more of color correction matrix (CCM) and lens shade correction (LSC).

[0205] For example, the color correction matrix of the current frame after white balance correction can be determined based on the third estimated color temperature, and the color correction of the current frame after white balance correction can be performed based on the color correction matrix.

[0206] Please refer to Figure 9 , Figure 9 This is a schematic diagram of a multi-camera color correction scene provided in an embodiment of this application. For example... Figure 9 As shown, the information of the memory frame previewed by the main camera at the default magnification (AWB color temperature, AWB white point coordinates, and LV) can be stored, as well as the ambient color temperature detected by the multispectral sensor. After the main camera is switched to the telephoto / ultra-wide-angle camera, the information of the current frame previewed by the current camera in real time (AWB color temperature, AWB white point coordinates, and LV), as well as the ambient color temperature detected by the multispectral sensor, can be obtained, and the white point of the main camera (the white point coordinates of the memory frame) can be mapped to the current camera to obtain the mapped white point. Then, the fusion weight can be calculated based on the information differences between the current frame and the memory frame (such as the difference in ambient color temperature, the difference in AWB color temperature, and the brightness information of the memory frame). Then, the white point of the current frame and the mapped white point are fused according to the fusion weight to obtain the actual white point, for example, by using the above formula (3) for fusion processing. Alternatively, the AWB color temperature of the current frame and the AWB color temperature of the memory frame are fused according to the fusion weight to obtain the actual AWB color temperature, for example, by using the above formula (3) for fusion processing.

[0207] Please refer to Figure 10 , Figure 10 This is a schematic diagram comparing the effects of color correction on image frames before and after, as provided in an embodiment of this application. It is assumed that the camera application switches from the main camera to the telephoto camera to take pictures in the same scene. Figure 10 Image (a) in the image is the memory frame captured by the main camera (i.e., the last frame captured by the main camera). Figure 10 Figure (b) in the figure is an image frame taken by a telephoto camera, that is, an image frame before color correction; Figure 10 Figure (c) shows the multi-camera color correction algorithm provided in the embodiments of this application. Figure 10 The image frame shown in Figure (b) is obtained after color correction. Figure 10 It can be seen that the color difference between the image frame before color correction and the memory frame is large, while the color difference between the image frame after color correction and the memory frame is small. This can improve the color consistency of images captured by multiple cameras.

[0208] Next, taking an electronic device equipped with one or more cameras, one of which supports multiple magnifications, as an example, the multi-magnification color correction algorithm involved in the embodiments of this application will be described in detail.

[0209] In this embodiment, for one of the one or more cameras in an electronic device that supports multiple magnification levels, a certain magnification level supported by the camera can be set as a reference magnification level. This allows the image parameters of the image captured by the camera at the reference magnification level to be used as a benchmark for color correction of images captured by the camera at other magnification levels. This ensures that the colors of images captured by the camera at other magnification levels are close to those captured at the reference magnification level, thereby reducing color differences in images captured by the same camera at different magnification levels in the same scene and improving color consistency. The reference magnification level can be any of the multiple magnification levels supported by the camera. In one example, the default magnification level used by the camera upon startup can be set as the reference magnification level. For instance, the default magnification level used by the camera upon startup is typically 1X, so 1X magnification can be set as the reference magnification level.

[0210] Figure 11 This is a flowchart illustrating a multi-magnification color correction algorithm provided in an embodiment of this application, as shown below. Figure 11 As shown, the method includes the following steps:

[0211] Step B1: Obtain and store the white dot information of the second white dot of the memory frame of the first camera at the reference magnification, and the second ambient color temperature of the shooting environment corresponding to the memory frame. The memory frame refers to the last frame captured by the first camera at the reference magnification.

[0212] The first camera is any one of one or more cameras configured in the electronic device that supports multiple magnification levels. In this embodiment, when the first camera is detected to switch from a reference magnification level to another magnification level, the ambient color temperature (i.e., the second ambient color temperature) of the shooting environment corresponding to the last frame (i.e., the memory frame) captured by the camera at the reference magnification level, and the white point information of the white point in the captured memory frame (i.e., the white point information of the second white point), are obtained and stored. For example, the memory frame is the last frame previewed by the first camera at the reference magnification level.

[0213] The magnification switching event, which switches from the reference magnification to other magnifications, is triggered by the user's magnification switching operation, or it can be automatically triggered by the camera application according to shooting needs. This application embodiment does not limit this.

[0214] Furthermore, other information corresponding to the memory frame can also be acquired and stored, such as one or more of the following: the second estimated color temperature of the memory frame, and the brightness information of the shooting environment corresponding to the memory frame.

[0215] It should be noted that the meanings of the second white point, the white point information of the second white point, the second ambient color temperature, the second estimated color temperature, and the brightness information mentioned above can be found in the above description. Figure 7 The relevant descriptions in the embodiments will not be repeated here.

[0216] Step B2: When the first camera takes a picture using a first magnification other than the reference magnification, determine the first white point of the current frame taken by the first camera using the first magnification, and the first ambient color temperature of the shooting environment corresponding to the current frame.

[0217] As an example, the current frame is the current frame previewed in real time at the first magnification after the first camera switches to the first magnification.

[0218] Furthermore, other information corresponding to the current frame captured by the first camera using the first magnification can also be obtained, such as the first estimated color temperature of the current frame.

[0219] It should be noted that the meanings of the first white point, the white point information of the first white point, the first ambient color temperature, and the first estimated color temperature mentioned above can be found in the above text. Figure 7 The relevant descriptions in the embodiments will not be repeated here.

[0220] Step B3: Determine the fusion weights based on the ambient color temperature difference between the first ambient color temperature and the second ambient color temperature of the shooting environment corresponding to the memory frame.

[0221] The ambient color temperature difference between the first and second ambient color temperatures represents the environmental difference between the current shooting environment and the shooting environment of the memory frame. The ambient color temperature difference is inversely proportional to the fusion weight; the smaller the ambient color temperature difference, the greater the fusion weight.

[0222] Thus, when the ambient color temperature difference is small, it means that the difference between the current shooting environment and the shooting environment of the memory frame is small. In this case, by applying a larger fusion weight to the first white point of the memory frame to a greater extent, the color correction of the current frame can be performed to a greater extent, making the color of the corrected image of the current frame closer to the color of the image of the memory frame. When the ambient color temperature difference is large, it means that the difference between the current shooting environment and the shooting environment of the memory frame is large. When the shooting environment difference is large, it is normal for the current frame and the memory frame to have color differences. In this case, by applying a smaller fusion weight to the first white point of the memory frame to a smaller extent, the color correction of the current frame can be performed to a smaller extent.

[0223] In one embodiment, the fusion weight can be determined based on the ambient color temperature difference and the correspondence between the ambient color temperature difference and the fusion weight. In the correspondence between the ambient color temperature difference and the fusion weight, the ambient color temperature difference is inversely proportional to the fusion weight; that is, the smaller the ambient color temperature difference, the larger the fusion weight.

[0224] In another embodiment, determining the blending weights based on the ambient color temperature difference may further include the following steps:

[0225] 1) Determine the first sub-weight based on the ambient color temperature difference. The smaller the ambient color temperature difference, the larger the first sub-weight.

[0226] 2) Determine the second sub-weight based on the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature of the memory frame. The larger the estimated color temperature difference, the larger the second sub-weight.

[0227] 3) Determine the fusion weight based on the first sub-weight and the second sub-weight.

[0228] For example, the product of the first sub-weight and the second sub-weight can be determined as the fusion weight. It should be understood that other methods can also be used to process the first sub-weight and the second sub-weight to obtain the fusion weight, and the embodiments of this application do not limit this.

[0229] In another embodiment, determining the blending weights based on the ambient color temperature difference may further include the following steps:

[0230] 1) Determine the first sub-weight based on the ambient color temperature difference. The smaller the ambient color temperature difference, the larger the first sub-weight.

[0231] 2) Determine the second sub-weight based on the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature of the memory frame. The larger the estimated color temperature difference, the larger the second sub-weight.

[0232] 3) Determine the third sub-weight based on the ambient brightness information of the shooting environment corresponding to the memory frame. The greater the brightness indicated by the ambient brightness information, the greater the third sub-weight.

[0233] 4) Determine the fusion weight based on the first sub-weight, the second sub-weight, and the third sub-weight.

[0234] For example, the product of the first sub-weight, the second sub-weight, and the third sub-weight can be determined as the fusion weight. It should be understood that other methods can also be used to process the first sub-weight, the second sub-weight, and the third sub-weight to obtain the fusion weight, and the embodiments of this application do not limit this.

[0235] The method for determining each sub-weight can be found above. Figure 7The relevant descriptions in the embodiments will not be repeated here.

[0236] Step B4: Based on the fusion weight, fuse the white point information of the first white point and the white point information of the second white point to obtain the white point information of the third white point.

[0237] For example, based on this fusion weight, the coordinates of the first white point and the second white point can be fused using the following formula (5) to obtain the logarithmic result of the coordinates of the third white point:

[0238] (5)

[0239] in, The result is the logarithm of the coordinates of the third white point. The logarithm of the coordinates of the first white point is given, and w is the fusion weight. This is the logarithmic result of the coordinates of the second white point.

[0240] Next, the coordinates of the third white point are determined based on the logarithm of its coordinates. For example, the logarithm of the third white point's coordinates can be used to perform an exponential operation to obtain the coordinates of the third white point, thus transforming the coordinates of the third white point from the log domain to the original domain.

[0241] It should be noted that, compared to the multi-camera color correction algorithm mentioned above, since the multi-magnification color correction algorithm performs color correction on images captured by the same camera, it is not necessary to map the second white point of the memory frame. The white point information of the first white point and the white point information of the second white point can be directly fused according to the fusion weight.

[0242] Step B5: Perform white balance correction on the current frame based on the white point information of the third white point.

[0243] For example, the white balance gain can be determined based on the white point information of the third white point, and the current frame can be white-balanced accordingly. For instance, the white point information of the third white point is its coordinates (R / G, B / G). The white balance gain determined based on this information includes: R channel gain R_Gain = G / R, and B channel gain B_Gain = G / B. Correspondingly, the R' of each pixel in the white-balanced current frame will be R' = R'. R_Gai, B'=B B_Gain, where the G channel value remains unchanged. Here, R and B represent the original R and B values ​​for each pixel in the current frame, respectively.

[0244] Step B6: Based on the fusion weight, the first estimated color temperature and the second estimated color temperature are fused to obtain the third estimated color temperature.

[0245] For example, based on the fusion weight, the first estimated color temperature and the second estimated color temperature can be fused using the above formula (4) to obtain the third estimated color temperature.

[0246] Step B7: Based on the third estimated color temperature, perform image processing on the current frame after white balance correction.

[0247] The image processing may include one or more of color correction and lens shading correction.

[0248] For example, the color correction matrix of the current frame after white balance correction can be determined based on the third estimated color temperature, and the color correction of the current frame after white balance correction can be performed based on the color correction matrix.

[0249] Next, in conjunction with the above Figure 5 and Figure 6 The multi-camera color correction method provided in the embodiments of this application will be illustrated by example.

[0250] Please refer to Figure 12 , Figure 12 This is a flowchart of a multi-camera color correction method provided in an embodiment of this application. This method is applied in electronic devices, and this embodiment uses the default-activated main camera as a reference camera. Figure 12 As shown, the method includes the following steps:

[0251] Step 1201: The user launches the camera application.

[0252] For example, users can tap the camera app icon to launch the camera app.

[0253] Step 1202: In response to the user's startup action, the camera application is launched.

[0254] Step 1203: The camera application sends a call command 1 to the main camera.

[0255] Command 1 is used to activate the main camera. After the camera app starts, it can use the main camera by default for taking pictures.

[0256] Step 1204: The main camera performs a preview at the default magnification according to the call command 1.

[0257] For example, the default magnification can be 1X.

[0258] Step 1205: Detect the ambient color temperature using a multispectral sensor.

[0259] During the main camera preview, the multispectral sensor can detect the ambient color temperature of the shooting environment.

[0260] As an example, once the camera application is launched, it can send a launch command to the multispectral sensor so that the multispectral sensor can detect the ambient color temperature based on the launch command.

[0261] Step 1206: The main camera sends preview image frames to the ISP.

[0262] The image frames previewed by the main camera can be sent to the ISP for processing first.

[0263] Step 1207: The ISP sends the image frame to the AWB module.

[0264] For example, the ISP can call the AWB module based on the image frame to send the image frame to the AWB module for processing.

[0265] Step 1208: The AWB module uses the AWB algorithm to calculate the AWB white point coordinates, AWB color temperature, and LV of the image frame.

[0266] For example, the AWB module can use the AWB algorithm to estimate the white point of an image frame, obtain the AWB white point coordinates, estimate the color temperature based on the AWB white point coordinates, obtain the AWB color temperature, and estimate the brightness of the image frame to obtain the brightness information LV.

[0267] It should be noted that the embodiments in this application only illustrate the method of obtaining the LV by estimating the brightness of an image frame using the AWB module. It should be understood that the LV of an image frame can also be determined in other ways. For example, the LV can be obtained through the AE module, i.e., obtaining the LV output by the AE module, or the LV can be obtained through a brightness sensor. The embodiments in this application do not limit this method.

[0268] Step 1209: The user switches to the telephoto camera.

[0269] Users can switch the main camera to the telephoto camera as needed. For example, they can switch the main camera to telephoto mode by clicking the option corresponding to the telephoto camera displayed in the camera app's preview frame.

[0270] Step 1210: In response to the user's camera switching operation, the camera application sends call command 2 to the telephoto camera.

[0271] Instruction 2 is used to invoke the telephoto camera.

[0272] Step 1211: The camera application sends a camera switching notification to the multi-camera color correction module.

[0273] The camera switching notification may carry the identifier of the camera after switching, or it may carry the identifier of the camera before switching. This application embodiment does not limit this.

[0274] Step 1212: The multi-camera color correction module sends a memory frame data acquisition request to the AWB module based on the camera switching notification.

[0275] Among them, the memory frame refers to the last frame of the main camera preview. In other words, the memory frame data acquisition request is used to request the data of the last frame of the main camera preview, such as the AWB white point coordinates, AWB color temperature, LV, and other data of the last frame.

[0276] Step 1213: Based on the memory frame data acquisition request, the AWB module sends the AWB white point coordinates 1, AWB color temperature 1, and LV1 corresponding to the memory frame to the multi-camera color correction module.

[0277] Step 1214: The multi-camera color correction module sends a color temperature data acquisition request to the multispectral sensor.

[0278] Among them, the color temperature data acquisition request is used to request the ambient color temperature of the shooting environment, such as obtaining the ambient color temperature 1 of the shooting environment corresponding to the memory frame.

[0279] Step 1215: Based on the color temperature data acquisition request, the multispectral sensor sends the detected ambient color temperature 1 to the multi-camera color correction module.

[0280] Among them, ambient color temperature 1 is used to indicate the ambient color temperature of the shooting environment corresponding to the memory frame of the main camera.

[0281] Step 1216: The multi-camera color correction module stores the AWB white point coordinates 1, AWB color temperature 1, LV1, and ambient color temperature 1 corresponding to the memory frame.

[0282] In other words, the multi-camera color correction module stores the data corresponding to the memory frame so that the image frames previewed by other cameras can be color corrected later based on the data corresponding to the memory frame.

[0283] Step 1217: The telephoto camera performs a real-time preview according to the call command 2.

[0284] Step 1218: The telephoto camera sends the current frame of the real-time preview to the ISP.

[0285] Step 1219: The ISP sends the current frame to the AWB module.

[0286] The ISP can invoke the AWB module based on the current frame to send the current frame to the AWB module for processing.

[0287] Step 1220: The AWB module uses the AWB algorithm to calculate the AWB white point coordinates 2 and AWB color temperature 2 of the current frame.

[0288] For example, the AWB module can use the AWB algorithm to estimate the white point of the image frame, obtain the AWB white point coordinates 2, and then estimate the color temperature based on the AWB white point coordinates 2 to obtain the AWB color temperature 2.

[0289] Step 1221: The AWB module sends the AWB white point coordinates 2 and the AWB color temperature 2 to the multi-camera color correction module.

[0290] Step 1222: The multispectral sensor sends the detected ambient color temperature 2 to the multi-camera color correction module.

[0291] Among them, ambient color temperature 2 is used to indicate the ambient color temperature of the shooting environment corresponding to the current frame.

[0292] Step 1223: The multi-camera color correction module calculates the coordinates of the fused white point and the fused color temperature using the multi-camera color correction algorithm based on the memory frame and the corresponding data of the current frame.

[0293] The specific implementation process of using a multi-camera color correction algorithm to calculate the coordinates of the fused white point and the fused color temperature can be found above. Figure 7 The relevant descriptions in the embodiments will not be repeated here.

[0294] Step 1224: The multi-camera color correction module sends the coordinates of the fused white point and the fused color temperature to the AWB module.

[0295] Step 1225: The AWB module performs white balance correction on the current frame based on the coordinates of the merged white point.

[0296] Step 1226: The AWB module sends the current frame with fused color temperature and white balance correction to the image processing module.

[0297] Step 1227: The image processing module processes the current frame after white balance correction based on the fused color temperature to obtain the target image frame.

[0298] For example, the image processing module may include one or more image processing modules such as CCM module and LSC module. For instance, based on the fused color temperature, color correction and lens shading correction can be performed on the current frame after white balance correction to obtain the processed target image frame.

[0299] Step 1228: The image processing module sends the target image frame to the camera application.

[0300] For example, the image processing module can first return the target image frame to the ISP, and then the ISP can send the target image frame to other subsequent processing modules.

[0301] Step 1229: The camera application displays the target image frame.

[0302] For example, a camera app can display a frame of the target image in the preview interface.

[0303] Next, in conjunction with the above Figure 5 and Figure 6 The multi-magnification color correction method provided in the embodiments of this application will be illustrated by example. Figure 13 This is a flowchart of a multi-magnification color correction method provided in an embodiment of this application. This method is applied in electronic devices, and this embodiment uses the default magnification after the camera is started as a reference magnification. Figure 13 As shown, the method includes the following steps:

[0304] Step 1301: The user launches the camera application.

[0305] For example, users can tap the camera app icon to launch the camera app.

[0306] Step 1302: In response to the user's startup action, the camera application is launched.

[0307] Step 1303: The camera application sends a call command 1 to the main camera.

[0308] Command 1 is used to activate the main camera. After the camera app starts, it can use the main camera by default for taking pictures.

[0309] Step 1304: The main camera performs a preview at the default 1X magnification according to the call command 1.

[0310] Step 1305: Detect the ambient color temperature using a multispectral sensor.

[0311] During the main camera preview, the multispectral sensor can detect the ambient color temperature of the shooting environment.

[0312] As an example, once the camera application is launched, it can send a launch command to the multispectral sensor so that the multispectral sensor can detect the ambient color temperature based on the launch command.

[0313] Step 1306: The main camera sends preview image frames to the ISP.

[0314] The image frames previewed by the main camera can be sent to the ISP for processing first.

[0315] Step 1307: The ISP sends the image frame to the AWB module.

[0316] For example, the ISP can call the AWB module based on the image frame to send the image frame to the AWB module for processing.

[0317] Step 1308: The AWB module uses the AWB algorithm to calculate the AWB white point coordinates, AWB color temperature, and LV of the image frame.

[0318] For example, the AWB module can use the AWB algorithm to estimate the white point of an image frame, obtain the AWB white point coordinates, estimate the color temperature based on the AWB white point coordinates, obtain the AWB color temperature, and estimate the brightness of the image frame to obtain the brightness information LV.

[0319] It should be noted that the embodiments in this application only illustrate the method of obtaining the LV by estimating the brightness of an image frame using the AWB module. It should be understood that the LV of an image frame can also be determined in other ways. For example, the LV can be obtained through the AE module or through a brightness sensor, and this application embodiment does not limit this method.

[0320] Step 1309: The user switches to 2X magnification.

[0321] Users can switch from 1X magnification to 2X magnification for shooting as needed. For example, they can switch from 1X to 2X magnification by clicking the 2X magnification option displayed in the preview frame of the camera application. Alternatively, they can switch from 1X to 2X magnification by zooming in and out on the preview screen. This application embodiment does not limit the operation of switching magnification.

[0322] Step 1310: In response to the user's magnification switching operation, the camera application sends a magnification switching command to the camera, which carries the 2X magnification to be switched.

[0323] Step 1311: The camera application sends a magnification switching notification to the multi-magnification color correction module.

[0324] The magnification switching notification may carry the magnification after the switch, or it may carry the magnification before the switch. This application embodiment does not limit this.

[0325] Step 1313: The multi-magnification color correction module sends a memory frame data acquisition request to the AWB module based on the magnification switching notification.

[0326] Among them, the memory frame refers to the last frame of the main camera preview using the default 1X magnification. In other words, the memory frame data acquisition request is used to request the data of the last frame of the main camera preview using the default 1X magnification, such as the AWB white point coordinates, AWB color temperature, LV, and other data of the last frame.

[0327] Step 1313: Based on the memory frame data acquisition request, the AWB module sends the AWB white point coordinates 1, AWB color temperature 1, and LV1 corresponding to the memory frame to the multi-magnification color correction module.

[0328] Step 1314: The multi-magnification color correction module sends a color temperature data acquisition request to the multispectral sensor.

[0329] Among them, the color temperature data acquisition request is used to request the ambient color temperature of the shooting environment, such as the ambient color temperature of the shooting environment corresponding to the memory frame.

[0330] Step 1315: Based on the color temperature data acquisition request, the multispectral sensor sends the detected ambient color temperature 1 to the multi-magnification color correction module.

[0331] Among them, ambient color temperature 1 is used to indicate the ambient color temperature of the shooting environment corresponding to the memory frame of the main camera.

[0332] Step 1316: The multi-rate color correction module stores the AWB white point coordinates 1, AWB color temperature 1, LV1 and ambient color temperature 1 corresponding to the memory frame.

[0333] In other words, the multi-magnification color correction module stores the data corresponding to the memory frame so that the image frames previewed by other cameras can be color corrected later based on the data corresponding to the memory frame.

[0334] Step 1317: The main camera switches to 2X magnification for real-time preview according to the magnification switching command.

[0335] Step 1318: The main camera sends the current frame of the real-time preview to the ISP.

[0336] Step 1319: The ISP sends the current frame to the AWB module.

[0337] The ISP can invoke the AWB module based on the current frame to send the current frame to the AWB module for processing.

[0338] Step 1320: The AWB module uses the AWB algorithm to calculate the AWB white point coordinates 2 and AWB color temperature 2 of the current frame.

[0339] For example, the AWB module can use the AWB algorithm to estimate the white point of the image frame, obtain the AWB white point coordinates 2, and then estimate the color temperature based on the AWB white point coordinates 2 to obtain the AWB color temperature 2.

[0340] Step 1321: The AWB module sends the AWB white point coordinates 2 and the AWB color temperature 2 to the multi-magnification color correction module.

[0341] Step 1322: The multispectral sensor sends the detected ambient color temperature 2 to the multi-magnification color correction module.

[0342] Among them, ambient color temperature 2 is used to indicate the ambient color temperature of the shooting environment corresponding to the current frame.

[0343] Step 1323: The multi-rate color correction module calculates the coordinates of the blended white point and the blended color temperature using the multi-rate color correction algorithm based on the memory frame and the corresponding data of the current frame.

[0344] The specific implementation process of calculating the blended white point coordinates and blended color temperature using the multi-magnification color correction algorithm can be found above. Figure 7 The relevant descriptions in the embodiments will not be repeated here.

[0345] Step 1324: The multi-magnification color correction module sends the coordinates of the blended white point and the blended color temperature to the AWB module.

[0346] Step 1325: The AWB module performs white balance correction on the current frame based on the coordinates of the merged white point.

[0347] Step 1326: The AWB module sends the current frame with fused color temperature and white balance correction to the image processing module.

[0348] Step 1327: The image processing module processes the current frame after white balance correction based on the fused color temperature to obtain the target image frame.

[0349] For example, the image processing module may include one or more image processing modules such as CCM module and LSC module. For instance, based on the fused color temperature, color correction and lens shading correction can be performed on the current frame after white balance correction to obtain the processed target image frame.

[0350] Step 1328: The image processing module sends the target image frame to the camera application.

[0351] For example, the image processing module can first return the target image frame to the ISP, and then the ISP can send the target image frame to other subsequent processing modules.

[0352] Step 1329: The camera application displays the target image frame.

[0353] For example, a camera app can display a frame of the target image in the preview interface.

[0354] This application also provides a chip coupled to a memory, which is used to read and execute computer programs or instructions stored in the memory to perform the methods described in the above embodiments.

[0355] This application also provides an electronic device including a chip for reading and executing computer programs or instructions stored in a memory, such that the methods in the various embodiments are performed.

[0356] This embodiment also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the methods described in the above embodiments.

[0357] This embodiment also provides a computer program product, which is a computer-readable storage medium storing program code. When the computer program product is run on a computer, it causes the computer to perform the above-described related steps to implement the method in the above embodiment.

[0358] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component, or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the methods in the above-described method embodiments.

[0359] In this embodiment, the electronic device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0360] This application does not specifically limit the structure of the execution subject of the method provided in this application embodiment. As long as a program containing the code of the method provided in this application embodiment can be run to perform video processing according to the method provided in this application embodiment, it is acceptable. For example, the execution subject of the method provided in this application embodiment can be an electronic device, or a functional module in an electronic device that can call and execute a program.

[0361] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0362] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0363] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0364] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium and includes several instructions that cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0365] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, Applied to an electronic device, the electronic device including a plurality of cameras, the plurality of cameras including a reference camera, the method includes: When the first camera is used to take a picture, the first white point of the current frame captured by the first camera and the first ambient color temperature of the shooting environment corresponding to the current frame are determined. The first camera is any one of the plurality of cameras except the reference camera. Based on the difference in ambient color temperature between the first ambient color temperature and the second ambient color temperature of the shooting environment corresponding to the memory frame of the reference camera, the white point information of the first white point and the white point information of the second white point in the memory frame are fused to obtain the white point information of the third white point. Based on the white point information of the third white point, white balance correction is performed on the current frame.

2. The method as described in claim 1, characterized in that, The step of fusing the white point information of the first white point and the white point information of the second white point in the memory frame based on the ambient color temperature difference between the first ambient color temperature and the second ambient color temperature of the shooting environment corresponding to the memory frame of the reference camera to obtain the white point information of the third white point includes: Based on the ambient color temperature difference, a first fusion weight corresponding to the first white point and a second fusion weight corresponding to the second white point are determined; wherein, the smaller the ambient color temperature difference, the larger the second fusion weight; the larger the second fusion weight, the smaller the first fusion weight. Based on the first fusion weight and the second fusion weight, the white point information of the first white point and the white point information of the second white point are fused to obtain the white point information of the third white point.

3. The method as described in claim 2, characterized in that, Before fusing the white point information of the first white point and the white point information of the second white point according to the first fusion weight and the second fusion weight, the method further includes: The second white dot is mapped onto the first camera to obtain the mapped white dot; The step of fusing the white point information of the first white point and the white point information of the second white point according to the first fusion weight and the second fusion weight to obtain the white point information of the third white point includes: Based on the first fusion weight and the second fusion weight, the white point information of the first white point and the white point information of the mapped white point are fused to obtain the white point information of the third white point.

4. The method as described in claim 3, characterized in that, The step of mapping the second white point to the first camera to obtain the mapped white point includes: Acquire calibration data for the reference camera and the first camera, the calibration data including white point information and color temperature under multiple standard light sources; Based on the color temperature difference between the estimated color temperature of the memory frame and the color temperature of the memory frame, at least one standard light source is determined from the plurality of standard light sources; Based on the white point information of the reference camera under at least one standard light source and the white point information of the first camera under at least one standard light source, a white point mapping matrix between the reference camera and the first camera is determined; Based on the white point mapping matrix, the second white point is mapped to the first camera to obtain the mapped white point.

5. The method as described in claim 2, characterized in that, Before determining the second fusion weight corresponding to the second white point based on the ambient color temperature difference, the method further includes: Obtain the first estimated color temperature of the current frame; The step of determining the second fusion weight corresponding to the second white point based on the ambient color temperature difference includes: A first sub-weight is determined based on the ambient color temperature difference; wherein, the smaller the ambient color temperature difference, the larger the first sub-weight. A second sub-weight is determined based on the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature of the memory frame; wherein, the larger the estimated color temperature difference, the larger the second sub-weight. The second fusion weight is determined based on the first sub-weight and the second sub-weight.

6. The method as described in claim 5, characterized in that, Before determining the second fusion weight based on the first sub-weight and the second sub-weight, the method further includes: A third sub-weight is determined based on the ambient brightness information of the shooting environment corresponding to the memory frame. The greater the brightness indicated by the ambient brightness information, the greater the third sub-weight. The step of determining the second fusion weight based on the first sub-weight and the second sub-weight includes: The second fusion weight is determined based on the first sub-weight, the second sub-weight, and the third sub-weight.

7. The method as described in claim 6, characterized in that, Determining the second fusion weight based on the first sub-weight, the second sub-weight, and the third sub-weight includes: The product of the first sub-weight, the second sub-weight, and the third sub-weight is determined as the second fusion weight.

8. The method as described in claim 3, characterized in that, The second fusion weight, the white point information of the first white point, and the white point information of the mapped white point satisfy the following formula: The white point information of the third white point = the white point information of the first white point * (1-W) + the white point information of the mapped white point * W; where W is the second fusion weight.

9. The method according to any one of claims 1-8, characterized in that, The step of performing white balance correction on the current frame based on the white point information of the third white point includes: Based on the white point information of the third white point, determine the white balance gain; The current frame is corrected based on the white balance gain.

10. The method according to any one of claims 1-8, characterized in that, Before performing white balance correction on the current frame based on the white point information of the third white point, the method further includes: The first estimated color temperature of the current frame and the second estimated color temperature of the memory frame are fused to obtain a third estimated color temperature; After performing white balance correction on the current frame based on the white point information of the third white point, the method further includes: Based on the third estimated color temperature, color correction is performed on the current frame after white balance correction.

11. An image processing method, characterized in that, The method is applied in an electronic device, the electronic device including a first camera, the first camera supporting multiple magnifications, the multiple magnifications including a reference magnification, and the image output mode corresponding to the reference magnification is a first image output mode, the method including: When the first camera takes a picture using the first magnification, the first white point of the current frame taken by the first camera using the first magnification and the first ambient color temperature of the shooting environment corresponding to the current frame are determined. The first magnification is a magnification other than the reference magnification among the multiple magnifications and the corresponding output mode is different from the first output mode. Based on the difference in ambient color temperature between the first ambient color temperature and the second ambient color temperature of the shooting environment corresponding to the memory frame of the first camera, the white point information of the first white point and the white point information of the second white point of the memory frame are fused to obtain the white point information of the third white point. The memory frame refers to the last frame captured by the first camera using the reference magnification. Based on the white point information of the third white point, white balance correction is performed on the current frame.

12. The method as described in claim 11, characterized in that, The step of fusing the white point information of the first white point and the white point information of the second white point in the memory frame based on the ambient color temperature difference between the first ambient color temperature and the second ambient color temperature of the shooting environment corresponding to the memory frame of the first camera to obtain the white point information of the third white point includes: Based on the ambient color temperature difference, a first fusion weight corresponding to the first white point and a second fusion weight corresponding to the second white point are determined; wherein, the smaller the ambient color temperature difference, the larger the second fusion weight; the larger the second fusion weight, the smaller the first fusion weight. Based on the first fusion weight and the second fusion weight, the white point information of the first white point and the white point information of the second white point are fused to obtain the white point information of the third white point.

13. The method as described in claim 12, characterized in that, Before determining the second fusion weight corresponding to the second white point based on the ambient color temperature difference, the method further includes: Obtain the first estimated color temperature of the current frame; The step of determining the second fusion weight corresponding to the second white point based on the ambient color temperature difference includes: A first sub-weight is determined based on the ambient color temperature difference; wherein, the larger the ambient color temperature difference, the smaller the first sub-weight. A second sub-weight is determined based on the estimated color temperature difference between the first estimated color temperature and the second estimated color temperature of the memory frame; wherein, the larger the estimated color temperature difference, the larger the second sub-weight. The second fusion weight is determined based on the first sub-weight and the second sub-weight.

14. The method as described in claim 13, characterized in that, Before determining the second fusion weight based on the first sub-weight and the second sub-weight, the method further includes: A third sub-weight is determined based on the ambient brightness information of the shooting environment corresponding to the memory frame; wherein, the greater the brightness indicated by the ambient brightness information, the greater the third sub-weight. The step of determining the second fusion weight based on the first sub-weight and the second sub-weight includes: The second fusion weight is determined based on the first sub-weight, the second sub-weight, and the third sub-weight.

15. The method as described in claim 14, characterized in that, Determining the second fusion weight based on the first sub-weight, the second sub-weight, and the third sub-weight includes: The product of the first sub-weight, the second sub-weight, and the third sub-weight is determined as the second fusion weight.

16. The method as described in claim 12, characterized in that, The second fusion weight, the white point information of the first white point, and the white point information of the second white point satisfy the following formula: The white point information of the third white point = the white point information of the first white point * (1-W) + the white point information of the second white point * W; where W is the second fusion weight.

17. The method as described in claim 11, characterized in that, The step of performing white balance correction on the current frame based on the white point information of the third white point includes: Based on the white point information of the third white point, determine the white balance gain; The current frame is corrected based on the white balance gain.

18. The method as described in any one of claims 11-17, characterized in that, Before performing white balance correction on the current frame based on the white point information of the third white point, the method further includes: The first estimated color temperature of the current frame and the second estimated color temperature of the memory frame are fused to obtain a third estimated color temperature; After performing white balance correction on the current frame based on the third white point information, the method further includes: Based on the third estimated color temperature, color correction is performed on the current frame after white balance correction.

19. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the method as claimed in any one of claims 1-10 or 11-18.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the method as claimed in any one of claims 1-10 or 11-18.

Images