Shielding identification method and equipment of camera and storage medium

Abstract

The invention discloses a camera shielding identification method and device and a storage medium, and belongs to the technical field of shooting. The method is applied to the electronic equipment comprising a plurality of cameras, and comprises the following steps: under the condition that the electronic equipment is switched from a first camera to a second camera for shooting, determining first statistical information of a memory frame of the first camera and second statistical information of a current frame shot by the second camera; wherein the memory frame is the last frame shot by the first camera, the first statistical information comprises a first target white point and a plurality of first statistical white points of the memory frame, and the second statistical information comprises a second target white point and a plurality of second statistical white points of the current frame; according to the difference between the first statistical information and the second statistical information, a shielding recognition result of the first camera is determined, and the shielding recognition result represents whether the first camera is shielded or not. According to the invention, the accuracy of identifying whether the camera is shielded can be improved.

Description

Technical Field

[0001] This application relates to the field of shooting technologies, and particularly to a method, device, and storage medium for identifying occlusion of a camera. Background Art

[0002] With the improvement of users' shooting requirements, the camera modules of electronic devices such as mobile phones have developed from single cameras to multi-cameras, that is, electronic devices are configured with multiple cameras, such as main cameras, telephoto cameras, and ultra-wide-angle cameras. In some scenarios, for example, when the electronic device switches from one camera to another for shooting, it may be necessary to identify whether the camera before the switch is occluded, so as to perform corresponding processing according to the recognition result. For example, according to the recognition result, it is determined whether to use the image captured by the camera before the switch as a reference image to correct the image captured by the camera after the switch.

[0003] Currently, when the electronic device switches from one camera to another for shooting, the image brightness difference between the image captured by the camera before the switch and the image captured by the camera after the switch can be determined, and whether the camera before the switch is occluded is identified according to the image brightness difference. However, in the case where the occlusion of the camera before the switch is not strict or not complete, the image brightness difference between the images captured by the two cameras before and after the switch will be very small, and it is difficult to distinguish whether the camera before the switch is occluded according to the image brightness difference, resulting in incorrect occlusion recognition. Therefore, the accuracy of identifying whether a camera is occluded according to the image brightness difference is relatively low. Summary of the Invention

[0004] This application provides a method, device, and storage medium for identifying occlusion of a camera, which can improve the accuracy of identifying whether a camera is occluded. The technical solutions are as follows:

[0005] In a first aspect, a method for identifying occlusion of a camera is provided, which is applied to an electronic device. The electronic device includes multiple cameras. The method includes: after the electronic device switches from a first camera to a second camera for shooting, determining first statistical information of a memory frame of the first camera and second statistical information of a current frame captured by the second camera. According to the difference between the first statistical information and the second statistical information, determining an occlusion recognition result of the first camera, that is, determining whether the first camera is occluded.

[0006] Wherein, the memory frame is the last frame captured by the first camera, the first statistical information includes a first target white point and multiple first statistical white points of the memory frame, and the second statistical information includes a second target white point and multiple second statistical white points of the current frame.

[0007] Since the statistical information of an image can more accurately reflect image features such as the color temperature of the image, and there will be obvious differences in the statistical information of the images captured by the corresponding camera when the camera is occluded and when it is not occluded, therefore, based on the difference in the statistical information of the images captured by the cameras before and after the switch, it is possible to more accurately identify whether the camera before the switch is occluded, thereby improving the accuracy of identifying whether the camera is occluded. In addition, according to the occlusion recognition result identified by the present application, the image captured by the camera after the switch can be more accurately corrected, improving the accuracy of image correction. For example, to a certain extent, it can avoid the serious color cast situation caused by using the image captured by the occluded camera as a reference image for color correction.

[0008] Among them, the scenario of switching from the first camera to the second camera for shooting may include: the scenario of directly switching from the first camera to the second camera for shooting, or the scenario of first switching from the first camera to other cameras and then switching from other cameras to the second camera. The embodiments of the present application do not limit this. The first camera can be any one of the multiple cameras configured for the electronic device, and the second camera is any one of the multiple cameras other than the first camera.

[0009] As an example, a certain camera among the multiple cameras configured for the electronic device can be pre-set as a reference camera, so as to correct the colors of the images captured by other cameras based on the image parameters of the reference camera, making the colors of the images captured by other cameras all approach the colors of the images captured by the reference camera, thereby reducing the color differences in multi-camera shooting in the same scene and improving the color consistency of multi-camera shooting. Among them, the reference camera can be any one of the multiple cameras, such as the camera defaultly called by the camera application. Usually, when the camera application is started, the main camera is defaultly called for shooting, so the main camera can be set as the reference camera. Correspondingly, the first camera is the pre-set reference camera. That is to say, in the embodiments of the present application, when the electronic device switches from the reference camera to other cameras for shooting, the method for identifying occlusion of the camera provided by the embodiments of the present application can be used to determine whether the reference camera is occluded.

[0010] In one embodiment, the target quantity ratio can be determined according to the first statistical information; if the target quantity ratio is less than the first threshold, then the first ratio and the second ratio are determined according to the first statistical information and the second statistical information; if the ratio of the first ratio to the second ratio is greater than the first ratio threshold or less than the second ratio threshold, it is determined that the first camera is occluded.

[0011] Among them, the target quantity ratio is the ratio of the target quantity to the total quantity of multiple first statistical white points. The target quantity refers to the quantity of first statistical white points among multiple first statistical white points whose white point coordinates are within the range centered on the white point coordinates of the first target white point and with a preset distance as the radius. The preset distance can be set in advance according to needs. For example, the preset distance can be an empirical value obtained by statistically analyzing the target white points and statistical white points in the images captured by the first camera under multiple standard light sources.

[0012] It should be noted that in the case of occlusion of the main camera, the image it captures is basically a solid color. For example, when the camera is blocked by a hand, the captured image is reddish, the B / G values of its statistical white points are similar, and the statistical white points generally show a horizontal straight-line distribution in the coordinate system with R / G as the abscissa and B / G as the ordinate. This distribution method will cause the statistical white points to be far from the target white points, and the number of statistical white points falling within the circle centered on the target white point and with a preset distance r as the radius is small. Therefore, the embodiment of the present application can determine whether the first camera is occluded according to the target quantity ratio.

[0013] For example, when the target quantity ratio is less than the first threshold, it indicates that the statistical white points of the memory frame are far from the target white points, and the first camera may be occluded. When the target quantity ratio is greater than or equal to the first threshold, it indicates that the statistical white points of the memory frame are close to the target white points, and the first camera is not occluded.

[0014] Among them, the first ratio is the ratio of the mean value of the first color statistical values of multiple first statistical white points to the mean value of the second color statistical values, and the second ratio is the ratio of the mean value of the first color statistical values of multiple second statistical white points to the mean value of the second color statistical values. The first color statistical value is the ratio of the R value to the G value of the corresponding statistical white point, and the second color statistical value is the ratio of the B value to the G value of the corresponding statistical white point.

[0015] In the embodiment of the present application, when the target quantity ratio is less than the first threshold, it can be preliminarily determined that the first camera may be occluded, and then other information can be combined to further determine whether the first camera is occluded. For example, the first ratio and the second ratio can be determined according to the first statistical information and the second statistical information, and whether the first camera is occluded can be determined according to the first ratio and the second ratio.

[0016] As an example, after determining that the target quantity ratio is less than the first threshold, it can also be first determined whether the target quantity ratio is less than or equal to the second threshold. If so, it is determined that the first camera is occluded; if not, the first ratio and the second ratio are further determined according to the first statistical information and the second statistical information, so as to determine whether the first camera is occluded according to the ratio of the first ratio to the second ratio. In this way, the recognition efficiency can be improved.

[0017] In one embodiment, after determining the occlusion recognition result of the first camera, it is also possible to determine whether to execute an image correction process based on the camera recognition result, that is, to determine whether to use the memory frame of the first camera as a reference image to correct the image frame captured by the second camera. Since the camera occlusion recognition method provided in the embodiments of the present application can more accurately identify whether the camera is occluded, the occlusion recognition result identified according to the present application can more accurately correct the image captured by the switched camera, improving the accuracy of image correction. For example, it can, to a certain extent, avoid the serious color cast caused by using the image captured by the occluded camera as a reference image for color correction.

[0018] As an example, if the occlusion recognition result indicates that the first camera is not occluded, the white point information of the first target white point and the white point information of the second target white point of the memory frame are fused, and the current frame is white balance corrected according to the fusion result; if the occlusion recognition result indicates that the first camera is occluded, the current frame is white balance corrected according to the white point information of the second target white point.

[0019] As an example, fusing the white point information of the first target white point and the white point information of the second target white point of the memory frame and white balance correcting the current frame according to the fusion result includes: fusing the white point information of the first target white point and the white point information of the second target white point of the memory frame according to the ambient color temperature difference between the first ambient color temperature of the shooting environment corresponding to the memory frame and the second ambient color temperature of the shooting environment corresponding to the current frame to obtain the white point information of the third target white point; white balance correcting the current frame according to the white point information of the third target white point.

[0020] In this way, when the ambient color temperature difference between the second camera and the first camera is small, it can be determined that the shooting scene difference is small, and it is likely to be shooting in the same scene. Combining the target white point information of the memory frame of the first camera and the target white point information of the current frame, color correction is performed on the current frame by white balance correcting the current frame to reduce the color difference between the current frame and the memory frame of the first camera, thereby reducing the color difference and the resulting visual difference between the images captured by other cameras and the first camera in the same scene, improving the color consistency of multi-camera shooting in the same scene, and further improving the user's visual experience and shooting experience.

[0021] In one embodiment, the fusion weight can be determined according to the ambient color temperature difference; the white point information of the first target white point and the white point information of the second target white point are fused according to the fusion weight to obtain the white point information of the third target white point.

[0022] Among them, the ambient color temperature difference between the second ambient color temperature and the first ambient color temperature can represent the ambient difference between the current shooting environment and the shooting environment of the memory frame. The ambient color temperature difference is inversely proportional to the fusion weight. The smaller the ambient color temperature difference, the larger the fusion weight.

[0023] In this way, when the ambient color temperature difference is small, it means that the difference between the current shooting environment and the shooting environment of the memory frame is small. In this case, by fusing the second target white point of the memory frame to a greater extent according to the larger fusion weight, a greater degree of color correction can be performed on the current frame, so that the color of the corrected image of the current frame is closer to the color of the image of the memory frame to a greater extent; when the ambient color temperature difference is large, it means that the difference between the current shooting environment and the shooting environment of the memory frame is large. In the case of a large difference in the shooting environment, it is normal for the current frame and the memory frame to have a color difference. In this case, a smaller degree of color correction can be performed on the current frame by fusing the second target white point of the memory frame to a smaller extent according to the smaller fusion weight.

[0024] In one embodiment, the first target white point can also be mapped to the second camera first to obtain a mapped white point; then, according to the fusion weight, the white point information of the mapped white point and the white point information of the second target white point are fused to obtain the white point information of the third target white point.

[0025] By first mapping the first target white point of the memory frame to the second camera to obtain a mapped white point, and then fusing the white point information of the mapped white point with the white point information of the second target white point of the current frame, the accuracy of multi-camera color correction can be further improved.

[0026] In one embodiment, mapping the first target white point to the second camera to obtain a mapped white point includes: obtaining the calibration data of the first camera and the first camera, where the calibration data includes the white point information and color temperature under multiple standard light sources; determining at least one standard light source from the multiple standard light sources according to the color temperature difference from the estimated color temperature of the memory frame; determining the white point mapping matrix between the first camera and the second camera according to the white point information of the first camera under at least one standard light source and the white point information of the second camera under at least one standard light source; and mapping the first target white point to the second camera according to the white point mapping matrix to obtain a mapped white point. In this way, the accuracy of white point mapping can be improved.

[0027] In one embodiment, determining the fusion weight according to the ambient color temperature difference includes: determining a first sub-weight according to the ambient color temperature difference; determining a second sub-weight according to the estimated color temperature difference between the first estimated color temperature of the memory frame and the second estimated color temperature of the current frame; and determining the fusion weight according to the first sub-weight and the second sub-weight.

[0028] Among them, the smaller the environmental color temperature difference, the larger the first sub-weight; the larger the estimated color temperature difference, the larger the second sub-weight.

[0029] Among them, the estimated color temperature difference between the second estimated color temperature and the first estimated color temperature can indicate the shooting effect difference between the second camera and the first camera. The estimated color temperature difference is proportional to the second sub-weight, and the larger the estimated color temperature difference, the larger the second sub-weight.

[0030] In this way, when the estimated color temperature difference is small, it means that the shooting effect difference between the second camera and the first camera is small. In this case, a relatively small degree of color correction can be performed on the current frame according to the smaller second sub-weight. When the estimated color temperature difference is large, it means that the shooting effect difference between the second camera and the first camera is large. In this case, a relatively large degree of color correction is performed on the current frame according to the larger second sub-weight to improve the shooting effect difference.

[0031] In one embodiment, the third sub-weight can also be determined first according to the environmental brightness information of the shooting environment corresponding to the memory frame. Then, according to the first sub-weight, the second sub-weight, and the third sub-weight, the fusion weight is determined. For example, the product of the first sub-weight, the second sub-weight, and the third sub-weight is determined as the fusion weight. For example, the white point information of the fusion weight, the white point information of the first target white point, and the white point information of the mapped white point satisfy the following formula: the white point information of the third target white point = the white point information of the second target white point * (1 - W) + the white point information of the mapped white point * W; where W is the fusion weight.

[0032] Among them, the greater the brightness indicated by the environmental brightness information, the larger the third sub-weight. The environmental brightness information of the shooting environment corresponding to the memory frame is used to indicate the environmental brightness of the shooting environment corresponding to the memory frame, such as brightness parameters such as LV. Generally, the smaller the environmental brightness, the more complex the shooting environment and the worse the shooting effect. The greater the environmental brightness, the better the shooting effect. The brightness indicated by the environmental brightness information is proportional to the third sub-weight, and the greater the brightness indicated by the environmental brightness information, the larger the third sub-weight.

[0033] In this way, when the brightness indicated by the environmental brightness information is smaller, the image effect of the memory frame of the first camera is worse. In this case, a relatively small degree of color correction can be performed on the current frame according to the smaller third sub-weight. When the brightness indicated by the environmental brightness information is larger, it means that the image effect of the memory frame of the first camera is better. In this case, a relatively large degree of color correction can be performed on the current frame according to the larger third sub-weight. Adopting this strategy can ensure the accuracy of color correction.

[0034] In one embodiment, performing white balance correction on the current frame according to the white point information of the third target white point includes: determining a white balance gain according to the white point information of the third target white point; and correcting the current frame according to the white balance gain.

[0035] In one embodiment, the first estimated color temperature of the memory frame and the second estimated color temperature of the current frame can also be fused to obtain a third estimated color temperature; according to the third estimated color temperature, color correction is performed on the current frame after white balance adjustment. In this way, the accuracy of color correction can be improved, thereby further improving the color consistency of multi-camera images.

[0036] In a second aspect, an occlusion recognition device for a camera is provided. The occlusion recognition device for the camera has a function of implementing the behavior of the occlusion recognition method for the camera in the first aspect above. The occlusion recognition device for the camera includes at least one module, and the at least one module is used to implement the occlusion recognition method for the camera provided in the first aspect above.

[0037] In a third aspect, an occlusion recognition device for a camera is provided. The structure of the occlusion recognition device for the camera includes a processor and a memory. The memory is used to store a program that supports the occlusion recognition device for the camera to execute the occlusion recognition method provided in the first aspect above, and to store data related to implementing the occlusion recognition method for the camera described in the first aspect above. The processor is configured to execute the program stored in the memory. The occlusion recognition device for the camera may further include a communication bus, and the communication bus is used to establish a connection between the processor and the memory.

[0038] In a fourth aspect, a computer-readable storage medium is provided. Instructions are stored in the computer-readable storage medium, and when it runs on a computer, it causes the computer to execute the occlusion recognition method for the camera described in the first aspect above.

[0039] In a fifth aspect, a computer program product containing instructions is provided. When it runs on a computer, it causes the computer to execute the occlusion recognition method for the camera described in the first aspect or the second aspect above.

[0040] The technical effects obtained in the second aspect, the third aspect, the fourth aspect, and the fifth aspect are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be elaborated here. Description of the Drawings

[0041] Figure 1 is a schematic diagram of the spatial distribution of the cameras of an electronic device provided by an embodiment of the present application;

[0042] Figure 2 is a schematic diagram of the comparison of the images captured before and after the camera switch provided by an embodiment of the present application;

[0043] Figure 3It is a schematic diagram of the statistical information difference of the images captured before and after the camera switch provided by the embodiment of the present application;

[0044] Figure 4 It is another schematic diagram of the statistical information difference of the images captured before and after the camera switch provided by the embodiment of the present application;

[0045] Figure 5 It is a schematic structural diagram of an electronic device provided by the embodiment of the present application;

[0046] Figure 6 It is a block diagram of the software system of an electronic device provided by the embodiment of the present application;

[0047] Figure 7 It is a schematic logical diagram of an image processing process provided by the embodiment of the present application;

[0048] Figure 8 It is a schematic flowchart of a method for identifying occlusion of a camera provided by the embodiment of the present application;

[0049] Figure 9 It is a comparative schematic diagram of the white point distribution of the images captured by two main cameras and the images captured by a telephoto camera provided by the embodiment of the present application;

[0050] Figure 10 It is a schematic logical diagram of a method for identifying occlusion of a camera provided by the embodiment of the present application;

[0051] Figure 11 It is a schematic flowchart of a multi-camera color correction algorithm provided by the embodiment of the present application;

[0052] Figure 12 It is a comparative schematic diagram of the effects before and after color correction of an image frame provided by the embodiment of the present application;

[0053] Figure 13 It is a schematic flowchart of image correction for the image captured by the current camera based on the occlusion recognition result provided by the embodiment of the present application. Detailed implementation manners

[0054] To make the objectives, technical solutions, and advantages of the present application clearer, the embodiments of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0055] It should be understood that the "multiple" mentioned in this application refers to two or more. In the description of this application, unless otherwise specified, " / " means "or". For example, A / B can mean A or B; "and / or" in this text is just a relational description of related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, for the convenience of clearly describing the technical solution of this application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and roles. Those skilled in the art can understand that the terms such as "first" and "second" do not limit the quantity and execution order, and the terms such as "first" and "second" do not necessarily limit to being different.

[0056] For the convenience of understanding, first, the names involved in the embodiments of this application will be explained.

[0057] White point: That is, the point on a white object. White refers to the visual response formed by the light reflected into the human eye due to the same proportion of blue, green, and red light and a certain brightness. A white object refers to an object that a person thinks is white when seen by the human eye in any scene, such as a white wall, a gray tabletop, white paper, etc. However, it should be noted that these white objects generally have colors when photographed by a camera (that is, the corresponding red (R), green (G), and blue (B) values are not equal), and their color values are determined by the light source. Therefore, the white objects (human eye perception) in the scene are generally not white (image value) in the image. The white point, like the white object, actually has a color.

[0058] White point information: White point information is used to indicate the color information of the corresponding white point. For example, the white point information can be the white point coordinates of the corresponding white point. The white point coordinates in this application refer to the coordinates of the white point in a coordinate system with R / G as the abscissa and B / G as the ordinate, that is, the white point coordinates are (R / G, B / G).

[0059] Target white point: The target white point in this application refers to the white point obtained by estimating the white point of an image using a white balance (WB) algorithm. For example, the WB algorithm can be an auto white balance (AWB) algorithm. Correspondingly, the target white point can be called the AWB white point. Exemplarily, the AWB algorithm can be a gray world algorithm, a perfect reflection algorithm, a dynamic threshold algorithm, etc. The embodiments of this application do not limit this. Generally, the number of target white points of an image is one.

[0060] Statistical white point: The statistical white point refers to the pixel points that meet the white point conditions statistically obtained based on the RGB values of each pixel point in the image, such as the white point statistically obtained using the WB algorithm. Generally, the number of statistical white points of an image is multiple.

[0061] Color temperature: Color temperature is a measure of the light color of a light source, with the unit of Kelvin (K). Color temperature is defined based on an absolute black body. When the radiation of the light source in the visible region is exactly the same as that of the absolute black body, the temperature of the black body at this time is called the color temperature of the light source. The higher the color temperature, the bluer the white light tone; the lower the color temperature, the yellower the white light tone.

[0062] White balance: Since the sensor in the camera cannot change its photosensitive characteristics according to the change of ambient light like the human eye, under light sources with different color temperatures, the response of white in the camera's sensor will be bluish or reddish. White balance is to restore the white color after imaging under ambient light with different color temperatures to the real white (usually the white observed by the human eye under natural daylight ambient light) through the white balance algorithm.

[0063] The white balance algorithm can make the white color truly presented by adjusting the intensities of the three color channels of R, G, and B. For example, for the image to be processed, the white point in the image can be calculated first, and then the white balance gain can be determined according to the white point information of the white point. The values of the three color channels of R, G, and B of the image are adjusted according to the white balance gain. In addition, the color temperature of the image can also be estimated through the white balance algorithm, and the estimated color temperature of the image can be output, such as estimating the color temperature according to the white point information.

[0064] The image processing method provided by the embodiments of this application is applicable to any electronic device with a shooting function and configured with multiple cameras. The electronic device can be a terminal such as a mobile phone, a tablet computer, a camera, or a smart wearable device. The embodiments of this application do not make any limitations in this regard. In addition, the embodiments of this application are mainly applied to the single-camera shooting scenario of the electronic device, that is, the scenario of separately calling one of the multiple cameras for shooting, such as the scenario of switching from one camera to another for shooting.

[0065] As an example, different cameras among the multiple cameras configured in the electronic device have different shooting capabilities. For example, the electronic device is configured with, but not limited to, a wide-angle camera, a telephoto camera (such as a periscope telephoto camera), an ultra-wide-angle camera, and a depth camera, etc.

[0066] As an example, the multiple cameras configured in the electronic device include cameras arranged on different sides, or cameras arranged on the same side. That is to say, the embodiments of this application can be applied to the scenario of calling any camera on different sides (such as the front camera or the rear camera) for shooting, and can also be applied to the scenario of calling any camera on the same side (such as the rear camera) for shooting.

[0067] As an example, an electronic device is configured with multiple cameras on the same side, and any camera on the same side can be called for shooting. For example, the electronic device is configured with multiple rear cameras on the back, and any rear camera can be called for shooting. Or, the electronic device is configured with multiple front cameras on the front side, and any front camera can be called for shooting separately.

[0068] Generally, an electronic device is configured with a main camera and at least one auxiliary camera. For example, please refer to Figure 1 , the spatial position distribution of multiple cameras can be as shown in Figure (a) of Figure 1 , or the spatial position distribution of multiple cameras can also be as shown in Figure (b) of Figure 1 . The multiple cameras are respectively camera 00, camera 01, camera 02, and camera 03. Exemplarily, camera 00 is the main camera, and the others are auxiliary cameras.

[0069] After the electronic device starts the camera application, it usually defaults to shooting through the main camera, and then can automatically switch according to the shooting requirements or switch to a certain auxiliary camera according to the user's switching operation for shooting. For example, please refer to Figure 1 , shooting through camera 00 by default, and then switching to camera 01, camera 02, or camera 03 for shooting according to the user's switching operation.

[0070] When the electronic device switches from one camera to another for shooting, the camera before the switch may be blocked by the user's hand or other objects during shooting, such as being blocked by the user's finger or palm and other hand positions, causing the camera to be in a blocked situation. If the camera before the switch is blocked during shooting, the image it shoots may not accurately reflect the shooting scene, and thus does not meet the standard of being a reference image for correcting the image shot by the camera after the switch. For example, if the image shot by the camera before the switch is still used as the reference image to correct the color of the image shot by the camera after the switch, it may cause the color-corrected image to be severely color-biased.

[0071] In order to identify the occlusion situation of the camera, a method has been proposed in the related art to identify whether the camera is occluded based on the image brightness difference between the images shot before and after the camera switch. For example, if the image brightness difference is large, it is determined that the camera before the switch is occluded. However, in the case where the occlusion of the camera before the switch is not strict or not complete, the image brightness difference between the images shot by the two cameras before and after the switch will be very small, and it is difficult to distinguish whether the camera before the switch is occluded based on the image brightness difference, resulting in incorrect occlusion recognition. For example, please refer to Figure 2 , Figure 2It is a comparison schematic diagram of images captured before and after the camera switch provided by an embodiment of the present application. As Figure 2 shown, the electronic device first calls the main camera for shooting, and then switches to the telephoto camera for shooting. Figure 2 In Figure (a) of Figure 2 , the image captured by the main camera when it is blocked (blocked by the user's finger) has a light value (LV) = 40; Figure 2 In Figure (b) of Figure 2 , the image captured by the switched telephoto camera has an LV = 56. Comparing

[0072] Figure (a) of

[0073] with Figure (b) of

[0074] , it can be seen that the difference in the light values of the two is small. If occlusion recognition is performed according to the difference in the image brightness before and after the camera switch in the related art, it is possible to misjudge that the main camera is not occluded, resulting in incorrect occlusion recognition.

[0075] Please refer to Figure 3 , Figure 3It is a schematic diagram of the statistical information difference of the images captured before and after the camera switch provided by an embodiment of the present application. As Figure 3 shown, the electronic device first calls the main camera for shooting, and then switches to the telephoto camera for shooting a close-up view. Figure 3 In Figure (a) of Figure 3 Figure 3 is the image captured by the main camera without occlusion, Figure 3 and in Figure (b) of Figure 3 Figure 3 is the image captured by the switched telephoto camera; Figure 3 In Figure (c) of Figure 3 Figure 3 is a schematic diagram of the distribution of the statistical white points of the image captured by the main camera,

[0076] Please refer to Figure 4 , Figure 4 is another schematic diagram of the statistical information difference of the images captured before and after the camera switch provided by an embodiment of the present application. As Figure 4 shown, the electronic device first calls the main camera for shooting and then switches to the telephoto camera for shooting. Figure 4 In Figure (a) of Figure 4 Figure 4 is the image captured by the main camera under occlusion; Figure 4 In Figure (b) of Figure 4 Figure 4 is the image captured by the switched telephoto camera. Figure 4 As shown in Figure (a) of Figure 4 Figure 4 when the main camera is occluded, the captured image is close to a pure color scene; correspondingly, as Figure 4 shown in Figure (c) of Figure 4 Figure 4 the ordinate of the statistical white points of the captured image in the coordinate system with rg as the abscissa and bg as the ordinate changes little, that is, the B / G values of each statistical white point change little, and the image color is close to red (reddish). As Figure 4By comparing with Figure (d), it can be seen that when the main camera is blocked, the distribution of the statistical white points of the images captured by the main camera and the telephoto camera is significantly different, that is, the statistical information of the images captured before and after the camera switch is significantly different. Therefore, in the present application, by comparing the differences in the statistical information of the images captured before and after the camera switch, and identifying the occlusion situation of the camera before the switch based on the differences in the statistical information, it is possible to more accurately identify whether the camera before the switch is occluded.

[0077] It should be noted that the method for identifying the occlusion of the camera provided in the present application will be described in detail in the following Figure 8 embodiments, and the embodiments of the present application will not be elaborated here.

[0078] For the sake of convenience of description, hereinafter, an example will be introduced in which the electronic device is provided with multiple cameras.

[0079] Figure 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Refer to Figure 5 , the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0080] It can be understood that the structure schematically shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0081] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0082] Among them, the controller may be the nerve center and command center of the electronic device 100. The controller may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.

[0083] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory may save the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from this memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0084] The wireless communication function of the electronic device 100 may be implemented by the antenna 1, antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc.

[0085] The antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas may also be multiplexed to improve the utilization rate of the antennas. For example, the antenna 1 may be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna may be used in combination with a tuning switch.

[0086] The mobile communication module 150 may provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 may receive electromagnetic waves through the antenna 1, filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 may also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves through the antenna 1 and radiate it out. In some embodiments, at least some functional modules of the mobile communication module 150 may be disposed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be disposed in the same device.

[0087] The wireless communication module 160 may provide solutions for wireless communications such as wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. applied to the electronic device 100. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves through the antenna 2, frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive the signal to be transmitted from the processor 110, frequency-modulate and amplify it, and convert it into electromagnetic waves through the antenna 2 and radiate it out.

[0088] The electronic device 100 realizes the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and is used for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0089] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is an integer greater than 1.

[0090] The electronic device 100 can implement the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor, etc.

[0091] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light passes through the lens and is transmitted to the camera photosensitive element. The optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize parameters such as the exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0092] The camera 193 is used to capture static images or videos. An object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in standard RGB, YUV, etc. formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is an integer greater than 1.

[0093] The digital signal processor is used to process digital signals. Besides being able to process digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0094] The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple coding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0095] The NPU is a neural-network (NN) computing processor. By drawing on the structure of the biological neural network, such as the transmission pattern between human brain neurons, it can quickly process the input information and can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as: image recognition, face recognition, speech recognition, text understanding, etc.

[0096] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. For example, files such as music and videos are saved in the external memory card.

[0097] The internal memory 121 can be used to store computer-executable program code, and the computer-executable program code includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, image playback function, etc.). The data storage area can store the data created during the use of the electronic device 100 (such as audio data, phone book, etc.). In addition, the internal memory 121 can include high-speed random access memory and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0098] The electronic device 100 can implement audio functions, such as music playback, recording, etc., through the audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and the application processor, etc.

[0099] The pressure sensor 180A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be disposed on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor may include at least two parallel plates having conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 can also calculate the position of the touch according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the pressure threshold acts on the short message application icon, the instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.

[0100] The touch sensor 180K, also referred to as the "touch panel". The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also referred to as the "touch screen". The touch sensor 180K is used to detect touch operations acting thereon or nearby. The touch sensor 180K can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In some other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100, at a different position from the display screen 194.

[0101] Next, the software system of the electronic device 100 will be described.

[0102] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of the present application, the Android system with a layered architecture is taken as an example to exemplarily describe the software system of the electronic device 100.

[0103] Figure 6 is a block diagram of a software system of an electronic device 100 provided by an embodiment of the present application. Refer to Figure 6 In the layered architecture, the software is divided into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, such as Figure 6As shown, the system architecture of the electronic device 100 includes an application layer 510, an application framework layer 520, a hardware abstraction layer (HAL) 530, and a driver layer 540.

[0104] It can be understood that Figure 6 merely as an example, the layers divided in the electronic device 100 are not limited to Figure 6 the layers shown. For example, between the application framework layer and the HAL layer, there may also be included an Android runtime and a libraries layer, etc.

[0105] The application layer 510 may include a series of application program packages. As Figure 6 shown, the application program packages may include a camera, a gallery, and other application programs. Other application programs include but are not limited to: calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, and other application programs.

[0106] The application framework layer 520 provides application programming interfaces (APIs) and programming frameworks for the application programs in the application layer. The application framework layer 520 includes some predefined functions. For example, the application framework layer 520 may include a camera access interface. The camera access interface may include camera management and a camera device. Among them, the camera management can be used to provide an access interface for managing the camera; the camera device can be used to provide an interface for accessing the camera.

[0107] In addition, the application program framework layer 520 may also include a content provider, a resource manager, a notification manager, a window manager, a view system, a telephone manager, etc. Similarly, the camera application can also call the content provider, the resource manager, the notification manager, the window manager, the view system, etc. according to actual business needs. The embodiments of the present application do not make any restrictions on this.

[0108] The hardware abstraction layer 530 is used to abstract the hardware. For example, it can encapsulate the driver programs in the driver layer and provide an interface for the application framework layer to call, shielding the implementation details of the lower-level hardware. For example, the hardware abstraction layer 530 may include a camera hardware abstraction layer (Camera HAL) and other hardware device abstraction layers. The camera hardware abstraction layer can be connected to an algorithm library to call the algorithms in the algorithm library.

[0109] For example, please refer to Figure 6, the algorithm library may include a camera occlusion recognition module, which integrates the occlusion recognition algorithm of the camera provided in the embodiments of the present application and is used to identify whether a specific camera is occluded. Additionally, the algorithm library may further include a color correction module, which integrates a color correction algorithm and is used to perform color correction on the image captured by the current camera according to the image captured by a reference camera (such as the camera before switching). For example, the color correction module integrates the multi-camera color correction algorithm provided in the embodiments of the present application and is used to perform color correction on the image frame captured by the current camera using the multi-camera color correction algorithm. As an example, in the case where the electronic device switches from the first camera to the second camera for shooting, the camera occlusion recognition module may trigger the color correction module to perform color correction on the image captured by the current second camera according to the image captured by the first camera before switching when it is recognized that the camera before switching is not occluded. However, when it is recognized that the camera before switching is occluded, the color correction module is not triggered to perform color correction on the image captured by the current second camera according to the image captured by the first camera before switching. For ease of description, the first camera before switching may also be referred to as the reference camera.

[0110] In addition, the algorithm library may further include an auto white balance (AWB) module and other image processing modules. The AWB module integrates the AWB algorithm and can calculate the statistical information of the image frame captured by the current camera using the AWB algorithm, and send specific statistical information to the color correction module. The color correction module processes the specific statistical information of the current image frame using a suitable color correction algorithm according to the specific statistical information of the memory frame (the last frame) captured by the reference camera to obtain the corrected statistical information, and then returns the corrected statistical information to the AWB module. The AWB module can perform white balance correction on the current image frame according to the corrected statistical information to obtain an AWB image frame. Additionally, the AWB module can continue to send the AWB image frame and the corrected statistical information to other image processing modules so that the other image processing modules can perform image processing on the AWB image frame according to the corrected statistical information, such as performing color correction or lens shadow correction, etc.

[0111] As an example, please refer to Figure 7, when the electronic device switches from the reference camera to the current camera for shooting, the AWB module can use the AWB algorithm to calculate the statistical information of the image frame captured by the current camera and send the statistical information to the camera occlusion recognition module. The camera occlusion recognition module identifies whether the reference camera before switching is occluded based on the statistical information of the image frame captured by the current camera and the statistical information of the memory frame captured by the pre-stored reference camera. When it is recognized that the reference camera before switching is not occluded, the AWB module is triggered to send the specific statistical information (such as AWB white point and AWB color temperature, etc.) of the current image frame calculated by the AWB algorithm to the color correction module, so that the color correction module can perform color correction on the current image frame according to the specific statistical information of the memory frame of the image captured by the reference camera before switching. For example, the color correction module performs a fusion process on the AWB color temperature difference between the memory frame and the current image frame, and the AWB white point of the memory frame and the AWB white point of the current image frame to obtain a fused white point, and performs a fusion process on the AWB color temperature of the memory frame and the AWB color temperature of the current image frame to obtain a fused color temperature; then, the fused white point and the fused color temperature are fed back to the AWB module so that the AWB module can perform white balance correction on the current image frame according to the fused white point to obtain an AWB image frame. In addition, the AWB module can also continue to send the fused color temperature and the AWB image frame to other image processing modules so that other image processing modules can perform other processing on the AWB image frame according to the fused color temperature, such as performing color correction or lens shadow correction, etc. In addition, when the camera occlusion recognition module recognizes that the reference camera before switching is occluded, the camera occlusion recognition module does not trigger the AWB module to send the specific statistical information (such as AWB white point and AWB color temperature, etc.) of the current image frame calculated by the AWB algorithm to the color correction module, so that the color correction module does not perform color correction on the current image frame.

[0112] The driver layer 540 is used to provide drivers for different hardware devices. For example, the driver layer 540 may include a camera device driver, a digital signal processor driver, a graphics processor driver, etc.

[0113] In addition, the hardware layer 550 includes hardware modules that can be driven, such as camera devices. For example, the camera device includes multiple cameras, such as camera 1, camera 2,..., camera n. In addition, the camera device may also include a multispectral sensor, a depth sensor (time of flight, TOF), etc., which are not limited in the embodiments of the present application.

[0114] In the present application, by calling the hardware abstraction layer interface in the hardware abstraction layer 530, the connection between the application layer 510 and the application framework layer 520 above the hardware abstraction layer 530 and the driver layer 550 and the hardware layer 550 below can be realized, and camera data transmission and function control can be realized.

[0115] The following exemplarily describes the working processes of the software and hardware of the electronic device 100 in combination with the capture and photographing scenario.

[0116] The camera application in the application layer 510 can be displayed on the screen of the electronic device 100 in the form of an icon. When the icon of the camera application is clicked and triggered by the user, the electronic device 100 starts to run the camera application. When the camera application runs on the electronic device 100, the camera application calls the corresponding interface of the camera application in the application framework layer 520, and then starts the camera device driver by calling the hardware abstraction layer 530, turns on any camera 193 on the electronic device 100, and captures an image through the camera 193. Exemplarily, the camera hardware abstraction layer 530 can issue an instruction to call a certain camera to the camera device driver to obtain a captured image through the called camera.

[0117] Taking the example that the main camera is defaultly called after the camera application is launched. After the main camera is called, for example, when the main camera captures the last frame of an image (the memory frame), the multispectral sensor can be called to obtain the ambient color temperature at the time of capturing the memory frame. Meanwhile, the image signal processor is called to perform white balance processing on the memory frame captured by the main camera, and obtain the statistical information of the memory frame (such as the target white point, statistical white point, estimated color temperature, etc.) calculated by the white balance algorithm during the white balance processing. Then, the statistical information of the memory frame, the ambient color temperature and other information are stored. After that, when any camera other than the main camera in the hardware layer is called, that is, when the main camera is switched to another camera, during the process of capturing the current frame by the current camera, the multispectral sensor can be called to obtain the ambient color temperature at the time of capturing the current frame. Meanwhile, the image signal processor is called to perform white balance processing on the current frame captured by this camera, and obtain the statistical information of the current frame calculated by the white balance algorithm during the white balance processing. Then, the statistical information of the memory frame is obtained, and based on the difference between the statistical information of the memory frame and the statistical information of the current frame, it is determined whether the main camera is occluded (that is, whether the main camera is occluded when capturing the memory frame). If it is determined that the main camera is not occluded, the current image frame is color-corrected according to the memory frame. For example, according to the ambient color temperature difference between the ambient color temperature of the memory frame and the ambient color temperature of the current frame, the white point information of the target white point of the memory frame and the white point information of the target calibration of the current frame are fused, and the fused white point information is fed back to the image signal processor, so that the image signal processor performs white balance correction and other processing on the current frame according to the fused white point information to obtain the target image; then the image signal processor returns the target image to the camera hardware abstraction layer through the camera device driver, and the camera hardware abstraction layer further processes the target image, and sends the processed image back to the camera application through the camera access interface for display and storage. Additionally, if it is determined that the main camera is occluded, the step of color-correcting the current image frame according to the memory frame is not executed. That is, the image signal processor can directly perform white balance correction and other processing on the current frame according to the statistical information of the current frame calculated by the white balance algorithm to obtain the target image, and then the image signal processor returns the target image to the camera hardware abstraction layer through the camera device driver, and the camera hardware abstraction layer further processes the target image, and sends the processed image back to the camera application through the camera access interface for display and storage.

[0118] The execution subject of the image processing method provided in the embodiments of the present application can be the above-mentioned electronic device, or a functional module and / or functional entity in the electronic device that can implement the image processing method. And the solution of the present application can be implemented in a hardware and / or software manner, which can be specifically determined according to actual usage requirements, and the embodiments of the present application do not make limitations.

[0119] Next, the method for identifying occlusion of the camera provided in the embodiments of the present application will be described in detail.

[0120] Figure 8 is a schematic flowchart of a method for identifying occlusion of a camera provided by an embodiment of the present application. As shown in Figure 8 the figure, the method includes the following steps:

[0121] Step A1: When the electronic device switches from the first camera to the second camera for shooting, determine the first statistical information of the memory frame of the first camera and the second statistical information of the current frame captured by the second camera.

[0122] Among them, the scenario of switching from the first camera to the second camera for shooting may include: the scenario of directly switching from the first camera to the second camera for shooting, or the scenario of first switching from the first camera to other cameras and then switching from other cameras to the second camera. The embodiments of the present application do not limit this.

[0123] Among them, the first camera can be any one of multiple cameras configured for the electronic device, and the second camera is any one of the multiple cameras other than the first camera.

[0124] As an example, a certain camera among the multiple cameras configured for the electronic device can be set as a reference camera in advance, so as to correct the colors of the images captured by other cameras based on the image parameters of the reference camera, so that the colors of the images captured by other cameras all approach the colors of the images captured by the reference camera, thereby reducing the color difference of multi-camera shooting in the same scene and improving the color consistency of multi-camera shooting. Among them, the reference camera can be any one of the multiple cameras, such as the camera defaultly called by the camera application. Usually, the main camera is defaultly called for shooting after the camera application is started, so the main camera can be set as the reference camera.

[0125] Correspondingly, the first camera is the reference camera set in advance. That is, in the embodiments of the present application, when the electronic device switches from the reference camera to other cameras for shooting, the method for identifying occlusion of the camera provided by the embodiments of the present application can be used to determine whether the reference camera is occluded.

[0126] Among them, the memory frame is the last frame captured by the first camera, such as the last frame previewed by the first camera. That is, the embodiments of the present application can identify whether the first camera is occluded when shooting the last frame before switching to other cameras. It should be understood that the memory frame can also be replaced with other image frames captured by the first camera as needed, so as to identify whether the first camera is occluded when shooting other image frames. For example, the memory frame can be replaced with the first frame captured by the first camera, etc. The embodiments of the present application do not limit this.

[0127] As an example, the current frame can be the first frame captured by the second camera, such as the first frame of the preview. That is, after the electronic device switches from the first camera to the second camera for shooting, it can identify whether the first camera is occluded based on the statistical information of the last frame captured by the first camera and the statistical information of the first frame captured by the second camera, so as to determine whether to start the image correction process according to the occlusion recognition result, that is, whether to correct the image frame captured by the second camera with the memory frame of the first camera as the reference image.

[0128] Among them, the first statistical information includes the first target white point of the memory frame and multiple first statistical white points, such as the white point coordinates including the first target white point and multiple first statistical white points. The second statistical information includes the second target white point of the current frame and multiple second statistical white points, such as the white point coordinates including the second target white point and multiple second statistical white points.

[0129] After determining the first statistical information of the memory frame of the first camera and the second statistical information of the current frame captured by the second camera, the occlusion recognition result of the first camera can be determined according to the difference between the first statistical information and the second statistical information. Among them, the occlusion recognition result indicates whether the first camera is occluded.

[0130] As an example, the operation of determining the occlusion recognition result of the first camera according to the difference between the first statistical information and the second statistical information can be implemented through the following steps A2 - A8.

[0131] Step A2: According to the first statistical information, determine the target quantity ratio. The target quantity ratio is the ratio of the target quantity to the total quantity of multiple first statistical white points. The target quantity refers to the quantity of first statistical white points among multiple first statistical white points whose white point coordinates are within the range centered on the white point coordinates of the first target white point with a preset distance as the radius.

[0132] Among them, the preset distance can be set in advance according to needs. For example, the preset distance can be an empirical value obtained by statistically analyzing the target white point and the statistical white point of the images captured by the first camera under multiple standard light sources. The multiple standard light sources can be specified according to needs. The standard light source is a light source specified by the standard, such as the light source specified by the International Commission on Illumination for unified color detection. The color temperatures of these multiple standard light sources are different. For example, these multiple standard light sources can be at least two of the standard light sources such as D75, D65, D50, CWF, TL84, U30, A, H, etc. Of course, other standard light sources can also be included, and the embodiments of the present application do not limit this. Among them, the color temperatures of the standard light sources D75, D65, D50, CWF, TL84, U30, A, H decrease in sequence, which are 7500K, 6500K, 5000K, 4150K, 4100K, 3000K, 2856K, and 2300K in sequence.

[0133] As an example, the standard light sources ranked second and second from the bottom in descending order of color temperature can be determined from multiple standard light sources, and the preset distance can be determined according to the difference between the target white point and / or the statistical white point of the first camera under these two standard light sources. For example, the preset distance is determined according to the coordinate distance of the target white point of the images captured by the first camera under these two standard light sources. It should be understood that other methods can also be used to determine the preset distance, and the embodiments of the present application do not limit this.

[0134] Among them, the target quantity ratio is used to indicate the degree of aggregation of multiple first statistical white points around the first target white point. The larger the target quantity ratio, the higher the degree of aggregation of multiple first statistical white points around the first target white point, and the closer the first statistical white point is to the first target white point. The smaller the target quantity ratio, the lower the aggregation of multiple first statistical white points around the first target white point, and the farther the first statistical white point is from the first target white point.

[0135] Taking the main camera switching to the telephoto camera as an example, please refer to Figure 9 , Figure 9 is a schematic comparison diagram of the white point distributions of the images captured by two main cameras provided by the embodiments of the present application and the images captured by the telephoto camera. Figure 9 In (a) of Figure 9 shows a schematic comparison diagram of the white point distribution of the image captured by the main camera without occlusion and the white point distribution of the image captured by the telephoto camera. As shown in (a) of Figure 9 In (b) ofFigure 9 As shown in Figure (b) in , when the main camera is blocked, the captured image is basically a solid color. Therefore, the B / G values of the statistically determined white points are similar, and the statistically determined white points are basically linearly distributed. As a result, the statistically determined white points are far from the target white point, and the number of statistically determined white points falling within a circle centered on the target white point with a preset distance r as the radius is small.

[0136] It should be noted that when the main camera is blocked, the captured image is basically a solid color. For example, when the camera is blocked by a hand, the captured image is reddish, the B / G values of the statistically determined white points are similar, and the statistically determined white points generally show a horizontal linear distribution in a coordinate system with R / G as the abscissa and B / G as the ordinate. This distribution pattern will result in the statistically determined white points being far from the target white point, and the number of statistically determined white points falling within a circle centered on the target white point with a preset distance r as the radius is small. Therefore, the embodiment of the present application can determine whether the first camera is blocked according to the target quantity ratio.

[0137] For example, the target quantity ratio can be represented by refNumRatio, and refNumRatio = the ratio of the target quantity to the total quantity of multiple first statistically determined white points.

[0138] Step A3: Determine whether the target quantity ratio is less than a first threshold.

[0139] Among them, the first threshold can be set in advance according to needs, and is generally a relatively small value. For example, the first threshold can be 0.06, 0.08, or 0.1, etc.

[0140] When the target quantity ratio is less than the first threshold, it indicates that the statistically determined white points of the memory frame are far from the target white point, and the first camera may be blocked. When the target quantity ratio is greater than or equal to the first threshold, it indicates that the statistically determined white points of the memory frame are close to the target white point, and the first camera is not blocked.

[0141] For example, if refNumRatio < 0.08, it is determined that the first camera may be blocked; if refNumRatio ≥ 0.08, it is determined that the first camera is not blocked.

[0142] Step A4: If the target quantity ratio is greater than or equal to the first threshold, determine that the first camera is not blocked.

[0143] Step A5: If the target quantity ratio is less than the first threshold, determine a first ratio and a second ratio according to the first statistical information and the second statistical information. The first ratio is the ratio of the mean value of the first color statistical values of multiple first statistically determined white points to the mean value of the second color statistical values, and the second ratio is the ratio of the mean value of the first color statistical values of multiple second statistically determined white points to the mean value of the second color statistical values.

[0144] Wherein, the first color statistical value is the ratio of the R value to the G value of the corresponding statistically white point, and the second color statistical value is the ratio of the B value to the G value of the corresponding statistically white point.

[0145] In the embodiment of the present application, when the target quantity ratio is less than the first threshold, it can be preliminarily determined that the first camera may be blocked, and then other information can be combined to further determine whether the first camera is blocked. For example, according to the first statistical information and the second statistical information, the first ratio and the second ratio can be determined, and whether the first camera is blocked can be judged according to the first ratio and the second ratio.

[0146] For example, the operation of determining the first ratio and the second ratio according to the first statistical information and the second statistical information may include the following steps:

[0147] 1) Calculate the R / G value and B / G value of each first statistically white point among multiple first statistically white points, and the R / G value and B / G value of each second statistically white point among multiple second statistically white points.

[0148] 2) Calculate the average value refRgAvg of the R / G values of multiple first statistically white points and the average value refBgAvg of the B / G values respectively, and calculate the average value curRgAvg of the R / G values of multiple second statistically white points and the average value curBgAvg of the B / G values respectively.

[0149] 3) Calculate the first ratio refDeltaMean = refRgAvg / refBgAvg and the second ratio curDeltaMean = curRgAvg / curBgAvg respectively.

[0150] As an example, after determining that the target quantity ratio is less than the first threshold, it can also be first judged whether the target quantity ratio is less than or equal to the second threshold. If so, it is determined that the first camera is blocked; if not, the first ratio and the second ratio are further determined according to the first statistical information and the second statistical information, so as to judge whether the first camera is blocked according to the ratio of the first ratio to the second ratio. In this way, the recognition efficiency can be improved.

[0151] Wherein, the second threshold is less than the first threshold. For example, the second threshold can be 0.01 or 0, etc.

[0152] Step A6: Judge whether the ratio of the first ratio to the second ratio is greater than the first ratio threshold, or whether the ratio of the first ratio to the second ratio is less than the second ratio threshold.

[0153] Wherein, the first ratio threshold is greater than 1, for example, it can be 2 or 3, etc. The second ratio threshold is less than 1, for example, it can be 0.5 or 0.3, etc. The first ratio threshold and the second ratio threshold can be set as needed, and the embodiment of the present application does not limit this.

[0154] If the ratio of the first ratio to the second ratio is greater than the first ratio threshold, it indicates that the R / G mean value of the statistical white point in the image captured by the first camera is relatively large, and the captured image is reddish. Therefore, it can be determined that the first camera is blocked. If the ratio of the first ratio to the second ratio is less than the second ratio threshold, it indicates that the B / G mean value of the statistical white point in the image captured by the first camera is relatively large, and the captured image is bluish. Therefore, it can be determined that the first camera is blocked. If the ratio of the first ratio to the second ratio is greater than or equal to the second ratio threshold and less than or equal to the first ratio threshold, it indicates that the RGB values of the statistical white point in the image captured by the first camera are within the normal range. Therefore, it can be determined that the first camera is not blocked.

[0155] Step A7: If the ratio of the first ratio to the second ratio is greater than the first ratio threshold, or the ratio of the first ratio to the second ratio is less than the second ratio threshold, then it is determined that the first camera is blocked.

[0156] For example, assume that the first ratio threshold and the second ratio threshold are 2 and 0.5 respectively. If refDeltaMean > 2 * curDeltaMean or refDeltaMean < 0.5 * curDeltaMean, then it is determined that the first camera is blocked.

[0157] In addition, if the ratio of the first ratio to the second ratio is greater than or equal to the second ratio threshold and less than or equal to the first ratio threshold, then it is determined that the first camera is not blocked.

[0158] For example, if 0.5 * curDeltaMean ≥ refDeltaMean ≤ 2 * curDeltaMean, then it is determined that the first camera is not blocked.

[0159] Please refer to Figure 10 , Figure 10 which is a logical schematic diagram of a method for identifying occlusion of a camera provided by an embodiment of the present application. As Figure 10As shown, assume that the electronic device switches from the main camera to the telephoto camera for shooting. First, the target quantity ratio refNumRatio can be calculated according to the statistical information of the memory frame of the main camera. Then, it is determined whether refNumRatio is less than 0.08. If not, it is determined that the main camera is not occluded. If so, it is further determined whether refNumRatio is equal to 0. If so, it is determined that the main camera is occluded. If not, the refDeltaMean corresponding to the main camera and the curDeltaMean corresponding to the telephoto camera are further calculated. Then, it is determined whether refDeltaMean is greater than 2*curDeltaMean or refDeltaMean is less than 0.5*curDeltaMean. If so, it is determined that the main camera is occluded. If not, it is determined that the main camera is not occluded.

[0160] It should be understood that in the embodiments of the present application, only an example is given in which after switching from the first camera to the second camera, the occlusion recognition result of the first camera is determined according to the difference between the first statistical information of the image captured by the first camera and the second statistical information of the image captured by the second camera. In other embodiments, it may also be assumed that the first camera is not occluded, and the occlusion recognition result of the second camera is determined according to the difference between the first statistical information of the image captured by the first camera and the second statistical information of the image captured by the second camera. The embodiments of the present application do not limit this.

[0161] In the embodiments of the present application, the occlusion of the camera can be recognized according to the difference between the statistical information of the images captured before and after the camera switch. Since the statistical information of the image can more accurately reflect the image characteristics such as the color temperature of the image, and there will be obvious differences in the statistical information of the images captured by the corresponding camera when the camera is occluded and not occluded, the occlusion of the camera before the switch can be more accurately recognized according to the difference in the statistical information of the images captured by the camera before and after the switch, thereby improving the accuracy of recognizing whether the camera is occluded.

[0162] In addition, in the embodiments of the present application, the image correction process can also be determined according to the camera recognition result, that is, it is determined whether to use the memory frame of the first camera as a reference image to correct the image frame captured by the second camera. Since the method for recognizing the occlusion of the camera provided by the embodiments of the present application can more accurately recognize whether the camera is occluded, the image captured by the camera after the switch can be more accurately corrected according to the occlusion recognition result recognized by the present application, improving the accuracy of image correction. For example, it can avoid to a certain extent the serious color cast caused by using the image captured by the occluded camera as a reference image for color correction.

[0163] As an example, according to the color temperature difference between the first ambient color temperature corresponding to the memory frame of the first camera and the second ambient color temperature of the current frame captured by the second camera, the white point information of the first target white point of the memory frame and the white point information of the second target white point of the current frame can be fused, and the current frame can be corrected for white balance according to the fused white point information.

[0164] Next, taking the first camera as a pre-set reference camera as an example, the application scenarios involved in the image correction provided by the embodiments of the present application will be illustrated by examples.

[0165] When an electronic device is equipped with multiple cameras, if the sensors of each camera have different responses to colors, it will cause certain deviations in the colors of the images captured by different cameras among these multiple cameras in the same scene. The visual differences brought about by such color differences in multi-camera shooting in the same scene will affect the user's shooting experience. For example, when the user switches cameras to shoot in the same scene, the colors of the pictures taken before and after the camera switch may deviate. For the user, the user will see that the colors of the same object taken before and after are inconsistent, resulting in a poor visual experience for the user. Moreover, this situation will also cause certain doubts to the user, and may make the user suspect that their shooting operation is incorrect, affecting the user's shooting experience.

[0166] Please refer to Figure 3 , in the same scene, the user can first use the main camera to shoot, and then switch to the telephoto camera to shoot. Figure 3 Figure (a) in Figure 3 is the image captured by the main camera,

[0167] Regarding the color differences in multi-camera shooting in the same scene, the embodiments of the present application also provide a multi-camera color correction method, namely the multi-camera color correction method. In this method, a certain camera among the multiple cameras configured in the electronic device can be set as the reference camera in advance, and the white point information of the first target white point of the memory frame (i.e., the last frame captured) of the reference camera and the first ambient color temperature of the shooting environment corresponding to the memory frame are stored, so as to perform color correction on the images captured by other cameras based on the relevant information of the memory frame of the reference camera. For example, when the electronic device calls any other camera other than the reference camera for shooting, the white point information of the second target white point of the current frame captured by the other camera and the second ambient color temperature of the shooting environment corresponding to the current frame can be obtained. Then, according to the color temperature difference between the first ambient color temperature of the shooting environment corresponding to the memory frame of the reference camera and the second ambient color temperature, the white point information of the first target white point of the memory frame and the white point information of the second target white point of the current frame are fused, and the current frame is corrected for white balance according to the fused white point information.

[0168] In this way, when the ambient color temperature of other cameras is close to that of the reference camera, it can be determined that the shooting scene difference is small, and it is likely to be shooting in the same scene. By combining the target white point information of the memory frame of the reference camera and the target white point information of the current frame, color correction is performed on the current frame by correcting the white balance of the current frame, so as to reduce the color difference between the current frame and the memory frame of the reference camera, thereby reducing the color difference between the images captured by other cameras and the reference camera in the same scene and the resulting visual difference, improving the color consistency of multi-camera shooting in the same scene, and further improving the user's visual experience and shooting experience.

[0169] It should be noted that the specific algorithm of this multi-camera color correction method will be described in detail in the following Figure 11 embodiments, and the embodiments of the present application will not elaborate on it here.

[0170] It should be understood that the above method for identifying camera occlusion can also be applied to other application scenarios, such as real-time detection of the images captured by the camera. When abnormal situations such as occlusion occur, a camera occlusion warning is triggered, etc.

[0171] As an example, a certain camera among multiple cameras configured in an electronic device can be set as a reference camera in advance, so as to correct the colors of the images captured by other cameras based on the image parameters of the reference camera, making the colors of the images captured by other cameras approach the colors of the images captured by the reference camera, thereby reducing the color difference in multi-camera shooting under the same scene and improving the color consistency of multi-camera shooting. Among them, the reference camera can be any one of the multiple cameras. In one example, the camera that the camera application defaults to start can be set as the reference camera. For example, usually, the camera that the camera application defaults to start is usually the main camera, so the main camera can be set as the reference camera.

[0172] Figure 11 is a schematic flowchart of a multi-camera color correction algorithm provided by an embodiment of the present application. As Figure 11 shown, the method includes the following steps:

[0173] Step B1: Obtain and store the white point information of the first target white point of the memory frame of the reference camera, and the first ambient color temperature of the shooting environment corresponding to the memory frame.

[0174] In an embodiment of the present application, when it is detected that the camera application switches from the reference camera to other cameras for shooting, the ambient color temperature (i.e., the first ambient color temperature) of the shooting environment corresponding to the last frame (i.e., the memory frame) captured by the reference camera, and the white point information of the white point of the memory frame (i.e., the white point information of the first target white point) can be obtained, and the white point information of the first target white point of the memory frame and the first ambient color temperature are stored. Exemplarily, the memory frame can be the last frame previewed by the reference camera.

[0175] Among them, the camera switching event of switching from the reference camera to other cameras for shooting can be triggered by the user's camera switching operation, or can be automatically triggered by the camera application according to the shooting requirements. The embodiment of the present application does not limit this.

[0176] Among them, the second target white point can be the AWB white point of the memory frame, that is, the white point represented by the white point information output by the AWB module in the image signal processor after estimating the white point of the memory frame. The white point information of the second target white point is used to indicate the second target white point, and can be the color information of the second target white point, such as the white point coordinates (R / G, B / G) of the second target white point. The white point coordinates refer to the coordinates in a coordinate system with R / G as the abscissa and B / G as the ordinate. The first ambient color temperature is used to indicate the color temperature of the corresponding shooting environment, and can be the correlated color temperature (CCT) of the shooting environment. Exemplarily, the first ambient color temperature is the color temperature detected by a multi-spectral sensor in the corresponding shooting environment.

[0177] Furthermore, other information corresponding to the memory frame can also be obtained and stored. For example, one or more of the following information can be obtained: the first estimated color temperature of the memory frame, and the brightness information of the shooting environment corresponding to the memory frame.

[0178] Among them, the first estimated color temperature is the color temperature obtained by estimating the color temperature of the memory frame. For example, it is the color temperature obtained by estimating the color temperature based on the first target white point. Exemplarily, the first estimated color temperature can be the color temperature obtained by estimating the color temperature of the memory frame using a white balance algorithm based on the first target white point. For example, the first estimated color temperature is the AWB color temperature, that is, the color temperature output after the AWB module in the image signal processor estimates the color temperature of the memory frame.

[0179] Among them, the brightness information is used to indicate the ambient brightness of the corresponding shooting environment. For example, the brightness information can be a brightness value (LV), and of course, it can also be other parameters used to measure the ambient brightness. This brightness information can be the brightness information obtained by statistically analyzing the brightness of the memory frame. For example, this brightness information is the brightness information output by the auto exposure (AE) module in the image signal processor after statistically analyzing the brightness of the memory frame.

[0180] Step B2: After the second camera other than the reference camera is called for shooting, obtain the white point information of the second target white point of the current frame captured by the second camera, and the second ambient color temperature of the shooting environment corresponding to the current frame.

[0181] Among them, the second camera is the current camera, and can be any one of the multiple cameras other than the reference camera. For example, the second camera is the current camera after the camera is switched. Exemplarily, the call scenario of the second camera can include any one of the following scenarios: directly switching from the reference camera to the second camera for shooting; or, first switching from the reference camera to other cameras, and then switching from other cameras to the second camera for shooting, etc. Exemplarily, the current frame can be the current frame of the real-time preview of the second camera.

[0182] Among them, the second target white point can be the AWB white point, that is, the white point represented by the white point information output by the AWB module in the image signal processor after estimating the white point of the memory frame. The white point information of the second target white point is used to indicate the second target white point, and can be the color information of the second target white point, such as the white point coordinates (R / G, B / G) of the second target white point. The second ambient color temperature is used to indicate the color temperature of the corresponding shooting environment, and can be the correlated color temperature (CCT) of the shooting environment. Exemplarily, the second ambient color temperature is the color temperature detected by a multispectral sensor in the corresponding shooting environment.

[0183] Further, other information corresponding to the current frame captured by the second camera can also be obtained, such as obtaining the second estimated color temperature of the current frame. The second estimated color temperature is the color temperature obtained by estimating the color temperature of the current frame, for example, the color temperature obtained by estimating the color temperature based on the second target white point. Exemplarily, the second estimated color temperature can be the color temperature obtained by estimating the color temperature of the current frame using a white balance algorithm. For example, the second estimated color temperature is the AWB color temperature, that is, the color temperature output after the AWB module in the image signal processor estimates the color temperature of the current frame.

[0184] Step B3: Map the first target white point of the memory frame of the reference camera to the second camera to obtain a mapped white point.

[0185] By first mapping the first target white point of the memory frame of the reference camera to the second camera to obtain a mapped white point, and then fusing the white point information of the mapped white point with the white point information of the second target white point of the current frame, the accuracy of multi-camera color correction can be further improved.

[0186] As an example, the operation of mapping the first target white point of the memory frame of the reference camera to the second camera may include the following steps:

[0187] 1) Obtain the calibration data of the reference camera and the second camera. The calibration data of each camera includes the white point information and color temperature of each camera under multiple standard light sources.

[0188] In the embodiments of the present application, for multiple cameras configured in the electronic device, the white point information and color temperature of each camera under multiple standard light sources can be pre-calibrated, and the calibration data of each camera can be stored.

[0189] Among them, multiple standard light sources can be specified as needed. The standard light source is a light source specified by the standard, such as the light source specified by the International Commission on Illumination for unified color detection. The color temperatures of these multiple standard light sources are different. For example, these multiple standard light sources can be at least two of the standard light sources such as D75, D65, D50, CWF, TL84, U30, A, H, etc. Of course, other standard light sources can also be included, and the embodiments of the present application do not limit this. Among them, the color temperatures of the standard light sources D75, D65, D50, CWF, TL84, U30, A, H decrease in sequence, which are 7500K, 6500K, 5000K, 4150K, 4100K, 3000K, 2856K, and 2300K in sequence.

[0190] As an example, when calibrating the white point information and color temperature of a certain camera under multiple standard light sources, the camera can be used to take pictures under each standard light source respectively, and the white point information of the white point of the captured image can be used as the white point information of the camera under the corresponding standard light source. The color temperature of the camera under each standard light source can be the estimated color temperature of the image captured by the camera under each standard light source, or the color temperature detected by an illuminometer under the corresponding standard light source, or the color temperature of the corresponding standard light source. The embodiments of the present application do not limit this. For example, assuming that the color temperature of each standard light source is used as the color temperature of the camera under each standard light source, the color temperatures of the camera under the standard light sources D75, D65, D50, CWF, TL84, U30, A, and H are 7500K, 6500K, 5000K, 4150K, 4100K, 3000K, 2856K, and 2300K respectively.

[0191] 2) Determine at least one standard light source from multiple standard light sources according to the color temperature difference between the first estimated color temperature of the memory frame of the reference camera.

[0192] That is, determine at least one standard light source from multiple standard light sources whose corresponding color temperature is closest to the first estimated color temperature. The number of these at least one standard light sources can be preset, such as preset to 1, 2, or 3, etc. The embodiments of the present application do not limit this.

[0193] For example, assuming that the number of these at least one standard light sources is 2, then the first 2 standard light sources ranked in order can be determined from these multiple standard light sources according to the ascending order of the color temperature difference between the corresponding color temperature and the first estimated color temperature. That is, determine 2 standard light sources from multiple standard light sources whose color temperature is closest to the first estimated color temperature.

[0194] 3) Determine the white point mapping matrix between the reference camera and the second camera according to the white point information of the reference camera under these at least one standard light sources and the white point information of the second camera under these at least one standard light sources.

[0195] As an example, the logarithm of the white point information of the reference camera under these at least one standard light sources can be taken, and the logarithm of the white point information of the second camera under these at least one standard light sources can be taken. According to the logarithm results, the white point mapping matrix between the reference camera and the second camera can be determined. Among them, taking the logarithm can be taking the common logarithm (i.e., log), or taking the natural logarithm (i.e., ln). The embodiments of the present application do not limit this.

[0196] For example, assume that the number of the at least one standard light source is 2, and the two standard light sources are denoted as L1 and L2. Given the white point information, i.e., the white point coordinates (R / G, B / G), the white point mapping matrix between the reference camera and the second camera can be determined by the following formula (1):

[0197]

[0198] where T is the white point mapping matrix. W dst is the logarithm of the white point coordinates of the reference camera under L1 and L2, that is where is the logarithm of the white point coordinates (R / G, B / G) of the reference camera under L1; is the logarithm of the white point coordinates (R / G, B / G) of the reference camera under L2.

[0199] where W src is the logarithm of the white point coordinates of the second camera under L1 and L2, that is where is the logarithm of the white point coordinates (R / G, B / G) of the second camera under L1; is the logarithm of the white point coordinates (R / G, B / G) of the second camera under L2.

[0200] 4) Map the first target white point to the second camera according to the white point mapping matrix to obtain the mapped white point.

[0201] For example, according to the white point mapping matrix, the first target white point can be mapped to the second camera by the following formula (2) to obtain the logarithm of the white point coordinates of the mapped white point:

[0202]

[0203] where W sync is the logarithm of the white point coordinates of the mapped white point, T is the white point mapping matrix, and log(rg ref ), log(bg ref ) are the logarithms of the white point coordinates of the first target white point in the memory frame of the reference camera.

[0204] After that, the white point coordinates of the mapped white point can be determined according to the logarithm of the white point coordinates of the mapped white point. For example, determine the opposite number of the logarithm of the white point coordinates of the mapped white point, and use the opposite number of the logarithm of the white point coordinates of the mapped white point as the white point coordinates of the mapped white point to convert the mapped white point from the log domain to the original domain.

[0205] Step B4: Determine the fusion weight according to the environmental color temperature difference between the second environmental color temperature and the first environmental color temperature of the shooting environment corresponding to the memory frame.

[0206] Among them, the environmental color temperature difference between the second environmental color temperature and the first environmental color temperature can represent the environmental difference between the current shooting environment and the shooting environment of the memory frame. The environmental color temperature difference is inversely proportional to the fusion weight. The smaller the environmental color temperature difference, the greater the fusion weight.

[0207] In this way, when the environmental color temperature difference is small, it means that the difference between the current shooting environment and the shooting environment of the memory frame is small. In this case, by fusing the second target white point of the memory frame to a greater extent according to the larger fusion weight, a greater degree of color correction can be performed on the current frame, so that the color of the corrected image of the current frame is closer to the image color of the memory frame to a greater extent; when the environmental color temperature difference is large, it means that the difference between the current shooting environment and the shooting environment of the memory frame is large. In the case of a large difference in the shooting environment, it is normal for the current frame and the memory frame to have a color difference. In this case, a smaller degree of fusion of the second target white point of the memory frame can be performed according to the smaller fusion weight to perform a smaller degree of color correction on the current frame.

[0208] In one embodiment, the fusion weight can be determined according to the environmental color temperature difference and the corresponding relationship between the environmental color temperature difference and the fusion weight. In the corresponding relationship between the environmental color temperature difference and the fusion weight, the environmental color temperature difference is inversely proportional to the fusion weight, that is, the smaller the environmental color temperature difference, the greater the fusion weight.

[0209] In another embodiment, determining the fusion weight according to the environmental color temperature difference may further include the following steps:

[0210] 1) Determine the first sub-weight according to the environmental color temperature difference. Among them, the smaller the environmental color temperature difference, the greater the first sub-weight.

[0211] For example, the first sub-weight can be determined according to the current environmental color temperature difference and the corresponding relationship between the environmental color temperature difference and the sub-weight. In the corresponding relationship between the environmental color temperature difference and the sub-weight, the environmental color temperature difference is inversely proportional to the sub-weight.

[0212] For example, the corresponding relationship may include multiple environmental color temperature differences and their respective corresponding sub-weights. If the corresponding relationship includes the sub-weight corresponding to the current environmental color temperature difference, the sub-weight corresponding to the current environmental color temperature difference can be used as the first sub-weight. In addition, if the corresponding relationship does not include the sub-weight corresponding to the current environmental color temperature difference, the sub-weights corresponding to the multiple environmental color temperature differences in the corresponding relationship can be used to interpolate the sub-weight corresponding to the current environmental color temperature difference to obtain the first sub-weight.

[0213] 2) Determine a second sub-weight according to the estimated color temperature difference between the second estimated color temperature and the first estimated color temperature of the memory frame. Wherein, the larger the estimated color temperature difference, the larger the second sub-weight.

[0214] Wherein, the estimated color temperature difference between the second estimated color temperature and the first estimated color temperature can indicate the shooting effect difference between the second camera and the reference camera. The estimated color temperature difference is proportional to the second sub-weight, and the larger the estimated color temperature difference, the larger the second sub-weight.

[0215] In this way, when the estimated color temperature difference is small, it means that the shooting effect difference between the second camera and the reference camera is small. In this case, a relatively small degree of color correction can be performed on the current frame according to the relatively small second sub-weight. When the estimated color temperature difference is large, it means that the shooting effect difference between the second camera and the reference camera is large. In this case, a relatively large degree of color correction is performed on the current frame according to the relatively large second sub-weight to improve the shooting effect difference.

[0216] As an example, the second sub-weight can be determined according to the current estimated color temperature difference and the corresponding relationship between the estimated color temperature difference and the sub-weight. In the corresponding relationship between the estimated color temperature difference and the sub-weight, the estimated color temperature difference is proportional to the sub-weight. Exemplarily, the corresponding relationship can include multiple estimated color temperature differences and their respective corresponding sub-weights. If the corresponding relationship includes the sub-weight corresponding to the current estimated color temperature difference, the sub-weight corresponding to the current estimated color temperature difference can be used as the second sub-weight. Additionally, if the corresponding relationship does not include the sub-weight corresponding to the current estimated color temperature difference, interpolation can be performed on the sub-weights corresponding to the multiple estimated color temperature differences in the corresponding relationship to obtain the second sub-weight.

[0217] 3) Determine a fusion weight according to the first sub-weight and the second sub-weight.

[0218] For example, the product of the first sub-weight and the second sub-weight can be determined as the fusion weight. It should be understood that other methods can also be used to process the first sub-weight and the second sub-weight to obtain the fusion weight, and the embodiments of the present application do not limit this.

[0219] In another embodiment, determining the fusion weight according to the ambient color temperature difference may further include the following steps:

[0220] 1) Determine a first sub-weight according to the ambient color temperature difference. Wherein, the smaller the ambient color temperature difference, the larger the first sub-weight.

[0221] 2) Determine a second sub-weight according to the estimated color temperature difference between the second estimated color temperature and the first estimated color temperature of the memory frame. Wherein, the larger the estimated color temperature difference, the larger the second sub-weight.

[0222] 3) Determine a third sub-weight according to the ambient brightness information of the shooting environment corresponding to the memory frame. Among them, the greater the brightness indicated by the ambient brightness information, the greater the third sub-weight.

[0223] Among them, the ambient brightness information of the shooting environment corresponding to the memory frame is used to indicate the ambient brightness of the shooting environment corresponding to the memory frame. For example, it can be a brightness parameter such as LV. Generally, the smaller the ambient brightness, the more complex the shooting environment and the worse the shooting effect. The greater the ambient brightness, the better the shooting effect. The brightness indicated by the ambient brightness information is proportional to the third sub-weight. The greater the brightness indicated by the ambient brightness information, the greater the third sub-weight.

[0224] In this way, when the brightness indicated by the ambient brightness information is smaller, the image effect of the memory frame of the reference camera is worse. In this case, a smaller degree of color correction can be performed on the current frame according to the smaller third sub-weight. When the brightness indicated by the ambient brightness information is greater, it means that the image effect of the memory frame of the reference camera is better. In this case, a greater degree of color correction can be performed on the current frame according to the larger third sub-weight. Adopting this strategy can ensure the accuracy of color correction.

[0225] As an example, the third sub-weight can be determined according to the target ambient brightness information of the shooting environment corresponding to the memory frame and the corresponding relationship between the ambient brightness information and the sub-weight. Among them, in the corresponding relationship between the ambient brightness information and the sub-weight, the brightness indicated by the ambient brightness information is proportional to the sub-weight. Exemplarily, the corresponding relationship can include multiple ambient brightness information and their respective corresponding sub-weights. If the corresponding relationship includes the sub-weight corresponding to the target ambient brightness information, the sub-weight corresponding to the target ambient brightness information can be used as the third sub-weight. Additionally, if the corresponding relationship does not include the sub-weight corresponding to the target ambient brightness information, interpolation can be performed according to the sub-weights corresponding to the multiple ambient brightness information in the corresponding relationship to obtain the third sub-weight.

[0226] 4) Determine a fusion weight according to the first sub-weight, the second sub-weight, and the third sub-weight.

[0227] For example, the product of the first sub-weight, the second sub-weight, and the third sub-weight can be determined as the fusion weight. It should be understood that other methods can also be used to process the first sub-weight, the second sub-weight, and the third sub-weight to obtain the fusion weight, and the embodiments of the present application do not limit this.

[0228] Step B5: According to the fusion weight, perform a fusion process on the white point information of the second target white point and the white point information of the mapped white point to obtain the white point information of the third target white point.

[0229] For example, according to the fusion weight, the white point information of the second target white point and the white point information of the mapped white point can be fused using the following formula (3) to obtain the logarithm result of the white point information of the third target white point:

[0230] W cur = W cur *(1 - w)+W sync *w(3)

[0231] Wherein, W act is the logarithm result of the white point information of the third target white point, W cur is the logarithm result of the white point information of the second target white point, w is the fusion weight, and W sync is the logarithm result of the white point information of the mapped white point.

[0232] After that, the white point information of the third target white point can be determined based on the logarithm result of the white point information of the third target white point. For example, the opposite number of the logarithm result of the white point information of the third target white point can be determined as the white point information of the third target white point to convert the white point information of the third target white point from the log domain to the original domain.

[0233] Step B6: Perform white balance correction on the current frame according to the white point information of the third target white point.

[0234] As an example, the white balance gain can be determined according to the white point information of the third target white point, and the current frame can be white balance adjusted according to the white balance gain.

[0235] For example, the white point information of the third target white point is the white point coordinates (R / G, B / G). The white balance gains determined according to the white point coordinates of the third target white point include: R channel gain R_Gain = G / R, B channel gain B_Gain = G / B. Correspondingly, for each pixel point in the current frame after white balance adjustment, R' = R * R_Gai; B' = B * B_Gain; the G channel value remains unchanged. Wherein, R and B are the original R value and B value corresponding to each pixel point in the current frame respectively.

[0236] Step B7: Fuse the first estimated color temperature and the second estimated color temperature according to the fusion weight to obtain the third estimated color temperature.

[0237] For example, according to the fusion weight, the first color temperature and the second color temperature can be fused using the following formula (4) to obtain the third color temperature:

[0238] CCT3 = CCT2*(1 - w)+CCT1*w(4)

[0239] Among them, CCT3 is the third estimated color temperature, CCT1 is the first estimated color temperature, w is the fusion weight, and CCT2 is the second estimated color temperature.

[0240] Step B8: Perform image processing on the current frame after white balance correction according to the third estimated color temperature.

[0241] Among them, the image processing may include one or more of color correction (color correction matrix, CCM) and lens shade correction (LSC).

[0242] For example, according to the third estimated color temperature, the color correction matrix of the current frame after white balance adjustment can be determined, and the current frame after white balance adjustment can be color-corrected according to the color correction matrix.

[0243] Please refer to Figure 12 , Figure 12 which is a schematic diagram of the effect comparison before and after color correction of the image frame provided by the embodiment of the present application. It is assumed that the camera switches from the main camera to the telephoto camera for shooting in the same scene. Among them, Figure 12 Figure (a) in Figure 12 is the memory frame taken by the main camera (i.e., the last frame taken by the main camera); Figure 12 Figure (b) in Figure 12 is the image frame taken by the telephoto camera, that is, the image frame before color correction; Figure 12 Figure (c) in

[0244] Next, combining the above Figure 6 and Figure 7 , an example is given to illustrate the process of image correction of the image taken by the current camera based on the occlusion recognition result provided by the embodiment of the present application.

[0245] Please refer to Figure 13 , Figure 13 which is a schematic flowchart of image correction of the image taken by the current camera based on the occlusion recognition result provided by the embodiment of the present application. This method is applied to an electronic device. In the embodiment of the present application, the default-started main camera is used as the reference camera. As Figure 13 shown, the method includes the following steps:

[0246] Step 1301: The user starts the camera application.

[0247] For example, the user can click on the application icon of the camera application to start the camera application.

[0248] Step 1302: In response to the user's start operation, the camera application starts.

[0249] Step 1303: The camera application sends Call Instruction 1 to the main camera.

[0250] Call Instruction 1 is used to call the main camera. After the camera application starts, it can default to calling the main camera for shooting.

[0251] Step 1304: The main camera previews at the default magnification according to Call Instruction 1.

[0252] For example, the default magnification can be 1X, etc.

[0253] Step 1305: The multispectral sensor detects the ambient color temperature.

[0254] During the main camera preview process, the multispectral sensor can detect the ambient color temperature of the shooting environment.

[0255] As an example, after the camera application starts, it can send a start instruction to the multispectral sensor so that the multispectral sensor detects the ambient color temperature according to this start instruction.

[0256] Step 1306: The main camera sends the preview image frame to the ISP.

[0257] The image frame previewed by the main camera can be sent to the ISP for processing first.

[0258] Step 1307: The ISP sends this image frame to the AWB module.

[0259] For example, the ISP can call the AWB module according to the image frame to send the image frame to the AWB module for processing.

[0260] Step 1308: The AWB module uses the AWB algorithm to calculate the statistical information of the image frame. The statistical information includes the AWB white point coordinates, AWB color temperature, statistical white point coordinates, and LV.

[0261] For example, the AWB module can use the AWB algorithm to estimate the white point of the image frame to obtain the AWB white point coordinates, estimate the color temperature according to the AWB white point coordinates to obtain the AWB color temperature, estimate the white point of the image frame to obtain the white point coordinates of multiple statistical white points, and estimate the brightness of the image frame to obtain the brightness information LV.

[0262] It should be noted that, in the embodiments of the present application, only the case where the LV is obtained by estimating the brightness of the image frame through the AWB module is taken as an example for illustration. It should be understood that the LV of the image frame can also be determined by other means. For example, the LV can be obtained through the AE module, or the LV can be obtained through a brightness sensor. The embodiments of the present application do not limit this.

[0263] Step 1309: The user switches to the telephoto camera.

[0264] The user can, according to needs, switch the main camera to the telephoto camera for shooting. For example, the main camera can be switched to the telephoto camera by clicking on the option corresponding to the telephoto camera displayed in the preview interface frame of the camera application.

[0265] Step 1310: In response to the user's camera switching operation, the camera application sends a call instruction 2 to the telephoto camera.

[0266] Among them, the call instruction 2 is used to call the telephoto camera.

[0267] Step 1311: The camera application sends a camera switching notification to the AWB module.

[0268] Among them, the camera switching notification can carry the identifier of the switched camera and can also carry the identifier of the camera before switching. The embodiments of the present application do not limit this.

[0269] Step 1312: According to the camera switching notification, the AWB module sends the statistical information 1 of the memory frame to the camera occlusion recognition module and the color correction module, including the AWB white point coordinate 1, the AWB color temperature 1, the statistical white point coordinate 1, and the LV1.

[0270] Among them, the memory frame refers to the last frame previewed by the main camera.

[0271] As an example, the AWB module can send the AWB white point coordinate 1 and the statistical white point coordinate 1 of the memory frame to the camera occlusion recognition module, and send the AWB white point coordinate 1, the AWB color temperature 1, and the LV1 to the color correction module.

[0272] Step 1313: According to the camera switching notification, the AWB module sends a color temperature data acquisition request to the multispectral sensor.

[0273] Among them, the color temperature data acquisition request is used to request the acquisition of the ambient color temperature of the shooting environment, such as acquiring the ambient color temperature 1 of the shooting environment corresponding to the memory frame.

[0274] Step 1314: According to the color temperature data acquisition request, the multispectral sensor sends the detected ambient color temperature 1 to the color correction module.

[0275] Among them, the ambient color temperature 1 is used to indicate the ambient color temperature 1 of the shooting environment corresponding to the memory frame of the main camera.

[0276] Step 1315: The color correction module stores the statistical information 1 of the memory frame and the ambient color temperature 1.

[0277] For example, it stores the AWB white point coordinates 1, AWB color temperature 1, and LV1 of the memory frame, as well as the ambient color temperature 1.

[0278] That is to say, the color correction module can store the data corresponding to the memory frame so as to perform color correction on the image frames previewed by other cameras according to the data corresponding to the memory frame subsequently.

[0279] Step 1316: The telephoto camera performs real-time preview according to the call instruction 2.

[0280] Step 1317: The telephoto camera sends the current frame of the real-time preview to the ISP.

[0281] Step 1318: The ISP sends the current frame to the AWB module.

[0282] The ISP can call the AWB module according to the current frame to send the current frame to the AWB module for processing.

[0283] Step 1319: The AWB module calculates the statistical information 2 of the current frame using the AWB algorithm. The statistical information 2 includes the AWB white point coordinates 2, AWB color temperature 2, and statistical white point coordinates 2.

[0284] For example, the AWB module can perform white point estimation on the image frame using the AWB algorithm to obtain the AWB white point coordinates 2, perform color temperature estimation based on the AWB white point coordinates 2 to obtain the AWB color temperature 2, and perform white point statistics on the image frame to obtain the white point coordinates of multiple statistical white points (i.e., the statistical white point coordinates 2).

[0285] Step 1320: The AWB module sends the statistical information 2 to the camera occlusion recognition module.

[0286] For example, the AWB module sends the AWB white point coordinates 2 and the statistical white point coordinates 2 to the camera occlusion recognition module.

[0287] Step 1321: The camera occlusion recognition module determines the occlusion recognition result of the main camera according to the difference between the statistical information 1 of the memory frame and the statistical information 2 of the current frame. This occlusion recognition result is used to represent whether the main camera is occluded.

[0288] The camera occlusion recognition module can determine the occlusion recognition result of the main camera according to the camera occlusion recognition method described in the above Figure 8 embodiment. The specific implementation process can refer to the above Figure 8For the relevant descriptions of the embodiments, the embodiments of the present application will not be elaborated herein again.

[0289] Step 1322: The camera occlusion recognition module sends the occlusion recognition result of the main camera to the AWB module and the multispectral sensor.

[0290] Step 1323: If the occlusion recognition result indicates that the main camera is not occluded, the AWB module can trigger the color correction process, that is, execute the following steps 1324 - 1331.

[0291] When the occlusion recognition result indicates that the main camera is not occluded, the AWB module can trigger the color correction process, that is, trigger the color correction module to perform color correction on the current frame according to the relevant parameters of the memory frame.

[0292] Step 1324: The AWB module sends the AWB white point coordinates 2 and the AWB color temperature 2 to the color correction module.

[0293] Step 1325: If the occlusion recognition result indicates that the main camera is not occluded, the multispectral sensor sends the detected ambient color temperature 2 to the color correction module.

[0294] Among them, the ambient color temperature 2 is used to indicate the ambient color temperature of the shooting environment corresponding to the current frame.

[0295] Step 1326: The color correction module calculates the fused white point coordinates and the fused color temperature by using the multi-camera color correction algorithm according to the memory frame and the data corresponding to the current frame.

[0296] Among them, the specific implementation process of calculating the fused white point coordinates and the fused color temperature by using the multi-camera color correction algorithm can refer to the relevant descriptions in the above Figure 11 For the relevant descriptions of the embodiments, the embodiments of the present application will not be elaborated herein again.

[0297] Step 1327: The color correction module sends the fused white point coordinates and the fused color temperature to the AWB module.

[0298] Step 1328: The AWB module performs white balance correction on the current frame according to the fused white point coordinates.

[0299] Step 1329: The AWB module sends the fused color temperature and the current frame after white balance correction to the image processing module.

[0300] Step 1330: The image processing module processes the current frame after white balance correction according to the fused color temperature to obtain the target image frame 1.

[0301] For example, the image processing module may include one or more of the image processing modules such as a CCM module and an LSC module. Exemplarily, color correction and lens shading correction may be performed on the current frame after white balance correction according to the fused color temperature, so as to obtain the processed target image frame 1.

[0302] Step 1331: The image processing module sends the target image frame 1 to the camera application.

[0303] For example, the image processing module may first return the target image frame 1 to the ISP, and then the ISP sends the target image frame 1 to other subsequent processing modules.

[0304] Step 1332: The camera application displays the target image frame 1.

[0305] For example, the camera application may display the target image frame 1 on the preview interface.

[0306] Step 1333: If the occlusion recognition result indicates that the main camera is occluded, the AWB module does not trigger the color correction process, that is, the AWB module performs white balance correction and other subsequent processing on the current frame according to the original algorithm.

[0307] Step 1334: The AWB module performs white balance correction on the current frame according to the AWB white point coordinates 2.

[0308] Step 1335: The AWB module sends the AWB color temperature 2 and the current frame after white balance correction to the image processing module.

[0309] Step 1336: The image processing module processes the current frame after white balance correction according to the AWB color temperature 2 to obtain the target image frame 2.

[0310] For example, the image processing module may include one or more of the image processing modules such as a CCM module and an LSC module. Exemplarily, color correction and lens shading correction may be performed on the current frame after white balance correction according to the AWB color temperature 2, so as to obtain the processed target image frame 2.

[0311] Step 1337: The image processing module sends the target image frame 2 to the camera application.

[0312] For example, the image processing module may first return the target image frame 2 to the ISP, and then the ISP sends the target image frame 2 to other subsequent processing modules.

[0313] Step 1338: The camera application displays the target image frame 2.

[0314] For example, the camera application may display the target image frame 2 on the preview interface.

[0315] In the embodiments of the present application, by identifying the difference in the statistical information of the images captured before and after the camera switch, it is possible to determine whether the camera before the switch is occluded, thereby improving the accuracy of identifying whether the camera is occluded. Additionally, based on the camera identification result, it is possible to determine whether to execute the image correction process, that is, to determine whether to use the memory frame of the camera before the switch as a reference image to correct the image frame captured by the camera after the switch. Since the camera occlusion identification method provided by the embodiments of the present application can more accurately identify whether the camera is occluded, the occlusion identification result identified according to the present application can more accurately correct the image captured by the camera after the switch, improving the accuracy of image correction. For example, it can, to a certain extent, avoid the serious color cast situation caused by using the image captured by the occluded camera as a reference image for color correction.

[0316] The present application also provides a chip that is coupled to a memory. The chip is configured to read and execute computer programs or instructions stored in the memory to perform the methods in the above embodiments.

[0317] The present application also provides an electronic device that includes a chip. The chip is configured to read and execute computer programs or instructions stored in the memory, such that the methods in the embodiments are executed.

[0318] This embodiment also provides a computer-readable storage medium that stores computer instructions. When the computer instructions are run on an electronic device, the electronic device is caused to execute the above-related method steps to implement the methods in the above embodiments.

[0319] This embodiment also provides a computer program product. The computer-readable storage medium stores program code. When the computer program product is run on a computer, the computer is caused to execute the above-related steps to implement the methods in the above embodiments.

[0320] Additionally, the embodiments of the present application also provide a device, which may specifically be a chip, a component, or a module. The device may include a processor and a memory connected thereto. The memory is configured to store computer execution instructions. When the device runs, the processor may execute the computer execution instructions stored in the memory, causing the chip to execute the methods in the above method embodiments.

[0321] Among them, the electronic device, computer-readable storage medium, computer program product, or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be elaborated here.

[0322] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more integrated available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as Digital Versatile Disc (DVD)), or semiconductor media (such as Solid State Disk (SSD)), etc.

[0323] The above are optional embodiments provided by the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the technical scope disclosed in the present application shall be included in the protection scope of the present application.

Claims

1. A method for occlusion recognition of a camera, characterized in that, Applied to an electronic device, the electronic device includes a plurality of cameras, and the method includes: When the electronic device switches from a first camera to a second camera for shooting, determining first statistical information of a memory frame of the first camera and second statistical information of a current frame captured by the second camera; Wherein, the memory frame is the last frame captured by the first camera, the first statistical information includes a first target white point of the memory frame and a plurality of first statistical white points, and the second statistical information includes a second target white point of the current frame and a plurality of second statistical white points; Determining an occlusion recognition result of the first camera according to a difference between the first statistical information and the second statistical information, where the occlusion recognition result indicates whether the first camera is occluded.

2. The method according to claim 1, wherein The determining the occlusion recognition result of the first camera according to the difference between the first statistical information and the second statistical information includes: Determining a target quantity ratio according to the first statistical information, where the target quantity ratio is a ratio of a target quantity to a total quantity of the plurality of first statistical white points, and the target quantity refers to the quantity of first statistical white points among the plurality of first statistical white points whose white point coordinates are within a range centered on the white point coordinates of the first target white point and with a preset distance as the radius; If the target quantity ratio is less than a first threshold, then determining a first ratio and a second ratio according to the first statistical information and the second statistical information, where the first ratio is a ratio of an average value of first color statistical values of the plurality of first statistical white points to an average value of second color statistical values, and the second ratio is a ratio of the average value of the first color statistical values of the plurality of second statistical white points to the average value of the second color statistical values, the first color statistical value is a ratio of the R value to the G value of the corresponding statistical white point, and the second color statistical value is a ratio of the B value to the G value of the corresponding statistical white point; If a ratio of the first ratio to the second ratio is greater than a first ratio threshold or less than a second ratio threshold, then determining that the first camera is occluded; wherein, the first ratio threshold is greater than 1 and the second ratio threshold is less than 1.

3. The method according to claim 2, wherein The if the target quantity ratio is less than the first threshold, then determining the first ratio and the second ratio according to the first statistical information and the second statistical information includes: If the target quantity ratio is less than the first threshold and greater than a second threshold, then determining the first ratio and the second ratio according to the first statistical information and the second statistical information, and the second threshold is less than the first threshold.

4. The method according to claim 3, wherein The method further includes: If the target quantity ratio is less than or equal to the second threshold, then determining that the first camera is occluded.

5. The method according to any one of claims 1-4, characterized in that, After determining the occlusion recognition result of the first camera according to the first statistical information and the second statistical information, it further includes: If the occlusion recognition result indicates that the first camera is not occluded, then performing fusion processing according to white point information of the first target white point and white point information of a second target white point of the memory frame, and performing white balance correction on the current frame according to a fusion processing result. If the occlusion recognition result indicates that the first camera is occluded, perform white balance correction on the current frame according to the white point information of the second target white point.

6. The method according to claim 5, wherein The performing of the fusion processing on the white point information of the first target white point and the white point information of the second target white point of the memory frame, and performing white balance correction on the current frame according to the fusion processing result includes: Performing fusion processing on the white point information of the first target white point and the white point information of the second target white point of the memory frame according to the environmental color temperature difference between the first environmental color temperature of the shooting environment corresponding to the memory frame and the second environmental color temperature of the shooting environment corresponding to the current frame, to obtain the white point information of a third target white point; Performing white balance correction on the current frame according to the white point information of the third target white point.

7. The method according to claim 6, wherein The performing of the fusion processing on the white point information of the first target white point and the white point information of the second target white point of the memory frame according to the environmental color temperature difference between the first environmental color temperature of the shooting environment corresponding to the memory frame and the second environmental color temperature of the shooting environment corresponding to the current frame, to obtain the white point information of a third target white point includes: Determining a fusion weight according to the environmental color temperature difference; wherein, the smaller the environmental color temperature difference, the larger the fusion weight; Performing fusion processing on the white point information of the first target white point and the white point information of the second target white point according to the fusion weight, to obtain the white point information of the third target white point.

8. The method according to claim 7, wherein Before the performing of the fusion processing on the white point information of the first target white point and the white point information of the second target white point according to the fusion weight, it further includes: Mapping the first target white point to the second camera to obtain a mapped white point; The performing of the fusion processing on the white point information of the first target white point and the white point information of the second target white point according to the fusion weight, to obtain the white point information of the third target white point includes: Performing fusion processing on the white point information of the mapped white point and the white point information of the second target white point according to the fusion weight, to obtain the white point information of the third target white point.

9. The method according to claim 8, wherein The mapping of the first target white point to the second camera to obtain a mapped white point includes: Obtaining the calibration data of the first camera and the first camera, where the calibration data includes the white point information and color temperature under multiple standard light sources; Determining at least one standard light source from the multiple standard light sources according to the color temperature difference from the estimated color temperature of the memory frame; Determining a white point mapping matrix between the first camera and the second camera according to the white point information of the first camera under the at least one standard light source and the white point information of the second camera under the at least one standard light source; Mapping the first target white point to the second camera according to the white point mapping matrix to obtain the mapped white point.

10. The method according to any one of claims 7-9, characterized in that, The determining of the fusion weight according to the environmental color temperature difference includes: Determining a first sub-weight according to the environmental color temperature difference; wherein, the smaller the environmental color temperature difference, the larger the first sub-weight; Determine a second sub-weight according to the estimated color temperature difference between the first estimated color temperature of the memory frame and the second estimated color temperature of the current frame; wherein, the larger the estimated color temperature difference, the larger the second sub-weight; Determine the fusion weight according to the first sub-weight and the second sub-weight.

11. The method according to claim 10, wherein Before determining the fusion weight according to the first sub-weight and the second sub-weight, it further includes: Determine a third sub-weight according to the ambient brightness information of the shooting environment corresponding to the memory frame, and the larger the brightness indicated by the ambient brightness information, the larger the third sub-weight; The determining the fusion weight according to the first sub-weight and the second sub-weight includes: Determine the fusion weight according to the first sub-weight, the second sub-weight and the third sub-weight.

12. The method according to claim 11, wherein The determining the fusion weight according to the first sub-weight, the second sub-weight and the third sub-weight includes: Determine the product of the first sub-weight, the second sub-weight and the third sub-weight as the fusion weight.

13. The method according to claim 8 or 9, characterized in that, The fusion weight, the white point information of the second target white point, the white point information of the mapped white point and the white point information of the third target white point satisfy the following formula: The white point information of the third target white point = the white point information of the second target white point * (1 - W) + the white point information of the mapped white point * W; where W is the fusion weight.

14. The method according to any one of claims 6-13, characterized in that The performing white balance correction on the current frame according to the white point information of the third target white point includes: Determine the white balance gain according to the white point information of the third target white point; Perform correction on the current frame according to the white balance gain.

15. The method according to claim 14, wherein The method further includes: Perform a fusion process on the first estimated color temperature of the memory frame and the second estimated color temperature of the current frame to obtain a third estimated color temperature; Perform color correction on the current frame after white balance adjustment according to the third estimated color temperature.

16. The method according to any one of claims 1-15, characterized in that, The first camera is a pre-set reference camera among the multiple cameras, and the second camera is any one of the multiple cameras other than the reference camera.

17. An electronic device, characterized in that, The electronic device includes: one or more processors, and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the electronic device to execute the method according to any one of claims 1-16.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions, and when the instructions run on the electronic device, the electronic device is caused to execute the method according to any one of claims 1-16.

Images